Why Don't PCs Have Hardware Video Decoders?

Robert X. Cringely speculates in his latest column that Apple may soon be adding H.264 hardware decoding and encoding to its Macintosh line.  Cringely is wrong more often than he is right so the truth of this rumor is unknown.  I'm sure someone at Apple is at least studying the idea.  More interesting to me than whether or not Apple is shipping a $50 chip in all Macs to do H.264 is his talk about why no one has done this before.  He hints at a conspiracy of sorts to keep video-accelerating hardware away from the public.  I worked on DVD playback at Microsoft from nearly the inception.  There are good reasons you don't have hardware decoders in your system right now.  Let me explain.

Cringely says, "Maybe the reason is economic (save the $7) or maybe it is political (Microsoft or maybe Apple are for some reason opposed to hardware decoding). But like a lot of real reasons, I think it probably comes down to hubris and the simple fact that by decoding video in software, road warriors have another incentive to buy a more expensive -- and more powerful -- computer."  The truth is, Microsoft did work back in 1997 to enable hardware decoding in Windows.  The support shipped in the form of DirectX Media around Thanksgiving of that year.  I think it was in version 5.2 of the SDK.  This functionality first shipped with Windows in Windows 98.  Several vendors took advantage of this support.  Toshiba had an early system which shipped with DVD decoding in hardware.  I still have one of their full-length PCI boards hanging in my office.  IBM shipped the Thinkpad with hardware decoding for several years.  Dell offered an upgrade to a Luxsonor decoder card.  Creative shipped the DXR2 and DXR3 at retail.

Despite these many offerings, the market for hardware decoders never grew very big.  Why not?  Because software could do the job and it was substantially cheaper.  Whereas a hardware decoder might cost $10-$50 in COGS (just a guess), a software decoder could be had for something like $1 in volume.  Even on early PCs, the CPU was fast enough to decode the MPEG2 streams for DVD.  A Pentium 2 - 266 could almost keep up and a 333 had plenty of horsepower.  Today's operating systems and background processes sap a lot more power than did Windows 98 but there's still plenty of power there.

Rarely, however, does DVD play on a computer though without some sort of hardware assistance.  As early as the ATI Rage II+, display cards included special hardware designed to accelerate MPEG2 decoding.  On Windows 95 and its variants, this hardware was accessed in a propreitary fashion.  With Windows 2000, Microsoft created a standard called DirectX Video Acceleration (DXVA) which is used today by all DVD decoders to utilize this hardware.  Depending on the display chip, you may get more or less acceleration but the software is always offloading at least a portion of the work to the GPU.

In June of 2005 Microsoft shipped DXVA support for WMV-9.  With Windows Vista, we also support accelerating VC-1 (the variant of WMV used in HD-DVD and BluRay) and H.264 via DXVA.

So, is there a conspiracy to get you to buy more expensive PCs just to play video?  No.  It saves money to decode video without a specialized device just for video.  It is for economic reasons that there are not dedicated hardware decoders.  It's cheaper (and just as effective) to decode video in software with a hardware boost than to ship a chip to do just decoding.

So why then is Apple possibly considering a video acceleration chip in the Mac?  There are two possible reasons.  The first is that H.264 is really, really hard to decode.  It is a lot harder to decode than MPEG-2, DIVX, or even WMV9/VC-1.  Support for H.264 acceleration on graphics chips is also behind the support for other codecs.  Probably the more likely reason is for encoding.  Encoding is a lot harder than decoding (because it requires a full decode in addition to the encode portions).  If Apple wants to do something like timeshifting in H.264, it will need hardware to do it.  Media Center (now included in Vista Home Premium) usually uses dedicated MPEG-2 encoding chips for the TV functionality.

Microsoft Announces HD-Photo

Once known as Windows Media Photo, Microsoft just released a new photo format called HD-Photo.  Bill Crow has a good writeup on the benefits of the new format.  Basically it allows for high-fidelity photo editing and storage.  While those of us in the normal world do all of our photography in JPEG format, professionals do not.  They use TIFF or a format called RAW (which is propreitary to each camera).  These formats are less aggressive in their compression and allow for greater color detail.  HD-Photo is similar but attempts to be a more interoperable standard.  It also supports better compression than the standards I mention.  This is a technology to watch.  JPEG has some serious deficiencies which something like HD-Photo can alleviate.  Two questions come to my mind:

1. Will camera makers and photo editing suites adopts HD-Photo?


2. Will consumers care?  Is JPEG good enough for them?

Only time will tell if HD-Photo takes off or not.  If it does, I think everyone will benefit.

TechFest 2007

Microsoft has a large number of pure researchers working for it.  They are collectively known as Microsoft Research (MSR).  Once a year a large contingent of these researchers gather in Redmond for an event called TechFest.  All full time employees are invited over to attend lectures and wander the "show floor" where various teams have set up booths.  The researchers are available to talk to and are more than happy to describe what it is they are working on.  There are people working on 3D graphics, podcasting, video, image detection, automated error detection in code, operating system concepts, etc.  You can see some videos of the items available here.

TechFest is interesting.  I think it is Microsoft's attempt to avoid being Xerox PARC.  For those unfamiliar with PARC, it was the Palo Alto Research Center that Xerox set up in the 1970s to explore the future of computing.  Amazing things happened there.  Among the inventions were the personal computer (Alto), Ethernet, the laser printer, the bitmapped display, and the modern WIMP GUI.  With all this cool stuff being invented, why isn't Xerox's name associated with computers today?  They were a photocopying company and didn't have an interest in actually developing any of their inventions.  Techfest is an opportunity for people in the product teams to meet the people in the research teams and hopefully cross-pollinate.  Many items from research have made their way into our products via just this method.

The Ephemeral Nature of Computers

My wife and I are talking about getting some landscaping done.  One of the interesting options we have is to get a computerized sprinkler system.  The system connects to your PC and uses the web to determine the weather.  It can use this information to make intelligent choices.  For instance, if rain is coming, it won't bother to water.  This sounds really cool until you stop to think about it.  How long do we expect the sprinkler system to last?  10 years?  20 years?  What is the chance that computers will be around in 20 years that can still run this irrigation system?  Even if we preserve one, what is the chance that whatever web service is behind it will still exist and will still have the same interface?  Zero?

One problem computers face is that they accelerate the pace of change.  Sometimes this can be very good.  It is the secret to much of the large increase in productivity this country has experienced.  However, because things change so quickly, nothing stays around very long.  Within the lifetime of everyone reading this, there will likely be several major computing platforms.  So far, we are not very good at preserving the old ones.  Try finding a machine to read a 5 1/4" floppy disk.

This, by the way, is the main reason I won't buy DRM'd music.  When formats change, I can always re-rip my CDs.  What if Apple stumbles and the iPod becomes uncool?  What will I be allowed to do with my FairPlay-encrypted music files?  I can't transcode them because they are encrypted.  I still have the CD's I bought 15 years ago in High School.  They still work.  Does anyone think that the iPod will still be using the same music format in 15 years?  If you think it will, just go talk to anyone who bought music from MSN music.

Will the web change this?  Will it preserve things better or worse?  My vote is for worse.  Take for example our digital photos.  When I have 3 gigs of pictures on my hard drive I have some hope of preserving them.  Let's say that .jpg dies out and some new format comes to dominate.  I can always run a conversion program to keep my pictures around.  Now assume that all of the photos are on Flickr instead of my hard drive.  How do I preserve them?  Sure, Flickr could just convert them for me but what if Yahoo goes out of business?  What if they just decide that they are not making money on the free storage of photographs model?  Then I have to find a way to download them all, convert them, then upload them to another site.  It can be done, but it's more custom work than just getting a photo converter program to run locally.  When we put our data into someone else's hands we lessen our ability to manipulate it which lessens our ability to preserve it.

Consider gaming.  I can still find ways to play games written 25 years ago for the Commodore 64.  What's the chance anyone will be able to experience World of Warcraft in the year 2030?

Soundscapes 101

Back in May of 2006 I was invited to a recording session at Microsoft Studios (yes, we have out own studios--very cool place) with Robert Fripp.  He's the artist brought in to help develop the Windows sound for Vista.  This was the 2nd of two sessions I'm aware he had here at Microsoft.  Robert Scoble was invited to the session and videotaped it.  I'm told that this is a rare recording of Fripp at work.  He goes through the creation of a soundscape and gives an explanation the gear he is using.  He also plays a few early versions of what became the Vista "Pearl Sound."  It amazes me that all this work was distilled down to just 4 notes.  I kinda like the power chord versions.  The video of the recording session is followed by an interview with Fripp.  The person working closest with Fripp here is Steve Ball who is one of the Program Managers I work closely with as part of the audio team in Windows.  If you have any interest in Fripp or the sound production process, I suggest you give this video a view.

P.S.  You can spot me in the white t-shirt at about the 7:30 mark.

Geeks Wanted

I thought this was too cool not to post.  For those that don't have the ASCII table memorized, the sign says "Now Hiring".

Hat tip to Kotaku.

Teaching My Son To Program

A few months back I read the article, "Why Johnny can't code" by David Brin.  He talks about the trials he had teaching his son to BASIC.  Just yesterday, I came across this article by Nat Torkington entitled, "Why Johnny Can't Program."  This one talks about his adventures teaching programming to youngsters.  He didn't use BASIC but rather Logo and Lego Mindstorms.  The comments there are a good source for other suggestions.  Some suggested languages aimed at kids like KPL or Squeak.  Others suggested that Ruby or Python were the way to go.  I think this latter group probably doesn't have young children.

My son is almost seven and insists that he's going to work at Microsoft when grows up.  Makes a father proud.  He enjoys math and for a while I've thought about how to introduce him to the concept of programming.  I thought about Basic, but after reading Nat's article, I thought better of it.  "Hello World" probably doesn't appeal to the average 7 year old.  Something with visuals might be more interesting to start with.  Many of the suggestions were too complex for a first language.

I decided to start small and settled on Logo as my introductory language of choice.  Logo also happens to be the first language I used back in 4th grade.  I recall finding it being taught without much context but I enjoyed it.  I thought I'd give it a shot.  After some looking around, I chose the free MSWLogo.

Tonight I introduced it to him.  He picked it up quickly and really enjoys it so far.  We'll see how long it holds his attention.  I suspect we'll have to move on from Logo before too long.  If things go well, I'll return to this topic in the future.

If you have suggestions on language or techniques that have worked for you, please send me mail or leave a comment.