Optimal Video Playback in Managed Desktop Application

Recently I had to build a desktop application that allows users to watch video files. After becoming familiar with Microsoft’s Windows Presentation Foundation, this seemed to be really easy: .NET Framework 3.5 provides a MediaElement control which you place in your UI and then assign a source file to play. It is simple to build your own transport controls right into the UI that match your design and there are lots of examples of how to do this on MSDN.

As I guessed, this was easy to wire up and my first pass looked like this:

WPF Example Using MediaElement

Here is a complete Visual Studio 2008 project (MediaElementPlayer) that you can build to see how it works.

Great.  Game Over, right?

Well, we watched it for a while, and found that the media player was dropping frames.  The video files would playback perfectly in Windows Media Player, but when run through this WPF app, it just didn’t work as well.  In many applications, maybe this is not a big deal, but we always want to achieve the best possible playback quality.  I tried several different attempts to optimize the application: attempting to force a specific rendering framerate, stripping down the application to just the MediaElement so that no other compositing layers or animations would tax the processor.  I tried playing lower bitrate media files, but nothing worked.  The playback still consistently dropped frames and stuttered.

I commissioned a special video file that is designed to make dropped frames and stuttering really noticeable. It features a vertical line which scrolls back and forth slowly – motion should always be perfectly smooth. You can download this video and try it with the project above.

Perhaps you can see the same behavior on your system if you build the project above.  Despite its ease of use, the MediaElement control does not allow a lot of flexibility in terms of tweaking its performance or finding out metrics on how well it is actually playing.

So I tried a different approach.  Based on some research from Jeremiah Morrill it seemed like we could use the older ActiveX Windows Media Player component to programmatically playback video inside our application. I found a tutorial and fairly quickly added the ActiveX control to playback the video files. This required building a separate forms library in order to automatically expose the necessary components as references, but it worked. This implementation seemed to harness the native Windows Media rendering pipeline and the playback performance was exactly the same as playing the video in Windows Media Player. The video was perfectly smooth again. But that wasn’t the complete solution.

The catch is that the older Windows Media wrapper was designed for Windows Forms and is only supported inside a Windows Forms Host control. This is a major problem because Windows Forms hosts can not be layered in a WPF UI layout the same way other controls can. They always show up on top, and everything else is obscured behind it. This is ultimately because of a fundamental difference between how WPF is rendered and the legacy windowing system. In this case, it meant that there was no way to add custom transport controls on top of the video. We could use the default Windows Media Player skin and transport controls, but then it gives away the secret and makes the app look thrown together. It would be much better to have the play and stop buttons match the look and feel of the rest of the application.

The final solution involved creating the transport controls in a separate transparent window and layering that on top of the video player, programmatically repositioning it automatically to create the illusion that they are part of the video player.

<Window x:Class="ActiveXMediaPlayer.TransportControlWindow"
Background="Transparent" WindowStyle="None" AllowsTransparency="True"
ShowInTaskbar="False" ResizeMode="NoResize" SnapsToDevicePixels="True" Topmost="True">

This ends up requiring listening to all of the re-size and layout events from the main window and responding appropriately. It took a lot of attention to corner cases when the video player goes full screen and when the primary window loses focus, but ultimately this solution achieves the necessary effect.

You can download the improved Visual Studio project (ActiveXMediaPlayer), and compare the quality for yourself.

Many thanks to Jeremiah Morrill for his very in-depth blog that covers all aspects of video and rendering in Windows.