Video formats: Open video – what exactly do we mean?
We have been having our periodical video formats review internally and in the interest of open practice we wanted to précis this for people as well as keep clients up to date with our thinking. Brace yourselves: this one might get a bit technical. In writing this I am synthesizing the more technical thoughts and knowledge of Lewis, Ady, Adam and Steph – they will get into answering more technical questions below!
In case anyone just wants the bottom line, our conclusion is that we will be encoding to a H264 standard and creating MPEG4 files. For archive content these will be played out using Flash Player on desktop machines and HTML5 on mobile devices. Live content will use Flash Player on the desktop, Flash Player on Android and the apple native player on iOS devices. We will be doing extensive testing of the HTML5 format against various Android builds to check that it will give us enough reach – if not then we may need to revert to Flash Player for Android devices for streaming of archive content as well.
This set of decisions provides Live and Archive streaming to most mobile and desktop devices and we will publish a full test matrix for clients to review. The exceptions are devices running Windows Mobile, which will not be able to view live streams. This is a limitation with the Windows Mobile platform as there are currently no viable live streaming options for Windows Mobile devices.
There is an elephant in the room on all this, however, which is that it is difficult to make good choices for the public at the same time as making good choices for corporate infrastructure. Please skip to the bottom if this is your priority.
How we got there
As anyone reading probably knows, we provide a streaming service for around 70 local authorities and a number of other clients. Streaming media – like any technology – is a fairly fast moving field and our policy is to make researched choices about the best option and then recommend a couple of options to clients. We do this rather than deciding on a single format/approach because there is no perfect answer to this question. The streaming field has a number of dominant proprietary formats but no single open format. The fact that mobile devices are now an essential part of the mix has further complicated things. Essentially this means that there is no single format that will serve all devices/states that we want to reach without having to make (and charge for) a far greater hardware investment than we think the market will bear. This is an assumption which is of course open to challenge and we do offer this as a choice to people who might want to spend their way out of the problem.
What are we really talking about?
Video formats are actually about three different things:
- The encoding algorithm
- The file that this creates
- The video player that is used by the viewer to watch the content
These are a set of interrelated choices – some simpler than others.
In trying to agree on a format recommendation here are the parameters that we work to:
- Content should be available on all platforms – mobile and desktop. We define this against a testing matrix that includes device and O/S
- We want to use industry standards not proprietary formats
- We also want to use open standards where possible – see below
- We want to provide the best – and most efficient video experience possible
- We want to minimize storage and hardware costs which means we are looking for a solution that involves just one encoding process and a single file for storage
- We want to future proof our work – we try to review these standards around every two years but keep an eye on technology shift in the meantime
- We want to prioritize the viewing public over the needs of internal networks (we will come back to this – its a biggie)
How ‘open’ is all this?
We get asked about Open Standards, Open Source and Open Data frequently and this has a three part answer:
- Open Standards – absolutely. Where a standard exists we use it (hence adoption of H264).
- Open Source – trickier. There is no viable open source product at any of the three stages described above that we have found. We came across an open source video player (The WebM Project) but on further investigation this wasn’t going to work on mobile devices and would require a large deployment of software updates if we used this as the desktop option. If we have missed something in this area then please let us know
- Open Data – interesting. We are currently looking at how we would create two new product elements. The first would be an option to download and store content that is created in our system offline so that it is more accessible (we need to consult with clients more on this one) and we are also looking at creating an API for the core platform that will mean that we can open things up more – these are all plans for the next 18 months so we will post more when they are further along.
We really believe in being open – hence this blog post in fact – but there is not a clear ‘open’ choice with respect to the video format question that we can see. What we are working on instead is making sure that the content is available to as many people as possible.
Accessibility is to some extent about how many people can view the content and we think that we have made a lot of improvements on this with our move to responsive design and also the fact that we are becoming pretty much device agnostic. However we are not yet able to offer things like transcription as standard so we are some way off having a standard solution which can do that at a price that clients think is reasonable. This is still an area of R&D for us (we are part of an EU Research Project called I2Web which is looking at these issues).
One thing that we do think improves the accessibility of the content is the ability to do live social reporting of the content alongside the video and we would like to run some trials to explore to what extent this could plug the gap left by the absence of cost-effective live transcription. If anyone is interested then please get in touch.
More detail on those choices:-
The encoding algorithm
This was the easy one! H264 is an industry standard and a fairly obvious choice.
The file that this creates
Again – MPEG4 is now also the most obvious – and open – standard to choose and we are pleased that this moves us away from the previously most accessible choice which was a proprietary Windows Media format.
The video player that is used by the viewer to watch the content
This was more difficult. We know that a lot of people have reservations about Flash Player but it really is now the dominant video player on people’s desktops and we believe its early accessibility issues have been addressed as long as it is deployed properly.
HTML5 should be the answer here but it is yet to prove itself for live content and also on the desktop – it does not work across all browsers and the implementation is often inconsistent.
Windows Media Player is no longer dominant in this market because of its inadequacies on mobile platforms and the fact that Microsoft are no longer pushing this technology forward.
The elephant in the room…
….is the fact that very few clients have deployed Flash Player on the desktop as yet. Hmmmm….
There are good and bad reasons for this but the effect is that there is considerable drift between in the technologies which the public are using online and the ones that are used on corporate networks (this is not just a Public Sector problem but a problem for anyone who needs to get content to two very different audiences). This is a problem which is further compounded by the fact that we are seeing iPads, etc, turn up in meetings of all kinds and so we have competing needs within a single client – the same client who can’t view HTML5 or Flash on the desktop wants to be able to view the same content on their mobile device – which demands one of these two formats.
We have assumed that clients don’t want to have to double up on the encoding process and storage costs and so our decision has been to create a recommendation (in the Spoiler) which prioritizes the end viewer. We will keep the legacy infrastructure running for those clients that cannot leave Windows Media Player or Silverlight behind on the desktop for a reasonable period but our aim will be to be migrate people away from the Windows Media infrastructure. Clients who stay with this legacy option will be able to continue to view content within their own infrastructure but will only be able to offer mobile content to viewers who use Windows Media Player – which is a tiny proportion of this market. They will not have the option of offering content for Flash or HTML5 as these depend on an MPEG4 storage format rather than the WMV format which is what the Windows architecture produces.
We can’t force people to move formats but we do feel strongly that we need to ensure that this content works for the external audience – and we simply can’t ignore the fact that many people want to be able to view content on mobile devices.
Please let us know if you have any views about any of this. Rousing cries of support would also be welcome as we start to persuade dozens of ICT departments that they need to update their desktop install to take into account the needs of the public – wish us luck!