June 8, 2015 - Luis de Bethencourt
Getting Things Done at the 2015 GStreamer Hackfest
Over the weekend of March 13-15th, the Samsung Open Source Group office in Staines-upon-Thames, UK, hosted 34 developers of the GStreamer project for a hackfest. GStreamer is a library for constructing graphs of media-handling components, and its uses range from simple music file playback and audio/video streaming to complex audio mixing and video processing.
A lot of familiar faces showed up, as well as an unusual number of new people, and it was a very productive hackfest. While everybody hammered away on laptops, we worked on and discussed a variety of topics related to both the framework and applications.
Discussions to Be Had…
Some of the discussions that took place on the framework side included:
- How to move forward with the DASH common encryption – Patches have been sitting in Bugzilla for this for a while. An agreement was reached on how to simplify things and make them more generic so its possible to add a minimal, new API rather than needing to add specialized libraries for each encryption scheme.
- Moving bug tracking from Bugzilla to Phabricator – There seems to be a general consensus this is what we need to do; the main question is what needs to be done before this is possible. We still need to determine where to host it, developing scripts for porting the current bug information, and any potential new maintenance needs.
- New design for how glimagesink outputs textures into applications – The first application for this is webkitgtk. Instead of having a custom webkitvideosink (defined and registered in WebKit code), you can now have glimagesink set up a GL context for resource sharing. Glimagesink does not create an internal window, but rather creates a GL context which shares resources with WebKit’s GL context.
- Plans for gap events supported by the aggregator base class – This is so it can be used with sparse streams, and the goal is to use this in a new SPU/subtitles base class.
- How the RTP stack deals with various network conditions and how to improve this functionality – Currently, when there is a network burst (like when a stream starts) the jitterbuffer deals with it very poorly.
- Customary discussion about documentation – The Pitivi people have implemented a more dynamic method for dealing with API docs, linking examples to API docs in a more modern way. The general consensus is that it could be a good framework to write more docs. These efforts have been postponed for a future Hackfest.
There were also a few questions and ideas related to PulseAudio development and some discussion about this topic. Finally, we had mandatory talks about ideas for GStreamer 2.0 and what we want to change in the future. Nothing was urgent, so 1.0 is here to stay for the foreseeable future.
…And Work to Be Done
Check out the event page to see all of the developers who contributed to this hackfest. Here are some of the highlights of our hands-on work:
- Tim fixed some long-standing issues with the MPEG-TS muxer. He made the “alignment” property work for UDP streaming, and fixed a problem with pass through delta unit / keyframe flagging so that tcpserversink can keep a backlog from the last keyframe and do burst-on-connect when new clients connect.
- Anton ported all of the 15 shader effects of GLeffects from GL to GLES2.
- Arun prototyped his work on profiles in audio sinks/sources: work he revived from a long pending bug report. The idea is that we should allow clients to specify what kind of media they are playing and adapt our filtering and buffering appropriately. He wants to make setting up the sink easy for developers and save power where possible.
- While fixing a bug, Arun also stumbled on a problem with the way PulseAudio sets up latency in capture devices for recording. He wrote a solution for this, and after some testing will submit it for review.
- Thibault refactored the ges-launch command line interface branch which involved some internal cleanups and refactoring.
- Wim reviewed and merged the patches from William that improved file descriptor passing between GStreamer processes. William has been toying with pulsevideo for his project which involves capturing video and passing it between containers. The captured video is turned into a file descriptor which is passed between 2 GStreamer pipelines using sockets.
- Jan and Nirbeek took a look at the alpha blending code in the compositor element and found many bugs. They fixed the alpha blending implementation in ORC for SSE and PPC versions and briefly looked at the ARM code. Nirbeek also wrote some unit tests for a video-converter and found some rounding errors that looked larger than expected. Wim investigated a little and experimented with different ways to make the rounding errors smaller. This presents a trade off between speed and accuracy, and no real solution has been found yet.
- Nicolas, Thiago, and Victor were hacking on v4l2src and working to find methods that can be used to implement renegotiation of the format. The problem is that with the v4l2 API, its only possible to change the capture format after all buffers are returned to the driver, so it’s necessary to drain all the buffers from the pipeline.
- Jan and Wim briefly looked at the current state of the stereoscopic video patches. The biggest challenge is to represent all possible layouts in the caps and negotiate them between elements. Jan made a new GstFlagSet type to GStreamer in order to simplify negotiation. There is some more work to do before this is ready to be merged, but it looks like they are making progress and we hope those patches will soon be ready.
- Sebastian started merging and cleaning up an Android camera source element.
- Thiago refactored camerabin to simplify its pipeline and be able to use bufferpools.
- Finally, there were a lot of other bug fixes and patch reviews all around.
The Best Part
Since the second day of the Hackfest happened to be Pi Day (March 14th). We enjoyed some delicious pieces of pie.
Our next event, a GStreamer Conference, is scheduled to coincide with LinuxCon EU in Dublin, Ireland which is scheduled for October 7th – 9th. Looking forward to seeing you all there. Stay tuned for more details as this date approaches. In the meantime, check out this time lapse video of the event, and thank you everybody for coming!
About Luis de Bethencourt
Luis de Bethencourt is a freedom-loving technocrat who works for the Samsung Open Source Group in London. He has always enjoyed programming and playing around with video. Since he discovered GStreamer 5 years ago he's been hooked. Originally from the Canary Islands, computers felt like a door to the world, and Luis saw open source software as the best way to enter the innovative technology community. He enjoys being in front of the screen, behind the screen, Friday beers, Sunday ice-creams, walks in the park, and people who read bios to the end.
Image Credits: Christophe Dumas -via Wikimedia Commons