December 2, 2015 - Shuah Khan
Media Resource Sharing Through The Media Controller API
Media devices have hardware resources that are shared across several functions. However, media drivers have no knowledge of which resources are shared. At the root of this problem is a lack of a common locking mechanism that can provide access control.
I have been working on solving the sharing problem for a year or so. At the Media Summit in San Jose back in March of this year, we reviewed my Media Token API patch series. It solved the problem with minimal changes to drivers, however, it introduced a new framework, in addition to the existing ones. Every new framework adds maintenance cost. This led to a discussion that identified the existing Media Controller API as a better alternative in the interest of avoiding adding a new framework. Since then, I’ve worked on implementing the solution using the Media Controller API, and I’ve ported it over to Media Controller Next Gen API.
In this article, I will cover how ALSA and au0828 drivers share media resources using the Media Controller Next Gen API on Win-TV HVR 950Q Hybrid USB TV stick. In addition, I will share how media-ctl/mc_nextgen_test tool can be used to generate media graphs for a media device. Let’s first start looking at the differences between Multi-function Device (MFD) and Media devices.
How do Media Devices Differ from a Multi-function Device (MFD)?
Media devices can be very small, yet complex as they consist of a group of independent devices that share an attach point. Each device implements a unique function, and sharing isn’t limited to the attach point. Functions can be shared, and media drivers have to coordinate this sharing. In some cases, non-media drivers are leveraged as in the case of Win-TV HVR 950Q, snd-usb-audio for audio is leveraged to drive the audio chip on the device.
MFD presents itself to the kernel as a single device. The Linux Kernel MFD framework helps identify functions as discrete platform devices. Sharing is limited to the attach point with sub-devices attached to a shared bus. Each function is independent with no shared resources other than the attach point. MFD drivers don’t need to coordinate sharing resources.
Challenges with Sharing Media Resources
Drivers are unaware of shared resources on a media device and they don’t have common data structures to ensure exclusive access to shared resource/function. The fine grain locks that exist today don’t ensure exclusive access and are local to function drivers; there are no global locks or data structures that span all drivers. For example, starting a digital TV application disrupts video streaming. This is because the digital driver can access/change modes while a video (Analog) TV application is using the tuner resource.
Exclusive Access Use-Cases:
- Starting Digital application should not disrupt active Video and Audio streams.
- Starting Video application should not disrupt active Digital and Audio streams.
- Starting Audio application should not disrupt Video and Digital streams.
- Querying current configuration (read only access) should work even when the tuner resource is busy.
What is the Media Controller API?
The Media Controller API is a relational media graph framework. Media functions and resources can be represented as nodes on a graph. The media device sits at the top as the root node, and nodes can be connected using media links to represent data paths. These links can be marked active to indicate them as active data paths between two or more nodes; this allows data pipelines to be started.
For example, a tuner node can be linked to a decoder with the tuner operating as the source and the decoder operating as the sink. Then, an additional video sink node can be linked to decoder to display output. A data pipeline can be started between the video node and the tuner once the link between the tuner and decoder is marked as active. As long as this pipeline is active, no other node will be allowed to activate their link to the tuner, thereby reserving the tuner resource for exclusive use.
Using The Media Controller API to Share Resources on a Win-TV HVR 950Q
To illustrate this, take a look at the Win-TV HVR 950Q device/driver view:
As you can see in this device/driver view, USB Device (struct usb_device) and USB Device Parent (struct device) are the only two common data structures across all the drivers for this device. If we were to create a media device at the USB Device Parent, then the media device will be common across all drivers. The Managed Media Controller API allows the media device to be created as a USB Device Parent device resource. This Media Device is created by either au0828 or snd-usb-audio as their probe routines run. The first probe routine that runs wins. This Managed Media Device serves as the root node for the media graph.
Managed Media Controller API:
Drivers use probe routines to create media graph nodes for their functions and resources. Much like when drivers create new media nodes, other drivers might need to take some action. The Media Controller Entity Notify API are new interfaces that enable drivers to register callbacks to take action as entities get added by other drivers.
Media Controller Entity Notify API:
Drivers add add entities for their functions and the bridge driver (au0828) creates links between entities via the entity_notify API. Please note, the bridge driver takes ownership for creating links as it is the master driver that knows how resources are shared and the relationships between them.
HVR 950Q Media Graph
This has given us a media graph that is common and shared by all drivers, but the next step is to provide a means to acquire and release shared resources. A new Media Controller Enable source handler grants access to shared resources or returns -EBUSY if not possible. Its counterpart, the Disable source marks shared resource as free. Bridge drivers (e.g au0828) implement and register enable/disable source handlers to manage access to shared resources on the device. Other drivers call the enable source handler from the media device to gain access to the resources and the disable source handler to release it.
snd-usb-audio and au0828 drivers have been changed to take advantage of the new Media Controller API. Now we look at a few media graphs when various applications own the tuner resource.
Audio Application Controls the Tuner
Audio Application Releases the Tuner
Video Application is Running
Video Application Releases the Tuner
Digital Application is Running
Digital Application Releases the Tuner
Video Application is Started while an Audio Application is Running
Digital Application is Started while an Audio Application is Running
Audio Application is Started while a Digital Application is Running
Video Application is Started while a Digital Application is Running
Digital Application is Started while a Video Application is Running
MC Next Gen Test Tool
Mauro Carvalho Chehab has developed a new mc_nextgen_test tool to generate the media graphs illustrated above.
Usage: mc_nextgen_test [OPTION…]
A testing tool for the MC next geneneration API
-d, –device=DEVICE media controller device (default: /dev/media0
-D, –dot show in Graphviz format
-e, –entities show entities
-i, –interfaces show pads
-I, –intf-links show interface links
-l, –data-links show data links
-?, –help Give this help list
–usage Give a short usage message
-V, –version Print program version
Demystifying Media Devices
Media devices can be small, yet very complex. I hope this post helps understand the complexity of these devices and how the Media Controller API helps solve the resource sharing problem across media and non-media drivers that control the device.
About Shuah Khan
Shuah Khan is a Senior Linux Kernel Developer at Samsung's Open Source Group. She is a Linux Kernel Maintainer and Contributor who focuses on Linux Media Core and Power Management. She maintains Kernel Selftest framework. She has contributed to IOMMU, and DMA areas. In addition, she is helping with stable release kernel testing. She authored Linux Kernel Testing and Debugging paper published on the Linux Journal and writes Linux Journal kernel news articles. She has presented at several Linux conferences and Linux Kernel Developer Keynote Panels. She served on the Linux Foundation Technical Advisory Board. Prior to joining Samsung, she worked as a kernel and software developer at HP and Lucent.
Image Credits: Derivative of work by Subcommandante