These APIs generally offer a number of functionalities, like video decoding, post-processing, or presentation of the decoded frames. Correspondingly, plugins generally offer a different GStreamer element for each of these functions, so pipelines can be built to accommodate any need.

For example, the gstreamer-vaapi plugin offers the vaapidecode, vaapipostproc and vaapisink elements that allow hardware-accelerated decoding through VAAPI, upload of raw video frames to GPU memory, download of GPU frames to system memory and presentation of GPU frames, respectively.

It is important to distinguish between conventional GStreamer frames, which reside in system memory, and frames generated by hardware-accelerated APIs. The latter reside in GPU memory and cannot be touched by GStreamer. They can usually be downloaded to system memory and treated as conventional GStreamer frames when they are mapped, but it is far more efficient to leave them in the GPU and display them from there.

GStreamer needs to keep track of where these “hardware buffers” are though, so conventional buffers still travel from element to element. They look like regular buffers, but mapping their content is much slower as it has to be retrieved from the special memory used by hardware accelerated elements. This special memory types are negotiated using the allocation query mechanism.

This all means that, if a particular hardware acceleration API is present in the system, and the corresponding GStreamer plugin is also available, auto-plugging elements like playbin are free to use hardware acceleration to build their pipelines; the application does not need to do anything special to enable it. Almost:

When playbin has to choose among different equally valid elements, like conventional software decoding (through vp8dec, for example) or hardware accelerated decoding (through vaapidecode, for example), it uses their rank to decide. The rank is a property of each element that indicates its priority; playbin will simply select the element that is able to build a complete pipeline and has the highest rank.

So, whether playbin will use hardware acceleration or not will depend on the relative ranks of all elements capable of dealing with that media type. Therefore, the easiest way to make sure hardware acceleration is enabled or disabled is by changing the rank of the associated element, as shown in this code:

https://gstreamer.freedesktop.org/documentation/tutorials/playback/hardware-accelerated-video-decoding.html?gi-language=c