Skip to content

Using GStreamer with Python Part 2

November 20, 2014

Now that you’ve gotten a taste of what you can do, it’s on to learn some better functionality of GStreamer.  There is really a lot you can do with GStreamer, especially when it comes to video processing.  In this guide, we will be focusing on streaming with the playbin2 element. We will also be working with Pads that have Sometimes along with GhostPads and Bins.

Playbin2 Project

Playbin2 is a great element for GStreamer 0.10.  I believe that in GStreamer 1 and up Playbin2 has been renamed to Playbin. Playbin2 is a single element that can stream media from the internet and play it with little configuration.  Ontop of that, it has several built-in features that let you do a lot with the single element.

Now for our next project we will be streaming video from a website and changing the video using another element and then outputting that product.  To do this, we will be using 5 elements.

  • playbin2 which will get our multimedia from the internet, play the audio on speakers and feed the video to the rest of our elements.
  • decodebin which will decode the video to raw media.
  • autoconvert which we use mostly to connect decodebin to solarize (I’m not sure what its doing behind the scenes)
  • solarize which adds a neat video effect to the video.
  • autovideosink which outputs the video to the monitor.

Now we will take a moment to look at all of our elements Pad Templates.  run gst-inspect-0.10 on each of our elements.  First off, you will notice that playbin2 has no elements.  This still has be a little dumbfounded since we will be giving it a pad in this project.  This will require us to build the project a little differently that our previous one.  Next look at decodebin, it has an Always Sink pad and a Sometimes Source pad. Here the Sometimes pads will only be created when the stream is ready.  It takes some time for that to happen so it will require a callback function which I will explain later.  This Sometimes pad will make connecting to the autoconvert element a little more difficult but we can do it. Autoconvert has Always Sink and Source pads.  Solarize has Always Sink and Source pads with some different capabilities but we wont need to pay attention to that because the next element, autovideosink has an Always Sink pad with ANY capabilities.

Now we will put our project together.  Import gst and create a Bin object.  We will use this to encapsulate all of our elements except our playbin2.  Later we will make the bin connect to the playbin2 through a special GhostPad which we will create.  Now create all of the elements we need and make sure they call created properly. And last, we will tell the playbin2 to stream a file from the internet.  Please note that there is a commented out line for media_source.set_property.  If the steam jitters at all, it might be an issue with a slow internet connection and downloading the webm file and playing it locally will also work.

From here we need to add all the elements to our output_bin except the playbin2 element. We’ll see how playbin2 connects to this output_bin later on.

Now, you remember how the decodebin element had a Sometimes Source pad, right? The Sometimes pads may take some time to appear because they are only created when the input stream has been handled and different pads may appear depending on that stream.  When they have been created, decodebin will emit a singal, “new-decoded-pad” to tell you when it is ready.  We will connect a callback function to this signal and handle the pad linking within this callback function.

Here in the function we are given the pad, we use that to get the parent which is our decodebin. Then we pipeline in order to get the instance of the element we want to connect our decodebin to.  We link the elements. Fairly simple, we waited until the decodebin had source pads and linked them using a callback function.  Now we link the rest of our elements in the output_bin.

What we have now is a playbin2 element which can stream video, and a bin full of elements that can manipulate those streams.  We still don’t have playbin2 connected to our output_bin.  What we need to do in order to connect these two, is create a sink pad for our Bin and tell playbin2 it has elements it can send its data to.  First we will get a static pad from the first element in our bin, decode.  We then use the gst.GhostPad method to create a special pad which a bin can handle. This GhostPad appears to just forward the data to the element which you made the GhostPad from. We then activate it and add it to the bin.

Now in order to connect the playbin2 to our output_bin, we set the playbin2’s property ‘video-sink’ to our bin.  I don’t really know why the property isn’t named video-source, that would seem more consistent to me.  Lastly, we set playbin2’s state to gst.STATE_PLAYING.  We get the playbin2’s bus, wait until theres an error or EOS and finally Free the resources.

When you run your project you should see some altered video from the Sintel trailer and normal audio.  To recap, playbin2 retrieves the multimedia from the website, plays the audio to the speakers and forwards the video feed to our output_bin.  It only feeds the video because that’s all that’s been requested because of the solarize and autovideosink elements. Once the video feed is in our bin it gets handled by decodebin and autoconvert, then manipulated by solarize, and output to the monitor by autovideosink.

If you’d like to see the project as a whole go to https://gist.github.com/markwingerd/a3621cf1dc9d379afcb6

One issue I ran into was that decodebin stopped linking randomly.  After spending some time figuring out why it wasn’t working, it started randomly working again.  No actual code was changed to my knowledge.  What I think happened was that it took too long for decodebin to create the pads but I can’t be sure since I cannot reproduce the issue.

Next we will learn how to split data so we can do multiple things with it.

Advertisements

From → GStreamer, Python

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: