Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Porting to gst1.0 #7

Open
wants to merge 138 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
138 commits
Select commit Hold shift + click to select a range
776432c
Initial import
Nov 2, 2010
d2a41b0
Add autogen scripts
Nov 2, 2010
afa8379
Add ChangeLog
Nov 2, 2010
1c99329
Fix spacing and alignment.
Nov 2, 2010
82659b6
Need __init__.py in encoders directory.
Nov 3, 2010
4c6a882
Add debian packaging files.
Nov 3, 2010
eb7735c
Fixed maintainer email.
Nov 3, 2010
05b8f9c
Fix more deps for debian package
Nov 8, 2010
e680b54
Add mp3 and aac fake muxers for audio streaming
Nov 3, 2010
f19200b
Add missing makefiles to AC_OUTPUT.
wulczer Mar 27, 2011
2151747
x264enc module doesn't take a profile.
mithro May 16, 2011
4e38770
Adding a proper gitignore file.
mithro May 17, 2011
78f0dda
Merge branch 'master' of git+ssh://github.com/mithro/flumotion-ugly
mithro May 17, 2011
27e0a0f
Adding an append-options method.
mithro May 26, 2011
1106422
Reduced icecast maximum retry interval after disconnect
Mar 8, 2012
8fac77c
Corrected retry attempt errback being called twice
Mar 8, 2012
38677a2
Adding a justintv component. Fixes #13
mithro Mar 14, 2012
d4a09ca
Fix a couple of typos in feedcomponent
Mar 26, 2012
0a3ecad
Restart video and audio rate elements when we receive a reset event
Mar 29, 2012
ef154c7
Check if the event has an structure asociated
Mar 29, 2012
b850927
Restart the encoder element when we receive a flumotion-reset event
Mar 29, 2012
ce8a37a
Fix where we log when using unwindGenerator/inlineCallbacks
Apr 12, 2012
c038e49
Update vorbis encoder name to be consistent with other encoders
Apr 17, 2012
f85a535
Add new command to get downstream components from a component id
Apr 18, 2012
0ad0f58
Add icystreamer's admin UI into a bundle
Apr 25, 2012
f198b52
Fix a typo left when refractoring _duration attribute
Apr 25, 2012
af3b27c
Add mimetypes for HLS
May 4, 2012
9157531
Ignore old feeders (which are None) when printing the downstreamlist
May 7, 2012
f5984bd
Force a revision of moap's command module
Jun 14, 2012
5e7fe93
Add files from external to ignore
Jun 14, 2012
a11f603
Fix icystreamer tests.
Jun 18, 2012
0880d6a
Some fixes for twisted 12
Jun 18, 2012
57b2a88
The changes were introduced in 11.1.0
Jun 19, 2012
b66b3a2
Do the version comparison properly (fix previous commit)
Jun 19, 2012
395d71f
Update common submodule
Jul 5, 2012
80f4c3b
Update common submodule
Jul 5, 2012
8d1a6b3
clean up docs a little; add anonymous checkout info
Jul 13, 2012
6aaa04e
Allow logWrite to take fragment duration for time_connected
Jul 18, 2012
ddfca43
Fix tests for hls resource
Jul 25, 2012
d91841e
Don't crash when fetching stream data without a plug
Jul 25, 2012
79a0990
Revert "Allow logWrite to take fragment duration for time_connected"
Jul 26, 2012
405cadb
Revert "Fix tests for hls resource"
Jul 26, 2012
fb9e1f7
We should store the uid as session-id to not break logging
Jul 26, 2012
d699994
add section about i18n and admin ui
Jul 26, 2012
a55735c
update
Jul 26, 2012
ee7ecc8
Added extra attributes option to component command
Aug 7, 2012
c2fffe3
fix output of flumotion-nagios -h by importing gst non-globally
Aug 27, 2012
8637181
remove ChangeLog; add note on imports
Oct 7, 2012
886031a
update to common on github and with efc.py
Jan 3, 2014
35f7f05
Fixed .travis.yml
ApsOps Mar 8, 2014
7b7625b
Using publicly accessible flumotion-common repo
ApsOps Mar 10, 2014
c8e6a4f
Added required packages to .travis.yml
ApsOps Mar 10, 2014
f93d2b6
Added faulthandler module to get error details on travis
ApsOps Mar 11, 2014
c30f35a
Changed flumotion reactor to be used for tests on travis
ApsOps Mar 12, 2014
7fa3095
Edited .travis.yml for parallelizing the builds
ApsOps Mar 14, 2014
5a15777
Using xvfb as display for testing on travis
ApsOps Mar 16, 2014
7686b0c
Added missing dependencies in .travis.yml
ApsOps Mar 16, 2014
821dc69
Fixed .travis.yml
ApsOps May 7, 2014
5a3487e
Travis build porting branch
ApsOps May 9, 2014
15b14eb
gstreamer.py ported
ApsOps May 23, 2014
74e0be1
Changed Gtk2Reactor to Gtk3Reactor
ApsOps May 25, 2014
d76ab6c
boot.py ported
ApsOps May 28, 2014
97e021c
pygobject.py ported
ApsOps May 29, 2014
7800772
feeder.py ported
ApsOps May 29, 2014
be7714a
padmonitor.py ported
ApsOps May 30, 2014
632534c
missed while porting gstreamer.py
ApsOps May 30, 2014
cc53caa
feedcomponent ported
ApsOps May 30, 2014
7193402
Fixed .travis.yml
ApsOps May 31, 2014
8725365
Moved TIME_ARGS function definition to __init__.py
ApsOps Jun 3, 2014
53142b3
Imported GstNet module in boot.py
ApsOps Jun 3, 2014
e67face
Run tests for ported modules only
ApsOps Jun 4, 2014
4808cf7
Run travis tests for ported modules only
ApsOps Jun 4, 2014
fed4322
fixed whitespace in __init__.py
ApsOps Jun 4, 2014
edda81d
Build gst-python1.0 from source
ApsOps Jun 4, 2014
1f2feab
removing TIME_ARGS definition since gst-python1.0 installed
ApsOps Jun 5, 2014
85eb124
Added required package to .travis.yml
ApsOps Jun 5, 2014
6514428
Fixed .travis.yml
ApsOps Jun 5, 2014
364566b
Missed while porting feedcomponent
ApsOps Jun 5, 2014
4a19b1b
Ported Videotest component
ApsOps Jun 5, 2014
7d28ee0
Ported check.py
ApsOps Jun 5, 2014
29f59d9
Fixed 'pspec is Nonetype' error
ApsOps Jun 6, 2014
4f77172
Added test config file
ApsOps Jun 6, 2014
f120b2b
changes missed while porting
ApsOps Jun 9, 2014
898392e
Ported theora encoder
ApsOps Jun 9, 2014
31d841b
Ported httpstreamer
ApsOps Jun 11, 2014
6f1e924
fixed 'set_blocked_async' change
ApsOps Jun 12, 2014
1445aad
Ported ogg muxer
ApsOps Jun 12, 2014
ac8f046
Updated test configuration
ApsOps Jun 12, 2014
8dd1823
fixed gst.TYPE_ELEMENT_FACTORY error
ApsOps Jun 14, 2014
a30d49f
ported volume effects
ApsOps Jun 14, 2014
e12abe2
ported audiotest producer
ApsOps Jun 14, 2014
f32abab
Ported Vorbis encoder
ApsOps Jun 14, 2014
2f78eca
FIXME: manually assigning correct value of Gst.CLOCK_TIME_NONE
ApsOps Jun 14, 2014
8223a46
Updated test_config for test video+audio stream
ApsOps Jun 17, 2014
0698152
Ported VP8 Encoder
ApsOps Jun 27, 2014
8fe990e
Merge branch 'master' of https://github.com/timvideos/flumotion into …
ApsOps Jun 27, 2014
70c97d3
Disker consumer ported
ApsOps Jun 30, 2014
104d421
Fixed the test for feeder component
ApsOps Jun 30, 2014
42b8555
Ported FGDP component
ApsOps Jul 7, 2014
38d1892
Ported Dirac encoder
ApsOps Jul 8, 2014
b029961
Ported effects modules
ApsOps Jul 21, 2014
6bfe84d
Ported producer checks
ApsOps Jul 21, 2014
9fa3fdb
Porting DVSwitch component
ApsOps Jul 21, 2014
ddcdfeb
Porting DVSwitch component - more changes
ApsOps Jul 22, 2014
cf3cfbf
Porting generic decoder
ApsOps Jul 25, 2014
cd9c9cf
fixed usage of deprecated functions listenWith and connectWith
Jan 30, 2014
c3a75ca
Use hashlib instead of the deprecated md5 library.
mithro Jul 27, 2014
ebd3890
Removing the _PatchedModuleImporter hook.
mithro Jul 27, 2014
0e2839f
ported av producer
ApsOps Jul 29, 2014
ef73d8c
ported effects
ApsOps Jul 29, 2014
35659f2
Merge branch 'modern-twisted-fix' of https://github.com/timvideos/flu…
ApsOps Jul 29, 2014
0377bc6
More changes in effects modules
ApsOps Jul 30, 2014
aa04bda
more changes in decodercomponent and generic.py
ApsOps Jul 30, 2014
e3c7a51
Other feedcomponent changes for generic decoder
ApsOps Jul 30, 2014
77ab735
Fixing merge conflicts
ApsOps Jul 31, 2014
40c6bad
more fixes in effects
ApsOps Aug 3, 2014
a008519
Merge branch 'modern-twisted-fix' of github.com:timvideos/flumotion i…
ApsOps Aug 3, 2014
1ea3fa7
some fixes in eventfunc
ApsOps Aug 16, 2014
358f639
remove fixme comments
ApsOps Aug 16, 2014
918d8e5
Remove eventfunc from effects
ApsOps Dec 2, 2014
18be35f
Fix porting error in videoscale effect
ApsOps Dec 2, 2014
4db5e02
Merge branch 'porting-dvswitch' into porting-to-gst1.0
ApsOps Dec 2, 2014
fc1abec
Update README
ApsOps Dec 10, 2014
5908d23
Use Github flavoured README
ApsOps Dec 10, 2014
a1d4885
Prepare for merge with flumotion
ApsOps Dec 18, 2014
65e239b
Merge flumotion-ugly into branch ugly-components
ApsOps Dec 18, 2014
b938280
Add missing dependency to README
ApsOps Dec 19, 2014
28facaf
Remove project name from ugly components xml
ApsOps Dec 19, 2014
7e66618
Fix error in handle_reset_event
ApsOps Dec 22, 2014
2fd02b3
Ported x264 encoder
ApsOps Dec 22, 2014
05820bf
Ported lamemp3 encoder
ApsOps Dec 22, 2014
787cb74
Change faac encoder to voaac
ApsOps Dec 22, 2014
36dd43c
Change Faac muxer to Voaac
ApsOps Dec 22, 2014
7816325
Port and add mp4 muxer
ApsOps Dec 22, 2014
0fb4f34
Merge branch 'ugly-components' into porting-to-gst1.0
ApsOps Dec 22, 2014
7d92dfd
Fix x264 encoder
ApsOps Feb 10, 2015
37f3c36
Add new muxer: Matroska(mkv)
ApsOps Feb 10, 2015
406f192
Fix errors in MKV muxer
ApsOps Feb 16, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 1 addition & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -66,13 +66,7 @@ doc/reference/html/

# Ignore external command
flumotion/extern/command.revision
flumotion/extern/command/ChangeLog
flumotion/extern/command/README
flumotion/extern/command/__init__.py
flumotion/extern/command/command.py
flumotion/extern/command/help2man.py
flumotion/extern/command/manholecmd.py
flumotion/extern/command/test_command.py
flumotion/extern/command/*

# Misc ignores
data/flumotion-admin.desktop
Expand Down
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "common"]
path = common
url = git://code.flumotion.com/flumotion-common.git
url = https://github.com/Flumotion/flumotion-common.git
33 changes: 33 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
language: python
python:
- "2.7"
virtualenv:
system_site_packages: true
# installing missing packages
before_install:
- sudo add-apt-repository ppa:gstreamer-developers/ppa -y
- sudo apt-get update -qq
- "export DISPLAY=:99.0"
- "sh -e /etc/init.d/xvfb start"
install:
- sudo apt-get install -qq subversion autopoint python-gst0.10 gstreamer0.10* python-gi python3-gi python-gobject-dev gstreamer1.0* gir1.2-gstreamer-1.0 gir1.2-gst-plugins-base-1.0 libglib2.0-dev gir1.2-glib-2.0 libgirepository1.0-dev libglib2.0-0 gir1.2-gtk-3 libxml-parser-perl python-twisted python-gtk2 python-glade2 python-kiwi pkg-config libglib2.0-dev liborc-0.4-dev bison flex
- sudo pip install icalendar==2.2 pyparsing python-dateutil
- wget https://launchpadlibrarian.net/170386464/gst-python1.0_1.2.0.orig.tar.gz
- tar xvzf gst-python1.0_1.2.0.orig.tar.gz
- cd gst-python-1.2.0/
- ./configure
- make
- sudo make install
- cd ..
# command to run tests
script:
- ./autogen.sh
- make
- ./run_tests.sh
# specified which branches to test
branches:
only:
- porting-to-gst1.0
# parallelizing the builds
env:
- TEST_TYPE=test
173 changes: 73 additions & 100 deletions README
Original file line number Diff line number Diff line change
@@ -1,129 +1,122 @@
Flumotion - your streaming media server
Flumotion - Your streaming media server
=======================================

WHAT IT IS
----------
[![Build Status](https://travis-ci.org/aps-sids/flumotion-orig.svg?branch=porting-to-gst1.0)](https://travis-ci.org/aps-sids/flumotion-orig)

Flumotion is a streaming media server based on GStreamer and Twisted.

This is the first public development series of Flumotion.
It will probably still contain bugs, be difficult to install (since
you need recent dependencies), and cause you some headaches.
This is the first public development series of **Flumotion ported to Gstreamer 1.x API**.
It is still a **work in progress** and will probably still contain bugs, be difficult to install (since some parts still need to be ported), and cause you some headaches.

On the other hand, when it works and you are capturing your Firewire camera
on one machine, encoding to Theora on a second (with an overlay, of course),
encoding the audio to Vorbis on a third, muxing to Ogg on a fourth, and then
streaming both audio and video from a fifth, audio only from a sixth,
video only from a seventh, and capturing all three streams to disk from
an eigth, you feel very good about yourself.
an eighth, you feel very good about yourself.

And you also have too many computers.

REQUIREMENTS
Requirements
------------

You need:
* Gstreamer 1.2.4 or higher
* Gstreamer plugins - good, bad and ugly
* Python bindings GObject Introspection Libraries
* Twisted 13 or higher
* and some other python packages

Gstreamer: you need GStreamer version 0.10.10 or higher. 0.10.11 or higher is
strongly recommended.
**Note:** You might have to use Gstreamer Developers PPA to get the latest packages

For 0.10.x:
GStreamer: 0.10.10 or higher
GStreamer plugins base 0.10.10 or higher
GStreamer plugins good 0.10.1 or higher
GStreamer python bindings 0.10.1 or higher
To install system requirements on Ubuntu, run follow commands:

PyGTK - 2.8.3 or higher
Python - 2.4 or higher
Twisted - 2.5.0 or higher
Kiwi - 1.9.13 or higher
PyCairo- 1.2.0 or higher
```bash
$ sudo add-apt-repository ppa:gstreamer-developers/ppa
$ sudo apt-get update
$ sudo apt-get install libtool autopoint autoconf libxml-parser-perl python-gi python-gobject-dev python-gst-1.0 gir1.2-gstreamer-1.0 gir1.2-gst-plugins-base-1.0 libglib2.0-dev gir1.2-glib-2.0 libgirepository1.0-dev libglib2.0-0 gir1.2-gtk-3 pkg-config libglib2.0-dev liborc-0.4-dev python-twisted python-pyparsing python-kiwi python-dateutil

# we can't use latest python-icalendar till a few things are fixed

$ sudo pip install icalendar==2.2
```

And depending on what codecs you want to use:

libvorbis 1.0.1 or higher
libogg 1.1 or higher
libtheora 1.0alpha3 or higher
* libvorbis 1.0.1 or higher
* libogg 1.1 or higher
* libtheora 1.0alpha3 or higher

And if you want to build documentation:

epydoc
* epydoc

And if you want support for java applets:

cortado
* cortado

ISSUES
------
Quick Start
-----------

Some issues have been brought to our attention. Please take note if you
run into them:
- locales with a comma as a decimal point trigger problems in the XML parsing.
Run with LANG=C if you run into this
- When encoding to Vorbis, prefer to use quality instead of absolute bitrate.
For absolute bitrate, for each sample rate a limited range of bitrates is
allowed.
- You may see memory leaks if you use the soundcard or firewire source
components, and you use PyGTK prior to 0.8.5, or gst-python prior to 0.10.3,
so upgrading to these versions is strongly recommended.

A FIRST TEST
------------
```bash
# Build source
$ ./autogen.sh
$ make

Once everything is built and installed,
you can try this to start the server:
# To run the tests
$ ./run_tests.sh

terminal 1:
flumotion-manager -v -T tcp conf/managers/default/planet.xml
# in terminal 1
$ ./env bash
$ flumotion-manager -v -T tcp test_config.xml

terminal 2:
flumotion-worker -v -T tcp -u user -p test
# in terminal 2
$ ./env bash
$ flumotion-worker -v -T tcp -u user -p test

terminal 3:
flumotion-admin
Unselect 'Secure connection via SSL', press Next, then enter "user" and
"test" in the dialog.
# in terminal 3
$ gst-launch playbin2 uri=http://localhost:8800/ogg-audio-video/
```

Go through the wizard, chosing defaults.
Some systems may not have `gst-launch`, but only `gst-launch-0.10` or `gst-launch-1.0`. In that case you can substitute either of those commands.

Double-click the http component after the wizard has completed
Alternatively, you can use any theora-enabled network player or web browser to see the stream.

terminal 4:
gst-launch playbin2 uri=http://localhost:8800/ogg-audio-video/
```bash
# in terminal 4
$ for a in `seq 1 9`; do ( flumotion-tester http://localhost:8800/ogg-audio-video/ & ); done
```

Some systems may not have gst-launch, but only gst-launch-0.8 or
gst-launch-0.10, in that case you can substitute either of those commands.
This will throw 9 processes with 100 clients each at the server. Hopefully, all of them will return success at the end of their run!

Alternatively, you can use any theora-enabled network player to see the
stream.
We use 900 clients rather than a nice round number such as 1000 because, with a standard unix system, you'll usually be limited to slightly under 1000 clients by default due to limits on open file descriptors (each client requires a file descriptor). This limit is changeable, but it's simpler to just test with slightly fewer clients.

terminal 5:
for a in `seq 1 9`; do ( flumotion-tester http://localhost:8800/ogg-audio-video/ & ); done
Issues
------

This will throw 9 processes with 100 clients each at the server. Hopefully,
all of them will return success at the end of their run!
Take a look at the Github issues. Please create any new issues you encounter.

We use 900 clients rather than a nice round number such as 1000 because, with
a standard unix system, you'll usually be limited to slightly under 1000
clients by default due to limits on open file descriptors (each client
requires a file descriptor). This limit is changable, but it's simpler to just
test with slightly fewer clients.
Hacking
-------

DOCUMENTATION
-------------
The user manual is available [here](http://www.flumotion.net/doc/flumotion/manual/en/trunk/html/), but it's probably outdated since currently there is no admin GUI to work with. Still, it might be helpful if you are new to Flumotion.

A manual is being written. You can access the current work in progress at
http://www.flumotion.net/doc/flumotion/manual/en/trunk/html/
The developer documentation is available [here](https://code.flumotion.com/trac/wiki/Documentation/DeveloperIntroduction). More information will be added as the work progresses.

Flumotion also comes with API documentation. You need epydoc to build them.
Contributing
------------

HACKING
-------
The project had been quite dead for a while. This port was done as a [GSoC 2014](https://www.google-melange.com/gsoc/homepage/google/gsoc2014) project under [TimVideos](http://code.timvideos.us/).

You can subscribe to the official development list. Information is at
http://lists.fluendo.com/mailman/listinfo/flumotion-devel

You can use jhbuild with the flu.modules and jhbuildrc file provided in misc/
You can visit us on IRC: [#fluendo](irc://irc.freenode.net/#fluendo) and [#timvideos](irc://irc.freenode.net/#timvideos) on irc.freenode.org

This will set up everything except for Twisted for you from source.
You can visit our trac installation for the Wiki, source browsing and
ticket tracking:
https://core.fluendo.com/flumotion/trac

SECURITY
Security
--------

Read the security chapter in the aforementioned manual to get started.
Expand All @@ -142,32 +135,12 @@ sure you change the configuration before moving it into production.
Remove the host entry from the sample planet.xml file to allow other hosts
to connect.

ABOVE AND BEYOND
----------------

You can subscribe to our development list. Information is at
http://lists.fluendo.com/mailman/listinfo/flumotion-devel

You can visit us on IRC: #fluendo on irc.freenode.org

You can visit our trac installation for the Wiki, source browsing and
ticket tracking:
https://core.fluendo.com/flumotion/trac

LICENSING
---------
Licence
-------

This version of the Flumotion Streaming Server is licensed under the the
terms of the GNU Lesser General Public License version 2.1 as published
by the Free Software Foundation. See LICENSE.LGPL

If the conditions of this licensing strategy are not clear to you, please
contact Thomas Vander Stichele ([email protected]).

TESTING
-------
For testing purposes of your streams, you can run some GStreamer pipelines
to verify if things work as they should.

- multipart/jpeg stream (video only):
gst-launch -v gnomevfssrc location=http://localhost:8802 ! multipartdemux ! jpegdec ! xvimagesink
Loading