Error "no fmt_url_map or conn information found in video info" with specific video

[xmagnum]

Well, with this video http://www.youtube.com/watch?v=1FX4zco0ziY, I was getting an error because it seams that youtube-dl is not finding any valid format.

$ youtube-dl http://www.youtube.com/watch?v=1FX4zco0ziY
[youtube] Setting language
[youtube] 1FX4zco0ziY: Downloading video webpage
[youtube] 1FX4zco0ziY: Downloading video info webpage
[youtube] 1FX4zco0ziY: Extracting video information
ERROR: no fmt_url_map or conn information found in video info

But, debugging I found that the problem seams to happen in the line 1082
if 'fmt_url_map' in video_info and len(video_info['fmt_url_map']) >= 1 and ',' in video_info['fmt_url_map'][0]:

For this specific video, the variable video_info['fmt_url_map'] has value:
['5|http://v10.lscache7.c.youtube.com/videoplayback?sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0hPR1dRVl9FSkNOOV9PS1pB&fexp=901033%2C904531%2C902300&algorithm=throttle-factor&itag=5&ipbits=0&burst=40&sver=3&signature=BF2B1CC7E1919209CAEF7208B1E83C3BC54E50DE.8705DABF3E0172EF8CC92B6828EEFB278BF03D0B&expire=1302768000&key=yt1&ip=0.0.0.0&factor=1.25&id=d455f8cdca34ce26']

So, I just commented the last part of the if that checked for a comma and it worked fine. I don't know why the comma checking is being done, especially because we make a call to split(',') latter. Maybe some special case :D

Well, hope it helps. I'm new to github so I didn't know if there is other way to suggest a code change then creating an issue, so sorry if I posted in the wrong place.

[adamgibbins]

I'm getting the exact same error on this video:
% ./youtube-dl http://www.youtube.com/watch\?v\=L3hRToC23mQ
[youtube] Setting language
[youtube] L3hRToC23mQ: Downloading video webpage
[youtube] L3hRToC23mQ: Downloading video info webpage
[youtube] L3hRToC23mQ: Extracting video information
ERROR: no fmt_url_map or conn information found in video info

[kenrestivo]

Yep, this one too: http://www.youtube.com/watch?v=TkPiXRNee7A

This patch fixes it, as noted above:

diff --git a/bin/youtube-dl b/bin/youtube-dl
index 3ac27a8..5946ed4 100755
--- a/bin/youtube-dl
+++ b/bin/youtube-dl
@@ -1079,7 +1079,7 @@ class YoutubeIE(InfoExtractor):
# Decide which formats to download
req_format = self._downloader.params.get('format', None)

•   if 'fmt_url_map' in video_info and len(video_info['fmt_url_map']) >= 1 and ',' in video_info['fmt_url_map'][0]:
  

•   if 'fmt_url_map' in video_info and len(video_info['fmt_url_map']) >= 1 :
      url_map = dict(tuple(pair.split('|')) for pair in video_info['fmt_url_map'][0].split(','))
      format_limit = self._downloader.params.get('format_limit', None)
      if format_limit is not None and format_limit in self._available_formats:
  

[rifter]

Any chance a new release could come out with this fix in?
Even the link to the cvs commit seems to be broken for me.

[phihag]

@rifter Should be fixed in my fork (download). Does that work for you?

[gitupandgo]

@philhag, @kenrestivo

even with your latest update, it is failing on the new format youtube videos such as:
http://www.youtube.com/watch?v=gELpXbIbtPc
(this video is part of a live broadcast, and the url will permanently expire very soon, probably at the end of July 2011 or early August)

[phihag]

@gitupandgo I can reproduce the error with the mentioned youtube URL. The video is completely blocked for requests from Germany, so it'll take some time for me. It looks like there is a promising fmt_stream_map parameter we could fall back to.

[phihag]

@gitupandgo Should work now, although I haven't downloaded the whole video yet, as my US server's connection is really slow. Can you test again with my fork?

[gitupandgo]

Yes, that's fixed it, although it now says while downloading that the file size is 2GB, which is a bit odd because it is a live webcam, streaming continuously without any fixed size or time limit.

[phihag]

@gitupandgo Oh, ok. I'll leave my test running and see if it gets over 2GB. Can you watch more than 2GB at a time?

[phihag]

@gitupandgo My test finished: youtube-dl creates a 2GB file. I'm suspecting that the youtube flash client requests a new file every 2GB. How do you think youtube-dl should handle those infinite streams? (Reassembling two or more files is quite hard). At the moment, you can call youtube-dl in a loop in a shell script or so.

[gitupandgo]

I dunno, but I'd say it would be more "Unixy" if youtube-dl was able to handle the situation without stopping after every 2GB. Since the youtube flash player will play a live webcam stream indefinitely without any user interaction to make it continue playing after receiving every 2GB, I think youtube-dl should do the same. youtube-dl could simply append a new suffix when it is saving each 2GB chunk with filenames like filename_1, filename_2, ..., filename_N for the N-th. Would that work?

[gitupandgo]

or perhaps: filename, filename_2, ..., filename_N
to retain compatibility with filenames for sizes below 2GB.

[phihag]

That's a reasonable proposal (we've already a similar functionality with --auto-number). There will be a short gap in between successive files, but otherwise, it should work. I'll have a shot at it, but probably not before the weekend (plus, testing it will take quite some time).

[mooffie]

@phihag Your fork works for me as well.

[pleira]

@phihag, with your version, I could get rid of the error.

[rifter]

@phihag I tried your patched version and it seems to work. Incidentally, I noticed that there is a release today but it is giving a 404 error.

[anondavid]

youtube-dl is broken! For most videos it gives this ERROR: no fmt_url_map or conn information found in video info. Anyone working on fixing this?

[phihag]

@anondavid A new version is out by now and should have fixed the problem. If it still persists, open a new bug, and include:

• The youtube URL (or better, the whole command line)
• The output of youtube-dl --version
• The output of python --version

[anondavid]

Thanks. The new version is working for me now. Will let you know if i hit any problems.

[girlpunk]

I've still got problems with this error

[girlpunk]

oh wait, I didn't have the updated version. turns out I updated about 10 mins before it was released.

[chartley]

@phihag thanks mate, excellent work

[camilopereira]

youtube-dl --update could fix this issue

[gitupandgo]

@philhag
If it's threaded properly, why would there be any gap in between successive 2GB files? Is it an issue with python?

[phihag]

@gitupandgo Well, there are a lot of corner cases that have to be considered. For example, how do you detect that a stream is infinite? How do you prevent infinite loops of a finite file/stream? What do you do if the title changes from request to request? What do you do on errors? Do you splice the files together? How do you detect a lagging stream? Note that there is no notion of 2GB "files" on youtube's side, it's just packaging the stream into chunks.

The problem is not Python-specific at all, but I'm afraid quite complex - too complex for me. I'd appreciate a code suggestion (if possible, with tests).

[gitupandgo]

Ok, I think I see what you're thinking. But I'm thinking to myself do these sorts of examples of possible problems actually occur on youtube, do they occur often enough to worry about, and should it be youtube-dl's job to worry about any of them?

I like the simplest approach of keeping youtube-dl at the minimum complexity for downloading: just blindly download each 2GB chunk into a separate file - don't worry about loops, lags, name changes. Errors are potentially troublesome, though. If an error occurs in downloading one 2GB chunk, just report it and give the user the option of either stopping or continuing. That probably needs further thought.

Once youtube-dl has to start worrying about detecting infinite loops in a finite stream, it is getting to be more like an image processing problem, bringing the risks of over-developing / over-complicating youtube-dl.

[aageboi]

it's worked for me : youtube-dl --update
thx.

[0031NL]

May I thank you phihag, your version works perfect

[ghost]

thanks!!

[marupingwl]

youtube-dl --update

Is the easiest and perfect way to fix that problem

[rg3]

AFAIK this is fixed.

[saulbass]

i'm still having issues with this. running the latest version. works fine with recorded videos but not working with live streams.

$ youtube-dl -U
youtube-dl is up-to-date (2013.02.25)
$ youtube-dl http://www.youtube.com/watch?v=y_Rd2hByRyc
[youtube] Setting language
[youtube] y_Rd2hByRyc: Downloading video webpage
[youtube] y_Rd2hByRyc: Downloading video info webpage
[youtube] y_Rd2hByRyc: Extracting video information
ERROR: no conn or url_encoded_fmt_stream_map information found in video info

[DaveJMir]

Likewise:

$ youtube-dl -v --console-title -c -t http://www.youtube.com/watch?v=oLzaaLev-yc
[debug] youtube-dl version 2013.02.25
[debug] Python version 2.7.3 - Darwin-12.3.0-x86_64-i386-64bit
[debug] Proxy map: {}
[youtube] Setting language
[youtube] oLzaaLev-yc: Downloading video webpage
[youtube] oLzaaLev-yc: Downloading video info webpage
[youtube] oLzaaLev-yc: Extracting video information
ERROR: no conn or url_encoded_fmt_stream_map information found in video info
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/Users/davidm/install/bin/youtube-dl/main.py", line 17, in
youtube_dl.main()
File "/Users/davidm/install/bin/youtube-dl/youtube_dl/init.py", line 516, in main
_real_main()
File "/Users/davidm/install/bin/youtube-dl/youtube_dl/init.py", line 500, in _real_main
retcode = fd.download(all_urls)
File "/Users/davidm/install/bin/youtube-dl/youtube_dl/FileDownloader.py", line 507, in download
videos = ie.extract(url)
File "/Users/davidm/install/bin/youtube-dl/youtube_dl/InfoExtractors.py", line 93, in extract
return self._real_extract(url)
File "/Users/davidm/install/bin/youtube-dl/youtube_dl/InfoExtractors.py", line 560, in _real_extract
self._downloader.trouble(u'ERROR: no conn or url_encoded_fmt_stream_map information found in video info')
File "/Users/davidm/install/bin/youtube-dl/youtube_dl/FileDownloader.py", line 230, in trouble
tb_data = traceback.format_list(traceback.extract_stack())

[DaveJMir]

In particular:
video_info['hlsvp'] is a URL to a index.m3u8 playlist (with one entry per video resolution) which appears to be what we're after.
(at the moment the stream is between rebroadcasts so I end up with a 5-second, 720p H264 video of the static splashscreen) Will retry once the stream comes to life again.

[saulbass]

^ cheriff i successfully captured http://www.youtube.com/watch?v=oLzaaLev-yc with ffmpeg by plugging in the m3u8 file. maybe that's worth looking into for the youtube-dl solution. i did run into an issue where ffmpeg i was capturing to .ts instead of .mp4. i was told to use the "-absf aac_adtstoasc" to do the mp4 capture on the fly.

[DaveJMir]

Brilliant! I'll give it a shot, an and this will definitely work as a stopgap solution. Thanks, saulbass.