can not combine --playlist-reverse and --playlist-start?

[mikk88]

Hello everyone

i was playing with the --playlist-reverse and the --playlist start NUMBER options to try to download in chronological order and resume where i stopped (like the next day continue at the 26th video) and found that these two options do not work together.

im not sure if this is a bug or if they are not supposed to be combined at all (maybe the help should say so then): in this case i would like to make a feature request for it!

some examples to show what i mean:
1: playlist-start 1 - no reverse
C:\youtubedl>youtube-dl.exe -e --max-downloads 3 --playlist-start 1 https://www.youtube.com/user/youtube
Introducing YouTube Gaming

ProudToLove - Celebrating Marriage Equality and LGBT Pride Month

The A-Z of YouTube: Celebrating 10 Years

2: playlist-start 3 - no reverse (works as expected #1 video now = #3 of the previous command)
C:\youtubedl>youtube-dl.exe -e --max-downloads 3 --playlist-start 3 https://www.youtube.com/user/youtube
The A-Z of YouTube: Celebrating 10 Years

DearMe - What Advice Would You Give Your Younger Self?

Highlights From the YouTube Interview with President Obama

3: playlist-start 1 + reverse
C:\youtubedl>youtube-dl.exe -e --max-downloads 3 --playlist-reverse --playlist-start 1 https://www.youtube.com/user/youtube
YouTube on the tube!
Hammer Time!
A Message From Chad and Steve

and finally
4: playlist-start 3 + reverse (BUG HERE: this gives the same result as command #3 above)
C:\youtubedl>youtube-dl.exe -e --max-downloads 3 --playlist-reverse --playlist-start 3 https://www.youtube.com/user/youtube
YouTube on the tube!
Hammer Time!
A Message From Chad and Steve

thanks in advance for feedback!

[jaimeMF]

It works that way because it first applies --playlist-start and then it reverses the resulting list. I guess the order could be swapped, or we could support negative indexes.

[mikk88]

thanks for the fast feedback!

I guess swapping the order would be how most people would expect the combination of these two options to work... at least I did :)

a solution would be great because the --playlist-start option (in not-reversed downloads) isn't that suitable on very active channels, index will change as new videos get added all the time leading to wrong starting points which need to be handled manually then.

thanks in advance!

[radoslawg]

Had same issue. Kinda workaround that works for me is to use "--playlist-end 100" option to download first 100 videos in reverse order.

I would also think that --playlist-start and --playlist-end should be applied to reversed list though.

[weedy]

Please change the order of enumeration. I just wanted to play the last 20 videos in a 1000+ playlist (the entire channel) and there doesn't seem to be a way to do it.

For me --playlist-reverse (for youtube sources) exists to download things in chronological order.
I don't think it's unreasonable to want to watch the newest few videos from a playlist.

[vitorgalvao]

One more request for this.

Also, this and #514 are duplicates.

[dmxraj]

For command: --playlist-start 2 --playlist-end 3 --playlist-reverse -o "%(playlist_index)s - %(title)s.%(ext)s"
Lets say original playlist is like this:

1. a
2. b
3. c
4. d
5. e

I think -playlist-reverse should be applied to the whole list, first. Playlist will become like this:

1. e
2. d
3. c
4. b
5. a

--playlist-start 2 --playlist-end 3 should be applied to it. So video d and c will be downloaded.
Then file name for those videos should be like this ("%(playlist_index)s - %(title)s.%(ext)s"):

2 - d.mp4
3 - c.mp4

[dmxraj]

For command: --playlist-start 2 --playlist-reverse -o "%(playlist_index)s - %(title)s.%(ext)s"
Lets say original playlist is like this:

1. a
2. b
3. c
4. d
5. e

I think -playlist-reverse should be applied to the whole list, first. Playlist will become like this:

1. e
2. d
3. c
4. b
5. a

--playlist-start 2 should be applied to it. So video d, c, b, a will be downloaded.
Then file name for those videos should be like this ("%(playlist_index)s - %(title)s.%(ext)s"):

2 - d.mp4
3 - c.mp4
4 - b.mp4
5 - a.mp4

[dmxraj]

For command: --playlist-end 3 --playlist-reverse -o "%(playlist_index)s - %(title)s.%(ext)s"
Lets say original playlist is like this:

1. a
2. b
3. c
4. d
5. e

I think -playlist-reverse should be applied to the whole list, first. Playlist will become like this:

1. e
2. d
3. c
4. b
5. a

--playlist-end 3 should be applied to it. So video e, d and c will be downloaded.
Then file name for those videos should be like this ("%(playlist_index)s - %(title)s.%(ext)s"):

1 - e.mp4
2 - d.mp4
3 - c.mp4

[mihawk90]

This report is sort of old, but I didn't want to start a new one because it is essentially the same issue.
This issue also affects the --playlist-items option (and probably --playlist-end, but I did not test this).

Here is some output to illustrate:

Without --playlist-items:

[t@localhost ASOT]$ youtube-dl https://www.youtube.com/playlist?list=PL0cWlOyqP6_PKyS76fDsafkj_ho4KLmEA -f bestaudio --download-archive PL0cWlOyqP6_PKyS76fDsafkj_ho4KLmEA.log --playlist-reverse
[youtube:playlist] PL0cWlOyqP6_PKyS76fDsafkj_ho4KLmEA: Downloading webpage
[download] Downloading playlist: A State Of Trance - by Armin van Buuren
[youtube:playlist] PL0cWlOyqP6_PKyS76fDsafkj_ho4KLmEA: Downloading page #1
[youtube:playlist] playlist A State Of Trance - by Armin van Buuren: Downloading 200 videos
[download] Downloading video 1 of 200
[youtube] LLlmtBHo8Mc: Downloading webpage
[youtube] LLlmtBHo8Mc: Downloading video info webpage
[download] Destination: A State Of Trance Episode 757 (#ASOT757)-LLlmtBHo8Mc.webm
[download] 100% of 108.46MiB in 00:07
[download] Downloading video 2 of 200
[youtube] QPkAi6QBcqI: Downloading webpage
[youtube] QPkAi6QBcqI: Downloading video info webpage
[download] Destination: A State of Trance Episode 758 (#ASOT758)-QPkAi6QBcqI.webm
[...]

With --playlist-items:

[t@localhost ASOT]$ youtube-dl https://www.youtube.com/playlist?list=PL0cWlOyqP6_PKyS76fDsafkj_ho4KLmEA -f bestaudio --download-archive PL0cWlOyqP6_PKyS76fDsafkj_ho4KLmEA.log --playlist-reverse --embed-thumbnail --playlist-items 1-43
[youtube:playlist] PL0cWlOyqP6_PKyS76fDsafkj_ho4KLmEA: Downloading webpage
[download] Downloading playlist: A State Of Trance - by Armin van Buuren
[youtube:playlist] playlist A State Of Trance - by Armin van Buuren: Downloading 43 videos
[download] Downloading video 1 of 43
[youtube] HG9VOWM9AP4: Downloading webpage
[youtube] HG9VOWM9AP4: Downloading video info webpage
[youtube] HG9VOWM9AP4: Downloading thumbnail ...
[youtube] HG9VOWM9AP4: Writing thumbnail to: A State Of Trance Episode 907 [#ASOT907] - Armin van Buuren-HG9VOWM9AP4.jpg
[download] Destination: A State Of Trance Episode 907 [#ASOT907] - Armin van Buuren-HG9VOWM9AP4.webm
[download] 100% of 113.42MiB in 00:06
ERROR: Only mp3 and m4a/mp4 are supported for thumbnail embedding for now.
[...]

Notice it starts at episode 907 (the latest as of the time of this writing), instead of 757 (the first on youtube and in the playlist).
Nevermind the thumbnail error.

[PaperBag42]

@mihawk90 I had also struggled with this issue for a long time. Merging #17347 would solve it. However, I'm afraid the maintainers will never merge it since it's buried under thousands of newer pull requests (it's currently at page 13 of the pull requests list). Up until now I've been using the fork directly.

[ALurker]

At this point, it seems likely that the youtube-dl folk either 1) don't view this as an issue, or 2) don't want to provide reasons on why the code I submitted to fix this issue is unacceptable.

My original PR of #17347 corrected the issue present after some iteration to resolve legitimate issues, and then waited over a year with no review. The second PR of #24487 has now been closed to discussion after being opened just shy of a month ago with no review / comments by the developers.

[Anon-Exploiter]

A quick solution, if someone came across the above^ (and doesn't want to use to build from the fork by @ALurker)

The following python3 script will generate commands for you, create a .sh file and execute them, you can remove the starting vids, which have already been downloaded.

import os

playlist 	= "https://www.youtube.com/user/..."
command 	= f"youtube-dl -j --flat-playlist --playlist-reverse {playlist} | jq -r '.id' | sed 's_^_https://youtu.be/_'"

output 		= os.popen(command).read()
print(output)

ytUrls 		= []

for count, urls in zip(range(1, 1000), output.strip().split("\n")):
	count 	= str(count).zfill(3)
	cmd 	= f'youtube-dl -f mp4 -i -v -R 3 --fragment-retries 3 -c -o "{count} - %(title)s.%(ext)s" {urls} --external-downloader "aria2c" --external-downloader-args "-j 10 -s 10 -x 16 --file-allocation=none -c"'
	print(cmd)

Gist: https://gist.github.com/Anon-Exploiter/ca3ab8f7a7edee8227622c7a6a97e396

[dlevear]

Another solution that only uses bash. I also felt frustrated with this, so I wrote a straightforward workaround. Maybe others will find it useful.

# Downloads final 3 videos in playlist
function ytdl_recent() {
    TOTAL=$(youtube_dl --flat-playlist $1 | tail -n2 | head -n1 | cut -d ' ' -f4);
    youtube_dl $1 --playlist-start $(($TOTAL-2)); #subtract N to download the final N+1 videos
}
# example usage: ytdl_recent 'https://www.youtube.com/...'

[rautamiekka]

Another solution that only uses bash. I also felt frustrated with this, so I wrote a straightforward workaround. Maybe others will find it useful.

# Downloads final 3 videos in playlist
function ytdl_recent() {
    TOTAL=$(youtube_dl --flat-playlist $1 | tail -n2 | head -n1 | cut -d ' ' -f4);
    youtube_dl $1 --playlist-start $(($TOTAL-2)); #subtract N to download the final N+1 videos
}
# example usage: ytdl_recent 'https://www.youtube.com/...'

Always double-quote var refs unless it's specifically necessary to allow splitting. The YT URL's usually have a question mark among others, which could break the script.

Would recommend checking for the emptiness of TOTAL cuz anything could go wrong:

ytdl_recent() {
    for _link in "${@}"; do
        # shellcheck disable=SC2155
        _total=$(youtube-dl --flat-playlist "${_link}" | tail -n2 | head -n1 | cut -d ' ' -f4)
        [ ! "${_total}" ] && {
            echo 'Failed to find the total number of videos for the link "'"${_link}"'". Skipping.'

            continue
        }

        youtube-dl "${_link}" --playlist-start $((_total - 2)) #subtract N to download the final N+1 videos
    done
}

Now it also works with Dash/POSIX sh, and can handle any number of links.