Feature: 4K Video Support

[peterrussell]

Looks like we'll finally get high resolution videos as part of the Apple TV 4K.
https://9to5mac.com/2017/09/11/tvos-11-4k-apple-tv-screensaver/

Can't wait to get them on my iMac

[holgr]

I wonder if anyone has found/published the new URL(s) to these yet?

[quantumgolem]

Great question. I'm looking for the links for the 4K videos, too.

[AddoSolutions]

The race is on!

[JohnCoates]

I'm looking as well. Seems like they've obfuscated the URLs in tvOS 11

[AddoSolutions]

That just means it's over SSL. Just need to do a MITM attack on a demo ATV and should be clear as day

[JohnCoates]

@AddoSolutions Haha, "Just". Apple tends to use SSL pinning. Hopefully they didn't in this case, but I wouldn't hold my breath on that route.

[AddoSolutions]

Yeah, I was thinking about that in the shower. On the normal device, you can add a SSL authority, and just do it that way. However, in this case, it may be more complicated because I don't think you can do that on Apple TV

[JohnCoates]

Even on regular iOS devices, Apple uses SSL pinning so that even root certs won't fool their critical services. Try opening the App Store on macOS or iOS while running your MITM proxy. You need to disable pinning with https://github.com/nabla-c0d3/ssl-kill-switch2 for example

[quantumgolem]

Can't you find the screensavers in the IPSW file? I can't check as I'm not a developer. That's what 9to5Mac said though (at least for the screensaver they put on YouTube)

[kjg]

The IPSW has a file called Default.mov in the screensaver app that I was able to find. This is probably what 9to5Mac found as well, but it does not include all of the videos, just that one demo. The rest of the videos probably still get streamed from web locations like with tvOS 10.

I wasn’t able to find any URLs in the IPSW that look like they point to a manifest file, but this is my first time looking through an IPSW and I’m not sure how to extract absolutely everything.

Maybe someone else can find it?

[JohnCoates]

Good news! I got the URLs. It wasn't in the IPSW, and the URL was transmitted using SSL pinning. Very hard to get.
These 4k videos are really beautiful. Now I just need to figure out how I'm going to play them.

[holgr]

Any chance you can share the URL(s)? :)

[finngaida]

I'd be curious, too!

[AddoSolutions]

Yeah! Thats a total tease :)

[msheaver]

Me three! :) Wondering how you found them as well! Michael Sheaver 703-200-3008 [email protected] www.michaelsheaver.com

[AddoSolutions]

Sounds like he used this https://github.com/nabla-c0d3/ssl-kill-switch2

[AddoSolutions]

C'mon, you are killing me!

[parsamivehchi]

Please add the new 4K screensavers. They're amazing.

[JohnCoates]

I'll post the URL here as soon as I release support for 4K. They're using HEVC which doesn't come with system support until High Sierra, so it's taking me a little longer.

As for how I found it: It was very hard, took me a long time to figure out, and requires a long explanation. To put it as simple as I can, I had to put my Apple TV running tvOS 11 through a proxy, but since I couldn't use SSL Kill Switch a lot of the connections were hidden. I had to correlate the request for the videos to the request for the sources URL, then since I couldn't see into that request I had to replicate it on the tvOS simulator, where I used a version of SSL Kill Switch that I modified to support this operation. The right modifications were found by using IDA Pro with Hex-Rays to reverse the simulator binaries resposible for checking the trust chain. Then I forged a request as if it came from the new 4K Apple TV.

If you have any questions or want more details feel free to DM me https://twitter.com/johncoatesdev

[spitfire]

https://sylvan.apple.com/Aerials/2x/entries.json

[SuperDOS]

any idea how to play them?

===================== Video =====================
Id                          : 1
Format                      : HEVC
Format info                 : High Efficiency Video Coding
Formatprofile               : Main 10@L5@High
Codec Id                    : dvh1
Duration (ms)               : 3mn 41s
Bit rate                    : 8 297 Kbps
Width                       : 3 840 pixels
Height                      : 2 160 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Constant
Frame rate                  : 29.970 (30000/1001) fps
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 10 bits
Compression mode            : Lossy
Bits/(Pixel*Frame)          : 0.033
Stream size                 : 219 MB (100%)
Title                       : Core Media Video
Encoded date                : UTC 2017-07-23 19:18:01
Tagged date                 : UTC 2017-07-23 19:18:02
Color range                 : Limited
Color primaries             : BT.2020
Transfer characteristics    : SMPTE ST 2084
Matrix coefficients         : BT.2020 non-constant
Mastering display color pri : R: x=0.680000 y=0.320000, G: x=0.265000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000
Mastering display luminance : min: 0.0050 cd/m2, max: 4000.0000 cd/m2
Maximum Content Light Level : 10000 cd/m2
Maximum Frame-Average Light : 111 cd/m2

[alistairmcmillan]

@SuperDOS If you mean as part of the screensaver then all you need to do is be a little patient and let @JohnCoates work his magic. :)

If you mean outwith the screensaver then you can download them and they play in VLC fine.

[SuperDOS]

@alistairmcmillan tried VLC but won't show anything, Potplayer just displays artifacts.

[spitfire]

@SuperDOS even Safari/QuickTime player fail to open them on High Sierra. VLC displays just a few colored blocks. Haven't tried https://nightlies.videolan.org/build/macosx-intel/ yet

[alistairmcmillan]

@SuperDOS @spitfire Try the SDR versions they work fine in VLC right now. For everything else I'm afraid you'll need a little patience. :)

[JohnCoates]

There's no way to play the HDR files currently. I opened up an issue with VLC, https://trac.videolan.org/vlc/ticket/18813

I'll be adding support for the SDR files by using VLC

[AddoSolutions]

Yeah, I am able to open the SDR files just fine in VLC. Will be interesting to see how that affects performance (if my CPU is gonna be at 100% from this on my 2x 4k monitors :P I have 2x 1070s but I kind of doubt VLC is GPU accelerated.

[hssuhas]

@esetnik

Thank you very much. The 4k videos are a delight!
Strangely, the issue I had earlier on my mbp 13' (with the workaround of having to enable the clock) is no longer seen!

I see that there are 15 videos now. Any estimate of when we might be treated to the rest of the videos?!! Thanks in advance.
Cheers mate!

[esetnik]

@hssuhas I’m glad you like it! Unfortunately Apple only shipped the limited set of 4K videos so that’s all we have for now. Maybe they will release more in the future.

[jyavenard]

HDR videos look crap on an IPad Pro 10.5"

[jyavenard]

@esetnik

2016 MBP should be capable of decoding HEVC

High Sierra for now only supports HEVC via Intel. The MBP 2016 skylake doesn't do HEVC HDR, only kabylake does (found in 2017 models)

[chipped]

@jyavenard

High Sierra for now only supports HEVC via Intel. The MBP 2016 skylake doesn't do HEVC HDR, only kabylake does (found in 2017 models)

Where did you read that? It also works on Radeon Pro 4xx and 5xx on 2016 and 2017 MacBook Pro.

[jyavenard]

That it works doesn't mean that decoding will be hardware accelerated.

As for where this info is from, it's in Intel spécialisé as well as in AMD.

[chipped]

@jyavenard

That it works doesn't mean that decoding will be hardware accelerated.

As for where this info is from, it's in Intel spécialisé as well as in AMD.

Yes it does, you are wrong.

Watch the WWDC keynote, 29min48sec, it clearly states hardware acceleration on MacBook Pro 2016.

https://www.apple.com/au/apple-events/june-2017/

[mattvlasach]

For what it is worth, I have the early 2016 MBP and the 4k playback is very jumpy.

So hardware accelerated or not, something about that model’s processor/gpu is not handling the format properly for smooth playback.

[jyavenard]

@chipped theres HEVC hardware decoding support for SDR, not HDR in the macbook pro 2016. There's no full HDR HEVC support in skylake Intel period. h265 main10 (the profile used in those videos) is only partially accelerated, just like VP9 was only partially accelerated (and pretty badly at that)
Only kabylake added full hardware VP9 and h265 main10

Check the Intel slide on the anandtech skylake presentation.
https://www.anandtech.com/show/9562/intels-skylake-gpu-analyzing-the-media-capabilities.

HEVC_VLD_Main10 has a DXVA profile, but it is done partially in the GPU (as specified in the slide above)

For a thorough presentation of performance of skylake VS kabylake https://youtu.be/cI-a4WZWwZc

A quick search of the difference between skylake and kabylake for hevc vp9 HDR will yield 100s of comparison
https://www.digitaltrends.com/computing/intel-kaby-lake-s-hdr/
http://www.fudzilla.com/news/graphics/40959-kaby-lake-gpu-learns-hardware-hdr

HEVC hdr on a macbook pro 2016 will play poorly. Including on late 2016 MBP.
Try the 2017 MBP, with the 8th gen Intel, it's night and day

[chipped]

@jyavenard You are looking at the wrong place mate.

It uses the Radeon Pro for HEVC hardware accelerated decoding.

@mattvlasach I also have the 2016 MacBook Pro with Radeon Pro 460. What type of 4K media were you using and does the player you use have Hardware Acceleration support?

[chipped]

@jyavenard

“macOS High Sierra adds support for High Efficiency Video Coding (HEVC), with hardware acceleration where available, as well as support for High Efficiency Image File Format (HEIF). Macs with the Intel Kaby Lake processor offer hardware support for Main 10 profile 10-bit hardware decoding, those with the Intel Skylake processor support Main profile 8-bit hardware decoding, and those with AMD Radeon 400 series graphics also support full HEVC decoding.[5] In addition, audio codecs FLAC and Opus will also be supported, but not in iTunes.[11][12]”

https://en.wikipedia.org/wiki/MacOS_High_Sierra

Changes > System > Media

[jyavenard]

@chipped did you even try to open those 4K HDR files on High-Sierra on a MBP 2016?
My guess is that you haven't despite your claims. There's no way you could even open those files on a MBP 2016 with any of the Apple player. (quicktime or safari) They are using dolbyvision variant, and there's no public decoders for those yet, not even VLC will play them.

For the others, why do you think this thread exists and so many people report playback issue? (stuttering and so forth).

Edit: on a MBP 15" late 2016 (i7 2.9Ghz skylake i7-6920HQ with Radeon Pro 460) the 4K SDR plays in Qt.
VTDecpderXPCService in Activity Monitor is around 108% CPU, QT itself around 8% CPU.
So all software decoding.

anyhow, I'm done.... obviously wasting my time

[mattvlasach]

Ok, my MBP is:
MacBook Pro (13-inch, 2016, Four Thunderbolt 3 Ports) - 10.13.2
Intel Iris Graphics 550 1536 MB

It does play in QuickTime Player, but shoots the processor well over 100%.

[chipped]

@jyavenard I figured out your stuttering problem. I have attached a screen recording, read this first and then watch it.

You can see the little "d" or "i" in my menu bar. This is an app called gfxCardStatus and it indicates Dedicated in use or Integrated in use. If you click on it shows it any Dependencies that are asking for the Dedicated graphics card. I always turn off Automatic Graphics Switching on my MacBook Pro so it runs off the Radeon Pro all the time using this app.

You guys having probably have Automated Graphics Switching turned on (you can control this in System Preferences > Energy Saver OR select Discrete only in gfxCardStatus menu bar menu. NOTE Must be done after each reboot for either method) and its causing your stuttering problem. Quicktime does not tell macOS it needs a Dependency, so the system doesn't switch to the Radeon Pro. This seems like a flaw and I may submit a bug report about it.

You can see when I manually switch to the Radeon Pro that the Frame Rate is perfect and the Processor on the Radeon is loaded up. This indicates Hardware Acceleration is working on the Radeon Pro. When I switch to Automated Graphics Switching the Integrated Intel is in use and it stutters, you can also see the intel GPU is idle as its being software decoded.

Link to video https://drive.google.com/file/d/1WfKLAp2NpYxs9RWaRk2bUoReOvxe0Ero/view?usp=sharing

Link to gfxCardStatus https://gfx.io

The app I'm using to monitor CPU and GPU stats is iStat Menus https://bjango.com/mac/istatmenus/

You're welcome 👍

[chipped]

@mattvlasach Does it stutter?

[chipped]

@mattvlasach I checked the screensaver and it is HEVC Main 10 profile, your Skylake CPU can only hardware accelerate up to Main 8 profile. That's why your CPU usage is high and it's probably stuttering.

There you go, stuttering mystery solved. Just have to do a bit of research :)

Screenshot attached.

Just incase you missed my post before referencing the Hardware Acceleration capabilities, I'll post it here again.

“macOS High Sierra adds support for High Efficiency Video Coding (HEVC), with hardware acceleration where available, as well as support for High Efficiency Image File Format (HEIF). Macs with the Intel Kaby Lake processor offer hardware support for Main 10 profile 10-bit hardware decoding, those with the Intel Skylake processor support Main profile 8-bit hardware decoding, and those with AMD Radeon 400 series graphics also support full HEVC decoding.[5] In addition, audio codecs FLAC and Opus will also be supported, but not in iTunes.[11][12]”

https://en.wikipedia.org/wiki/MacOS_High_Sierra

Changes > System > Media

[mattvlasach]

@chipped thanks for all of the info. I haven't had a chance to try your previous recommendation involving Auto Graphics Switching. That said, given your latest post, I assume that it will not make any improvement in my case due to lack of hardware support for Main 10?

I'm not sure if this even makes any sense, but is transcoding to Main 8 possible?

[chipped]

@mattvlasach You wouldn't have Automated Graphics Switching as you only have one GPU.

You can try downloads this 4K clip which is 8 bit (Main 8) and play it. I tried it using my Intel GPU and it was super smooth and low CPU usage. Hardware Acceleration working well.

Link http://4ksamples.com/4k-uhd-fireworks-sample/

[jyavenard]

The AMD card isn't used for decoding yet despite what the wikipedia entry state (since anyone can modify it, it's not a hard thing to do)

For the the 4K videos forcing the AMD cards with gfxStatus gives identical results above as the default, that is:
1- 10 bits videos above aren't played
2- 8 bits 4K ones shows very high CPU usage

Using a 2017 MBP with 7th gen intel however works as expected.

If Apple didn't automatically switch the GPU to one that can play the content, I would lodge a bug with Apple.
There should never be any needs to force which GPU is in use.

[chipped]

@jyavenard

1. Did you try 10 bit SDR or HDR?

2. Did you try the 8 bit one I linked above? Maybe your sample is incompatible. Download MediaInfo and provide a screenshot.

3. Where is your evidence that Apple is not using AMD card for decoding? If you watch my screen recording you can clearly see the AMD GPU processor is working when I play the 4K SDR screensaver with Automated Graphics Switching turned off.

4. gfxCardStatus simply toggles the Automated Graphics Switching in System Preferences. You don't actually need it to force the AMD GPU. I use it because its nice and easy and displays useful info.

[jyavenard]

All my comments are related to the issue at play. That is this bug and the videos provided by https://sylvan.apple.com/Aerials/2x/entries.json (listed in #378 (comment))
I don't care about other videos you may find elsewhere. It's about the videos HERE that don't play properly.

In your screen recording, we can clearly see the CPU increasing and staying steady the moment you start playback. 25%+ if the GPU was in use isn't normal, it should be around 5%.

3- talking at WWDC with the engineers involved.

I've asked my contacts at AMD about the state of HEVC playback and if it has dedicated decoding circuits, or if it's like VP9: they use an OpenCL decoder (so it's accelerated not fully decoded).
A clear issue with their VP9 HW decoder, is that the power usage is fairly high (above 30W) which makes it mostly unsuitable on MBP. The fans start spinning at a fairly high rate making the MBP very loud.
On Desktop it doesn't matter, on laptop that's not acceptable even if the capabilities are there.

4. You certainly need to make the switch manually, otherwise your entire earlier comment is irrelevant.

[chipped]

@jyavenard The CPU is at 25% because of the screen recording. Usually it’s around 15%.

The videos do play properly, the SDR videos play fine. You can see that in my screen recording.

I solved your “stuttering issue”, which was really that you just didn’t know you had to use the AMD GPU to fix the stutter.

HDR is not supported by QuickTime so I can’t test it.

[jyavenard]

My stuttering issue? When did I ever state I had a stuttering issue?

You solved nothing, nor should you assume what a person knows.

Forcing AMD serves no purpose. You keep creating strawman arguments by "solving" issues you invented and that people do not have in the first place.

People with 13" MBP don't care about AMD. If they have a MBP with a skylake they can't play some content.

Anyhow, I'm only restating what I first posted.

[chipped]

@jyavenard

My stuttering issue? When did I ever state I had a stuttering issue?

My mistake, that wasn't you.

You solved nothing, nor should you assume what a person knows.

I proved the below statement of yours to be incorrect.

High Sierra for now only supports HEVC via Intel. The MBP 2016 skylake doesn't do HEVC HDR, only kabylake does (found in 2017 models)

Forcing AMD serves no purpose. You keep creating strawman arguments by "solving" issues you invented and that people do not have in the first place.

Sure it serves a purpose, it eliminates the stutter. I thought we established that already???

People with 13" MBP don't care about AMD. If they have a MBP with a skylake they can't play some content.

No, but we do know that it reliably play 8 bit colour depth properly.

Anyhow, I'm only restating what I first posted.

You're argument has now changed, whether you like it or not I helped you and hopefully everyone else. Now I think I may have an idea which I will post next.

[chipped]

When the links were intercepted what was the Apple TV 4K plugged into?

My Apple TV 4K is plugged into a TV which only supports 4K@60Hz 8-bit.

I wonder if mine is pulling down 8 bit screen savers instead of the 10 bit ones that are linked above?

That would make it work a lot better with a larger variety of machines.

[jyavenard]

Sure it serves a purpose, it eliminates the stutter. I thought we established that already???

no it doesn't... and you haven't proven anything.

Once again:
https://sylvan.apple.com/Aerials/2x/Videos/LA_A006_C008_4K_SDR_HEVC.mov plays exactly the same weither you force AMD graphic or not: 108% CPU by VTDecpderXPCService (not QT)
That's on top of the range MBP late 2016.
While titled SDR, it's still a 10 bits video... Having an AMD doesn't help.
https://sylvan.apple.com/Aerials/2x/Videos/LA_A006_C008_4K_HDR_HEVC.mov won't play at all (it's DolbyVision)

The higher end MBP version will play them without stuttering, the lower end won't.

And yes, I have access to all types of MBP, so I can actually verify what I'm describing.

I'm starting to guess that you don't even own that machine, and making stuff as you go, trolling out of your a...
I re-iterate what I posted there: #378 (comment)
"High Sierra for now only supports HEVC via Intel. The MBP 2016 skylake doesn't do HEVC HDR, only kabylake does (found in 2017 models)"

[alistairmcmillan]

This argument has been going on for four days now. That's enough. This isn't a discussion forum.

[glouel]

Closing this as implemented in 1.4