World of Warcraft - DXVK_STATE_CACHE

[SveSop]

I have a problem enabling the new DXVK_STATE_CACHE when running World of Warcraft.

First time WoW is run and the cache file is not there, it gets created and i guess that works as intended.

The next time i start the game the screen is just black and i cant enter the game. Sometimes i actually gets as far as the character select screen before locking up.

Disabling cache by setting DXVK_STATE_CACHE=0 the game works and removing the Wow.dxvk-cache file also will make the game run.
Probably not of much help with the logfiles, as it only shows pipeline compiling stopping and no error messages.

Software information

World of Warcraft - Battle for Azeroth

System information

• GPU: nVidia GTX970 4GB
• Driver: 396.54.06
• Wine version: 3.16
• DXVK version: v0.80-4-ga950872

Log files

• d3d11.log:
  Wow_d3d11.log
  Wow_d3d11-State0.log

• dxgi.log:
  Wow_dxgi.log
  Wow_dxgi-State0.log

[Dehir]

Everything seems to work ok for me, so far atleast.

System information

OS: Arch Linux x86_64
GPU: Sapphire RX480 4GB
Driver: vulkan-radeon 18.2.1-1, mesa 18.2.1-1,
Wine version: staging-3.12 (lutris build)
DXVK version: v0.80 (arch aur binary build)

Eager to test thou does these patches fix problem that obviously pulls the whole pipe 100% usage and starts to cause stuttering even fps is ok. Happens when effects like blood magic(easy to test Uldir raid normal/HC first boss) fog etc happens and it gets worse when there are more ppl in screen. You can help the pipe littlle by changing camera angle to point at location where there is no effects present.

radeontop screenshot

*fixed typo 👍

[ryao]

@Dehir How did you get v0.82? That hasn't been invented yet.

[Dehir]

@ryao Apparently typo :D

[doitsujin]

Can you test if ddfa7e4 fixes the problem?

[andy-d1969]

I had the same issue in some games and ddfa7e4 fixes all of them for me.
Thanks!

[SveSop]

@doitsujin
Definate improvement with ddfa7e4

Was able to crash it a couple of times when changing wine version (between 3.16 and some git release..) Perhaps something to do when DXVK invalidates cached items? Staying with the same rev i could start/stop without issues tho.

I did the following experiment:
Clear out $HOME/.nv/ cache folder. Delete the wow.dxvk-cache file
Log into the game.. run a bit around. Exit the game
Rename $HOME/.nv/ cache folder.
Log into the game... crash.
Delete the $HOME/.nv/ cache folder, and move back the one i renamed above. Not touching DXVK cache.
Log into the game - OK

So, it MAY seem as whenever nVidia driver recreates its cached textures for some reason (wine version/driver/dxvk...), the DXVK cache crashes on some validation thing? Dunno..

@andy-d1969 Do you use a nVidia card? And could you possibly try to recreate my experiment above?

[doitsujin]

DXVK doesn't invalidate the cache when you update wine. Do I understand correctly that those crashes still only happen with the state cache enabled? Are you sure it's not some issue caused by wine updating its prefix?

8bcd47d implements more conservative locking, please let me know if that changes anything.

[SveSop]

It is when nVidia creates IT'S cache that seem to be the problem. nVidia creates new cache binaries whenever something changes - like wine/dxvk binaries/drivers and whatever.
I explained this in some other post, but quick ref.

Ex. on a "live" folder structure for me:
/.nv/GLCache/cc8669432ea68d4684a818385efa4034/757205617526815c/
with 2 files inside:
2c1308714869aab9.bin
2c1308714869aab9.toc

Whenever something happens those two .toc and .bin files gets a new "HashID" (Or the folder structure changes). Probably some nifty logic behind when what changes, but i know it changes upon wine/dxvk/kernel/driver or similar changes... thus nVidia cretes a NEW cache file.

When THAT happens, and the .dxvk-cache file is NOT cleared out, it seems as if something goes wrong. I get 8 threads going 100%, screen freezes and GPU goes to 0%. Sometimes just at a black screen, sometimes with some graphics "half-way ingame" so to speak. Clearly when DXVK tries to send its "data" to the nVidia driver, and the nVidia driver tries to create its cache from this, it goes tits up :)

PS. Just to clear up any misunderstandings - I do NOT have to switch any versions of anything to recreate the issue - its enough to delete the nVidia "GLCache" folder to make this happen.... It was just that changing driver/wine/whatever creates these files automagically in a new folder/filename.

[doitsujin]

Hm, that's annoying. Does that still happen with the latest commit?

[jp7677]

I did a double check from what I wrote here #647 (comment), but at least with Arkham Knight I cannot reproduce the behavior you describe.

I started Steam with a new newly created directory for the Nvidia cache
__GL_SHADER_DISK_CACHE_PATH=~/.steam/cache/tmp wine Steam.exe

When starting Arkham Knight with an existing BatmanAK.dxvk-cache file of ~12MB the DXVK log says:

info:  DXVK: Read 6316 valid state cache entries
info:  DXVK: Using 9 compiler threads

Running the game and built-in benchmark confirms my numbers from the link above. The Nvidia cache directory now contains the GLCache folder with ~7MB of different files.

I have a GTX 1080 with driver version 396.54 (short term branch, not the Vulkan development branch) and I run DXVK 0.80 release version (not master).

[SveSop]

@doitsujin Yeah, latest 8bcd47d
Infact, logging in/out/exit WoW will eventually lead to a hang.. but the next time i log in can work, so it seems more random than i initially thought.
Doing the "clear out .nv folder cache" trick makes it happen each time tho.

@jp7677 A lot less cache entries and size on the file i use tho
info: DXVK: Read 1219 valid state cache entries
so it does not seem as a size issue either.

Might very well be a World of Warcraft issue tho? (But havent tested this with non-dev driver)

[doitsujin]

Can you try to trigger a hang with DXVK_LOG_LEVEL=debug and attach the log files? I'm curious now whether it actually hangs during pipeline compilation or at some other point.

In the worst case I'll have to disable the state cache again, and I really don't want to.

[SveSop]

@doitsujin
I think i found something with World of Warcraft, so as of now i think it is 99% a "World of Warcraft" thing.

WoW has a config setting called: SET processAffinityMask This lets you kinda set what cores you want the game to utilize. Default i guess is "all awailable", but you can tweak this to limit cores. Eg. if you want only core 0-3 used, you would set SET processAffinityMask "15"

Sometimes gaming with wine, games can benefit from not using "all", or if you run multiple things and so on and so forth. However, to make the story shorter, mine was set to SET processAffinityMask "255" This means "8 cores"... should be plenty no? I had this set from when i had a i7 2600K 4cores/4ht

I removed the setting, thus allowing "all available" and for my 8700K that is all 12 (6cores/6ht).. And i cant seem to reproduce this problem anymore.

Conclusion: Crappy threading in World of Warcraft.exe, or some sort of strangeness happening with the way nVidia recreates shadercache? A pure bug with processors > 8 cores in WoW? Dunno...
Setting __GL_MULTITHREADED_OPTIMIZATIONS=0 while keeping SET processAffinityMask "255" actually made me stutter along with 2 fps a bit before locking with those 8 cores at 100%.

Anyway, not sure WHY that cache option would make 8 cores go crazy, but removing the setting (the default value), it seems to work for now :)

[mati865]

processAffinityMask has been deprecated for years because it's buggy.

[Leopard1907]

@SveSop Can you try it?

#665 (comment)

[SveSop]

@Leopard1907 The logfiles attached to when i opened the bug is from export DXVK_LOG_LEVEL=debug and nothing more useful than the pipeline compiling just stopping.

@mati865 Possibly. World of Warcraft is not really known for its awesome multithreading either.. But nevertheless, the setting was there from i guess my earlier experimentations :)
Still ppl are using this with some success due to blizzards cruddy multithread implementation maybe? https://us.battle.net/forums/en/wow/topic/20767788101

[SveSop]

Closing this, as this might not be a DXVK issue, but maybe a WoW-threading vs nVidia issue perhaps?