LiF MMO: rendering issues

[jayquonmarton]

There are rendering issues in LiF MMO as seen below. The dxvk HUD shows up as expected. I'm not sure what to make of it. It seems like the shaders for clothing/skin/vegetation aren't applying correctly?

I tried all DXVK_CONFIG_FILE settings, but none helped.

Software information

• Life is Feudal - MMO. https://play.lifeisfeudal.com
• I tried all DXVK_CONFIG_FILE settings, but none helped.

System information

• GPU: nvidia gtx 1070
• Driver: 396.45.0
• Wine version: 3.13-staging
• DXVK version: 0.70. Note I am using the d3d10 dlls in this new version also.

Apitrace file(s)

• https://mega.nz/#!zXxkAaII!9KFtHINn1TQ7j3Ku2p9Kh8mEhi3lCAPvYabPvHUJnUA

Log files

• d3d11.log: https://bpaste.net/show/ffebd3677bea
• dxgi.log: https://bpaste.net/show/1dabf4fdc6d1
• console output: https://bpaste.net/show/25cfc21ccf08
• console output with WINEDEBUG=-all: https://bpaste.net/show/c99c9b7da353
• wined3d (no dxvk) output: https://bpaste.net/show/a3ec2488aac4

wined3d

If it helps identifying whether the issue is in dxvk, the game renders similarly with wined3d (d3d11.dll, d3d10.dll, dxgi.dll, etc set to builtin).

With wined3d, there are numerous errors related to blitting that are not present in dxvk, e.g.

00a8:fixme:d3d:surface_convert_format Cannot find a conversion function from format WINED3DFMT_R32_TYPELESS to WINED3DFMT_R32_FLOAT.
00a8:fixme:d3d:surface_cpu_blt Cannot convert WINED3DFMT_R32_TYPELESS to WINED3DFMT_R32_FLOAT.
00a8:err:d3d:cpu_blitter_blit Failed to blit.

[adrianlizarraga]

Greetings good sir. I recommend updating your Nvidia drivers to the latest version. The recommended version is currently 396.54. Refer to https://github.com/doitsujin/dxvk/wiki/Driver-support

[jayquonmarton]

Thanks. I'll try that driver version when it is available in my distro. It isn't yet.

When running the game on windows, I noticed it looked like the dxvk screenshot initially, until the game fully loaded. For some reason, in wine it never appears to load in the high quality resources. I'm not sure if it is failing to load textures or shaders or what. Is there a way to determine this?

Here is pic from windows of how it should look.

[K0bin]

It is very unlikely that a shader would cause issues like that.

For some reason the game is not streaming in higher resolution textures.

[K0bin]

Maybe the game's log contains anything useful, would you mind uploading that?

The issue is baked into the apitrace you provided. Can you make another one but this time on Windows with normal Direct3D?

[jayquonmarton]

@K0bin Nice thinking. The game log under wine has errors about out of texture memory that are not present in the windows log.

ERRR 2018-08-23 15:19:15.135 {} <> [47056] Error: failed to load texture art/Textures/CharacterTextures/Customization/male/Male_Body_v1_DIFFUSE.dds: out of texture memory budget (0 / 3)!

ERRR 2018-08-23 15:19:23.971 {} <> [47430] Out of texture memory, trying to lower texture quality by 2

And the the texture memory budget is 0 under wine while it has reasonable numbers under windows, e.g.

ECHO 2018-08-23 15:17:58.664 {} <<Thread>> [] GFXStreamableTextureManager initialized. Texture memory budget: 0, memory warning level: 0 [T:WorkerThread0:0x95DE2C8]

ECHO 2018-08-23 11:33:50.762 {} <<Thread>> [] GFXStreamableTextureManager initialized. Texture memory budget: 1327, memory warning level: 884 [T:WorkerThread0:0x2BA5E3C8]

Any thoughts why the budget might be 0 under wine?

Game logs

• windows, game log: https://bpaste.net/show/dd1c27807b61
• wine, game log: https://bpaste.net/show/22f88abbced7

New API traces

In windows, it takes a couple mins before the high quality resources are switched in, so I redid the wine apitrace waiting idly for what should have be enough time for the texture error to occur. The win api trace waits in game long enough for the high quality resources to switch in before exiting the game.

• windows apitrace: https://mega.nz/#!njgFzbCJ!fxR-xmXdmtc-T4wao9-3bnyxQHM2hFBa6cqerikd_rA
• wine apitrace: https://mega.nz/#!nzBAzAZK!fxHDIs5I6hgtT6wzo_2NZfOwD4Pxv3O6iauPoAlwUxQ

Also, just curious.. Can you briefly explain what tool(s) you use to interpret the apitrace files? What sort of things do you look for in them to help identify issues? Perhaps, using these tools myself will help me identifying issues on my own.

Thanks

[K0bin]

The logs were indeed useful because it seems like the game can't get the amount of VRAM your graphics card has.

Apitrace allows you to look into all D3D calls the game makes and replay them without owning the game. I think I'm gonna look into the calls with which the game reads the amount of VRAM, look for differences between the DXVK one and the working D3D11 one and hope something sticks out.

[jayquonmarton]

Thanks looking into that.

At risk of being off-topic, is https://apitrace.github.io/ the tool you're using or are there others? I tried it briefly, and it seemed to only provide api calls for the first part of the recorded session. It didn't have api calls past the loading screen (though it did replay the full recorded session). Maybe the files I uploaded were simply too large.

[K0bin]

I assume LIF MMO is 64 bit, if so, please test:

lif_experiment_dxvk_x64.zip

[jayquonmarton]

That fixed it.

• FPS dropped from 35 to 11.
• GPU utilization dropped from 99% to 45%.
• CPU shows one thread in the process is pinned at 99%. Previously, that thread was around 85%.

Seems like it also causes the process to become CPU bound.

Which API was it that was reporting invalid VRAM?

[K0bin]

Which API was it that was reporting invalid VRAM?

This one:
https://docs.microsoft.com/en-us/windows/desktop/api/dxgi/nf-dxgi-idxgiadapter-getdesc

This isn't a proper fix, I just hardcoded all the values to what native D3D returned for your card to see if that works. The game apparently uses another api to get VRAM and tries to match that using something that D3D11 returns. (maybe the device name)

Seems like it also causes the process to become CPU bound.

I don't really know why that happens, I didn't really change anything performance. It's streaming in textures now which is obviously work but probably shouldn't be that bad. What's causing the cpu load, the game process or Wine server?

Can you test if this one also does the trick:
dxgi_nv_adapter_name.zip

[jayquonmarton]

dxgi_nv_adapter_name.zip didn't fix the issue (high quality textures not streaming in, 0 texture memory budget reported in logs), and it also caused the process to become CPU bound like the first zip. So worst of both worlds.

I've been using dxvk version 0.70. Are your changes based top of a different version?

What's causing the cpu load, the game process or Wine server?

It is a thread in the game process, cm_client.exe. Here is some top -H output. There is always one thread that jumps between 99.0 and 99.9% with your two zip files.

  PID  PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                          
 4712  20   0 6590656   4.5g 373296 R  99.0  19.3  13:23.92 cm_client.exe                                                                                                                    
 4760  20   0 6590656   4.5g 373296 S  18.8  19.3   2:34.21 cm_client.exe                                                                                                                    
 4802  20   0 6590656   4.5g 373296 S   7.9  19.3   1:13.47 cm_client.exe                                                                                                                    
 4759  20   0 6590656   4.5g 373296 S   4.0  19.3   0:31.37 cm_client.exe                                                                                                                    
 4975  20   0 6590656   4.5g 373296 S   1.0  19.3   0:08.68 cm_client.exe                                                                                                                    
 4976  20   0 6590656   4.5g 373296 S   1.0  19.3   0:05.74 cm_client.exe                                                                                                                    
 4750  20   0 6590656   4.5g 373296 S   0.0  19.3   0:00.00 cm_client.exe

[K0bin]

Try this one please:

dxgi_nv_adapter_name_luid.zip

I've tried this build in Witcher 3 and it runs exactly the same as v0.70. It's based of the latest master but that shouldn't impact performance either. Please also retest v0.70 to make sure that it's really running faster because there shouldn't be any reason for that.

[jayquonmarton]

dxgi_nv_adapter_name_luid.zip behaves similarly to dxgi_nv_adapter_name.zip.

The difference between your libs and the 0.70 release under the releases tab on github is less severe than I originally posted, but it does appear measurable and consistent.

I tried more carefully to reduce the number of variables in testing the two sets of dlls this time, e.g. made sure rendering the same scenery, waited a few mins until things stabilized, re-ran each test 4 times, switched back and forth between the two sets of libs for each test.

dxgi_nv_adapter_name_luid.zip

• 24-26 fps
• 98-99.9% cpu utilization in the most demanding process thread
• 71-81% gpu utilization

0.70 from github releases tab

• 33-35 fps
• 90-95% cpu utilization in the most demanding process thread
• 90-99% gpu utilization

I tested dxvk-0.65-state-cache-x64-simplified.zip briefly again, and it was practically the same as the dxgi_nv_adapter_name_luid.zip results above.

Maybe the difference is from compiler flags/optimizations present in the official release but not in yours?

[K0bin]

I'm using MSVC while doitsujins builds are done with MinGW. I've never seen such a big performance gap and if anything builds done on Windows have been slightly faster for some reason.

Let's concentrate on fixing the streaming issue first:
dxgi_nv_adapter_subsys.zip

[jayquonmarton]

The issue remains with dxgi_nv_adapter_subsys.zip. 0 texture memory budget reported in logs.

[Guy1524]

Could the game be using nvapi, if so, try setting these env variables:

DXVK_CUSTOM_VENDOR_ID=1002 DXVK_CUSTOM_DEVICE_ID=67DF

Also, disable both nvapi and nvapi64 from winecfg.

[jayquonmarton]

I disabled nvapi64 (there was no nvapi, perhaps because it is a 64-bit prefix?), and did

export DXVK_CUSTOM_VENDOR_ID=1002                                              
export DXVK_CUSTOM_DEVICE_ID=67DF

and used the following config

dxgi.customDeviceId = 67DF                                                        
dxgi.customVendorId = 1002

It didn't appear to have any effect.

[K0bin]

So if the first dlls I provided worked and the ones after didn't then the game is probably using SubSysId and/or Revision of the DXGI_ADAPTER_DESC struct or a combination of these with the device name or the luid.

[jayquonmarton]

So if the first dlls I provided worked and the ones after didn't

Yes. The first dlls (dxvk-0.65-state-cache-x64-simplified.zip) worked, and the later ones (dxgi_nv_adapter_name.zip, dxgi_nv_adapter_name_luid.zip, and dxgi_nv_adapter_subsys.zip) did not.

I see the comment that contained the link to the working dlls I downloaded, dxvk-0.65-state-cache-x64-simplified.zip, was modified. It now has lif_experiment_dxvk_x64.zip, which I haven't tested. Are those different dlls, or just a renamed zip filename?

[K0bin]

I just corrected the file name that I messed up.

[jayquonmarton]

the game is probably using SubSysId and/or Revision of the DXGI_ADAPTER_DESC struct or a combination of these with the device name or the luid.

@K0bin by this, do you mean that you suspect the game is using these values to lookup how much memory to reserve for textures based on pci ids, rather than a value like DedicatedVideoMemory?

[K0bin]

It seems like that's the case as the values DXVK reports are certainly not 0. I've also read somewhere that their engine is based on Ogre Torque which is really old so it might just use something like D3D9, DirectDraw or Dxdiag to read the amount of VRAM but select the adapter (gpu) based on what D3D11 reports.

Can you confirm that it's indeed based on the subsysid? Does this one works?
dxgi_nv_adapter_subsys.zip

(Sorry if I keep throwing new builds at you but this is kinda guesswork.)

[jayquonmarton]

I tried it. It didn't work. Sorry, if that wasn't clear in #591 (comment)

[ZeroFault]

Searching around, I've been reading that this game is using a heavily modified Torque3D game engine and Torque3D used to query the video device memory using the Win32_videoController class through WMI. I'm not sure if this engine is still using that.

[jayquonmarton]

@ZeroFault, I think @K0bin said he only stubbed the results from IDXGIAdapter::GetDesc in order to fix the issue. So it would only be a matter of figuring out which of those stubbed values are specifically required by the game (then properly settings those values, instead of hard-coding them).

[K0bin]

That's exactly what I was trying to do with all these builds.

I've found a forum post that said they deviated from Torque 3D in 2011. The earliest commit in the Torque3D GitHub repo is from 2012.

They do the following:

• list all adapters using dxgi and store their adapter index
• query adapter information using that index. First they try to use DXGI which should perfectly because the game would get a correct description that has the correct amount of VRAM. If DXGI fails, the game falls back to Dxdiag but for anything other than VRAM. If that also fails (or for VRAM) the game uses WMI as @ZeroFault says.

If they were still using that code, it would probably work. I'll just assume that the DXGI case was implemented after they forked. So it might be possible that DXVK has a different ordering for adapters than Wine but that doesn't explain why Wine doesn't work except if it was using an inconsistent ordering between WineD3D and WMI.

That also wouldn't explain why hardcoding everything fixed it.

I'm running out of ideas so let's try something a little bit weird: I'm gonna just hardcode the amount of memory. It's slightly less than DXVK reports and shouldn't work but, lets just see what happens:
dxgi_memory_hardcode.zip

[jayquonmarton]

@K0bin dxgi_memory_hardcode.zip works.

ECHO 2018-08-24 13:16:39.944 {} <<Thread>> [] GFXStreamableTextureManager initialized. Texture memory budget: 1327, memory warning level: 884 [T:WorkerThread0:0x80AE2C8]

[K0bin]

I'm confused.

• D3D11 returns 8470331392
• DXVK returns 8589934592

The engine divides that by 1048576 (Byte => MB), which gets us 8077 and 8192.

Why does it decided that the latter one only gets 0 MB for textures?

[jayquonmarton]

@K0bin Can you throw the code for dxgi_memory_hardcode.zip and lif_experiment_dxvk_x64.zip up in two branches so I can get a better idea of what each of these are doing?

[K0bin]

pDesc->DedicatedVideoMemory           = 8470331392;
pDesc->SharedSystemMemory             = 12611805184;

That's everything I changed in dxgi_memory_hardcode compared to master. Master uses the value that Vulkan gives us. I got these numbers from your D3D11 apitrace.

The newer open source version of Torque only uses DedicatedVideoMemory and I assume their version does the same.

[jayquonmarton]

And just for reference, from the DXVK apitrace, I see.

DedicatedVideoMemory = 8589934592,
SharedSystemMemory = 18946007040

Hopefully, other people familiar graphics APIs will have some ideas. The only thing I can think of is the engine falls back to 0 if it sees an unexpected value that doesn't match what it has hardcoded for the given pci id, though why it would do that, I don't know.

[K0bin]

This makes no sense to me so let's just see if it works if I subtract 8 bytes from the VRAM for no reason at all.

dxgi_reduce_vram.zip

[jayquonmarton]

That works. Please send a PR that arbitrarily subtracts 8 bytes from the VRAM. :)

ECHO 2018-08-24 14:19:05.155 {} <<Thread>> [] GFXStreamableTextureManager initialized. Texture memory budget: 1365, memory warning level: 910 [T:WorkerThread0:0x87EE2C8]

[jayquonmarton]

fwiw, nvidia-settings shows the dedicated memory as follows, which may suggest DXVK's value for DedicatedVideoMemory is incorrect -- DXVK is setting it to total memory instead of total dedicated memory.

Doesn't really explain why the game engine reserves 0 bytes for texture memory though.

Perhaps the game engine uses another api to get the total memory, detects the dedicated memory is equal to the total memory, and falls back to 0 because they shouldn't be equal?

[doitsujin]

fwiw, nvidia-settings shows the dedicated memory as follows, which may suggest DXVK's value for DedicatedVideoMemory is incorrect

No, it's not. DXVK just reports the value it gets from Vulkan, and this game is particularly stupid in that it seems to cut off the upper 32 bits from the reported value.

I'll add a config option for this, and limit this game to 4095 MB by default if that works.

[jayquonmarton]

Odd. It would be high 31 bits, since it doesn't overflow/fail at 4GB. It works up to but not including 8GB.

@doitsujin , Maybe should limit it to min(actual_value, 0x1ffffffff)? Otherwise, 4GB might be an issue if you don't in fact have at least 4GB? Plus that wouldn't limit normal users who have a modern gpu that has 8GB+ who aren't going to mess with config files.

[K0bin]

@doitsujin Implemented the option to limit memory, please verify that this works:

dxgi_limit_memory.zip

[jayquonmarton]

Three issues with the patch:

1. cm_client.exe is also used by this game studio's other games, so a simple comparison against the exe filename will match more than just LiF MMO fyi.
2. Why limit it to 4095, when @K0bin determined it works up to but not including 8GB? Shouldn't it be limited to 8191 instead? This way normal users who don't know about dxvk config files won't have their modern gpus unnecessarily limited.
3. Why limit maxSharedMemory for cm_client.exe at all? Didn't @K0bin determine the issue was specifically with maxDeviceMemory?

Checking the dlls here in a minute..

Update: yes. Those dlls worked.

[K0bin]

1 cm_client.exe is also used by this game studio's other games, so a simple comparison against the exe filename will match more than just LiF MMO fyi.

If they even have the same filename then I'd assume that they have the same problem

2 Why limit it to 4095, when @K0bin determined it works up to but not including 8GB? Shouldn't it be limited to 8191 instead? This way normal users who don't know about dxvk config files won't have their modern gpus unnecessarily limited?

4095 is the highest possible value for a 32bit unsigned integer. The 1070 has 8096 MB of VRAM, if you subtract 1 from that you get 8095 which when converted to a 32bit uint overflows to 4095.

[jayquonmarton]

4095 is the highest possible value for a 32bit unsigned integer. The 1070 has 8096 bytes of VRAM, if you subtract 1 from that you get 8095 which when converted to a 32bit uint overflows to 4095.

I see what you mean. Thanks

[K0bin]

It does work?

[jayquonmarton]

Yes (sorry, I updated the last comment with the results)

[earnaud]

Hello !
I am currently getting the described problems above, concerning the texture rendering issue. The logs seem to reveal the same problems.

I am running LiF:MMO from steam using the experimental proton.

Please find attached the logs file.

C_2020-02-28-11-24-28.log

[misyltoad]

This isn't a DXVK log so it isn't useful to us.

[lucianposton]

If in fact it is the same issue, you should be able to fix it by the same workaround i.e. setting the same config overrides to a value below 4GB that doesn't overflow 32 bits. See 73cbf5b

dxgi.maxDeviceMemory
dxgi.maxSharedMemory

dxvk/dxvk.conf

Lines 43 to 50 in 2133049

	# Override maximum amount of device memory and shared system memory
	# reported to the application. This may fix texture streaming issues
	# in games that do not support cards with large amounts of VRAM.
	#
	# Supported values: Any number in Megabytes.
	# dxgi.maxDeviceMemory = 0
	# dxgi.maxSharedMemory = 0

However, from a brief search online, I don't think your radeon hd 5600 video card has 4GB+ memory, which was part of this original issue. Wikipedia lists that card as 1 or 2 GB.

Attempting to create GFX device: ATI Radeon HD 5600 Series (D3D11)

[misyltoad]

The HD 5600 also doesn't support Vulkan 🐸

[earnaud]

Oh, thanks.

[Xatrongamer]

Hello. What is the finish fix to the problem?
I don't know anything about programming and I love LiF. Please, someone can help me... I don´t understand any of the last fixes

[Xatrongamer]

And I don´t speak english and the google translate doesn´t translate very good. So please write a simple awser ease to understand for me