v5.3 keys, v5.2 hdiff errors, as well as freezes/crashes with AnimeWwise v2.1

[Boyishdude]

There are just a few things I want to talk about (I also just woke up after some poor-quality sleep, so I may not be the best at articulating things).

---

It's been nearly two weeks since v5.3 launched, and there are still no keys for v5.3. I assume this is because you seem to be working on v2.2 of AnimeWwise? That being the case, I'll just have to update the game on PC now so I can do the event before it's gone. I've already downloaded v5.3 .hdiff and v5.3 complete English audio from Reddit, so extracting the v5.3 stuff later shouldn't be an issue provided that the program is working properly. Speaking of...

---

AnimeWwise v2.1 always gives me an error when I try to load the .hdiff files for v5.2:

This is a consistent problem when I try to so much as load .hdiff files, so I suspect it's some sort of bug with the program.

---

Whenever I try to extract the Complete English v5.2 audio after it's been loaded, the application always freezes up/locks up, and if I try to click on it at all, I get a pop-up from Windows about how the app is not responding (to which I get to choose whether to wait or to force the application to close):

One thing I did notice though is how the program seems to lag like hell whenever I click the checkboxes for some of the files, such as the checkbox for VO_AQ. The program is always unresponsive for a few seconds when I click on that and a couple of the other checkboxes in the list, then it becomes responsive again once it's figured out that I wanted to check/uncheck those things. My best guess is that the program is having difficulty dealing with the massive amount of files in general. It's either that, or the extraction function just isn't working (I extract as WAV/WAVE, rather than WEM).

I can also confirm that whatever causes this is not related to my RAM usage, since Python isn't leaking memory like it did on that older version of the application last year. This seems like an outright bug, or instability in the program compared to previous versions of it, and it's not something I can fix myself.

[Boyishdude]

I should also mention that v2.2 of AnimeWwise does not seem to currently work. All of the requirement.txt dependencies are already installed, but when I tried opening it, it gave me an error:

This is mildly annoying because the branch for v2.2 says that you fixed something with extract.py, so I was hoping that using v2.2 would address the problems I've been having these past couple of weeks. I guess I'll just have to wait until v2.2 gets done, since it appears to be unfinished right now.

[Boyishdude]

There are just a few things I want to talk about (I also just woke up after some poor-quality sleep, so I may not be the best at articulating things).

It's been nearly two weeks since v5.3 launched, and there are still no keys for v5.3. I assume this is because you seem to be working on v2.2 of AnimeWwise? That being the case, I'll just have to update the game on PC now so I can do the event before it's gone. I've already downloaded v5.3 .hdiff and v5.3 complete English audio from Reddit, so extracting the v5.3 stuff later shouldn't be an issue provided that the program is working properly. Speaking of...

AnimeWwise v2.1 always gives me an error when I try to load the .hdiff files for v5.2:

This is a consistent problem when I try to so much as load .hdiff files, so I suspect it's some sort of bug with the program.

Whenever I try to extract the Complete English v5.2 audio after it's been loaded, the application always freezes up/locks up, and if I try to click on it at all, I get a pop-up from Windows about how the app is not responding (to which I get to choose whether to wait or to force the application to close):

One thing I did notice though is how the program seems to lag like hell whenever I click the checkboxes for some of the files, such as the checkbox for VO_AQ. The program is always unresponsive for a few seconds when I click on that and a couple of the other checkboxes in the list, then it becomes responsive again once it's figured out that I wanted to check/uncheck those things. My best guess is that the program is having difficulty dealing with the massive amount of files in general. It's either that, or the extraction function just isn't working (I extract as WAV/WAVE, rather than WEM).

I can also confirm that whatever causes this is not related to my RAM usage, since Python isn't leaking memory like it did on that older version of the application last year. This seems like an outright bug, or instability in the program compared to previous versions of it, and it's not something I can fix myself.

I decided to just let the app do its thing and got an actual error for the third part about the program freezing/crashing during extraction (it presumably closed on its own):

This being some sort of permission error is.. weird. It's not like the directories where the files are located require Administrator privileges to access. This is all being done on a 5 TB external HDD, and I didn't have this problem with older versions from before Natlan (I haven't extracted anything using it since v4.8; I just didn't feel motivated to work on my creative project that required the audio files from the game for quite a while).

[Escartem]

Thanks for this long report, so i'll sumarize everything...

• 2.1 is completly broken right now, the loading system for files / folders was bad as hell and decided to fix it in 2.2. As for 2.2 it actually works fine the error you had is just a python thing, it's a change they made around python 3.12 or so and older versions don't handle it. I fixed it just now so it'll work with any version now. So i recommend you using 2.2 as it should have everything fixed.
• For the keys, they are planned in the 2.2 update too, I got help on a way to recover the names of general audio files like music and sound effects in addition to voicelines so i'm currently working on updating my file format to handle those and then i'll publish 2.2 with everything up to date.
• for the lag, it seems to be a python / qt issue, when the list have ten of thousands of files, clicking one may cause freeze and choosing "extract all" also causes it to freeze because it's loading every element and taking a while. I don't know how to fix it honestly because it's an issue from the window engine itself. As for the freeze it's just because this specific action (interacting with the file list) is not threaded, so the app pauses waiting for it to finish unlike other actions like extracting. And windows being windows, if the main window is paused waiting for a task to finish, windows will show that infamous "app is not responding" message when in reality it's working fine, just doing a heavy task in a non-threaded way...
• lastly for the errors you had, the first screenshot indicates it found 0 hdiff files which is weird and should be handled, and for the permission error it's up to windows to handle this, at best i can say to run the tool in admin but that's it.

hope this helps you 👍

[Boyishdude]

Kinda figured that v2.1 was completely broken, since nothing worked aside from loading non-.hdiff files.

I'm not sure how to run the program as an administrator when the context menu (when I right click on the folder containing your program) doesn't give me that option, it just gives me "Open in Terminal". Best I think I might be able to do is finding the location of the Terminal's .exe file and changing its properties to always run it as an administrator, but I don't actually know where that's located.

And ye, Windows fucking sucks donkey dick. It's so weirdly funny how everyone who engages with open source software in some capacity always seems to arrive at this conclusion organically. I would use Linux on my laptop if there weren't a few apps I specifically use that only work on Windows.

Edit: Figured out how to manually navigate to the directory of your program in the Terminal after looking it up. I'll see if running the terminal as as administrator addresses the .hdiff issue or not.

Edit 2: Turns out that I was just being dumb. I forgot that the Complete English Audio folder I needed to use to extract the v5.2 .hdiff files is the complete English audio for v5.1, not the complete English audio for v5.2. It's working now.

[Boyishdude]

Kinda figured that v2.1 was completely broken, since nothing worked aside from loading non-.hdiff files.

I'm not sure how to run the program as an administrator when the context menu (when I right click on the folder containing your program) doesn't give me that option, it just gives me "Open in Terminal". Best I think I might be able to do is finding the location of the Terminal's .exe file and changing its properties to always run it as an administrator, but I don't actually know where that's located.

And ye, Windows fucking sucks donkey dick. It's so weirdly funny how everyone who engages with open source software in some capacity always seems to arrive at this conclusion organically. I would use Linux on my laptop if there weren't a few apps I specifically use that only work on Windows.

Edit: Figured out how to manually navigate to the directory of your program in the Terminal after looking it up. I'll see if running the terminal as as administrator addresses the .hdiff issue or not.

Edit 2: Turns out that I was just being dumb. I forgot that the Complete English Audio folder I needed to use to extract the v5.2 .hdiff files is the complete English audio for v5.1, not the complete English audio for v5.2. It's working now.

Nope, even with administrator privileges in Terminal, it still gives me a permission error when I use Extract All as WAVE/WAV on the Complete English Audio for v5.2:

I'm at a loss here. I have no idea what's causing the problem or why. The file I apparently didn't even have permission for this time is different from the file I didn't have permission for last time as well, so that complicates things even further.

[Escartem]

well if the permission errors still persist, try copying the pck files on a new folder to check and if it's still doing it then it's likely a bug from my program that's probably reading the file and trying to open a new instance of it but it's locked or something, tell me if copying works otherwise I'll look for issues in the tool

[Boyishdude]

I think I actually found the source of the issue; it couldn't do anything with the files because the root directory and subfolders/files where the .pck files were located was set to Read-only. I turned it off but kept the Properties window open (closing it will reset the folders and files to Read-only, even if you hit Apply and then Okay) and was able to extract the complete v5.2 English audio twice without any issues (using the Extract All function; the second time around, I didn't even have anything selected in the list manually, I just hit Extract All from the toolbar and waited a while for it to do its thing).

If it's actually the Read-only thing, and I'm inclined to believe it is, it might need to be made special note of in the usage instructions on the main page for the project. Troubleshooting this was kinda frustrating, and I'd rather other people be spared the trouble.

I know the amount of files mapped and the total amount of files are different, but I didn't get any error messages in the Terminal. I just waited for the program to do what it did, then it used Focus Steal to let me know that it was done.

[Boyishdude]

To be precise about the special notes for the instructions on using the program, when you turn off Read-only for the root of the directory with the files (in this case, my folder called Audio_English(US)_5.2.0 is where all the audio files for v5.2 are located), you need to select the option that also applies this change to all files and subfolders in the extracted directory.

You MUST leave the properties window for the directory with the .pck files open after turning off Read-only properly until you're done using AnimeWwise to extract files because if you close the window before you're done doing whatever it is you need to do with the program, the directory will go back to being Read-only automatically.

You may also need to check the option inside AnimeWwise itself to include subfolders, and then load the files with the root directory (click on "Audio_English(US)_5.2.0" after hitting the 'Load file(s)' button, then click 'Select Folder') to make sure they can be extracted properly. I don't know if this last part is true, but that's just what I did to be on the better side of "this is more likely to work than this other approach". You might not need to go this extra mile at all.

There's also the distinct possibility that this might not happen to everyone who will try to use this program, so that's why I'm saying it just be a special note like this one:

...rather than being treated as fundamental instructions for using the program. This is just for if you get permission errors when trying to extract the files.

[Escartem]

i would need to add some sort of faq to explain this, even though I never encountered this problem and even by forcing it, I didn't have to keep the window open either, maybe it was only on your computer ? on the other hand, all mappings were updated to the latest version

[Boyishdude]

What ended up being the problem for me in this case might not happen to everyone, but that's just the nature of Windows, honestly. It's a terrible operating system that doesn't function well (I should note that I'm using Windows 11, though 11 is basically just Windows 10 under the hood). Did you try testing this on your main drive or an external drive? I have all of my AI stuff as well as your program on a 5 TB external HDD, unless it's something like UVR (which needs to be installed on the main/system drive in order to work).

I'll certainly check out the new v5.3 mappings in a jiffy.

[Escartem]

Windows 11 is a reskin of 10, which is 7 being xp being nt and so on, at least backwards compatibility is fine. On my side the tool is running on an external ssd, and the game files are in an internal ssd, none of them being my C drive