Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use libsamplerate on WAV loads #99572

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

DeeJayLSP
Copy link
Contributor

@DeeJayLSP DeeJayLSP commented Nov 23, 2024

Replaces the resampling procedure in AudioStreamWAV load with a libsamplerate resample, using its Medium Sinc Interpolator. Only on editor builds.

Instead of using a fast interpolation algorithm like Cubic, it uses the second best resampler that exists in the world of open-source. High-frequency sounds on lower sample rates should become slightly less silent.

The initial version of the PR used Sinc Best, but was replaced due to being too heavy and slow compared to the others, the gain being not that great compared to Medium. Sinc Medium is used instead.

@DeeJayLSP DeeJayLSP requested review from a team as code owners November 23, 2024 03:40
@fire
Copy link
Member

fire commented Nov 23, 2024

I have used "Secret Rabbit Code" https://libsndfile.github.io/libsamplerate/ before, and it is excellent.

It's worth it!

Copy link
Member

@fire fire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay with this, but others may have strong opinions about weight gain enhancement proposal-wise.

Needs a code review, too.

@fire
Copy link
Member

fire commented Nov 23, 2024

I used the Secret Rabit Code for VoIP, so could it be exposed somehow? I would be thrilled, but it can be another PR.

Godot has significant issues with sample rate conversion for microphone inputs vs speaker sample rate.

editor/import/resource_importer_wav.cpp Outdated Show resolved Hide resolved
editor/import/SCsub Outdated Show resolved Hide resolved
@DeeJayLSP
Copy link
Contributor Author

DeeJayLSP commented Nov 23, 2024

I did a few production builds to check the size penalty.

The approximate results:

Method Increase
Sinc Fast 24.31 KiB
Sinc Medium 100.31 KiB
Sinc Best 1.31 MiB

Note, however, that libsamplerate is only being applied on editor builds, for this importer. Size increases tend to be a concern on template builds mostly.

In my opinion, either Medium or Best should be the one used. While Fast is slightly better than the current Cubic, I think it's better to sacrifice a few more KiBs.

@Calinou
Copy link
Member

Calinou commented Nov 23, 2024

Sinc Medium is probably the right compromise here, considering a 1+ MB increase is very significant for mobile and especially the web editor (it directly impacts its startup times).

Import speeds also matter for ease of iteration, and Sinc Medium should import files much faster than Sinc Best too. I'd be surprised if the difference can be heard in a double blind test either way 🙂

@DeeJayLSP
Copy link
Contributor Author

DeeJayLSP commented Nov 23, 2024

I'd be surprised if the difference can be heard in a double blind test either way 🙂

By doing a blind shootout test between the three Sinc modes with a high-frequency heavy audio, I ended up picking Medium most of the times. Fast turned to be the most noticeable (and worst, obviously).

Considering the audio won't be upsampled using a heavy algorithm anyway, Medium really seems to be the best choice.

@DeeJayLSP
Copy link
Contributor Author

DeeJayLSP commented Nov 23, 2024

Rebased to use Sinc Medium instead of Sinc Best.

@DeeJayLSP DeeJayLSP requested review from a team as code owners December 4, 2024 02:47
@DeeJayLSP DeeJayLSP changed the title Use libsamplerate on WAV imports Use libsamplerate on WAV loads Dec 4, 2024
@DeeJayLSP
Copy link
Contributor Author

DeeJayLSP commented Dec 4, 2024

Addressing changes from #93831, which moved most of the WAV loading code from ResourceImporterWAV to AudioStreamWAV.

I didn't want libsamplerate's size penalty to affect template builds (which I believe isn't worth it at all), so I made the libsamplerate implementation editor-only. The old cubic interpolation method remains on template builds.

@fire
Copy link
Member

fire commented Dec 4, 2024

@DeeJayLSP, is there anything in particular you want me to review?

@DeeJayLSP
Copy link
Contributor Author

DeeJayLSP commented Dec 4, 2024

@DeeJayLSP, is there anything in particular you want me to review?

Now that the importer has been moved to non-editor builds, I wanted an opinion about Sinc being editor only (given it's a significant size increase for something that wouldn't be used that often), and the way it's implemented.

@fire
Copy link
Member

fire commented Dec 4, 2024

I'll probably force the libsamplerate code to work in the runtime templates for my use in problems like #99930 and voip, but use your best judgment.

@adamscott
Copy link
Member

Is there a way that each platform could define the quality needed for Sinc? It could be even exposed to the user at compile time.

@DeeJayLSP
Copy link
Contributor Author

Is there a way that each platform could define the quality needed for Sinc? It could be even exposed to the user at compile time.

Each Sinc quality is locked behind a define, so you could just swap them. Although I think Fast isn't that much of an improvement compared to Cubic (and it's much heavier).

As this is an import procedure, I think consistency between platforms should be respected.

@adamscott
Copy link
Member

As this is an import procedure, I think consistency between platforms should be respected.

Maybe. But we should at least set a SCons parameter of the chosen quality, with medium as default.

@DeeJayLSP
Copy link
Contributor Author

DeeJayLSP commented Dec 5, 2024

Added a builtin_libsamplerate variable to allow it to be provided by the system.

Checks failing to find samplerate.h, weird since it builds fine in my system. What did I do wrong?

Edit: Solved. I have to be more careful with prepends.

@DeeJayLSP DeeJayLSP force-pushed the secretrabbitcode branch 2 times, most recently from f43a435 to c8137b2 Compare December 5, 2024 14:12
@DeeJayLSP
Copy link
Contributor Author

DeeJayLSP commented Dec 7, 2024

As this is an import procedure, I think consistency between platforms should be respected.

Maybe. But we should at least set a SCons parameter of the chosen quality, with medium as default.

I'm honestly against exposing the other Sinc methods because:

  1. Sinc Best provides little benefit over Sinc Medium considering the additional import time, size penalty and, although not really an issue, git diff.
  2. Sinc Fastest isn't much better than Godot's built-in Cubic, which is a much faster method. Another case of little benefit.
  3. Some applications like sndfile-resample and LMMS are set to use Sinc Medium by default, despite providing both Sinc Best and Sinc Fastest as options. Godot isn't a software focused in audio, so I think it should go with the most "default" option.

If there is support, however, I would 100% add the other options. But as it is at the time of this writing, someone that really needs Sinc Best or Sinc Fastest could just drop the header for it in the folder, then change two lines of code (the define exposing the respective Sinc method and the method used AudioStreamWAV).

@DeeJayLSP
Copy link
Contributor Author

DeeJayLSP commented Dec 19, 2024

I was experimenting this PR with a few different sounds, and I noticed a significant difference in a specific one.

The following was taken from the TPS Demo, specifically the robot explosion sound:

samplerate.webm

All sounds above (apart from the original) were saved using Godot's save_to_wav function.

On Cubic, you can notice a high frequency sound at the beginning that you can't hear on Sinc Medium or Sinc Best (or even Sinc Fastest, but it muffles so much that I removed it from the equation). The sound is also noticeable in libsamplerate's Linear and ZOH converters.

To some extent, I think it's accurate to say this PR fixes something now, but only from the editor's side.

@Repiteo Repiteo modified the milestones: 4.4, 4.x Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants