Infinite recursion in SourcemappingUrlProcessor

[tomhughes]

We have an asset in our project that appears to be causing an infinite recursion in the new SourcemappingUrlProcessor with a stack that looks like:

/usr/local/share/gems/ruby/3.0.0/gems/sprockets-rails-3.4.0/lib/sprockets/rails/sourcemapping_url_processor.rb:15:in `block in call'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-rails-3.4.0/lib/sprockets/rails/sourcemapping_url_processor.rb:12:in `gsub'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-rails-3.4.0/lib/sprockets/rails/sourcemapping_url_processor.rb:12:in `call'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:84:in `call_processor'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:66:in `block in call_processors'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:65:in `reverse_each'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:65:in `call_processors'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:22:in `block in <class:CompositeProcessor>'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:33:in `call'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:84:in `call_processor'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:66:in `block in call_processors'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:65:in `reverse_each'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/processor_utils.rb:65:in `call_processors'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/loader.rb:182:in `load_from_unloaded'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/loader.rb:59:in `block in load'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/loader.rb:337:in `fetch_asset_from_dependency_cache'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/loader.rb:43:in `load'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/cached_environment.rb:44:in `load'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/context.rb:116:in `load'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/context.rb:144:in `depend_on_asset'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/context.rb:184:in `link_asset'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-rails-3.4.0/lib/sprockets/rails/context.rb:27:in `compute_asset_path'
/usr/local/share/gems/ruby/3.0.0/gems/actionview-6.1.4.1/lib/action_view/helpers/asset_url_helper.rb:203:in `asset_path'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-rails-3.4.0/lib/sprockets/rails/sourcemapping_url_processor.rb:18:in `block in call'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-rails-3.4.0/lib/sprockets/rails/sourcemapping_url_processor.rb:12:in `gsub'
/usr/local/share/gems/ruby/3.0.0/gems/sprockets-rails-3.4.0/lib/sprockets/rails/sourcemapping_url_processor.rb:12:in `call'

which then repeats over and over.

The asset that triggers it is https://github.com/Johann-S/bs-custom-file-input/blob/master/dist/bs-custom-file-input.js and although the url map comment seems sensible the preprocessor gets reinvoked for bs-custom-file-input.js rather than bs-custom-file-input.js.map when build the asset path.

[esb]

+1 Got the same problem yesterday and had to rollback to the previous version. I haven't had time to identify the asset that triggered the bug, but I'm getting the same type of recursion.

[chubchenko]

+1

[tomhughes]

This is the call where the problem starts:

/usr/local/share/gems/ruby/3.0.0/gems/sprockets-4.0.2/lib/sprockets/context.rb:144:in `depend_on_asset'

It is called with bs-custom-file-input.js.map but when it calls resolve on that it gets a URI for bs-custom-file-input.js which it then tries to load.

[ebenenglish]

Same issue after updating to 3.4.0, reverting to 3.2.2 solved the problem.

[rafaelfranca]

@dhh this seems to be caused by your change. I tried to understand how to fix but unfortunately I could not understand the requirements.

[dhh]

I'll investigate and fix.

[dhh]

The requirement is that sourceMappingURL needs to refer to digested assets, otherwise it's not going to work when those comments are added by esbuild/webpack/etc, because Sprockets will digest all assets in production. It's the same issue as with CSS url().

[dhh]

Trying to figure out exactly how this is triggered. How are you including bs-custom-file-input.js? Is it a local file?

Can you try #483 to see if it resolves the issue?

[tomhughes]

It's just installed in node_modules and then pulled into app/assets/javascripts/application.js with:

//= require bs-custom-file-input
//= require bs-custom-file-input-init

[tomhughes]

I've checked and #483 makes no difference but then I wouldn't expect it to as there is no path involved here.

[dhh]

Hmm, I figured the issue was that we were running the same replacement more than once. But guess not. I'll try to replicate your setup.

[dhh]

Found the issue. It's a bug deeper in Sprockets. If you do context.resolve("a/source.js.map") you'll correctly get a/source.js.map back. If you do context.resolve("a/b/source.js.map"), you'll incorrectly get a/b/source.js back instead 🤔. Vaguely remember hitting this bug before in another issue, but don't recall the specifics.

Anyway, I can work around this for now, but would be nicer to fix the underlying issue!

[dhh]

@tomhughes Can you give #484 a try?

[dhh]

Confirmed fixed on the reproduction and released as 3.4.1. Thanks for the help tracking this down!

Solving the underlying and long-standing issue with sprocket lookups on 2+ deep nesting is still pending. If anyone is interested in pursuing, please do.

[tomhughes]

Yes that seems to have fixed it for us as well, thanks.