Figure out what to do when different versions of json and json_pure are loaded

[byroot]

Ref: #646

There is no version constraints between the two, but they expose the same files in $LOAD_PATH, so if two different versions are loaded it can easily cause incompatibilities.

We should find a solution for this. Perhaps use require_relative? But even then, large parts of the implementation are in the global JSON module, so we can't load two namespaced versions of it.

[headius]

Normally I'd say the common parts should be in a separate gem json_base that both json and json_pure depend upon (with a strict version). If they tried to activate different versions it would be an error earlier in library management, and even better it would not silently appear to load ok and then break.

However it would mean another gem in stdlib.

The only other alternative is shipping the common parts namespaced too.

[Earlopain]

Since this is an unsupported use-case, can they both define a version constant somewhere independent from each other?

begin
  require "json/pure/from_gem/version"
  require "json/ext/version"
  if JSON::Ext::VERSION != JSON::Pure::FromGem::VERSION
    raise "Some descriptive error"
  end
rescue LoadError
end

This leaves people with older versions out of luck but maybe its acceptable? OTOH, loading different versions probably has just worked for a long time.

[byroot]

Yet another such issue #652

if JSON::Ext::VERSION != JSON::Pure::FromGem::VERSION

Yeah, this is kinda the obvious / easy thing to do. But I wonder if it's really what should be done.

I also wonder if the solution isn't simply to deprecate and remove json_pure.

After all, now that json is in the stdlib, why would you need json_pure specifically?

[byroot]

When I have some time, I'd like to go over look more into https://rubygems.org/gems/json_pure/reverse_dependencies, because at first sight I don't see a lot of legitimate use.

So I think we could just push a json_pure that's just a gemspec that require json. The Pure implementation would remain inside json but we'd enforce that only one implementation is ever loaded (e.g. use the pure generator on truffle, etc).

[headius]

why would you need json_pure specifically?

Didn't we just have an issue where someone wanted to use json_pure because they deployed on a system without build tools?

[byroot]

It was resolved by the fact that json is part of the stdlib, so there's no need to build it.

[eregon]

It was resolved by the fact that json is part of the stdlib, so there's no need to build it.

Mmh, but that requires to use the same version of json as in stdlib (which might not be an option if some bug fix or change is wanted) and recent Bundler always reinstalls default gems (json is a default gem): rubygems/rubygems#8180

Maybe we could have some documented env var (or a extconf.rb flag, should also work fine) to allow skipping the C extension when installing the json gem?

[byroot]

Maybe we could have some documented env var (or a extconf.rb flag, should also work fine) to allow skipping the C extension when installing the json gem?

That would seem like the better solution, I need to research how this is generally done.

[eregon]

There is enable_config(name, default) in mkmf, e.g. in https://github.com/ruby/prism/blob/ff26b40f5c8fe5458db6203de095ae83d3f921a8/ext/prism/extconf.rb#L123C4-L123C17

[byroot]

One tricky thing if we don't build the extension (and kinda already is an issue with json_pure today) is that we can't just require 'json/ext' and rescue LoadError, as we may end up loading the extension that is bundled with Ruby.

So we'd need to use require_relative or something like that, but I don't think we can really trust the .so is in a consistent relative location?

[eregon]

Mmh, good point, maybe we should remember we skipped the extension then e.g. by creating some file under the installed gem directory. And if that file is there, don't attempt to require any extension.
Or even creating lib/json/ext/{parser,generator}.rb to just raise LoadError from the extconf.rb if skipping extensions would be enough?

[byroot]

Seems brittle. I have a hard time trusting assumptions about how/when extconf.rb is invoked, etc.

But I think we might be able to rely on .so precedence over .rb?

[eregon]

But I think we might be able to rely on .so precedence over .rb?

It's the other way around, .rb always wins over .so (for require 'some/path/without/extension').

[byroot]

Arf, 😢 . I'm a bit out of idea then.

[eregon]

Quick though: what if the json_pure gem had files lib/json/ext/{parser,generator}.rb which both raise LoadError, and depended on the regular json gem (might not even need to be the same version)?
Then as long as json_pure would be on $LOAD_PATH before require 'json' it would ensure only the pure parser/generator are used, based on

json/lib/json/ext.rb

Lines 16 to 17 in 4af700e

	require 'json/ext/parser'
	require 'json/ext/generator'

[byroot]

Right, I think that would work? We'd need to make sure these files are only included in the pure package though, but that seems doable.

[byroot]

Alright, I opened #682, @eregon I'd appreciate your thoughts on it.

[eregon]

Maybe json could depend on json_pure? That sounds nice conceptually.
But not sure how that would play out with json being a default gem, it would mean json_pure becomes a default gem too I suppose.
And we'd likely still need exact version constraints to be safe.

---

I'm getting more of the opinion that there shouldn't be any json_pure gem (it just seems a workaround for broken environments), which we could handle by json_pure being an empty gem depending on json.

[Earlopain]

I don't see the point of json_pure. Under bundler, you just need one gem that depends on json to break your build if you can't compile, with faraday being the most common (though there also many development dependencies in that list). Since bundler doesn't allow substituting dependencies, there is not much you can do in that situation, only the env variable proposed above would help with that.

Without bundler, it's already installed anyways. So I guess you could use it bundler-less if you need the latest version for some reason? Or you just so happen to just not have json in your lockfile at this point which could change at any time.

[byroot]

Yeah, that's why I suggested to publish a final version of it that is just an empty gem that depends on json in #650 (comment).

But I'm not 100% sure of the consequences yet, so I think I'll delay this to the 3.0, and for 2.8 I'll do #682

[byroot]

Alright, I released json_pure 2.8.0 as basically an empty gem with a dependency on json and no version constraint: https://rubygems.org/gems/json_pure/versions/2.8.0

[Earlopain]

Doesn't depending on json defeat the whole point though? Now you also need to compile when installing the pure variant which I thought was its only actual "feature". Wouldn't it be better to remove the dependency and let it instead pick up the version shipped with Ruby?

[byroot]

Right...

[byroot]

Fixed in 2.8.1.

[Earlopain]

I wasn't quick enough to comment on the PR, appologies. Easy fix at least