Add support for the Bun and pnpm package managers.

[commiterate]

Add support for using the Bun and pnpm package managers to manage package.json and node_modules. These are popular Node.js package managers that are faster and more space efficient than npm and Yarn.

Users of the Bun package manager are probably also using the Bun runtime which supports partial CommonJS and ESM interop. This helps smooth over integration between CLJS and the JavaScript ecosystem (no need to mess with different Clojure requires syntax when handling ESM v. CommonJS).

This is not about adding support for running a CLJS REPL on the Bun runtime (e.g. #923 for a CLJS REPL on the Deno runtime). Bun users should use an existing REPL like the browser REPL.

[commiterate]

For those who don't want to wait for this, you can use the existing :node-modules key in the shadow-cljs.edn file.

{:node-modules
 {; Override the guessed Node.js package manager.
  ;
  ; Not needed if you specify :install-cmd.
  :managed-by
  :bun

  ; Override the guessed install command for the Node.js package manager.
  :install-cmd
  ["bun" "add"]}}

The corresponding documentation PR surfaces this in the user guide.

[thheller]

I'm unsure if I want to add this until there is some actual support for these runtimes.

I've been meaning to change how the install process works for a while now, see #998.

[commiterate]

This would only support additional Node.js package managers (i.e. manages package.json and node_modules), not JavaScript runtimes (i.e. executes JavaScript files).

• Node.js Package Managers
  • npm
  • bun
  • pnpm
  • yarn
• JavaScript Runtimes
  • node
  • bun (the binary has both the package manager and runtime)

Users are free to mix and match package managers and runtimes. This is like how in Clojure we can use tools.deps, Leiningen, or any other Clojure package manager with Corretto, OpenJDK, Zulu, or any other Java runtime distribution.

Bun having both the package manager and runtime in a single executable can be confusing since there isn't a CLJS REPL on Bun being added here (users need to use the browser REPL instead). I think we can just note that in the user guide (Node REPL requires node) unless a Bun REPL is a hard requirement.

As a side note, the Bun runtime has polyfills for Node.js runtime APIs.

https://bun.sh/docs/runtime/nodejs-apis

On the pure JavaScript side, most Node.js applications seem to just work when executed on the Bun runtime. It's not 100% coverage yet so there will be some gaps.

Likewise on the CLJS side, people can typically do bun run build/compiled-js-output/main.js and things will usually work as well.

tl;dr I don't think missing a CLJS REPL on the Bun runtime should be a blocker since we'd be complecting Bun package manager support with Bun runtime support.

---

For the linked issue on reworking :npm-deps handling, adding the Bun and pnpm package managers to the current implementation doesn't change the status quo on version resolution. They both follow the Node.js package semver standards so behavior wise they're the same as npm.

In other words, when shadow-cljs finishes deps.cljs Node.js package version resolution and calls bun add --exact {package}@{version} or pnpm add --save-exact {package}@{version}, the behavior will be the same.

I think it might be better for shadow-cljs to not pin exact versions in package.json to be nice to downstream JavaScript consumers. Node.js package manager lock files like package-lock.json or bun.lockb already lock exact versions for in-project building/testing but JavaScript consumers should solve and lock their own versions.

Clojure has similar behavior with deps expansion (e.g. if a library a project uses org.clojure/data.json:2.0.0 but the project declares org.clojure/data.json:2.5.0 in its deps.edn, 2.5.0 is used). deps.edn just always has exact versions because it doesn't use version ranges, opting for a different version resolution algorithm (usually the newest of all specified versions). Node.js package managers on the other hand find the intersection of all the specified version ranges for a given package.

There might be a different pattern where the deps.cljs file declares a dependency on a CLJS package's own package.json which controls actual Node.js dependencies. For example:

cljs-lib-a/src/deps.cljs

{:npm-deps
 {"cljs-lib-a" "^1"}}

cljs-lib-a/package.json

{
    "name": "cljs-lib-a",
    "version": "1.0.0",
    "dependencies": {
        "fraction.js": "^2"
    }
}

This would offload a fair amount of the version resolution logic to the Node.js package manager, but this won't fix existing packages whose deps.cljs files lock different exact versions of the same package.

Granted, this requires publishing effectively a placeholder package to the npm public registry (npmjs.org) that's otherwise empty (unless someone is trying to vend compiled CLJS artifacts).

The main problem is that it doesn't play nice with source-based libraries vended through Git. If they use a Git procurer in deps.edn, the Node.js package manager will try to talk to the npm public registry instead of letting people use a Git URL for the Node.js package.

Another approach for library vendors might be to just omit a deps.cljs completely and have people manually add the Node.js package portion to their package.json (e.g. npm install my-package@^1). For example:

cljs-lib-z/deps.edn

{:deps
 {cljs-lib-a
  {:mvn/version "5.0.0"}

  cljs-lib-b
  {:git/url "git+ssh://github.com/org/cljs-lib-b.git"
   :git/sha "{hash}"}}

cljs-lib-z/package.json

{
    "name": "cljs-lib-z",
    "version": "1.0.0",
    "dependencies": {
        "cljs-lib-a": "^5",
        "cljs-lib-b": "git+ssh://github.com/org/cljs-lib-b.git#{hash}"
    }
}

This seems to work decently well for both source (e.g. Git) and compiled artifact (e.g. Clojars) vending.

It's also problematic though because tools.deps and Node.js semver may yield different Maven and Node.js package coordinates (e.g. might get the 5.0.0 Maven/Clojars package with the 5.1.0 Node.js package).

In comparison, deps.cljs has an advantage. The Java classpath (and PATH for Unix processes) will use the first instance of a resource that shows up on the classpath. For example, if both org.package-1.0.jar/MyClass.class and org.package-1.1.jar/MyClass.class are on the classpath, whichever appears first will be used by the JVM.

Likewise for deps.cljs resolution, any CLJS compiler should prioritize the versions of Node.js packages in :npm-deps depending on which shows up first in the classpath (which is determined by deps.edn so this sits after tools.deps has done deps expansion). I think this is what the ClojureScript compiler and shadow-cljs do today and is the most Clojure/Java-like approach.

To extend that, maybe we should view package.json as a generated build artifact (like ClojureScript compilation output). Thus, the first pattern mentioned above (put your own Node.js package in your deps.cljs) is actually a circular dependency.

People would use their build.clj/bb.edn to generate an initial package.json with everything except dependencies populated (e.g. name, version, devDependencies). Afterwards, the ClojureScript compiler or shadow-cljs will resolve + flatten the various deps.cljs files on the classpath and run the Node.js package manager install commands to populate dependencies in package.json.

Since we start with a fresh package.json each time, we won't end up with orphaned dependencies entries when a CLJS dependency removes something from their deps.cljs.

Here's an example with the ClojureScript compiler (made for testing similar changes submitted to it):

https://github.com/commiterate/test-clojurescript-bun

The main ugliness with this is that deps.cljs usually sits in some source directory instead of at the root of the project alongside deps.edn.

This approach is the same as how we treat pom.xml and pom.properties files when we're generating a JAR with compiled Java classes. It just looks strange because JARs are self contained while Node.js packages use package.json to select/filter what project files are uploaded to an npm registry.