Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exec does not convert :env and :extra-env keywords #123

Closed
lread opened this issue May 11, 2023 · 1 comment · Fixed by #131
Closed

exec does not convert :env and :extra-env keywords #123

lread opened this issue May 11, 2023 · 1 comment · Fixed by #131

Comments

@lread
Copy link
Contributor

lread commented May 11, 2023

Problem

While working on #117 I have found that exec does not convert :env and :extra-env like process does.

Repro

This applies to all platforms, but I'll show behaviour on Linux.
It is easiest for me to demonstrate running from bb as exec only works when natively compiled:

❯ bb --version
babashka v1.3.179

Given file repro.clj

(require '[babashka.process :as p]
         '[clojure.edn :as edn])

(def opts (-> *command-line-args*
              first
              edn/read-string))

;; -cp '' so that bb does not require JAVA_HOME
(def cmd "bb -cp '' script/wd.clj :env")

(println "Testing with opts:" (pr-str opts))

(println "Running p/process")
(-> (p/process opts cmd)
    :out
    slurp
    println)

(println "Running p/exec")
(p/exec opts cmd)

:env

All strings - all good

If we use string keys and vals, all good:

> bb repro.clj '{:env {"k1" "v1" "k2" "v2" "k3" "v3"}}' 
Testing with opts: {:env {"k1" "v1", "k2" "v2", "k3" "v3"}}
Running p/process
{"k3" "v3", "k1" "v1", "k2" "v2"}
Running p/exec
{"k3" "v3", "k1" "v1", "k2" "v2"}%    

Keyword for key - exec fails

But if we use keyword for key, process works, but exec fails:

❯ bb repro.clj '{:env {"k1" "v1" :k2 "v2" "k3" "v3"}}' 
Testing with opts: {:env {"k1" "v1", :k2 "v2", "k3" "v3"}}
Running p/process
{"k3" "v3", "k1" "v1", "k2" "v2"}
Running p/exec
----- Error --------------------------------------------------------------------
Type:     java.lang.ClassCastException
Message:  clojure.lang.Keyword cannot be cast to java.lang.String
Location: /home/lee/proj/oss/babashka/process/repro.clj:20:1

----- Context ------------------------------------------------------------------
16:     slurp
17:     println)
18: 
19: (println "Running p/exec")
20: (p/exec opts cmd)
    ^--- clojure.lang.Keyword cannot be cast to java.lang.String

----- Stack trace --------------------------------------------------------------
babashka.process/exec - <built-in>
user                  - /home/lee/proj/oss/babashka/process/repro.clj:20:1

Keyword for val - exec fails

Same idea if we use keyword for val:

❯ bb repro.clj '{:env {"k1" "v1" "k2" :v2 "k3" "v3"}}' 
Testing with opts: {:env {"k1" "v1", "k2" :v2, "k3" "v3"}}
Running p/process
{"k3" "v3", "k1" "v1", "k2" ":v2"}
Running p/exec
----- Error --------------------------------------------------------------------
Type:     java.lang.ClassCastException
Message:  clojure.lang.Keyword cannot be cast to java.lang.String
Location: /home/lee/proj/oss/babashka/process/repro.clj:20:1

----- Context ------------------------------------------------------------------
16:     slurp
17:     println)
18: 
19: (println "Running p/exec")
20: (p/exec opts cmd)
    ^--- clojure.lang.Keyword cannot be cast to java.lang.String

----- Stack trace --------------------------------------------------------------
babashka.process/exec - <built-in>
user                  - /home/lee/proj/oss/babashka/process/repro.clj:20:1

:extra-env

Repros not shown but same idea for:

> bb repro.clj '{:extra-env {"k1" "v1" "k2" :v2 "k3" "v3"}}' 
> bb repro.clj '{:extra-env {"k1" "v1" :k2 "v2" "k3" "v3"}}'
> bb repro.clj '{:extra-env {"k1" "v1" "k2" :v2 "k3" "v3"}}'  

Expected Behaviour

exec should convert keywords to strings for :env and :env-var like process does.

Next Steps

An easy fix, I handle this with #117

@borkdude
Copy link
Contributor

Sounds good.

lread added a commit to lread/process that referenced this issue May 22, 2023
New bb tasks:
- clean - delete build work
- test:bb - runs all tests under babashka
- test:native - runs exec tests against native image of run_exec.clj
- test:jvm - (renamed from test) runs non-exec tests under jvm clojure

CI:
- GitHub Actions updated to run native image tests
  - jobs renamed to match CircleCI naming convention
- CircleCI was doing a subset of GitHub Actions work, it now is a full reflection.
  - added jdk install support in script/circle_ci.clj
  - bootstrapping CI to be able to run bb and then scripting in bb is
  much more pleasant/maintainable than scripting in bash+Powershell
  - used jdk install support even on linux, instead of trying to
  to select jdk via docker image. This made config cleaner, allowed for
  consitent jdk selection across OSes and had surprisingly little
  performance hit especially after caching jdks.
  - installing jdks via bb hugely more performant than doing so via
    brew/scoop/chocolatey
  - jdk download urls, and shas discovered by disco API. See:
  https://github.com/foojayio/discoapi
- Both CIs are temporarily using a SHAPSHOT release of bb for
`babashka.process/exec` reload support. We'll turf this after next
release of babashka.

Add `run_exec.clj` - a wee program to exercise `exec`
- source and build support under ./test-native
- includes script-adjacent `bb.edn` to suport running as a script and
reloading sources instead of using babashka built-in babashka.process
- specify `babashka.process.test.reload` system property to signal that
`run_exec.clj` should reload `babashka.process` from sources when
running from bb.
- note: `test:native` task does not natively compile tests, it only
natively compiles `run_exec.clj`. It launches natively compiled
`run_exec.clj` via jvm clojure and observes results for correctness.

Test utilities moved to `babashka.process.test-utils` so that they can
be used by existing `babashka.process-test` and new:
- `babashka.process-exec-test`
- bb build scripts (specifically: test-native/bb.edn)

Add :ps-me to wee dummy wd.clj to support `exec` arg0 testing for macOS
and Linux.

Add doc/dev.md with some relevant notes/pointers.

Other changes:
- deleted `script/test` bash script, it is replaced by bb tasks
- we now look for bb first in current dir, then on path. This seems the
safer way to choose bb when running babashka.process tests from babashka
- tests now printing bb version to give confidence we are using the
bb we want to be using

Bugs found and fixed as a result of testing:
- tests using java now understand that java executable path can contain
spaces.
- test using `javac` seem to be happier running `javac` from same dir as source
- `exec` now converting `:env` and `:extra-env` keywords the
same way `process` function do, fixes babashka#123

Closes babashka#117
lread added a commit to lread/process that referenced this issue May 22, 2023
New bb tasks:
- clean - delete build work
- test:bb - runs all tests under babashka
- test:native - runs exec tests against native image of run_exec.clj
- test:jvm - (renamed from test) runs non-exec tests under jvm clojure

CI:
- GitHub Actions updated to run native image tests
  - jobs renamed to match CircleCI naming convention
- CircleCI was doing a subset of GitHub Actions work, it now is a full reflection.
  - added jdk install support in script/circle_ci.clj
  - bootstrapping CI to be able to run bb and then scripting in bb is
  much more pleasant/maintainable than scripting in bash+Powershell
  - used jdk install support even on linux, instead of trying to
  to select jdk via docker image. This made config cleaner, allowed for
  consitent jdk selection across OSes and had surprisingly little
  performance hit especially after caching jdks.
  - installing jdks via bb hugely more performant than doing so via
    brew/scoop/chocolatey
  - jdk download urls, and shas discovered by disco API. See:
  https://github.com/foojayio/discoapi
- Both CIs are temporarily using a SHAPSHOT release of bb for
`babashka.process/exec` reload support. We'll turf this after next
release of babashka.

Add `run_exec.clj` - a wee program to exercise `exec`
- source and build support under ./test-native
- includes script-adjacent `bb.edn` to suport running as a script and
reloading sources instead of using babashka built-in babashka.process
- specify `babashka.process.test.reload` system property to signal that
`run_exec.clj` should reload `babashka.process` from sources when
running from bb.
- note: `test:native` task does not natively compile tests, it only
natively compiles `run_exec.clj`. It launches natively compiled
`run_exec.clj` via jvm clojure and observes results for correctness.

Test utilities moved to `babashka.process.test-utils` so that they can
be used by existing `babashka.process-test` and new:
- `babashka.process-exec-test`
- bb build scripts (specifically: test-native/bb.edn)

Add :ps-me to wee dummy wd.clj to support `exec` arg0 testing for macOS
and Linux.

Add doc/dev.md with some relevant notes/pointers.

Other changes:
- deleted `script/test` bash script, it is replaced by bb tasks
- we now look for bb first in current dir, then on path. This seems the
safer way to choose bb when running babashka.process tests from babashka
- tests now printing bb version to give confidence we are using the
bb we want to be using

Bugs found and fixed as a result of testing:
- tests using java now understand that java executable path can contain
spaces.
- test using `javac` seem to be happier running `javac` from same dir as source
- `exec` now converting `:env` and `:extra-env` keywords the
same way `process` function do, see babashka#123

Closes babashka#117, Fixes babashka#123
borkdude pushed a commit that referenced this issue May 27, 2023
New bb tasks:
- clean - delete build work
- test:bb - runs all tests under babashka
- test:native - runs exec tests against native image of run_exec.clj
- test:jvm - (renamed from test) runs non-exec tests under jvm clojure

CI:
- GitHub Actions updated to run native image tests
  - jobs renamed to match CircleCI naming convention
- CircleCI was doing a subset of GitHub Actions work, it now is a full reflection.
  - added jdk install support in script/circle_ci.clj
  - bootstrapping CI to be able to run bb and then scripting in bb is
  much more pleasant/maintainable than scripting in bash+Powershell
  - used jdk install support even on linux, instead of trying to
  to select jdk via docker image. This made config cleaner, allowed for
  consitent jdk selection across OSes and had surprisingly little
  performance hit especially after caching jdks.
  - installing jdks via bb hugely more performant than doing so via
    brew/scoop/chocolatey
  - jdk download urls, and shas discovered by disco API. See:
  https://github.com/foojayio/discoapi
- Both CIs are temporarily using a SHAPSHOT release of bb for
`babashka.process/exec` reload support. We'll turf this after next
release of babashka.

Add `run_exec.clj` - a wee program to exercise `exec`
- source and build support under ./test-native
- includes script-adjacent `bb.edn` to suport running as a script and
reloading sources instead of using babashka built-in babashka.process
- specify `babashka.process.test.reload` system property to signal that
`run_exec.clj` should reload `babashka.process` from sources when
running from bb.
- note: `test:native` task does not natively compile tests, it only
natively compiles `run_exec.clj`. It launches natively compiled
`run_exec.clj` via jvm clojure and observes results for correctness.

Test utilities moved to `babashka.process.test-utils` so that they can
be used by existing `babashka.process-test` and new:
- `babashka.process-exec-test`
- bb build scripts (specifically: test-native/bb.edn)

Add :ps-me to wee dummy wd.clj to support `exec` arg0 testing for macOS
and Linux.

Add doc/dev.md with some relevant notes/pointers.

Other changes:
- deleted `script/test` bash script, it is replaced by bb tasks
- we now look for bb first in current dir, then on path. This seems the
safer way to choose bb when running babashka.process tests from babashka
- tests now printing bb version to give confidence we are using the
bb we want to be using

Bugs found and fixed as a result of testing:
- tests using java now understand that java executable path can contain
spaces.
- test using `javac` seem to be happier running `javac` from same dir as source
- `exec` now converting `:env` and `:extra-env` keywords the
same way `process` function do, see #123

Closes #117, Fixes #123
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants