Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems packaging for NixOS #13

Closed
crinklywrappr opened this issue Dec 1, 2021 · 39 comments
Closed

Problems packaging for NixOS #13

crinklywrappr opened this issue Dec 1, 2021 · 39 comments

Comments

@crinklywrappr
Copy link
Contributor

User support record here

I get this error when running ./configure

./configure: line 14058: syntax error near unexpected token `3.0'
./configure: line 14058: `GUILE_PKG(3.0 2.2)'
@rlbdv
Copy link
Collaborator

rlbdv commented Dec 4, 2021

The git build dependency that's mentioned in the support record may be removed in a while. I think I mostly have that working, and it's likely that make dist will then work too.

Regarding the ./configure error, I'm not sure what's going on. If it helps, at the moment, I'm using autoconf 2.69, automake 1.16.3, and guile 3.0.7. Does configure fail in the same way if you clone the repo and follow the build instructions manually?

@crinklywrappr
Copy link
Contributor Author

Does configure fail in the same way if you clone the repo and follow the build instructions manually?

Yes.

@rlbdv
Copy link
Collaborator

rlbdv commented Dec 4, 2021

Presuming you're using a fresh clone, then I'm really not sure what's going on unless you have a guile install whose m4 macros have been adjusted in a way that's breaking things, or the upstream guile m4 macros have a bug with respect to your platform. Just guessing, though.

Could take a look at those lines in the configure script and see if you can spot anything that doesn't look OK, but from the error message I'd guess the real issue may be in the GUILE_PKG macro, which here is in /usr/share/aclocal/guile-3.0.m4. I suppose you could find the relevant file on your system, and compare it to the upstream copy diff -u ..., etc. for your guile version.

@crinklywrappr
Copy link
Contributor Author

It looks like the guile build might be doing something funny. Post-build steps here.

I will check out that file when I get a chance.

@rlbdv
Copy link
Collaborator

rlbdv commented Dec 4, 2021

Oh, if your guile package isn't installing that .m4 macro file in the right place for your system, then that might well be the problem. I believe autoconf has to be able to find it during the autoreconf, etc. I think the "normal" location is /usr/share/aclocal, but I don't know if that's what nix would expect.

@crinklywrappr
Copy link
Contributor Author

Ah, yeah that could be the reason. I did some light googling...

NixOS/nix#3658 is a ticket where some contributor states "the m4 files must be present in the tarball."

Also found this which suggests I need to provide nativeBuildInputs = [ ... autoreconfHook ]; in my default.nix. Something for me to try.

@rlbdv
Copy link
Collaborator

rlbdv commented Dec 4, 2021

I suspect that if you just copy the m4 file into ./m4 in the lokke tree it might work, that or put it "somewhere" and then ./configure -I somewhere, if I recall correctly (just to see if it'll fix that problem).

@crinklywrappr
Copy link
Contributor Author

crinklywrappr commented Dec 15, 2021

Got it to build w/ a lot of help from the nixos community on matrix. doesn't run, but problems for another day....

@crinklywrappr
Copy link
Contributor Author

runtime error after build

❱ /nix/store/rwijsdzlcy679k6959w1q1h9bqp9li8m-lokke/bin/lokke 
Backtrace:
In ice-9/boot-9.scm:
  1752:10  1 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
           0 (apply-smob/0 #<thunk 7fea05b7a080>)

ERROR: In procedure apply-smob/0:
In procedure public-lookup: Module named (lokke main) does not exist
Exception: /nix/store/rwijsdzlcy679k6959w1q1h9bqp9li8m-lokke/bin/lokke exited with 1
[tty 98], line 1: /nix/store/rwijsdzlcy679k6959w1q1h9bqp9li8m-lokke/bin/lokke 

@crinklywrappr
Copy link
Contributor Author

It looks like lokke's scm files aren't getting into the share dir, and therefore not in the load path. 🤔

@crinklywrappr
Copy link
Contributor Author

@rlbdv is there a make target which installs Lokke's scm files?

Here's a copy of my build output: https://gist.github.com/crinklywrappr/f91271b41355bdf8e3bf2bbde0ce91be

@crinklywrappr
Copy link
Contributor Author

crinklywrappr commented Dec 15, 2021

I ran make check and got missing test plan over and over.

image

This looks like the reason

;;; failed to create path for auto-compiled file "/build/lokke-repo/./test/lokke-destructure"

☝🏻 I see that for each test.

@crinklywrappr
Copy link
Contributor Author

Attempting to run the lokke repl by starting w/ the scm files in the load path.

❱ GUILE_LOAD_PATH="/home/crinklywrappr/Documents/lokke/lokke/mod/:/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/share/guile/3.0:/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/share/guile/site/3.0:/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/share/guile/site:/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/share/guile" /nix/store/lmrbb0bangqmsi06c230icv12nir5gv6-lokke/bin/lok
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/main.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/core.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/array.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/collection.scm
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/collection.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/array.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/destructure.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/reader/literal.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/hash-map.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/map.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/map-entry.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/pr.scm
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/pr.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/map-entry.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/map.scm failed:
;;; Unbound variable: <sequential>
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/hash-map.scm failed:
;;; Unbound variable: <coll>
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/reader/literal.scm failed:
;;; Unbound variable: <map>
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/transmogrify.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/hash-set.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/set.scm
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/set.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/hash-set.scm failed:
;;; Unbound variable: <coll>
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/transmogrify.scm failed:
;;; Unbound variable: <set>
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/ns.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/metadata.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/scm/atom.scm
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/scm/atom.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/metadata.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/ns.scm failed:
;;; Unbound variable: atom
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/destructure.scm failed:
;;; Unbound variable: <sequential>
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/collection.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/vector.scm
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/vector.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/collection.scm failed:
;;; Unbound variable: <lokke-vector>
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/core.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/main.scm failed:
;;; In procedure resolve-interface: no binding `aclone' in module (lokke array)
Backtrace:
In ice-9/boot-9.scm:
  3556:26 19 (_)
In unknown file:
          18 (primitive-load-path "lokke/main" #<procedure 7f8856fbf?>)
In ice-9/eval.scm:
   721:20 17 (primitive-eval (define-module (lokke main) # ((?) ?) ?))
In ice-9/psyntax.scm:
  1230:36 16 (expand-top-sequence ((define-module (lokke main) # ?)) ?)
  1222:19 15 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
   259:10 14 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) ?)
In ice-9/eval.scm:
   293:34 13 (_ #<module (#{ g16}#) 7f8856fbba00>)
In ice-9/boot-9.scm:
   3409:4 12 (define-module* _ #:filename _ #:pure _ #:version _ # _ ?)
  2594:24 11 (call-with-deferred-observers #<procedure 7f885482c690 ?>)
  3422:24 10 (_)
   222:29  9 (map1 (((ice-9 eval-string) #:select (eval-string)) # ?))
   222:29  8 (map1 (((ice-9 format) #:select (format)) ((# #) # #) ?))
   222:29  7 (map1 (((ice-9 rdelim) #:select (read-delimited)) (?) ?))
   222:29  6 (map1 (((ice-9 textual-ports) #:select (#)) ((# ?) ?) ?))
   222:29  5 (map1 (((lokke base dynamic) #:select (binding)) (# ?) ?))
   222:29  4 (map1 (((lokke config) #:select (ensure-config-dir)) # ?))
   222:17  3 (map1 (((lokke core) #:select (# prn)) ((lokke #) # ?) ?))
  3353:12  2 (resolve-interface (lokke core) #:select _ #:hide _ # _ ?)
   260:13  1 (for-each #<procedure 7f8854a78300 at ice-9/boot-9.scm?> ?)
  3360:19  0 (_ _)

ice-9/boot-9.scm:3360:19: In procedure resolve-interface: no binding `prn' in module (lokke core)

@crinklywrappr
Copy link
Contributor Author

crinklywrappr commented Dec 16, 2021

Basically the same issue when loading from the guile repl.

❱ GUILE_LOAD_PATH="/home/crinklywrappr/Documents/lokke/lokke/mod/" guile

GNU Guile 3.0.7Copyright (C) 1995-2021 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> %load-path 
$1 = ("/home/crinklywrappr/Documents/lokke/lokke/mod/" "/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/share/guile/3.0" "/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/share/guile/site/3.0" "/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/share/guile/site" "/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/share/guile")
scheme@(guile-user)> ,language lokke
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/language/lokke/spec.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/compile.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/hash-map.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/collection.scm
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/collection.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/hash-map.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/map.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/map-entry.scm
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/pr.scm
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/pr.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/map-entry.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/base/map.scm failed:
;;; Unbound variable: <sequential>
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/compile.scm failed:
;;; Unbound variable: <coll>
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/language/lokke/spec.scm failed:
;;; In procedure resolve-interface: no binding `get' in module (lokke hash-map)
;;; compiling /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/ns.scm
;;; WARNING: compilation of /home/crinklywrappr/Documents/lokke/lokke/mod/lokke/ns.scm failed:
;;; In procedure resolve-interface: no binding `assoc' in module (lokke hash-map)
While executing meta-command:
In procedure resolve-interface: no binding `assoc' in module (lokke hash-map)

@crinklywrappr
Copy link
Contributor Author

I've run out of things to try and am definitely open to ideas.

@crinklywrappr
Copy link
Contributor Author

crinklywrappr commented Dec 16, 2021

hm GUILE_LOAD_PATH=./lokke/mod guile + ,language lokke just works on pop-os.

I noticed that it uses guile 3.0.5, whereas nix uses 3.0.7. Wonder if the problem is with the new version of guile.

EDIT: actually there are some issues, the lok binary works fine but loading from guile fails

image

@crinklywrappr
Copy link
Contributor Author

crinklywrappr commented Dec 16, 2021

@rlbdv how should I set up the environment if I want to execute lokke from inside a guile repl? I tried this on pop-os and it doesn't quite work.

GUILE_LOAD_PATH=./mod LTDL_LIBRARY_PATH=./lib GUILE_LOAD_COMPILED_PATH=./mod guile

EDIT: Got that part working, kinda....

$ GUILE_LOAD_PATH=$(pwd)/mod GUILE_LOAD_COMPILED_PATH=$(pwd)/mod LTDL_LIBRARY_PATH=$(pwd)/lib/.libs guile
GNU Guile 3.0.5
Copyright (C) 1995-2021 Free Software Foundation, Inc.
 
Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.
 
Enter `,help' for help.
scheme@(guile-user)> ,module (lokke user)
scheme@(lokke user)> ,L lokke
Happy hacking with Lokke, a Clojure dialect!  To switch back, type `,L scheme'.
lokke@(lokke user)> [1 2 3]
;;; <unknown-location>: warning: possibly unbound variable `/lokke/reader-vector'
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Unbound variable: /lokke/reader-vector

EDIT2: that also happens with ./guile

@rlbdv
Copy link
Collaborator

rlbdv commented Dec 17, 2021

Hmm, #! scripts like those in test/ should work, but the ,L support does appear to be broken (here too). The repl support isn't nearly as well developed, and I believe we may need to make some improvements to guile itself to get everything working well (e.g. the recursive/error repl printer issue), but I'll try to see what's caused the regression when I get a chance. I'll also finish up and push the make dist fixes, but it may be a bit before I have time.

Thanks for reporting the issue.

@crinklywrappr
Copy link
Contributor Author

Thanks. But really all I was trying to do was take all the magic out of it so hopefully I could understand what was going wrong with my nix build and I'm not any closer tbh.

@rlbdv
Copy link
Collaborator

rlbdv commented Dec 18, 2021

OK, so the repl problem is just an issue with the docs. I changed the (lokke user) module to a clj module a bit back, i.e. lokke.user, and forgot to update the docs. Making it lokke.user meant moving the guile module to (lokke ns lokke user). So this should work:

,module (lokke ns lokke user)
,L lokke

I'll update the docs in a while.

@rlbdv
Copy link
Collaborator

rlbdv commented Dec 18, 2021

And regarding running lok, lokke, or ./guile, they should work as long as all the .scm and/or .go files are in the right arrangement and in the guile load path (perhaps via GUILE_LOAD_PATH and/or GUILE_LOAD_COMPILED_PATH), and the .so files are available via LTDL_LIBRARY_PATH.

You can see exactly what happens in lokke.c, and note that default values for the various paths are embedded in the binaries at compile time (e.g. LOKKE_MODULE_ROOT -- which is not currently affected by an environment var).

We may change some of that eventually, but if it helps, that's how it works right now.

In any case, here at least, and from a completely clean checkout, this allows me to build a lokke that will run from some other directory (using the current main branch):

./setup
autoreconf -fi
./configure --prefix /some/other/directory
make -j5 check
make install
cd /some
some/other/directory/bin/lok

Is that at all relevant to your current problem (not sure I know exactly where you're having trouble now).

@crinklywrappr
Copy link
Contributor Author

very cool. 😎

$ GUILE_LOAD_PATH=$(pwd)/mod LTDL_LIBRARY_PATH=$(pwd)/lib/.libs guile
...
scheme@(guile-user)> ,module (lokke ns lokke user)
...compiler messages...
scheme@(lokke ns lokke user)> ,L lokke
lokke@(lokke ns lokke user)> (inc 1)
$1 = 2

So, based on this I think I need to get some more assistance from the nix community to determine

  1. how to add the mod directory to the output
  2. wtf to do with the libs.

I suspect there is a strong likelihood the lokke executable won't work on nix. It still looks like magic to me (search the project for LOKKE_LIBDIR returns nothing,). I don't know how/where this local guile binary is coming from either, but if the lok/lokke binaries require it that would be yet another reason it probably won't work on nix.

Maybe I can package a simple lok shell script?

@rlbdv
Copy link
Collaborator

rlbdv commented Dec 19, 2021

Hmm, here I have:

(main)$ git grep LOKKE_LIBDIR
Makefile.am.in:lokke_cflags += -DLOKKE_LIBDIR='"$(modlibdir)"'
Makefile.am.in:lokke_dev_cflags += -DLOKKE_LIBDIR='"$(abs_top_srcdir)/lib/.libs"'
lokke.c:    setenv_or_die ("LTDL_LIBRARY_PATH", LOKKE_LIBDIR);
lokke.c:      int rc = asprintf (&path, "%s:%s", LOKKE_LIBDIR, env);

And it may already be possible, but if not we can arrange for it to be possible to set the defaults for the executable to be correct for nix, if you can find out where nix expects the lokke module source tree (the .scm files), the compiled tree (the .go files), and the shared libs (the .so files) to be installed. i.e. we can (or will be able to after a few changes) adjust the embedded defaults via suitable ./configure arguments, or similar.

@crinklywrappr
Copy link
Contributor Author

Oh okay. I guess github's search function is just unreliable.

image

@crinklywrappr
Copy link
Contributor Author

As a quick sanity check, I ran this test on nix by telling nix-build to save the build directory. Based on my prior experience I expected this to work flawlessly and I would then be on my way to actually writing a build file.

❱ GUILE_LOAD_PATH=/tmp/nix-build-lokke.drv-3/lokke-repo/mod/ LTDL_LIBRARY_PATH=/tmp/nix-build-lokke.drv-3/lokke-repo/lib/.libs/ guile
GNU Guile 3.0.7
Copyright (C) 1995-2021 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,module (lokke ns lokke user)
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/ns/lokke/user.scm
;;; compiling /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/ns.scm
;;; compiling /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/hash-map.scm
;;; compiling /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/base/collection.scm
;;; WARNING: compilation of /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/base/collection.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/hash-map.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; compiling /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/base/map.scm
;;; compiling /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/base/map-entry.scm
;;; compiling /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/pr.scm
;;; WARNING: compilation of /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/pr.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/base/map-entry.scm failed:
;;; Syntax error:
;;; unknown location: unexpected syntax in form ()
;;; WARNING: compilation of /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/base/map.scm failed:
;;; Unbound variable: <sequential>
;;; WARNING: compilation of /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/ns.scm failed:
;;; Unbound variable: <coll>
;;; WARNING: compilation of /tmp/nix-build-lokke.drv-3/lokke-repo/mod/lokke/ns/lokke/user.scm failed:
;;; In procedure resolve-interface: no binding `assoc' in module (lokke hash-map)
While executing meta-command:
In procedure variable-ref: Unbound variable: #<variable 7fced6ac63f0 value: #<undefined>>

😕

@crinklywrappr
Copy link
Contributor Author

Made a little more progress. nix removes the .git subdir from the build directory (i guess this is one step toward reproducible builds). So, in order for the call to git ls-files here to work, the builder needs to do a git init . before proceeding. HOWEVER, I did not think about git add -A.

So now I am getting an error with the builder.

libtool: link: ( cd "lib/.libs" && rm -f "lokke-vector.la" && ln -s "../lokke-vector.la" "lokke-vector.la" )
GUILE=/build/lokke-repo/guile \
GUILE_AUTO_COMPILE=0 \
/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/bin/guild compile \
  -Wunbound-variable -Wmacro-use-before-definition \
  -Warity-mismatch -Wformat \
-o mod/language/lokke/spec.go mod/language/lokke/spec.scm
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Syntax error:
unknown location: unexpected syntax in form ()
make[1]: *** [Makefile:2594: mod/language/lokke/spec.go] Error 1
make[1]: Leaving directory '/build/lokke-repo'
make: *** [Makefile:916: all] Error 2

@crinklywrappr
Copy link
Contributor Author

I am going to assume that this is due to some issue w/ Guile 3.0.7 ... Nixos is a couple versions ahead of Debian.

If I get some time I may try to compile 3.0.5 on nixos or vice-versa.

@rlbdv
Copy link
Collaborator

rlbdv commented Dec 21, 2021

I'm using Debian's 3.0.7-1+b1 here, so maybe that's fine. I forget, does lokke build and pass the tests fine "normally" for you? i.e. without involving nix (or whatever's removing the .git dir, etc.)?

And the patches I've mentioned that I'm still working on will remove the need for that ls-files, and for .git during the build process.

You might also want to make sure you don't have cached files interfering somehow. When it decides to autocompile, guile will store the result in ~/.cache/guile/ccache/PATH and may use it later. If you don't mind the risk of having to re-auto-compile things, you can just delete the ccache dir, or you can be more selective and delete sub-paths in there. I'm not suggesting that's the problem, but it's another variable that you could eliminate.

@rlbdv
Copy link
Collaborator

rlbdv commented Dec 21, 2021

Oh, and if you haven't, double-check that you have the latest commit (and maybe a clean tree) -- I did fix some syntax-related issues in the not too distant past.

@rlbdv
Copy link
Collaborator

rlbdv commented Jan 23, 2022

Not sure if it'll help, but just pushed changes that remove the git (and ./.git) build dependency, and support make distcheck, so it should build fine now from a make dist archive.

@crinklywrappr
Copy link
Contributor Author

Thanks. I will update and give it a shot.

@crinklywrappr
Copy link
Contributor Author

crinklywrappr commented Jan 26, 2022

I was able to simplify the build file a bit by removing the calls to git. Huge thanks for that.

let
  pkgs = import <nixpkgs> {};
  unstable = import <nixos-unstable> {};
in with pkgs;
  stdenv.mkDerivation {
    name = "lokke";
    src = pkgs.fetchgit {
      name = "lokke-repo";
      url = "https://github.com/lokke-org/lokke.git";
      rev = "1e14bd689d3f04d64b4ea4b481307e8d381a71d3";
      sha256 = "0p3z35fiwdsw6mkpbq3fn10g67y6klvvmvyc81xdw1lz55mazwi3";
    };
    nativeBuildInputs = [autoreconfHook];
    buildInputs = [man gettext pcre2 pcre2.dev
                   libunistring libunistring.dev
                   guile_3_0 guile_3_0.dev
                   autoconf automake
                   libtool pkg-config];
    postPatch = ''
      patchShebangs .
      ./setup
    '';
    doCheck = true;
  }

But it continues to error.

libtool: link: (cd "lib/.libs" && rm -f "lokke-vector.so" && ln -s "lokke-vector-0.0.1.so" "lokke-vector.so")
libtool: link: ( cd "lib/.libs" && rm -f "lokke-vector.la" && ln -s "../lokke-vector.la" "lokke-vector.la" )
GUILE=/build/lokke-repo/guile \
GUILE_AUTO_COMPILE=0 \
LTDL_LIBRARY_PATH="/build/lokke-repo/lib" \
/nix/store/rspf61hbvg5nd22lmj6f8196vnpkc9c3-guile-3.0.7/bin/guild compile \
  -Wunbound-variable -Wmacro-use-before-definition \
  -Warity-mismatch -Wformat \
  -o mod/language/lokke/spec.go mod/language/lokke/spec.scm
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Syntax error:
unknown location: unexpected syntax in form ()
make[1]: *** [Makefile:2662: mod/language/lokke/spec.go] Error 1
make[1]: Leaving directory '/build/lokke-repo'
make: *** [Makefile:989: all] Error 2

@rlbdv
Copy link
Collaborator

rlbdv commented Jan 27, 2022

That's likely because your guile 3.0.7 is "broken", i.e. you'll need to use a different version or add this patch: https://git.savannah.gnu.org/cgit/guile.git/commit/?id=d79a226359d28f4a1dc5df136e5544d699903a96. We added it to debian a while back, but a normal 3.0.7 won't have it.

@crinklywrappr
Copy link
Contributor Author

crinklywrappr commented Jan 27, 2022

There are tools to alter portions of a derivative in nixos. I will attempt to apply this patch.

That section of the learning material was out of reach for me, so I may need to ask someone for assistance.

@rlbdv
Copy link
Collaborator

rlbdv commented Jan 27, 2022

If it helps, if you can install the build dependencies, you can also just clone guile and check out 3.0.7, git cherry-pick HASH to add that commit, build and install it somewhere, say ~/opt/guile-3.0 and then build lokke against that. If you decide to try that, let me know and I can help with the details. The main thing for the guile part is to ./configure --prefix ~/opt/guile-3.0, and then you'll need some settings to get lokke to find it during the build process, and at runtime.

@crinklywrappr
Copy link
Contributor Author

Thanks @rlbdv

Applying that patch pushed the build process a bit further. Unfortunately, I am now seeing this build error.

GUILE=/build/lokke-repo/guile \
GUILE_AUTO_COMPILE=0 \
LTDL_LIBRARY_PATH="/build/lokke-repo/lib" \
/nix/store/78gqxak7m3hi43r2h7sksb24dsjgq89k-guile-3.0.7/bin/guild compile \
  -Wunbound-variable -Wmacro-use-before-definition \
  -Warity-mismatch -Wformat \
  -o mod/lokke/core.go mod/lokke/core.scm
Backtrace:
In ice-9/boot-9.scm:
   3409:4 19 (define-module* _ #:filename _ #:pure _ #:version _ # _ ?)
  3422:24 18 (_)
   222:29 17 (map1 (((ice-9 match) #:select (match-lambda*)) ((?)) ?))
   222:29 16 (map1 (((oop goops)) ((lokke collection) #:select #) # ?))
   222:29 15 (map1 (((lokke collection) #:select (lazy-seq seq)) # ?))
   222:29 14 (map1 (((lokke exception) #:select (ex-info throw)) # ?))
   222:29 13 (map1 (((lokke hash-map) #:select (hash-map)) ((?) ?) ?))
   222:17 12 (map1 (((lokke pcre2) #:select (# # # PCRE2_UNSET ?)) ?))
  3326:17 11 (resolve-interface (lokke pcre2) #:select _ #:hide _ # _ ?)
In ice-9/threads.scm:
    390:8 10 (_ _)
In ice-9/boot-9.scm:
  3252:13  9 (_)
In ice-9/threads.scm:
    390:8  8 (_ _)
In ice-9/boot-9.scm:
  3536:20  7 (_)
   2835:4  6 (save-module-excursion #<procedure 7ffff08f99f0 at ice-?>)
  3556:26  5 (_)
In unknown file:
           4 (primitive-load-path "lokke/pcre2" #<procedure 7ffff0e1?>)
           3 (load-extension "lokke-pcre2.so" "init_pcre2")
In system/foreign-library.scm:
   190:25  2 (load-foreign-library _ #:extensions _ # _ #:search-path ?)
In unknown file:
           1 (dlopen "/build/lokke-repo/lib/.libs/lokke-pcre2.so" 1)
In ice-9/boot-9.scm:
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure dlopen: file "/build/lokke-repo/lib/.libs/lokke-pcre2.so", message "/build/lokke-repo/lib/.libs/lokke-pcre2.so: undefined symbol: scm_i_string_wide_chars"
make[1]: *** [Makefile:2662: mod/lokke/core.go] Error 1
make[1]: Leaving directory '/build/lokke-repo'
make: *** [Makefile:989: all] Error 2

@rlbdv
Copy link
Collaborator

rlbdv commented Jun 12, 2022

Hmm, what do you see for ldd /build/lokke-repo/lib/.libs/lokke-pcre2.so?

And what happens if you try this from the lokke source tree:

$ ./guile
GNU Guile 3.0.8
Copyright (C) 1995-2021 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (load-extension "lokke-pcre2.so" "init_pcre2")
scheme@(guile-user)>

@rlbdv
Copy link
Collaborator

rlbdv commented May 21, 2023

Just checking back to see if this was still relevant.

@crinklywrappr
Copy link
Contributor Author

Closing for now. I'll probably take another crack at it after I replace my broken laptop (won't boot)

@crinklywrappr crinklywrappr closed this as not planned Won't fix, can't repro, duplicate, stale May 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants