Can't compile custom module includes ejabberd_sql_pt.hrl

[relbraun]

Environment

• ejabberd version: 22.05
• Erlang version: 24
• OS: MacOs (with docker ejabberd/ecs)
• Installed from: docker

loglevel: 5
...

Errors from CLI

error in parse transform 'ejabberd_sql_pt': {undef,
                                             [{erl_syntax_lib,map,
                                               [#Fun<ejabberd_sql_pt.27.33154580>,
                                                {attribute,
                                                 {1,1},
                                                 file,
                                                 {"/home/ejabberd/.ejabberd-modules/sources/mod_ariel/src/mod_ariel.erl",
                                                  1}}],
                                               []},
                                              {ejabberd_sql_pt,
                                               '-top_transform/1-fun-1-',1,
                                               [{file,
                                                 "src/ejabberd_sql_pt.erl"},
                                                {line,151}]},
                                              {lists,map,2,
                                               [{file,"lists.erl"},
                                                {line,1243}]},
                                              {ejabberd_sql_pt,
                                               parse_transform,2,
                                               [{file,
                                                 "src/ejabberd_sql_pt.erl"},
                                                {line,63}]},
                                              {compile,foldl_transform,3,
                                               [{file,"compile.erl"},
                                                {line,1145}]},
                                              {compile,fold_comp,4,
                                               [{file,"compile.erl"},
                                                {line,416}]},
                                              {compile,internal_comp,5,
                                               [{file,"compile.erl"},
                                                {line,400}]},
                                              {compile,
                                               '-internal_fun/2-anonymous-0-',
                                               2,
                                               [{file,"compile.erl"},
                                                {line,229}]}]}
Error: error

Bug description

I have a custom module who using the ejabberd_sql_pt.hrl and when I try to compile it using ejabberdctl module_upgrade mod_ariel I get the error above.

This is the module:

-module(mod_ariel).


-include("ejabberd_sql_pt.hrl").
-include("logger.hrl").

start(_Host, _Opts) ->
 
  ?INFO_MSG("Hello mod_ariel ~p", [_Host]),
  ok.
 

stop(Host) ->
 ok.

depends(_Host, _Opts) ->
  [].

mod_options(_Host) ->
  [].

Also, I tried to do the compilation using the ghcr.io/processone/ejabberd docker and received this response:

 error in parse transform 'ejabberd_sql_pt':
exception error: undefined function erl_syntax_lib:map/2
  in function  ejabberd_sql_pt:'-top_transform/1-fun-1-'/1 (src/ejabberd_sql_pt.erl, line 151)
  in call from lists:map/2 (lists.erl, line 1243)
  in call from ejabberd_sql_pt:parse_transform/2 (src/ejabberd_sql_pt.erl, line 63)
  in call from compile:foldl_transform/3 (compile.erl, line 1147)
  in call from compile:fold_comp/4 (compile.erl, line 418)
  in call from compile:internal_comp/5 (compile.erl, line 402)
  in call from compile:'-internal_fun/2-anonymous-0-'/2 (compile.erl, line 229)
Error: error

It is very weird because I looked at the erlang/OPT repository and saw this function exsisted.

[relbraun]

@prefiks do you have any idea?

[prefiks]

Possibly syntax_tools erlang app (where i think this module lives) is not included in erlang that your installation is using? Could you see if you run debug shell what executing this gives: m(erl_syntax_lib).

[relbraun]

@prefiks you are right and I get error on m(erl_syntax_lib). How can I fix it? (It's the erlang/OTP from the docker)

[relbraun]

@badlop I see in the latest commit that you removed the erlang-syntax-tools and other erlang libraries so it makes the compilation impossible.

[weiss]

I see in the latest commit that you removed the erlang-syntax-tools and other erlang libraries so it makes the compilation impossible.

As mentioned in the commit message, most of those packages no longer exist in Alpine 3.14 (which is used here), including erlang-syntax-tools. The contents were moved to the main erlang package, which should also include the erl_syntax_lib.beam file we're talking about.

[relbraun]

@weiss but running m(erl_syntax_lib) returns

** exception error: undefined function erl_syntax_lib:module_info/0
     in function  c:m/1 (c.erl, line 841)

[weiss]

Understood, and I'm not aware of the casue, just wanted to clarify that it's probably not the commit above.

[relbraun]

@weiss in addition, the general erlang package not exist in the Dockerfile as well.

[badlop]

ejabberd in the docker image (and the github container too) is compiled using the full erlang/OTP suite. Then an OTP release is generated, which includes only the strictly needed OTP libraries.

Strictly needed... as defined in src/ejabberd.app.src.script

I imagine adding this to ejabberd, then regenerating the image (which generates the OTP release) would solve the problem:

diff --git a/src/ejabberd.app.src.script b/src/ejabberd.app.src.script
index a657b640b..dd2cdff93 100644
--- a/src/ejabberd.app.src.script
+++ b/src/ejabberd.app.src.script
@@ -16,7 +16,7 @@ Vars = case file:consult(filename:join([filename:dirname(SCRIPT), "..", "vars.co
   {registered, []},
   {applications, [kernel, sasl, ssl, stdlib]},
   {included_applications,
-    [compiler, inets, mnesia, os_mon,
+    [compiler, inets, mnesia, os_mon, syntax_tools,
      cache_tab,
      eimp,
      fast_tls,

I can reproduce your problem if I attempt to compile that module with an ejabberd OTP release generated with "make dev".

But now I wonder, is it worth including syntax_tools in the general OTP release? Maybe there's a solution for your user case with docker that doesn't involve including it in all cases.

[prefiks]

We are using that module after all, so i think adding it to release is the correct thing to do here.

[badlop]

Try this image: https://github.com/processone/ejabberd/pkgs/container/ejabberd/29952195?tag=master