Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Main.seval fails to strip whitespace and therefore throws when it shouldn't. #379

Closed
LilithHafner opened this issue Oct 10, 2023 · 1 comment · Fixed by #380
Closed

Main.seval fails to strip whitespace and therefore throws when it shouldn't. #379

LilithHafner opened this issue Oct 10, 2023 · 1 comment · Fixed by #380
Labels
bug Something isn't working

Comments

@LilithHafner
Copy link
Contributor

Affects: JuliaCall

>>> from juliacall import Main
>>> Main.seval("""
... function f()
...     0
... end
... """)
Julia: f (generic function with 1 method)
>>> if True:
...     Main.seval("""
...     function g()
...         1
...     end
...     """)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/x/.julia/packages/PythonCall/qTEA1/src/jlwrap/module.jl", line 25, in seval
    return self._jl_callmethod($(pyjl_methodnum(pyjlmodule_seval)), expr)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
juliacall.JuliaError: Base.Meta.ParseError("extra token after end of expression")
Stacktrace:
 [1] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:272
 [2] parse
   @ ./meta.jl:266 [inlined]
 [3] pyjlmodule_seval(self::Module, expr::Py)
   @ PythonCall ~/.julia/packages/PythonCall/qTEA1/src/jlwrap/module.jl:13
 [4] _pyjl_callmethod(f::Any, self_::Ptr{PythonCall.C.PyObject}, args_::Ptr{PythonCall.C.PyObject}, nargs::Int64)
   @ PythonCall ~/.julia/packages/PythonCall/qTEA1/src/jlwrap/base.jl:62
 [5] _pyjl_callmethod(o::Ptr{PythonCall.C.PyObject}, args::Ptr{PythonCall.C.PyObject})
   @ PythonCall.C ~/.julia/packages/PythonCall/qTEA1/src/cpython/jlwrap.jl:47

The issue is that Python's multiline string parsing is different from Julia's and does not strip trailing spaces as much, so the strict behavior of Julia's Meta.parse does not make sense for strings that were written using Python's multiline string syntax.

I think PyCall doesn't have this issue, but I'm not positive.

I think that a good solution would be to strip inputs before parsing. It's hard to imagine a case when someone would want code with a leading or trailing space to throw a parse error.

@cjdoris
Copy link
Collaborator

cjdoris commented Oct 10, 2023

Having played around with various combinations of whitespace, this issue only occurs if there is a newline followed by any other character after the expression.

julia> m = Py(Main)
Python: Julia: Main

julia> m.seval("1")
Python: 1

julia> m.seval("1 ")
Python: 1

julia> m.seval("1  ")
Python: 1

julia> m.seval("1\n")
Python: 1

julia> m.seval("1\n ")
ERROR: Python: Julia: Base.Meta.ParseError("extra token after end of expression")

The documentation for Meta.parse says "An error is thrown if there are additional characters after the first expression." Presumably any whitespace before and any whitespace after up to a newline is being treated as part of the expression - which I suppose makes sense because newlines delimit expressions in Julia, but other whitespace does not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants