Different execution outcome between the Sandbox and Nashorn executed directly

[sinaa]

Hello again. I'm a bit at loss trying to figure out why the following script executes fine with Nashorn when executed directly, but does not work with the NashornSandbox.

The eval on NashornSandboxes triggers the following exception: javax.script.ScriptException: TypeError: null has no such function "type" in <eval> at line number 1.

Please find below a unit test for this:

  @Test
  public void testMapReduce() throws ScriptException {
    String script = "[1,2,3,4].map(function(n){return n+1}).reduce(function(prev,cur){return prev+cur}, 0)";
    String[] NASHORN_ARGS = {"--no-java", "--no-syntax-extensions"};

    Double nashornResult = (Double) new NashornScriptEngineFactory().getScriptEngine(NASHORN_ARGS).eval(script);
    assertEquals(14, nashornResult.intValue());

    Double sandboxResults = (Double) NashornSandboxes.create(NASHORN_ARGS).eval(script);
    assertEquals(14, sandboxResults.intValue());
  }

Sandbox version this was tried on: 0.1.13

[sinaa]

Strange enough, the same test above passes on 0.1.11 but not on 0.1.12 and above.

[mxro]

Thank you for reporting this issue and please excuse the delay.

Hmm, that's really interesting. Let me add a unit test for this ...

[mxro]

When I run this test, I get:

javax.script.ScriptException: ReferenceError: "Java" is not defined in <eval> at line number 1
	at jdk.nashorn.api.scripting.NashornScriptEngine.throwAsScriptException(NashornScriptEngine.java:470)
	at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:454)
	at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:406)
	at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:402)
	at jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155)
	at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
	at delight.nashornsandbox.internal.EvaluateOperation.executeScriptEngineOperation(EvaluateOperation.java:47)
	at delight.nashornsandbox.internal.NashornSandboxImpl.executeSandboxedOperation(NashornSandboxImpl.java:156)
	at delight.nashornsandbox.internal.NashornSandboxImpl.eval(NashornSandboxImpl.java:146)
	at delight.nashornsandbox.internal.NashornSandboxImpl.eval(NashornSandboxImpl.java:127)
	at delight.nashornsandbox.TestIssue56.testMapReduce(TestIssue56.java:25)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)
Caused by: <eval>:1 ReferenceError: "Java" is not defined
	at jdk.nashorn.internal.runtime.ECMAErrors.error(ECMAErrors.java:57)
	at jdk.nashorn.internal.runtime.ECMAErrors.referenceError(ECMAErrors.java:319)
	at jdk.nashorn.internal.runtime.ECMAErrors.referenceError(ECMAErrors.java:291)
	at jdk.nashorn.internal.objects.Global.__noSuchProperty__(Global.java:1441)
	at jdk.nashorn.internal.scripts.Script$4532$\^eval\_.:program(<eval>:1)
	at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:637)
	at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494)
	at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393)
	at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:449)
	... 31 more

[mxro]

When removing '--no-java' it works.

Might it be that Nashorn uses Java internally for implementing the map function?

[sinaa]

Thanks for following up on the issue.

The issue in here (and the reason I filed it as an issue on this repo) is that there is a difference between Nashorn and the sandbox.

Note that the NASHORN_ARGS I provided above are passed to both the Nashorn engine as well as the sandbox. Therefore, if this was a problem at engine level, then running it directly with nashorn should have also triggered the fault.

However, note that on the same test case, the failure actually happens on line 25 (at delight.nashornsandbox.TestIssue56.testMapReduce(TestIssue56.java:25)), therefore the first assertion on line 23 is passing. Either both should fail, or both should pass.

Could it be that the sandbox is injecting additional code that requires Java execution? And if so, can it be disabled when --no-java is on?

[mxro]

Thank you for further looking into this!

The sandbox indeed depends on a Java class to function properly. The best solution I could come up with is to disallow using the engine parameter --no-java.

I don't think any Java classes should be available anyway when .disallowAllClasses can be used. Or are there any other restrictions which can only be achieved with --no-java?

[sinaa]

Fair enough.

Is there a documentation of what java code (or additional code) sandbox executes along with the requested scripts?

Moreover, how confident can one be of the disallowAllClasses() to filter out everything?

[mxro]

If you add the following to your log4j properties it will output the code generated:

log4j.logger.delight.nashornsandbox.internal.NashornSandboxImpl=DEBUG

Please be welcome to do tests on how well disallowAllClasses works - to my knowledge, it shouldn't be possible to call any Java classes (but sandboxes are always tricky).

[sinaa]

Thanks, that's helpful.

I'll create new issues if I find any further challenges.