Spark integration fails due to web sockets not being supported

[mkjois]

I have a working hello-world-ish local Spark server, but when trying to run it on Lambda it throws the UnsupportedOperationException when the embedded server tries to configure web sockets.

Here's the exact error I'm getting:

START RequestId: <request_id_redacted> Version: $LATEST
[main] INFO <class_redacted> - Initializing routes
[Thread-0] ERROR spark.Spark - ignite failed
java.lang.UnsupportedOperationException
	at com.amazonaws.serverless.proxy.spark.embeddedserver.LambdaEmbeddedServer.configureWebSockets(LambdaEmbeddedServer.java:65)
	at spark.Service.lambda$init$2(Service.java:502)
	at java.lang.Thread.run(Thread.java:745)
END RequestId: <request_id_redacted>
REPORT RequestId: <request_id_redacted>	Duration: 1501.41 ms	Billed Duration: 1600 ms 	Memory Size: 512 MB	Max Memory Used: 54 MB	
RequestId: <request_id_redacted> Process exited before completing request

I can't see a current workaround other than some crazy reflection magic. I guess there are some potential fixes like:

• change LambdaEmbeddedServer.configureWebSockets() to not throw an exception
• use a custom initExceptionHandler on the Spark.Service instance to ignore the exception

Thoughts?

[sapessi]

API Gateway and Lambda do not support web sockets so you wouldn't be able to receive requests from clients. What are you trying to accomplish with web sockets?

[mkjois]

I'm not using web sockets. But the Spark service always calls configureWebSockets() on startup, which throws an exception, which kills the process.

[sapessi]

Can you show me the code for your LambdaHandler class?

[sapessi]

The one catch with spark is that you need to define the resources after you have initialized the handler: https://github.com/awslabs/aws-serverless-java-container/blob/master/samples/spark/pet-store/src/main/java/com/amazonaws/serverless/sample/spark/LambdaHandler.java#L45

[mkjois]

Sure. Thanks for the quick reply btw.

public class LambdaHandler
        implements RequestHandler<AwsProxyRequest, AwsProxyResponse> {

    private SparkLambdaContainerHandler<AwsProxyRequest, AwsProxyResponse> handler;

    @Override
    public AwsProxyResponse handleRequest(AwsProxyRequest input, Context context) {

        if (handler == null) {
            try {
                handler = SparkLambdaContainerHandler.getAwsProxyHandler();
                HelloSparkServer.start();
            } catch (ContainerInitializationException e) {
                throw new RuntimeException("Failed to initialize server container", e);
            }
        }

        return handler.proxy(input, context);
    }
}

public class HelloSparkServer {

    private static final Logger LOG = LoggerFactory.getLogger(HelloSparkServer.class);

    // not used on AWS Lambda
    public static void main(String[] args) {

        int port = getPort();
        if (port >= 0) {
            port(port);
        }

        start();
        LOG.info(format("Listening on port %d", port()));
    }

    public static void start() {

        LOG.info("Initializing routes");

        path("/api", () -> {

            path("/v1", () -> {

                before("/*", (req, res) -> {
                    LOG.info(format("%s %s %s",
                            req.ip(),
                            req.requestMethod(),
                            req.url()));
                });

                get("/things/:id/prop",
                        (req, res) -> format("Hello thing %s prop", req.params(":id")));
                post("/things/:id/prop/update",
                        (req, res) -> format("Hello thing %s prop update", req.params(":id")));

                notFound((req, res) -> "Not found");
                internalServerError((req, res) -> "Internal server error");

                after("/*", (req, res) -> res.type("application/json"));
            });
        });
    }

    private static int getPort() {
        String portFromEnv = System.getenv("FM_API_PORT");
        return portFromEnv == null ? -1 : Integer.parseInt(portFromEnv);
    }
}

[sapessi]

Thanks, I will test this. Which version of spark are you using?

[mkjois]

2.6.0

I'm looking right now to see if that's the issue. Their changelog is here: http://sparkjava.com/news#spark-kotlin-released

[sapessi]

Thanks. I'll test this and publish a fix in the next few days. Flying around right now.

[sapessi]

Meanwhile, could you try with spark 2.5.3 - the library is tested with this version. If you want to join the Gitter chat room I'll probably post updates there as I work on this.

[mkjois]

Getting a strange NPE with Spark 2.5.3:

Error while handling request: nulljava.lang.NullPointerException
	at com.amazonaws.serverless.proxy.internal.servlet.AwsLambdaServletContainerHandler.doFilter(AwsLambdaServletContainerHandler.java:179)
	at com.amazonaws.serverless.proxy.spark.SparkLambdaContainerHandler.handleRequest(SparkLambdaContainerHandler.java:158)
	at com.amazonaws.serverless.proxy.spark.SparkLambdaContainerHandler.handleRequest(SparkLambdaContainerHandler.java:57)
	at com.amazonaws.serverless.proxy.internal.LambdaContainerHandler.proxy(LambdaContainerHandler.java:119)
	at <package_redacted>.LambdaHandler.handleRequest(LambdaHandler.java:28)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at lambdainternal.EventHandlerLoader$PojoMethodRequestHandler.handleRequest(EventHandlerLoader.java:456)
	at lambdainternal.EventHandlerLoader$PojoHandlerAsStreamHandler.handleRequest(EventHandlerLoader.java:375)
	at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:1139)
	at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:278)
	at lambdainternal.AWSLambda.<clinit>(AWSLambda.java:62)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:94)

It's hard to tell but it looks like maybe filterChainManager isn't being set? Looking through the code, it looks like the implementation of SparkLambdaContainerHandler.handleRequest() doesn't set the httpServletContext with the setter here, which sets filterChainManager. I'm not certain though...have you seen this before?

[sapessi]

Yes, it looks like it's a bug with the latest release. I think FilterChainManager was introduced in 0.5. Try and use 0.4 of the library - I'll include this fix in the TT. Sorry about the churn on this.

[sapessi]

I've found the issue and have a fix - I will create some unit tests for this over the weekend and push it. I'll let you know when it's there and you can start using 0.6-SNAPSHOT.

The fix is very simple. I needed to call the

super.handleRequest(httpServletRequest, httpServletResponse, lambdaContext);

in the spark handler. If you want, you can add it yourself here.

[sapessi]

I've pushed a fix for the bugs in the 2.5.3 support to the servlet-improvements branch. You should be able to get started right away by cloning the branch, mvn install, and adding 0.6-SNAPSHOT as a dependency to your project.

I noticed that spark made some breaking changes in 2.6.0 that require bigger changes to the library, particularly if we want to support both 2.5.x and 2.6.x versions. I have created a separate issue to track this #42

[mkjois]

Thanks a bunch! Would it be possible to publish the 2.5.3-related bug fixes to the 0.5 version on maven central? I agree that supporting 2.6 would be appropriate for the 0.6 release. Perhaps it's a good idea to employ semver with patch updates?

[sapessi]

Yes, it may make sense to push out a fixed 0.5.1. I would also like to include #43 in the release, just waiting to get a little bit of feedback on the proposed changes then I'll push this out.

[dmcg]

+1 for a 0.5.1 to get us running with Spark 2.5.3

[sapessi]

@mkjois, @dmcg I've committed both fixes and it seems to be working fine for me. Would you mind testing with 0.6-SNAPSHOT from the servlet-improvements branch

[dmcg]

Will do. Do you expect it to work against Spark 2.6 or just 2.5.3?

[sapessi]

@dmcg only 2.5.3 - I have support for 2.6 in a separate issue for the next major release

[dmcg]

0.6-SNAPSHOT from the servlet-improvements branch works for me, thanks

[sapessi]

Good to know, thanks. If all goes well I will push out 0.5.1 today.

[sapessi]

Resolving now that the fix is verified and about to go out