Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The state of AWS Lambda Go Streaming support #565

Open
jmnarloch opened this issue Jul 10, 2024 · 2 comments
Open

The state of AWS Lambda Go Streaming support #565

jmnarloch opened this issue Jul 10, 2024 · 2 comments
Labels
question

Comments

@jmnarloch
Copy link
Contributor

jmnarloch commented Jul 10, 2024 •
edited
Loading

Is your feature request related to a problem? Please describe.
I had been trying make the AWS Lambda response streaming work with aws-lambda-go runtime and I was not able to succeed. Neither the Lambda Function URLs or direct API call using InvokeWithResponseStream work and instead I was retrieving the entire response serialized to JSON.

I made my Lambda function return the events.LambdaFunctionURLStreamingResponse, but the entire response had been serialized and returned as JSON. The passed io.Reader had not been proceesed and suprisingly enough the response contain JSON serialized representation of the io.Reader implementation i.e.

{"StatusCode":200,"Headers":{"Content-Type":"application/octet-stream"},"Body":{"ReadCloser":{}},"Cookies":null}

I had been runing the GO Lambda function on al2023 runtime, compiled the binary with -tags lambda.norpc option and set up the Function URL.

FunctionUrl:
    Type: AWS::Lambda::Url
    Properties:
      TargetFunctionArn: !Ref Function
      AuthType: NONE
      InvokeMode: RESPONSE_STREAM

Neither of this had any impact and the behavior persisted. Even the documented in code example is not working correctly:

// Example:
//
//	lambda.Start(func() (*events.LambdaFunctionURLStreamingResponse, error) {
//		return &events.LambdaFunctionURLStreamingResponse{
//			StatusCode: 200,
//			Headers: map[string]string{
//				"Content-Type": "text/html",
//			},
//			Body: strings.NewReader("<html><body>Hello World!</body></html>"),
//		}, nil
//	})
//
// Note: This response type requires compiling with `-tags lambda.norpc`, or choosing the `provided` or `provided.al2` runtime.

Which results in response from Function URL:

{"StatusCode":200,"Headers":{"Content-Type":"text/html"},"Body":{},"Cookies":null}

Is Response Streaming not supported with Go runtime?

Describe the solution you'd like
I would like to use Lambda Response Streaming with my Go runtime.

Describe alternatives you've considered
I am not using AWS Lambda for this use case today.

Additional context
github.com/aws/aws-lambda-go v1.47.0
@bmoffatt
Copy link
Collaborator

Streaming URLs is supported! I'm sorry to see that you're having issues!

The passed io.Reader had not been proceesed and suprisingly enough the response contain JSON serialized representation of the io.Reader implementation i.e.
{"StatusCode":200,"Headers":{"Content-Type":"application/octet-stream"},"Body":{"ReadCloser":{}},"Cookies":null}

Can you provide a code sample that produces this response? Also your Go version and version of aws-lambda-go from your go.mod

Neither of this had any impact and the behavior persisted. Even the documented in code example is not working correctly

{"StatusCode":200,"Headers":{"Content-Type":"text/html"},"Body":{},"Cookies":null}

The example code works for me when I tested today.

My main.go:

package main

import (
        "strings"

        "github.com/aws/aws-lambda-go/events"
        "github.com/aws/aws-lambda-go/lambda"
)

func main() {
        lambda.Start(func() (*events.LambdaFunctionURLStreamingResponse, error) {
                return &events.LambdaFunctionURLStreamingResponse{
                        StatusCode: 200,
                        Headers: map[string]string{
                                "Content-Type": "text/html",
                        },
                        Body: strings.NewReader("<html><body>Hello World!</body></html>"),
                }, nil
        })
}

My template.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  StreamingFunction:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: go1.x
    Properties:
      CodeUri: .
      Handler: .
      Runtime: provided.al2023
      Architectures: [ arm64 ]
      Timeout: 10
      FunctionUrlConfig:
        AuthType: AWS_IAM
        InvokeMode: RESPONSE_STREAM
Outputs:
  StreamingFunctionURL:
    Description: "Streaming Lambda Function URL"
    Value: !GetAtt StreamingFunctionUrl.FunctionUrl

Deployed and tested sam build && sam deploy --region us-west-2 --profile "$USER".

@jmnarloch
Copy link
Contributor Author

jmnarloch commented Jul 11, 2024 •
edited
Loading

Thank you for the comprehensive response. I realized that the main reason why I run into this problem is that I had use BuildMethod: makefile once I change it into BuildMethod: go1.x and remove the Makefile from the project the response has been process fully and was not serialized as JSON.

"{\"statusCode\":200,\"headers\":{\"Content-Type\":\"text/html\"}}\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000<html><body>Hello World!</body></html>"

I guess my question is, is there still possible to make this work with a custom Makefile?

The makefile I had been trying to use

build-Function:
	GOOS=linux GOARCH=arm64 go build -o bootstrap -tags lambda.norpc .
	cp ./bootstrap $(ARTIFACTS_DIR)/.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bmoffatt @jmnarloch