Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incomplete response when using mode=grpcwebtext #951

Open
anthonyrouseau opened this issue Sep 8, 2020 · 18 comments
Open

incomplete response when using mode=grpcwebtext #951

anthonyrouseau opened this issue Sep 8, 2020 · 18 comments

Comments

@anthonyrouseau
Copy link

When using mode=grpcwebtext for client configuration, responses are giving an error:

{code: StatusCode.UNKNOWN, message: 'Incomplete response'}

The response is received by the client, but is truncated (e.g. string should be 'hello world' but is 'hello'). This error does not occur when using mode=grpcweb. I'm unsure if additional configuration is needed to use grpcwebtext, but following the grpc-web examples it does not seem like it.

It looks like this may be related to #903 and issue #881

@stanley-cheung
Copy link
Collaborator

Can you give us more information please? There is not enough information to go on.

@adityavikasd
Copy link

adityavikasd commented Sep 27, 2020

I'm facing the same issue. I get an error in the response

{code: 2, message: 'Incomplete response'}

Screen Shot 2020-09-27 at 4 50 20 PM
I followed the hello world example just as mentioned and used the following nginx.conf file (along with some CORS config not provided below) in a docker image

master_process off;
daemon off;
worker_processes 1;
pid nginx.pid;
error_log stderr info;
env GRPC_VERBOSITY=DEBUG;
error_log /dev/stdout info;

events {
  worker_connections 1024;
}

http {
  access_log /dev/stdout;
  client_max_body_size 0;
  client_body_temp_path client_body_temp;
  proxy_temp_path proxy_temp;
  proxy_request_buffering off;
  server {
    listen 8080;
    listen 8443 http2;
    server_name localhost;
    
    location / {
        grpc_pass host.docker.internal:9090;
    }
  }
}

I have added a line to print the incoming request to the method handler in the server.js file to see what is being received on the server when we make a gRPC-web request

function doSayHello(call, callback) {
    callback(null, {
        message: 'Hello ! ' + call.request.name
    });
    console.log("Request: ", call.request)
}

I see that there is no name in the request when the request is being logged. And, there are no errors on the server. It makes sense because if there was no name sent, then the server should just say hello without a name.

❯ node server.js                                                                                          
Request:  { name: '' }

The Chrome Dev tools shows that a payload is being sent for the request and I'm not sure if the payload contains a name, and a 200 status code is received, with a hello ! response too

Screen Shot 2020-09-27 at 4 56 12 PM
Screen Shot 2020-09-27 at 4 56 17 PM
Screen Shot 2020-09-27 at 4 56 35 PM

I used the gRPC Web Dev tools chrome extension, found here https://github.com/SafetyCulture/grpc-web-devtools and I see that the name is being sent.

Screen Shot 2020-09-27 at 4 57 17 PM

Note

After coming across this issue, I tried using grpcweb as the wire format and this works just fine. But, if I use the grpcwebtext format I can still reproduce this issue.

@adam-rocska
Copy link

With our case it happens even if I have our protos compiled as mode=grpcweb & we get the same rubbish error in the grpc-web-devtools mumbo.

We can confirm however, that the message should be there, as other clients in more appropriate languages & stacks can understand the receive error; only js has problems ¯_(ツ)_/¯

@adam-rocska
Copy link

@stanley-cheung I can grant you access to our stuff and guide you around if you'd need a repro-environment.
It's quite annoying as it really works with all clients... even with the most disgusting php scripts 😆

This is how we do the rejection in the swift server for example (dumb down version of-course):

    func deAuthenticate(request: _21gram_Authentication_Token, context: StatusOnlyCallContext) -> EventLoopFuture<Google_Protobuf_Empty> {
        let status = GRPCStatus(code: .permissionDenied, message: "Pay up buttercup.")
        context.responseStatus = status
        return context.eventLoop.makeFailedFuture(status)
    }

@stanley-cheung
Copy link
Collaborator

Would help if there's a simple reproducible test case that I can clone, launch a docker image or something and reproduce the error.

Or is there a special situation that triggers this?

@adam-rocska
Copy link

@stanley-cheung I don't know if it's special or not; I don't think so. It shouldn't be.

You know what? I'll look into creating such a package later today (GMT+1) and ping you once it's done.
I'm only worried about the back-end service side of things, as we roll with Swift & I know it's not mainstream yet ¯_(ツ)_/¯
Oh, and we don't do docker or any local env. virtualization, so that'll also add some fun.

@remvst
Copy link

remvst commented Dec 14, 2020

I'm also facing the same issue. I am able to query my APIs using BloomRPC with the web toggle on, and targetting the envoy port, but when querying them from the web client, I get Object { code: 2, message: "Incomplete response" }. It doesn't seem like the request makes it even to the server, and I don't see anything in the network tab of the dev tools (in the client).

Both client and server are using typescript exports, in case that matters.

Server dependencies:

        "google-protobuf": "^3.14.0",
        "grpc": "^1.24.2",
        "grpc-tools": "^1.8.1",

Proto generated using:

protoc \
        --proto_path=../../proto \
        --js_out=import_style=commonjs:${PROTO_DEST} \
        --grpc-web_out=import_style=commonjs+dts,mode=grpcwebtext:${PROTO_DEST} \
        -I ../../proto \
        $file

Client dependencies:

        "google-protobuf": "^3.14.0",
        "grpc-web": "^1.2.1",

@remvst
Copy link

remvst commented Dec 14, 2020

Okay seems like I was able to fix my issue. In case anyone is experiencing something similar, make sure you do specify the HTTP protocol in your client:

// Works
const client = new VideoClient('http://localhost:9000');

// Doesn't work
const client = new VideoClient('localhost:9000');

Would it make sense to add some assertion to the constructor to avoid these kinds of issues?

@maxekman
Copy link

maxekman commented Dec 15, 2020

I'm facing the same issue. Running grpc-web 1.2.1 against an Istio 1.7.6 ingress gateway. In my case I create the clients without hostname:

const client = new BookingServicePromiseClient('')

The problem occurs in Safari 14.0.1.

@chriss2401
Copy link

+1 here, facing the same issues with localhost:${port} or http://localost:${port}

@bram-abe
Copy link

bram-abe commented Jun 1, 2021

I can solve the issue Object { code: 2, message: "Incomplete response" } same as @remvst stated above.
Specify http:// in your client target URL.

@LukeLaScala
Copy link

Specifying http:// didn't resolve the issue for me. Are there any other workarounds at the moment? This is a super frustrating bug.

@mateodelnorte
Copy link

I'm in the same boat, @LukeLaScala. Did you find any resolution?

@mateodelnorte
Copy link

mateodelnorte commented Nov 4, 2021

I'm in the same boat, @LukeLaScala. Did you find any resolution?

I was able to solve this with mode=grpcweb instead of mode=grpcwebtext. However, I'm concerned I'm limiting capabilities in doing so:

image

image
It appears long-running server side streaming is only available when using grpcwebtext to generate client code.

Note, the exact behavior is that any generated client calls to .unaryCall return a promise, but that promise never resolves. A 200 response is received with what looks like correct data, but it may be incomplete.

proto.company.proto.energy.thing.otherthing.v1.TheDomainThingAPIPromiseClient.prototype.getSomething =
    function(request, metadata) {
  return this.client_.unaryCall(this.hostname_ +
      '/company.proto.energy.thing.otherthing.v1.TheDomainThingAPI/GetSomething',
      request,
      metadata || {},
      methodDescriptor_TheDomainThingAPI_GetSomething);
};

The call, above, for instance, would return a Promise but it would never resolve or eventually resolve an error that looks like:
image

We are generating client code with protoc-gen-grpc-web-1.3.0-linux-x86_64 and leveraging latest grpc-web and google-protobufs versions in all code:

    "google-protobuf": "~3.14.0",
    "grpc-web": "^1.3.0"

@GoncaloHit
Copy link

Im having the same issue as above, i have tried, both grpcweb and grpcwebtext on the client url i use https
everything that is successfully works, but on error it doesnt, the gateway is Kong with grpcweb plugin

@patryksliwinski
Copy link

I faced the same issue, grpcweb resolved this for me but I cannot use streams now

@dalazx
Copy link

dalazx commented Oct 31, 2022

In case you see { code: 2, message: "Incomplete response" } while you expect a properly set error, it might mean that the CORS configuration of your gateway/proxy does not allow browsers read grpc-status and grpc-message response header values.
These two headers should be explicitly exposed as follows (along with any other headers that you might use):

access-control-expose-headers: grpc-status,grpc-message

This worked well for me using grpcwebtext.

See the example configuration for Envoy:

cors:
allow_origin_string_match:
- prefix: "*"
allow_methods: GET, PUT, DELETE, POST, OPTIONS
allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
max_age: "1728000"
expose_headers: custom-header-1,grpc-status,grpc-message

cc @GoncaloHit

@EByrdS
Copy link

EByrdS commented Oct 19, 2023

Adding to my Envoy proxy

expose_headers: grpc-status,grpc-message

actually solved the issue for me, thanks a lot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests