Apache Thrift <> fbthrift compatibility

[bigs]

Hello,

I'm working on a framework that uses fbthrift extensively. All of our internal services are built in C++ with fbthrift and are working wonderfully (truly!)

In order to be as flexible as is possible to clients using our framework, we wish to have our client libraries communicate to the framework via Apache Thrift. This is convenient for a variety of reasons, among them the easily available, well packaged client libraries (the updated python client module for fbthrift doesn't install out of the box, etc.)

So far, I've had no trouble connecting to a fbthrift service (built with the new python generator) from an Apache Thrift client (in this case, Ruby with the 0.9.2 gem). Unfortunately, I've not been able to connect to an Apache Thrift service (again, Ruby w/ 0.9.2 gem) from our C++ codebase.

The ruby service is using the Apache Thrift generated code & library to start a service (using the FramedTransport and BinaryProtocol, ThreadedServer). The C++ client is connecting via TAsyncSocket, and an async client to the server constructed with a HeaderClientChannel constructed with the socket.

The ruby server is able to receive the request (determined with simple logs that are, in fact, called), but when returning its response, I get a TransportException: Channel got EOF.

Does anyone have any insight as to what is required to get the C++(2) client communicating successfully with an Apache Thrift (ruby) service? Happy to provide details as requested.

[bigs]

Ah -- an update:

I'm actually not able to make a successful connection to the Apache Thrift server at all. I was simply misreading my logs.

bigs

[bigs]

Update: with some debugging, I was able to locate a (silent) failure on the client side. I'm able to successfully connect to my Apache Thrift service using the synchronous client, which is good enough for me!

AsyncClient doesn't work w/ Apache Thrift services, as expected.

[andrewcox]

You actually should be able to connect to Apache Thrift services using the async client, if you are making a client that uses a compatible protocol/transport. For example, if the server doesn't expect framed transport messages, and the async client is sending framed transport messages (or vice versa), that will be a problem.

If you paste the code used to create async & sync clients here, one of us can probably tell you why one worked and the other did not.

[bigs]

@andrewcox: thanks! the relevant code is here

https://gist.github.com/bigs/21dd95bd37b9e2b75f7d

[andrewcox]

Yeah, I suspect the issue is that Apache Thrift servers aren't using header transport.

For cpp2 async client, there is only HeaderClientChannel, but it can be manipulated to send non-header messages. Probably something like this will work (if your server uses framed transport):

channel->getHeader()->setClientType(THRIFT_FRAMED_DEPRECATED)

As for the "_DEPRECATED" part, sending non-header messages is deprecated in fbthrift, but will likely still be supported for a long time...

[bigs]

@andrewcox: could you elaborate on that a bit? afaik there is no "header transport" available in Apache Thrift.

Barring some direct compatibility with the new AsyncClient, I feel more comfortable using the synchronous client & TFramedTransport going forward.

Btw: Thanks so much for the help!

[emaxerrno]

@andrewcox doing a simple

      socket_ = TAsyncSocket::newSocket(ev_, addr_);
      clientTypes_[THRIFT_FRAMED_DEPRECATED] = 1;
      clientTypes_[THRIFT_HEADER_CLIENT_TYPE] = 1;
      channel_ = HeaderClientChannel::newChannel(socket_);
      channel_->getHeader()->setSupportedClients(&clientTypes_);
      channel_->getHeader()->setClientType(clientType_);
      // CHECK(channel_->getHeader()->isSupportedClient())
      client_ = std::make_shared<ClientType>(channel_);

Throws this excpt:

E0325 14:51:27.617950  7766 EventBase.cpp:733] runInEventBaseThread() std::function threw a N6apache6thrift21TApplicationExceptionE exception: Transport does not support this client type

[andrewcox]

Don’t know which client type you’re using, as I don’t see an assignment to clientType_.

Also, setSupportedClientTypes is for incoming requests. You don’t need to set this on the client.

[andrewcox]

As to your question about whether there is header transport in apache thrift, there isn't. But, as a client, header channel knows how to send non-header messages (to be compatible with non-header servers like apache).

But yeah, if you are comfortable sticking with sync, there's nothing wrong with that!