http server

[AdrianRossouw]

this is pretty much just a wrapper around http.createServer, that has a graft instance inside of it that it uses to resolve the request.

it will map the response into a returnChannel, and pass the message along.

It should allow you to use all the other stream-oriented http tools that exist, and stay the fsck out of everyone's way as much as possible.

bonus points: be able to pipe messages through a graft http request service, that has been piped into a locally running graft http server =P you know. for tests...

[mcollina]

Yeah. Should it be inside the main graft package, just because we have all other transports over there? I'm actually inclined to do that, because of completeness. It can also be a module used there though.
We might also move the main graft thing to 'graft-core', and leave there only the multi-transport bit.

See GraftJS/graft#8

[AdrianRossouw]

to me this feels more like way to map against other services, not a way for graft <-> graft communication.

i also think a case could be made for moving things required like 'graft/spdy' to 'graft-spdy' instead.
but it's not a priority.

[mcollina]

to me this feels more like way to map against other services, not a way
for graft <-> graft communication.

Right. The graft module is for graft-graft communication. This is for
interop.

i also think a case could be made for moving things required like
'graft/spdy' to 'graft-spdy' instead.
but it's not a priority.

I'm currently against that, mainly for testing purposes. Once the graft
interface is stable we can move those (and also split jschan).

[tamagokun]

Trying to whip up something quick to understand how this will work. Let me know if I am on the right track starting this example off:

'use strict';

var graft = require('graft')();
var http  = require('http');

var server = http.createServer(function(req, res) {
  graft.write({
    req: req,
    res: res
  });

  return graft
    .pipe(through.obj(function(msg, enc, cb) {
      msg.res.end("Hello World");
      cb();
    }));
});

server.listen(3000);

In terms of an api for the server, any thoughts on what would be better?

require('graft-http').server(3000) // <-- returns graft instance for piping, etc.

// this way is more inline with how the spdy/ws servers function

or

var server = http.createServer(require('graft-http'));
server.listen(3000);

// how do we map micro services to the server at this point. are we assigning services to url endpoints?

[AdrianRossouw]

We could do unwrap the req a little bit of before just putting it into the graft channel. Something like having the .path, and .query and .body available on the message itself.

As for mapping things... I'm just riffing, but I could imagine something similar to :

graft
  .pipe(route('/path/:arg', argHandler))
  .pipe(route('/path', pathHandler));

var pathHandler = through.obj(function(msg, enc, done) {
   graft.write({
     topic: 'something', // this message writes to the response
     response: msg.res
   });
});

[AdrianRossouw]

maybe we should start by getting some more concrete test cases out ?

I might have one in aetherboard.
In that app, the client has a subscribe message that they give the server a stream to write the initial png.
We should have a GET endpoint to retrieve that same initial image.

[tamagokun]

Yes, some things should definitely get pulled into the message. This way you could do some things like

graft
  .where({path: '/foo'}, handler);

[tamagokun]

@AdrianRossouw check out the draft branch.

Currently I am mixing the properties of a graft instance in with an object. Not sure that this is very clean. It is probably better to have graft as a property of the server, and then proxy graft's methods to it (pipe where etc)

[tamagokun]

Look at example.js to see how this would look when using.

You can do something like curl http://localhost:3000 to see the "Hello World" message.

[tamagokun]

Setting it up like this, we should be able to do things like:

var app = connect();

var graft = graftHttp();

app.use('/path', graft.pipe(through.obj(function(msg, enc, done) {

})));

Nevermind.. not working. I think by using the proxy method rather than merge object properties this will work.

[tamagokun]

Set up proxy methods for write and pipe, and got it working.

Now the above example would actually work.

[mcollina]

There is a problem with the response, you cannot set the headers across machines, as on the other end it will just appear as a standard stream.

I think the right message to be sent is something in the line:

{
  url: ..
  path: ...
  host: ...  // plus all the components
  headers: ...
  body: ...
  handler: handler
}

// on the other side

req.handler.write({
  headers: // my headers
  statusCode: 422
  body: // ...
  // other stuff...
})

I'm against hijacking write and pipe, as they have a very specific meaning in the node world, i.e. streams. We can definitely use a different vocabulary, as this component is a bridge.

We should consider the use case of /users/:id, which is captured by express. There are tons of http routers we can embed and reuse for this.

[tamagokun]

Ooooo, I like it. Reminds me of rack.

[tamagokun]

Was wondering about routing. Do we want to embed something like https://github.com/pillarjs/path-match ? Or leave it up to the user to pick their router of choice.

What if we set up methods for standard http methods?

graftHttp.get('/post/:id', through.obj(function(msg, enc, done) {})));

[mcollina]

I think we should dogfood this, and use microservices ;), so we might base it around three steps:

1. we should agree on the http-request-to-message serialization that can work, and iterate on it as soon as we have examples
2. provide a simple module that converts (req, res) requests to Graft sessions and channels
3. provide a simple router (based on path-match), as a Graft microservice.

[tamagokun]

Making some progress here. I think I am on the right track. Next up are writing tests.

More detail about my progress:

• I've created a preliminary micro service for handling connect-compatible middleware. https://github.com/GraftJS/graft-http/blob/draft/lib/middleware.js
• I've updated the example to demonstrate this: https://github.com/GraftJS/graft-http/blob/draft/example.js
• Converting requests to/from messages: https://github.com/GraftJS/graft-http/blob/draft/lib/request.js

The example will pipe graft to bodyParser.json() and then to a simple echo-er that just spits out whatever was in the body of the request. http client sends a request with {foo:bar} and you see that in the console (first time as an object, 2nd as a string of json)

I've noticed that middleware love attaching things directly to the request object, so I need a better way to go from http.IncomingMessage -> graft message. My initial thought was to black list properties we don't want, and filter that on the request.

---

More thinking about a router: would you prefer using something similar graft.branch where they would just pass in the route details rather then graft would only use those streams if the route matched? Or just stick to creating a micro service that will just call done() if the route doesn't match? I'm wondering if there would be any performance implications here.

[tamagokun]

Nevermind that last bit about the router. It will definitely need to be a micro service because when the router parses the route parameters, those will need to be pushed into the stream.

[tamagokun]

Router micro service: https://github.com/GraftJS/graft-http/blob/draft/lib/router.js

Example has been updated to run when path matches /post/:id

[tamagokun]

Last thing I need some opinions on for this server component:

When stacking micro services, the return is normally a stream so that you can pipe it through to other micro services. This looks weird in the example, because you see graftHttp.use().pipe().pipe()

Should we have it be the same as a stream and use pipe? graftHttp.pipe().pipe() @mcollina I agree with what you said about not using stream-oriented method names. Or perhaps stack/pipe the microservices on the fly when the http request comes in (similar to how connect processes middleware)? This would allow you to do things like:

var testService = graftHttp();
testService.use(...);
testService.use(...);
// ^ these get piped together later

testService.listen();

[AdrianRossouw]

I think we should stick to pipe. like :

var server = graftHttp();
var testService = server
  .pipe(middleware(bodyParser()))
  .pipe(middleware(static('/some/dir'));

testService.pipe(route('/post:id')).pipe(handler);

[mcollina]

@AdrianRossouw I think that is not the best approach, piping will slow things down if we pipe too much (let's say we have 20 uservices a message has to pass through at each request, that will be slow).

Here is another proposal along the line.

var server = graftHttp();
var router = graftRouter();
var testService = server
  .pipe(middleware(bodyParser()))
  .pipe(router)
  .pipe(middleware(static('/some/dir'));

router.route('/post:id', handler);

Actually, having a separate graft-router is kind of better than the integrated where approach.

[tamagokun]

Ahhhh, of course. handler is just another micro service, right?

[mcollina]

yes!
Il gio 20 nov 2014 alle 12:20 Mike Kruk [email protected] ha
scritto:

Ahhhh, of course. handler is just another micro service, right?

—
Reply to this email directly or view it on GitHub
#1 (comment).

[tamagokun]

done 👍 https://github.com/GraftJS/graft-http/blob/draft/example.js

ok, time for writing tests!

[mcollina]

Gooood! :)