Cafe Request Handler

[sanderpick]

Fixes #591
Fixes #661

[sanderpick]

Moving this original info to a comment:

The general idea here is to expose the pb.CafeRequests to a mobile API so that the native OS can process them accordingly.

Major ideas:

• The mobile OS should implement the cafe.RequestHandler interface, which requires methods for processing different types of requests
• The internal CafeService should also implement this interface, and it should do so w/ libp2p. This means Switch cafe service back to direct p2p #568.
• In order for others to handle cafe requests over HTTP, the Cafe HTTP API should be formalized / expanded beyond the temporary /service handler
• Because some methods are handled very differently over libp2p, the interface has to be pretty "light", e.g., STORE requests over p2p first kick off a request to see if the data is even needed (it might be already stored), and then each IPFS block pushed over the connection. Over HTTP, we just PUT the raw data behind the block and forget it
• The node should expose a couple new methods for the handler to grab requests
  • list
  • delete
  • statgroup (see below or code)

Some ongoing notes here, just c/p'd from cafe/main.go:

list: return a batch of requests by date until "next" is empty
complete: mark a single request as complete (does not delete until all in the group are deleted
so that we can know the total data size being handled)
stat a group: returns some info:
  - num total reqs
  - num complete
  - percent complete
  - will return nil if all done since they will have been deleted

Payloads for different requests (expose something like get payload?):
- store: PUT /store/:cid, body => raw object data
- unstore: DELETE /store/:cid, body => none
- store thread: PUT /thread/:id, body => encrypted thread object (snapshot)
- unstore thread: DELETE /thread/:id, body => none
- deliver message: POST /inbox/:pid, body => encrypted message

Other, related methods...
- stat a thread: are there any pending cafe requests (per cafe) / block messages for this thread?
- stat a files block: what percentage of total bytes for the group are complete, per cafe?
- when you add a new cafe, a bunch of new request should be created, which results in a not-as-synced state

Thoughts...

There are two types of things that need to be "stored" and count to users' data usage:
1. thread blocks (requests grouped by thread id)
2. files dags (requests grouped by target id)
(thread snapshots do not count toward data usage, nor do inboxed messages, ?)

[sanderpick]

This is ready for a look.

[carsonfarmer]

Love all the tests! While obviously none of the new endpoints are helpful to a local node (say via cli client), what do you think about exposing some of these new API endpoints in js-http-client? Do you think some of these could be precursors to a 'lite' browser client?

[carsonfarmer]

Cool!

[carsonfarmer]

Oh nice, that's slick

[sanderpick]

Thanks @carsonfarmer! Yeah, seems like having a JS client for the cafe would be slick. If you could somehow obtain a cafe session in the browser, you'd be off to the races.

[asutula]

Ok so CafeOutboxHandler just specifies that the object needs to implement Flush()... Is the idea that Flush will be called whenever new CafeRequests are added to the queue?

[sanderpick]

Yep, exactly.

[sanderpick]

Just pushed this guy: 6fb1e29

Need to include the core package with gomobile bind in order to access the CafeOutboxHandler interface.