gateway & p2p improvements

[Arqu]

Currently many requests go and invoke fetch_bitswap with an empty provider list (ref trace ID 2b90b66cb34b936f005cde120d35de0e). We should fail fast here and return 404 or something similar.

The response from the resolver on gateway should be a stream, both to avoid loading everything into memory and significantly reduce the time to first byte which is currently t(fetch_bitswap) + some_overhead while it could be t(fetch_bitswap_first_byte) + some_overhead. This also allows us to return metrics like time to first byte/block, latency vs time etc.

[dignifiedquire]

The response from the resolver on gateway should be a stream

Given that our blocks are limited at 2MiB, and that neither our DB nor the rpc layer are currently streaming them, I am not sure this is actually worth the work currently.

[Arqu]

The response from the resolver on gateway should be a stream

Given that our blocks are limited at 2MiB, and that neither our DB nor the rpc layer are currently streaming them, I am not sure this is actually worth the work currently.

Agree, this was more a "we should do it eventually thing"