Filter out un-actionable provider records #28

2color · 2024-01-12T12:55:13Z

Current situation

There are two scenarios in which someguy could filter out responses to reduce traffic and ensure higher success rates of clients relying on this:

Scenario one

The delegated providers endpoint contains a provider record with only private IPs or no addresses at all.
For example source had:

    {
      "Addrs": [
        "/ip6/::1/udp/4001/quic-v1/webtransport/certhash/uEiBRy1Y-qOilERGytvzaLVdqdy5DslwUypUTPIwMZpR4kw/certhash/uEiA4LFSdjmE2fej6x3PvvL3lPNI3UAJH1_UaxKaNi9mctg",
        "/ip6/::1/tcp/4001",
        "/ip6/::1/udp/4001/quic-v1",
        "/ip4/127.0.0.1/tcp/4001",
        "/ip4/192.168.0.110/tcp/4001",
        "/ip4/192.168.0.110/udp/4001/quic-v1",
        "/ip4/127.0.0.1/udp/4001/quic-v1/webtransport/certhash/uEiBRy1Y-qOilERGytvzaLVdqdy5DslwUypUTPIwMZpR4kw/certhash/uEiA4LFSdjmE2fej6x3PvvL3lPNI3UAJH1_UaxKaNi9mctg",
        "/ip4/192.168.1.76/udp/51508/quic-v1",
        "/ip4/192.168.1.76/tcp/51508",
        "/ip4/192.168.1.76/udp/51508/quic-v1/webtransport/certhash/uEiBRy1Y-qOilERGytvzaLVdqdy5DslwUypUTPIwMZpR4kw/certhash/uEiA4LFSdjmE2fej6x3PvvL3lPNI3UAJH1_UaxKaNi9mctg",
        "/ip4/127.0.0.1/udp/4001/quic-v1",
        "/ip4/192.168.0.110/udp/4001/quic-v1/webtransport/certhash/uEiBRy1Y-qOilERGytvzaLVdqdy5DslwUypUTPIwMZpR4kw/certhash/uEiA4LFSdjmE2fej6x3PvvL3lPNI3UAJH1_UaxKaNi9mctg"
      ],
      "ID": "12D3KooWGDMwwqrpcYKpKCgxuKT2NfqPqa94QnkoBBpqvCaiCzWd",
      "Schema": "peer"
    },

Or

    {
      "ID": "12D3KooWGDMwwqrpcYKpKCgxuKT2NfqPqa94QnkoBBpqvCaiCzWd",
      "Schema": "peer"
    },
    {
      "ID": "12D3KooWRBy97UB99e3J6hiPesre1MZeuNQvfan4gBziswrRJsNK",
      "Schema": "peer"
    },
    {
      "ID": "12D3KooWFtNWJWT3UgqeGDiyz41zCuoWQ3BUfGRr32i9hNYFR9dr",
      "Schema": "peer"
    },

Suggestion:

These provider record should not be returned, i.e. filtered out

scenario 2

The delegated providers endpoint contains a provider record with some private/loopback addrs and some public ones, e.g.

    {
      "Addrs": [
        "/ip4/192.168.1.76/tcp/51508",
        "/ip4/127.0.0.1/udp/4001/quic-v1/webtransport/certhash/uEiBRy1Y-qOilERGytvzaLVdqdy5DslwUypUTPIwMZpR4kw/certhash/uEiA4LFSdjmE2fej6x3PvvL3lPNI3UAJH1_UaxKaNi9mctg",
        "/ip6/::1/tcp/4001",
        "/ip4/192.168.1.76/udp/51508/quic-v1/webtransport/certhash/uEiBRy1Y-qOilERGytvzaLVdqdy5DslwUypUTPIwMZpR4kw/certhash/uEiA4LFSdjmE2fej6x3PvvL3lPNI3UAJH1_UaxKaNi9mctg",
        "/ip4/192.168.0.110/udp/4001/quic-v1",
        "/ip4/192.168.0.110/tcp/4001",
        "/ip4/127.0.0.1/tcp/4001",
        "/ip4/192.168.1.76/udp/51508/quic-v1",
        "/ip4/84.158.140.90/udp/4001/quic-v1",
        "/ip4/192.168.0.110/udp/4001/quic-v1/webtransport/certhash/uEiBRy1Y-qOilERGytvzaLVdqdy5DslwUypUTPIwMZpR4kw/certhash/uEiA4LFSdjmE2fej6x3PvvL3lPNI3UAJH1_UaxKaNi9mctg",
        "/ip6/::1/udp/4001/quic-v1",
        "/ip6/::1/udp/4001/quic-v1/webtransport/certhash/uEiBRy1Y-qOilERGytvzaLVdqdy5DslwUypUTPIwMZpR4kw/certhash/uEiA4LFSdjmE2fej6x3PvvL3lPNI3UAJH1_UaxKaNi9mctg",
        "/ip4/127.0.0.1/udp/4001/quic-v1"
      ],
      "ID": "12D3KooWGDMwwqrpcYKpKCgxuKT2NfqPqa94QnkoBBpqvCaiCzWd",
      "Schema": "peer"
    },

Note: this is a real example that I saw but was later unable to reproduce.

Suggestion

Filter out the private IPs addresses in the Addrs array, i.e. all addresses except for /ip4/84.158.140.90/udp/4001/quic-v1 should be filtered out.

The text was updated successfully, but these errors were encountered:

lidel · 2024-02-13T16:55:49Z

Agree, it sounds like a sensible default to reduce response size.

Only ask is that there should be a flag+env for controlling this behavior, as people may want to use someguy in private swarms, where they DO want private provider addrs. Maybe someguy start --include-private-addrs to bring them back?

aschmahmann · 2024-02-13T17:44:49Z

I don't think we can filter out returning just peerIDs unless we also attempt to resolve those peerIDs within someguy. DHT FindProviders queries may sometimes return just peerIDs and require a FindPeer query to follow up.

Note: While this is a bunch of extra work, it's not necessarily a bad idea since if someguy is tracking peerIDs it can also do things like check on if the peers are alive and sort response records accordingly (which will help clients be more efficient, and potentially workaround webtransport issues in chromium).

lidel · 2024-03-26T14:40:11Z

Triage notes:

filtering private addresses is 👍 and can be implemented
we don't want filtering out PeerIDs without multiaddrs, because even withotu active probing, client can always do FindPeerequivalent of /routing/v1/peers/{peerid}
- active probing will remove the need for this additional lookup, but that is out for scope here, see Add active peer probing and reputation caching #53

2color · 2024-04-15T10:23:06Z

we don't want filtering out PeerIDs without multiaddrs, because even withotu active probing, client can always do FindPeerequivalent of /routing/v1/peers/{peerid}

One thing I'd like to point out is that based on experience, in most cases the the peers returned without any addresses from the providers endpoint are also not resolvable through the peers endpoint

2color · 2024-04-26T14:02:31Z

I just made a call to the delegated routing endpoint that returned me a loopback address:

➜  ~ curl 'https://delegated-ipfs.dev/routing/v1/peers/bafzaajaiaejcav3fwj35j27gor72ap5aqhiz44qmje4gcxvo5wogjmczwhk4xp7p' \
           -H 'accept: application/x-ndjson' \
           -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
           -H 'cache-control: no-cache' \
           -H 'origin: http://localhost:3000' \
           -H 'pragma: no-cache' \
           -H 'priority: u=1, i' \
           -H 'referer: http://localhost:3000/' \
           -H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
           -H 'sec-ch-ua-mobile: ?0' \
           -H 'sec-ch-ua-platform: "macOS"' \
           -H 'sec-fetch-dest: empty' \
           -H 'sec-fetch-mode: cors' \
           -H 'sec-fetch-site: cross-site' \
           -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' -v | Jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 104.18.29.158:443...
* Connected to delegated-ipfs.dev (104.18.29.158) port 443 (#0)
* ALPN: offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [323 bytes data]
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [19 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [4174 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [79 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=delegated-ipfs.dev
*  start date: Apr 15 08:37:00 2024 GMT
*  expire date: Jul 14 08:36:59 2024 GMT
*  subjectAltName: host "delegated-ipfs.dev" matched cert's "delegated-ipfs.dev"
*  issuer: C=US; O=Let's Encrypt; CN=E1
*  SSL certificate verify ok.
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: delegated-ipfs.dev]
* h2 [:path: /routing/v1/peers/bafzaajaiaejcav3fwj35j27gor72ap5aqhiz44qmje4gcxvo5wogjmczwhk4xp7p]
* h2 [accept: application/x-ndjson]
* h2 [accept-language: en-GB,en-US;q=0.9,en;q=0.8]
* h2 [cache-control: no-cache]
* h2 [origin: http://localhost:3000]
* h2 [pragma: no-cache]
* h2 [priority: u=1, i]
* h2 [referer: http://localhost:3000/]
* h2 [sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"]
* h2 [sec-ch-ua-mobile: ?0]
* h2 [sec-ch-ua-platform: "macOS"]
* h2 [sec-fetch-dest: empty]
* h2 [sec-fetch-mode: cors]
* h2 [sec-fetch-site: cross-site]
* h2 [user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36]
* Using Stream ID: 1 (easy handle 0x12100f000)
> GET /routing/v1/peers/bafzaajaiaejcav3fwj35j27gor72ap5aqhiz44qmje4gcxvo5wogjmczwhk4xp7p HTTP/2
> Host: delegated-ipfs.dev
> accept: application/x-ndjson
> accept-language: en-GB,en-US;q=0.9,en;q=0.8
> cache-control: no-cache
> origin: http://localhost:3000
> pragma: no-cache
> priority: u=1, i
> referer: http://localhost:3000/
> sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"
> sec-ch-ua-mobile: ?0
> sec-ch-ua-platform: "macOS"
> sec-fetch-dest: empty
> sec-fetch-mode: cors
> sec-fetch-site: cross-site
> user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36
>
< HTTP/2 200
< date: Fri, 26 Apr 2024 14:02:13 GMT
< content-type: application/x-ndjson
< access-control-allow-origin: *
< cache-control: public, max-age=14400
< last-modified: Fri, 26 Apr 2024 13:58:19 GMT
< vary: Accept-Encoding
< vary: Origin
< vary: Accept
< x-ipfs-pop: someguy-am6
< cf-cache-status: HIT
< age: 234
< expires: Fri, 26 Apr 2024 18:02:13 GMT
< server: cloudflare
< cf-ray: 87a71bf8db418635-WAW
< alt-svc: h3=":443"; ma=86400
<
{ [488 bytes data]
100   488    0   488    0     0   3645      0 --:--:-- --:--:-- --:--:--  3782
* Connection #0 to host delegated-ipfs.dev left intact
{
  "Addrs": [
    "/ip4/147.28.186.157/udp/9095/quic-v1",
    "/ip4/147.28.186.157/udp/9095/quic-v1/webtransport/certhash/uEiCmLPMgXJ1F1wQ-OgOWJEVa_SYB_jLSf5IkQ_d3V98GBQ/certhash/uEiB-ti6URtr64LV8HYDMvZzzzrb1iNEIT-vGY0yd6UYk2g",
    "/ip4/127.0.0.1/udp/9095/quic-v1",
    "/ip4/127.0.0.1/udp/9095/quic-v1/webtransport/certhash/uEiCmLPMgXJ1F1wQ-OgOWJEVa_SYB_jLSf5IkQ_d3V98GBQ/certhash/uEiB-ti6URtr64LV8HYDMvZzzzrb1iNEIT-vGY0yd6UYk2g"
  ],
  "ID": "12D3KooWFhXabKDwALpzqMbto94sB7rvmZ6M28hs9Y9xSopDKwQr",
  "Schema": "peer"
}

I suspect that it may be related to Cloudflare caching because it's not easy to reproduce

2color · 2024-05-16T11:16:24Z

Re-opening this, as I keep on seeing some results with private IPs. I still haven't figured out why exactly, so I'll just share my findings:

For example, taking the following Curl command from the browser returns private IPs:

curl 'https://delegated-ipfs.dev/routing/v1/peers/bafzaajaiaejcbpw7xni2z7srzjy5tjsqktdf4o5c2d6wkxhvh7pniiegnowoheaf' \
  -H 'accept: application/x-ndjson' \
  -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'cache-control: no-cache' \
  -H 'origin: http://localhost:5173' \
  -H 'pragma: no-cache' \
  -H 'priority: u=1, i' \
  -H 'referer: http://localhost:5173/' \
  -H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  -H 'sec-fetch-dest: empty' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-site: cross-site' \
  -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36'

The response is a cache hit:

HTTP/2 200
date: Thu, 16 May 2024 11:01:26 GMT
content-type: application/x-ndjson
access-control-allow-origin: *
cache-control: public, max-age=300, stale-while-revalidate=172800, stale-if-error=172800
last-modified: Thu, 16 May 2024 11:00:07 GMT
vary: Accept-Encoding
vary: Origin
vary: Accept
x-ipfs-pop: someguy-am6
cf-cache-status: HIT
age: 78
server: cloudflare
cf-ray: 884adeab8e046a75-TXL
alt-svc: h3=":443"; ma=86400

and contains the following payload

{"Addrs":["/ip6/::1/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip4/192.168.1.140/udp/4001/quic-v1","/ip6/fd9f:3dd0:6bc2:0:bf9d:49af:51d9:2d17/tcp/4001","/ip4/213.171.213.29/udp/4001/quic-v1/webtransport/certhash/uEiB5T6G9Nf_21RTajh4LhmOrqh4Fh7nPUQiqe8-yjoNogg/certhash/uEiDVzZmI2sOeARG-6o8wto-cItetAENvWOQlKrck1jNDkw/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip6/2a00:da00:1800:103::2/udp/4001/quic-v1/webtransport/certhash/uEiB5T6G9Nf_21RTajh4LhmOrqh4Fh7nPUQiqe8-yjoNogg/certhash/uEiDVzZmI2sOeARG-6o8wto-cItetAENvWOQlKrck1jNDkw/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip6/::1/udp/4001/quic-v1","/ip4/127.0.0.1/udp/4001/quic","/ip6/2a00:da00:1800:103::2/tcp/4001/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip4/213.171.213.29/tcp/4001/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip6/::1/udp/4001/quic","/ip4/127.0.0.1/udp/4001/quic-v1","/ip6/fd9f:3dd0:6bc2:0:1e3e:6614:da82:dc6b/udp/4001/quic-v1","/ip6/fd9f:3dd0:6bc2:0:1e3e:6614:da82:dc6b/udp/4001/quic","/ip4/213.171.213.29/udp/4001/quic-v1/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip4/127.0.0.1/tcp/4001","/ip6/fd9f:3dd0:6bc2:0:bf9d:49af:51d9:2d17/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip6/::1/tcp/4001","/ip6/fd9f:3dd0:6bc2:0:bf9d:49af:51d9:2d17/udp/4001/quic","/ip4/192.168.1.140/tcp/4001","/ip4/23.94.202.252/udp/4001/quic/p2p/12D3KooWHwwULwN7WjMCy85ZQZgACiRv6o42Yp3WSiGFTaTgGr1o/p2p-circuit","/ip4/127.0.0.1/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip6/fd9f:3dd0:6bc2:0:1e3e:6614:da82:dc6b/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip6/fd9f:3dd0:6bc2:0:1e3e:6614:da82:dc6b/tcp/4001","/ip6/2a00:da00:1800:103::2/udp/4001/quic-v1/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip4/192.168.1.140/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip4/192.168.1.140/udp/4001/quic","/ip6/fd9f:3dd0:6bc2:0:bf9d:49af:51d9:2d17/udp/4001/quic-v1","/ip4/23.94.202.252/udp/4001/quic-v1/webtransport/certhash/uEiAEKl96gHrjYZrePVozMNgmcVKWPmZRAsFcgTtvf6Ww0w/certhash/uEiDNw97_EHVQ_kGhivLgd1r4yXvdBcX0TORgz7we2xi5XQ/p2p/12D3KooWHwwULwN7WjMCy85ZQZgACiRv6o42Yp3WSiGFTaTgGr1o/p2p-circuit","/ip4/23.94.202.252/udp/4001/quic-v1/p2p/12D3KooWHwwULwN7WjMCy85ZQZgACiRv6o42Yp3WSiGFTaTgGr1o/p2p-circuit"],"ID":"12D3KooWNfTW176i7oAEyY9yQQfSnfvyjFH9ytjBAh1rQxpFmkon","Schema":"peer"}

If I add a query param to the url to bust the cache, I get the following:

➜  ~ curl 'https://delegated-ipfs.dev/routing/v1/peers/bafzaajaiaejcbpw7xni2z7srzjy5tjsqktdf4o5c2d6wkxhvh7pniiegnowoheaf?ab=124124' \
           -H 'accept: application/x-ndjson' \
           -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
           -H 'cache-control: no-cache' \
           -H 'origin: http://testing.com' \
           -H 'pragma: no-cache' \
           -H 'priority: u=1, i' \
           -H 'referer: http://localhost:5173/' \
           -H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
           -H 'sec-ch-ua-mobile: ?0' \
           -H 'sec-ch-ua-platform: "macOS"' \
           -H 'sec-fetch-dest: empty' \
           -H 'sec-fetch-mode: cors' \
           -H 'sec-fetch-site: cross-site' \
           -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' -i
HTTP/2 200
date: Thu, 16 May 2024 11:03:24 GMT
content-type: application/x-ndjson
access-control-allow-origin: *
cache-control: public, max-age=300, stale-while-revalidate=172800, stale-if-error=172800
last-modified: Thu, 16 May 2024 11:03:24 GMT
vary: Accept-Encoding
vary: Origin
vary: Accept
x-ipfs-pop: someguy-am6
cf-cache-status: MISS
server: cloudflare
cf-ray: 884ae1895f6d2685-TXL
alt-svc: h3=":443"; ma=86400

However, someguy still returns private IPs:

{"Addrs":["/ip6/::1/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip4/192.168.1.140/udp/4001/quic-v1","/ip6/fd9f:3dd0:6bc2:0:bf9d:49af:51d9:2d17/tcp/4001","/ip4/213.171.213.29/udp/4001/quic-v1/webtransport/certhash/uEiB5T6G9Nf_21RTajh4LhmOrqh4Fh7nPUQiqe8-yjoNogg/certhash/uEiDVzZmI2sOeARG-6o8wto-cItetAENvWOQlKrck1jNDkw/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip6/2a00:da00:1800:103::2/udp/4001/quic-v1/webtransport/certhash/uEiB5T6G9Nf_21RTajh4LhmOrqh4Fh7nPUQiqe8-yjoNogg/certhash/uEiDVzZmI2sOeARG-6o8wto-cItetAENvWOQlKrck1jNDkw/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip6/::1/udp/4001/quic-v1","/ip4/127.0.0.1/udp/4001/quic","/ip6/2a00:da00:1800:103::2/tcp/4001/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip4/213.171.213.29/tcp/4001/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip6/fd9f:3dd0:6bc2:0:1e3e:6614:da82:dc6b/udp/4001/quic","/ip4/127.0.0.1/udp/4001/quic-v1","/ip6/fd9f:3dd0:6bc2:0:1e3e:6614:da82:dc6b/udp/4001/quic-v1","/ip6/::1/udp/4001/quic","/ip4/213.171.213.29/udp/4001/quic-v1/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip6/::1/tcp/4001","/ip6/fd9f:3dd0:6bc2:0:bf9d:49af:51d9:2d17/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip4/127.0.0.1/tcp/4001","/ip6/fd9f:3dd0:6bc2:0:bf9d:49af:51d9:2d17/udp/4001/quic","/ip4/192.168.1.140/tcp/4001","/ip4/23.94.202.252/udp/4001/quic/p2p/12D3KooWHwwULwN7WjMCy85ZQZgACiRv6o42Yp3WSiGFTaTgGr1o/p2p-circuit","/ip6/fd9f:3dd0:6bc2:0:1e3e:6614:da82:dc6b/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip4/127.0.0.1/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip6/fd9f:3dd0:6bc2:0:1e3e:6614:da82:dc6b/tcp/4001","/ip6/2a00:da00:1800:103::2/udp/4001/quic-v1/p2p/12D3KooWRgWYDt7QHvcETkN6YR3BUzoNGuo4pZNKPpXDH5RsKUaM/p2p-circuit","/ip4/192.168.1.140/udp/4001/quic-v1/webtransport/certhash/uEiAyim-vGUCGwqt1zJhFMyiqK-AgfZ16jfepJYNDY7NEDw/certhash/uEiAJKSgEXLMI1RZwYZmpEcNgpZ_F2Kit3rV5peMxEz2ldw","/ip6/fd9f:3dd0:6bc2:0:bf9d:49af:51d9:2d17/udp/4001/quic-v1","/ip4/192.168.1.140/udp/4001/quic","/ip4/23.94.202.252/udp/4001/quic-v1/webtransport/certhash/uEiAEKl96gHrjYZrePVozMNgmcVKWPmZRAsFcgTtvf6Ww0w/certhash/uEiDNw97_EHVQ_kGhivLgd1r4yXvdBcX0TORgz7we2xi5XQ/p2p/12D3KooWHwwULwN7WjMCy85ZQZgACiRv6o42Yp3WSiGFTaTgGr1o/p2p-circuit","/ip4/23.94.202.252/udp/4001/quic-v1/p2p/12D3KooWHwwULwN7WjMCy85ZQZgACiRv6o42Yp3WSiGFTaTgGr1o/p2p-circuit"],"ID":"12D3KooWNfTW176i7oAEyY9yQQfSnfvyjFH9ytjBAh1rQxpFmkon","Schema":"peer"}

Removing the 'accept: application/x-ndjson' header seems to resolve the problem and the response contains no private IPs.

I suspect that this bug might be related to how ndjson responses are streamed.

2color added this to IPFS Shipyard Team Jan 16, 2024

2color changed the title ~~Filter out private IPs~~ Filter out un-actionable results Feb 12, 2024

2color changed the title ~~Filter out un-actionable results~~ Filter out un-actionable provider records Feb 12, 2024

2color mentioned this issue Feb 12, 2024

Epic: Rainbow / Someguy Productionisation ipshipyard/waterworks-community#12

Open

17 tasks

lidel moved this to 🥞 Todo in IPFS Shipyard Team Feb 13, 2024

lidel assigned hacdias Mar 26, 2024

hacdias mentioned this issue Mar 27, 2024

feat: filter out private and loopback addresses #57

Merged

hacdias closed this as completed in #57 Apr 11, 2024

github-project-automation bot moved this from 🥞 Todo to 🎉 Done in IPFS Shipyard Team Apr 11, 2024

2color reopened this Apr 26, 2024

2color closed this as completed Apr 26, 2024

2color reopened this May 16, 2024

hacdias mentioned this issue May 17, 2024

fix: filter out private addresses when only using dht #68

Merged

hacdias closed this as completed in #68 May 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Filter out un-actionable provider records #28

Filter out un-actionable provider records #28

2color commented Jan 12, 2024

lidel commented Feb 13, 2024 •

edited

Loading

aschmahmann commented Feb 13, 2024 •

edited

Loading

lidel commented Mar 26, 2024

2color commented Apr 15, 2024

2color commented Apr 26, 2024 •

edited

Loading

2color commented May 16, 2024

Filter out un-actionable provider records #28

Filter out un-actionable provider records #28

Comments

2color commented Jan 12, 2024

Current situation

Scenario one

Suggestion:

scenario 2

Suggestion

lidel commented Feb 13, 2024 • edited Loading

aschmahmann commented Feb 13, 2024 • edited Loading

lidel commented Mar 26, 2024

2color commented Apr 15, 2024

2color commented Apr 26, 2024 • edited Loading

2color commented May 16, 2024

lidel commented Feb 13, 2024 •

edited

Loading

aschmahmann commented Feb 13, 2024 •

edited

Loading

2color commented Apr 26, 2024 •

edited

Loading