ipfs shutdown orphans 5001 #7865

NorseGaud · 2021-01-15T18:05:51Z

Version information:

❯ ipfs version --all
go-ipfs version: 0.7.0-ea77213
Repo version: 10
System version: amd64/darwin
Golang version: go1.14.12

❯ sw_vers
ProductName:    macOS
ProductVersion: 11.1
BuildVersion:   20C69

Description:

I ran ipfs daemon & which seems to have started:

❯ ipfs daemon &
[1] 2779

Initializing daemon...
go-ipfs version: 0.7.0-ea77213
Repo version: 10
System version: amd64/darwin
Golang version: go1.14.12
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.0.107/tcp/4001
Swarm listening on /ip4/192.168.0.107/udp/4001/quic
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip4/24.62.29.132/tcp/4001
Swarm announcing /ip4/24.62.29.132/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic
API server listening on /ip4/127.0.0.1/tcp/5001
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready

I then ran ipfs shutdown and it worked:

❯ ipfs shutdown

Received interrupt signal, shutting down...
(Hit ctrl-c again to force-shutdown the daemon.)

[1]  + 2779 done       ipfs daemon

However, I then try to start it again and see:

. . .
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip4/24.62.29.132/tcp/4001
Swarm announcing /ip4/24.62.29.132/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic

Error: serveHTTPApi: manet.Listen(/ip4/127.0.0.1/tcp/5001) failed: listen tcp4 127.0.0.1:5001: bind: address already in use

Why would this orphan the socket? How do I shut down ipfs daemon safely?

The text was updated successfully, but these errors were encountered:

lidel · 2021-01-15T19:19:39Z

@NorseGaud I don't think shutdown command should be even there – opened #7866 to track that.

I believe you should be able to stop daemon by interrupting ipfs daemon command or sending one of signals to the process without a problem:

graceful shutdown: SIGHUP, SIGINT or SIGTERM
forced shutdown: SIGKILL

Some questions:

Do you experience the same issue when you run ipfs daemon in foreground (without &) and interrupt it with ^C or killall ipfs ?
Are there any zombie processes (ps aux | grep ipfs)?
Is the port still taken by ipfs? (you can check with lsof -i :5001)

If the issue is occurring only with the shutdown command, we should close this and move to #7866

NorseGaud · 2021-01-15T20:09:26Z

Do you experience the same issue when you run ipfs daemon in foreground (without &) and interrupt it with ^C or killall ipfs ?

Can't restart my computer just yet to try. I will in a bit

Are there any zombie processes (ps aux | grep ipfs)?
Nope!

❯ ps aux | grep ipfs           
nathanpierce     77008   0.0  0.0  4416888    820 s003  S+    3:08PM   0:00.00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox ipfs

Is the port still taken by ipfs? (you can check with lsof -i :5001)
It's weird, the lsof was there right after the first daemon SIGTERM I sent. However, it's gone now. Here is what I see:

❯ ipfs daemon                             
Initializing daemon...
go-ipfs version: 0.7.0-ea77213
Repo version: 10
System version: amd64/darwin
Golang version: go1.14.12
Swarm listening on /ip4/10.8.1.9/tcp/4001
Swarm listening on /ip4/10.8.1.9/udp/4001/quic
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.0.107/tcp/4001
Swarm listening on /ip4/192.168.0.107/udp/4001/quic
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip4/24.62.29.132/tcp/4001
Swarm announcing /ip4/24.62.29.132/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic

Error: serveHTTPApi: manet.Listen(/ip4/127.0.0.1/tcp/5001) failed: listen tcp4 127.0.0.1:5001: bind: address already in use

❯ lsof -i tcp:5001

❯ netstat -vanp tcp | grep 5001
tcp6       0      0  ::1.5001               *.*                    LISTEN      131072 131072   2374      0 0x0080 0x00000006
tcp4       0      0  127.0.0.1.5001         *.*                    LISTEN      131072 131072   2374      0 0x0080 0x00000006

aschmahmann · 2021-01-15T20:14:27Z

@NorseGaud heads up that you accidentally did lsof on 5000 not 5001

NorseGaud · 2021-01-15T20:19:12Z

Thanks, @aschmahmann. Ran again with the right port and still an empty response.

NorseGaud · 2021-01-15T20:25:51Z

Ok, rebooted. Socket is gone. Here is the code story confirming that it's only on shutdown from what I can tell (maybe it's forced and not graceful?):

❯ ipfs daemon
Initializing daemon...
go-ipfs version: 0.7.0-ea77213
Repo version: 10
System version: amd64/darwin
Golang version: go1.14.12
Swarm listening on /ip4/10.8.1.9/tcp/4001
Swarm listening on /ip4/10.8.1.9/udp/4001/quic
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.0.107/tcp/4001
Swarm listening on /ip4/192.168.0.107/udp/4001/quic
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip4/24.62.29.132/tcp/4001
Swarm announcing /ip4/24.62.29.132/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic
API server listening on /ip4/127.0.0.1/tcp/5001
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready
^C
Received interrupt signal, shutting down...
(Hit ctrl-c again to force-shutdown the daemon.)

❯ netstat -vanp tcp | grep 5001

❯ lsof -i :5001   

❯ ipfs daemon &                
[1] 2391

❯ Initializing daemon...
go-ipfs version: 0.7.0-ea77213
Repo version: 10
System version: amd64/darwin
Golang version: go1.14.12
Swarm listening on /ip4/10.8.1.9/tcp/4001
Swarm listening on /ip4/10.8.1.9/udp/4001/quic
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.0.107/tcp/4001
Swarm listening on /ip4/192.168.0.107/udp/4001/quic
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip4/24.62.29.132/tcp/4001
Swarm announcing /ip4/24.62.29.132/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic
API server listening on /ip4/127.0.0.1/tcp/5001
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready

❯ kill -SIGTERM 2391

Received interrupt signal, shutting down...
(Hit ctrl-c again to force-shutdown the daemon.)

[1]  + 2391 done       ipfs daemon

❯ lsof -i :5001     

❯ netstat -vanp tcp | grep 5001

❯ ipfs daemon                  
Initializing daemon...
go-ipfs version: 0.7.0-ea77213
Repo version: 10
System version: amd64/darwin
Golang version: go1.14.12
Swarm listening on /ip4/10.8.1.9/tcp/4001
Swarm listening on /ip4/10.8.1.9/udp/4001/quic
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.0.107/tcp/4001
Swarm listening on /ip4/192.168.0.107/udp/4001/quic
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic
API server listening on /ip4/127.0.0.1/tcp/5001
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready
Received interrupt signal, shutting down...
(Hit ctrl-c again to force-shutdown the daemon.)

❯ netstat -vanp tcp | grep 5001
tcp4       0      0  127.0.0.1.5001         127.0.0.1.49619        TIME_WAIT   408077 146988   3051      0 0x2131 0x0000002c

❯ lsof -i :5001

NorseGaud · 2021-01-15T20:30:13Z

Looking at the pid that's using 5001 (3168), I see:

root              3168   0.0  0.0  4758748   8492   ??  S     3:23PM   0:00.23 /opt/cisco/anyconnect/bin/acwebsecagent -console

Killing that fixes it for me. Darn cisco...

Not sure what my brain was doing originally to miss this. Thanks for taking a look.

endomorphine · 2021-04-04T22:51:59Z

if you don't actually use 'acwebsecagent' component you can just
sudo /opt/cisco/anyconnect/bin/websecurity_uninstall.sh

NorseGaud added kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization labels Jan 15, 2021

ipfs deleted a comment from welcome bot Jan 15, 2021

lidel mentioned this issue Jan 15, 2021

Deprecate and remove the shutdown command #7866

Open

lidel added need/author-input Needs input from the original author need/analysis Needs further analysis before proceeding and removed need/triage Needs initial labeling and prioritization labels Jan 15, 2021

NorseGaud closed this as completed Jan 15, 2021

lidel mentioned this issue Jan 18, 2021

use standard, unassigned ports for the daemon #874

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ipfs shutdown orphans 5001 #7865

ipfs shutdown orphans 5001 #7865

NorseGaud commented Jan 15, 2021

lidel commented Jan 15, 2021 •

edited

Loading

NorseGaud commented Jan 15, 2021 •

edited

Loading

aschmahmann commented Jan 15, 2021

NorseGaud commented Jan 15, 2021

NorseGaud commented Jan 15, 2021

NorseGaud commented Jan 15, 2021 •

edited

Loading

endomorphine commented Apr 4, 2021

ipfs shutdown orphans 5001 #7865

ipfs shutdown orphans 5001 #7865

Comments

NorseGaud commented Jan 15, 2021

Version information:

Description:

lidel commented Jan 15, 2021 • edited Loading

NorseGaud commented Jan 15, 2021 • edited Loading

aschmahmann commented Jan 15, 2021

NorseGaud commented Jan 15, 2021

NorseGaud commented Jan 15, 2021

NorseGaud commented Jan 15, 2021 • edited Loading

endomorphine commented Apr 4, 2021

lidel commented Jan 15, 2021 •

edited

Loading

NorseGaud commented Jan 15, 2021 •

edited

Loading

NorseGaud commented Jan 15, 2021 •

edited

Loading