Port forwarding

[Saviq]

Forwarding instance ports out onto multipassd's listening address(es):

# can pass multiple --forward rules
$ multipass launch --forward <target_port>[:<listen_port>]
Port forwarding configured on:
Port <target_port>: <external_ip>:<listen_port>

$ multipass forward <name> <target_port>[:<listen_port>]
Port forwarding configured on: <external_ip>:<listen_port>

$ multipass unforward <name>[:<target_port>] [<name>[:<target_port>] ...]

$ multipass info <name>
[...]
Port forwarding:
<target_port>  <listen_port>

Where it's possible / makes sense, use hypervisor for this, otherwise have multipassd listen.

[setempler]

Possible w/ ssh local (or remote) port forwarding:

• forward requests from multipass host (port 9000) to multipass instance (port 80)

    sudo ssh \
        -i /var/root/Library/Application\ Support/multipassd/ssh-key/id_rsa \
        -L 9000:localhost:80 \
        multipass@<multipass instance ip>
  

• access multipass instance (port 80) from any box that can access multipass host

    curl <mutlipass host ip>:9000
  

Notes:

• ssh key path valid for macOS Where to find SSH keys for multipass? #913
• related issues access from public  #512 Support for bridged networking #118

[JonTheNiceGuy]

Can I also request another feature?

bridge <NICNAME> == vboxmanage modifyvm "VM name" --nic2 bridged --bridgeadapter2 NICNAME

unbridge <NICNAME> == vboxmanage modify "VM name" --nic2 none

[Saviq]

@JonTheNiceGuy that would be #118 - remember we need to be able to support the features across all of the hypervisors, not everywhere it is as easy as this :)

[JonTheNiceGuy]

not everywhere it is as easy as this :)

Fair comment :)

[rmetzger]

The ssh command posted above has slightly changed:

• it's ssh-keys instead of ssh-key
• the username is ubuntu instead of multipass.

Resulting in:

sudo ssh \
      -i /var/root/Library/Application\ Support/multipassd/ssh-keys/id_rsa \
      -L 9000:localhost:80 \
      ubuntu@<multipass instance ip>

[ktemby]

In addition to using the SSH tools, I've found it to be very helpful for server use cases to use nginx as well, for the use case of hosting multipass VM's from a headless ubuntu server.

e.g. I'm running a rails webserver in a multipass VM on port 3000, and will link that with ssh onto port 9000 on my host, and be able to connect to my host from anywhere at <host ip>:8080

1. Link multipass to host

sudo ssh -L 9000:localhost:3000 \ 
    -i /var/snap/multipass/common/data/multipassd/ssh-keys/id_rsa \
    ubuntu@<multipass instance ip> 

2. Enable access to multipass externally from host

Now using nginx, I'm going to connect to port 9000 using the host's IP and expose that on port 8080 for external connections.

server {
    listen 8080;
    server_name development;
    location / {
        proxy_pass http://localhost:9000;
    }
}

[Saviq]

@ktemby if you have <multipass instance ip>, you can give that to nginx directly, no?

[ktemby]

@Saviq no, this doesn't work for me. I think this is the point of multipass port forwarding being a limitation. While I can hit an IP, I can't tunnel to the port the server runs on without SSH.
e.g. my IP for the instance is 10.151.47.130, the server is running on port 3000 there

curl 10.151.47.130:3000
curl: (7) Failed to connect to 10.151.47.130 port 3000: Connection refused

It's easiest to repro the use case by running multipass VM's on a remote machine, then trying to view the webservers. This is how I imagine i'll use multipass for a home server cluster.

[Saviq]

@Saviq no, this doesn't work for me. I think this is the point of multipass port forwarding being a limitation. While I can hit an IP, I can't tunnel to the port the server runs on without SSH.
e.g. my IP for the instance is 10.151.47.130, the server is running on port 3000 there

curl 10.151.47.130:3000
curl: (7) Failed to connect to 10.151.47.130 port 3000: Connection refused

More likely your rails webserver only listens on localhost, rather than 0.0.0.0 :)

[ktemby]

Could be the case - but this is the vanilla rails config. I also can't curl port 80 from nginx running on the multipass VM either. As a non-expert, I like it when the tools just work without too much config... which is why I generally have been excited to find multipass!

[icokk]

Since multipass shell uses ssh anyway, why not create command multipass forward which would internally just start ssh with port forwarding? It would be very simple to add, completely cross platform, but it would still be very helpful for users (no need to search for instance IP and multipass ssh key).

[Saviq]

@icokk sure, that's the rough approach we're likely to take, we just never got to it yet.

[mikeubell]

sudo ssh
-i /var/root/Library/Application\ Support/multipassd/ssh-keys/id_rsa
-L 9000:localhost:80
ubuntu@

When I try this it asks for the password. Even after setting a password for user ubuntu and using that it fails to log in.

[Saviq]

@mikeubell it's best to import your own SSH key, no need to reach for the Multipass one. Password SSH is disabled by default. You can enable it manually or set it all up through cloud-init:

https://cloudinit.readthedocs.io/en/latest/topics/modules.html#set-passwords

[mikeubell]

@Saviq Thanks. I found a simpler solution:
ssh -L local_addr:local_port:remote_addr:remote_port -N 127.0.0.1
This does not require a login to the remote multipass instance.

[Saviq]

@mikeubell but how does it give you access to the services inside the instance? The line you wrote SSH's into your own host? If you have access to the instance directly, why do you need to do port forwarding?

[mikeubell]

I am running a web server in the instance that I need to access from outside the host. The ssh line forwards the port on that host to the port on the instance.

…

On Nov 2, 2021, at 4:09 AM, Michał Sawicz ***@***.***> wrote:  @mikeubell but how does it give you access to the services inside the instance? The line you wrote SSH's into your own host? If you have access to the instance directly, why do you need to do port forwarding? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[kycfeel]

Seriously? I don't understand that why everyone else is just talking about the SSH tunneling.

That is just a temporary solution to open the port, for a quick dev or testing environment.

Isn't doing a port forward for VM is a super common use case?

[Saviq]

@kycfeel outside of VirtualBox you get direct IP access to the instance. And even with that you can use --network to put the VM on your physical network and gain that access, so port forwarding wasn't a focus, is all.

[lethargosapatheia]

Hello. So is there any deadline or approximative time frame when this will be finished?

As an alternative I was just trying to understand the --network option, but the explanation is simply confusing to me. It's not clear at all (except for the bridge part I guess) how I could access the ip of the virtual machine directly:

--network Add a network interface to the
instance, where is in the
"key=value,key=value" format, with the
following keys available:
name: the network to connect to
(required), use the networks command for
a list of possible values, or use
'bridged' to use the interface
configured via multipass set local.bridged-network.
mode: auto|manual (default: auto)
mac: hardware address (default:
random).

[lethargosapatheia]

Ok, now I'm starting to get it. There's a local network created just for this, it's not a bridge that allows the VM to connect to your network - that's why I think this is rather ambiguous and it should be made clear in my opinion.

Any way I can assign a static IP to the virtual machine?

[ricab]

Hi @lethargosapatheia, when you provide a --network, Multipass creates an extra network interface inside that instance and bridges it to the specified network (interface) on your host. If you are using mode=auto (the default), DHCP is enabled for that additional network interface, inside the instance. It will therefore get an IP as any other machine on the same network (presumably dealt by your router).

If you want it to have a static IP, you need to configure that yourself on your network (e.g. you could configure your router to always provide the same IP to the right MAC address). You can specify a MAC address in your --network parameter, as well as mode=manual to configure networking yourself in the instance (e.g. with netplan). For example:

$ multipass networks
Name  Type      Description
eth0  ethernet  Ethernet device
[...]
$ multipass launch --network name=eth0,mode=manual,mac=ab:cd:ef:ab:cd:ef`

Have a look here for more info. Hope this helps.

[nreith]

FYI, VS Code does dynamic port forwarding, recognizing all the remote ports being used via ssh and automatically forwarding these back to local. There is probably an ssh setting that can do this in .ssh/config on your local.

[lethargosapatheia]

That would be just ssh tunnelling. I guess the consequence is the same, but it's still not port forwarding in the classical sense.
In any case, it's not that useful, bececause, by default the IP of the virtual machine is accessible, so vscode already knows that IP because you've given it to it (so you have to know the new IP beforehand - a script with awk that parses multipass list?). Having a consistent setup on (such as having the same IP each time) is different.

@ricab You mean to say, there's a dhcp server running inside the virtual machine, so inside Ubuntu? How is that called? I thought the dhcp ran somehow on the host's side.
Now that I think about it, that doesn't make sense, because it would mean you'd have several dhcp servers for the same network if you add more VMs or that the VMs depended on the dhcp of a some other VM.

[ricab]

@ricab You mean to say, there's a dhcp server running inside the virtual machine, so inside Ubuntu?

@lethargosapatheia no no, I meant the client side of it. With mode=auto, the interface tries to obtain an IP via DHCP. With mode=manual, it doesn't.

BTW, if you want to parse, you can use the --format option to list and info to get your output in json, csv, or yaml too.

[nreith]

Indeed. It would be reverse tunneling with socks proxy. NAT would give similar behavior in passing traffic through host to take advantage of network access. But a nice feature of vscode dynamic port forwarding and even wsl2 is WebApps appear on localhost. No need to edit url for the current IP of multipass vm to get it to work. I came across this thread because I was looking for ways to make multipass act a little more like wsl2. I'd like to use this with cloud-init to give a day zero dev box to software engineers or data scientists who are scared of Linux.

[yingshaoxo]

You don't need port forwarding by yourself

VScode can do it for you if you login by using vscode-remote:

https://github.com/yingshaoxo/ubuntu_ssh_server_for_the_stupid_apple_macos_m1_system#another-method-use-multipass

[NickFranceschina]

been monkeying with this stuff for days and the way @ricab explained it made more sense to me than the docs... so I used --network en1 for my 2020 Mac M1 OSX and then I got an actual IP address from my actual router and I could finally reach my containers directly. I just wanted to point out that from my home router's perspective, all the VMs seem to have the same MAC address.. the MAC of the host.. which isn't "wrong" (technically a single MAC can have multiple IPs) the "address reservation" tooling of the router is using the MAC address as an identifier, so, of course, it's not going to work if it's not unique. I built several more VMs where I passed in a MAC address to use, and when I shell into them I do indeed see that it's using the supplied MAC address, but I guess that's not the same thing since the VMs are living on their own internal network of sorts. Just leaving this note for anyone that might be googling these search terms. This networking stuff is maddening and well above my pay grade :)

here's the ARP I'm seeing externally

arp -a
? (192.168.68.99) at 14:98:77:5c:be:db on en0 ifscope [ethernet]  # static IP configured in VM1
? (192.168.68.112) at 14:98:77:5c:be:db on en0 ifscope [ethernet]  # DHCP IP of host (osx)
? (192.168.68.115) at 14:98:77:5c:be:db on en0 ifscope [ethernet]  # DHCP IP to VM2

[ricab]

Thank you @NickFranceschina, that's interesting, I also have just a basic internet-provider router, but it reports the instance's MAC address (even knows its name). That's with a bridge created by Multipass via NetworkManager...

[ricab]

Hi @NickFranceschina, I guess you're using the qemu driver (multipass get local.driver) on macOS, correct? We use vmnet in bridged mode, which is very poorly documented and essentially a black box to us what exactly Apple does. On top of that, routers can report DHCP leases in different ways and there can be inconsistencies at that level too.

I tried what you described and, when passing a wifi interface to --network my router doesn't even list the device! If I connect an ethernet cable over USB (no ethernet port on the macbook) and use that interface, then I do see the instance with the correct MAC address in the router's list. One other thing you could try is to create a virtual bridge manually and use that. I am afraid I don't have much more than that...

[Saviq]

Let me just add that bridging over WiFi is generally problematic, because of how packets are routed over the wireless link. Simplifying, the access point doesn't expect packets from a different MAC on the link (there are no WiFi switches like there are for Ethernet).

The fact that it works at all suggests macOS is doing some translation (think masquerade) under the hood - but that is going to be hit and miss on the other side.

Wonder if the IPs are actually/also sitting on the WiFi interface in some way…

[NickFranceschina]

Thanks guys... @Saviq yes I should actually have this box plugged into the ethernet instead, but at the moment I only have one port in the router (already used) and no extra switches laying around, so I tried with the wifi. @ricab I'm trying to do everything as "vanilla" as possible with multipass, falling back to the defaults that I'm sure have been thoroughly considered (so yeah I'm using the qemu driver). I don't want to dig into any specialty settings on OSX networking stuff as I tried that before and ended up in a corner where I had to re-install the OS to be sane again.

So when I get a chance I will try this again with the ethernet adapter and see if the mac addresses work as expected. until then I've opted to configure static IPs (which makes sense anyways since these services don't need DHCP) and it all appears to be working for now. I thank you for your feedback!

[ricab]

OK, glad you got it working for you @NickFranceschina. I am curious, how did you configure the static IP? Did you use cloud-init, netplan, ip commands directly? And did you tell Multipass to use mode=manual in your --network arg when launching?

[NickFranceschina]

@ricab yeah so again I haven't spent much of my life in Linux so I'm doing things as best I can figure out... I originally created the VM with just --network en1 (I didn't specify "manual" so it did auto).... then I created a sudo nano /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg file with the network: {config: disabled} (per the notes at the top of /etc/netplan/50-cloud-init.yaml) and then created a new /etc/netplan/01-netcfg.yaml with the interface name and the static ip/gateway/nameservers I wanted.... then sudo netplan apply

I think that's what I did anyways :) I should def try to break it all and see if I can fix it to make sure I really understand

[NickFranceschina]

@Saviq @ricab --- an update: added a switch and changed everything over to use ethernet DHCP instead of wifi and now I'm seeing different mac addresses from the router for the VMs! nice. thanks for your insights!

[ricab]

You're welcome, thanks for sharing your experience!

[zippy-zebu]

How does it supposed to work with exposing instances with MULTIPASS_SERVER_ADDRESS parameter ? Described here.