no eth0 in sys-net // no network traffic gets routed at all to the outside.

[yeoldegrove]

Qubes OS version:

4.0-rc3

Affected TemplateVMs:

sys-net

---

Steps to reproduce the behavior:

Install 4.0-rc3 on my hardware configuration.

[user@sys-net ~]$ lspci | grep -i eth
00:05.0 Ethernet controller: Broadcom Limited NetLink BCM57781 Gigabit Ethernet PCIe (rev 10)

Expected behavior:

There should be a eth0 present in sys-net VM.

Actual behavior:

There is no eth0 present in sys-net VM.

General notes:

The script /usr/lib/qubes/init/network-proxy-setup.sh which is triggered by qubes-networl.service does not find eth0.
Therefore no network traffic gets routed at all to the outside.

tg3 network module is loaded but eth0 still does not appear.

[user@sys-net ~]$ ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
3: vif3.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 32
    link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
    inet 10.137.0.5/32 scope global vif3.0
       valid_lft forever preferred_lft forever
    inet6 fe80::fcff:ffff:feff:ffff/64 scope link 
       valid_lft forever preferred_lft forever

[user@sys-net ~]$ lspci | grep -i eth
00:05.0 Ethernet controller: Broadcom Limited NetLink BCM57781 Gigabit Ethernet PCIe (rev 10)

[user@sys-net ~]$ sudo bash -x /usr/lib/qubes/init/network-proxy-setup.sh
++ qubesdb-read /qubes-netvm-network
+ network=10.137.0.5
+ '[' x10.137.0.5 '!=' x ']'
+ '[' -e /proc/sys/kernel ']'
+ '[' -e /proc/sys/kernel/modules_disabled ']'
+ readonly modprobe_fail_cmd=false
+ modprobe_fail_cmd=false
++ qubesdb-read /qubes-netvm-gateway
+ gateway=10.137.0.5
++ qubesdb-read /qubes-netvm-primary-dns
+ primary_dns=10.139.1.1
++ qubesdb-read /qubes-netvm-secondary-dns
+ secondary_dns=10.139.1.2
+ modprobe netbk
+ modprobe xen-netback
+ echo NS1=10.139.1.1
+ echo NS2=10.139.1.2
+ /usr/lib/qubes/qubes-setup-dnat-to-ns
+ echo 1
+ /sbin/ethtool -K eth0 sg off
Cannot get device feature names: No such device
+ true

---

Related issues:

[hast0011]

Maybe the same reason is true for me with Release 4.0 rc2:

Ethernet controller: Broadcom Limited NetXtreme BCM5761 Gigabit Ethernet PCIe (rev 10)

In release 3.2 the following helped me out but now it won't work anymore:

[Unit]
Description=Netvm fixup
Before=qubes-netvm.service

[Service]
ExecStart=/bin/sh -c 'echo 0000:04:00.0 > /sys/bus/pci/drivers/pciback/permissive'
Type=oneshot
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Then enable it with "systemctl enable qubes-pre-netvm.service".

[marmarek]

Check kernel messages (sudo dmesg) - tg3 driver probably put some messages there why it doesn't work

[yeoldegrove]

[user@sys-net ~]$ sudo dmesg |grep tg3
[   12.826961] tg3.c:v3.137 (May 11, 2014)
[   24.912236] tg3 0000:00:05.0: tg3_test_dma: Buffer write failed. err = -19
[   24.912295] tg3 0000:00:05.0: DMA engine test failed, aborting

[hast0011]

For me the output of dmesg in sys-net was like (I cant remember the exact output):

tg3: Problem fetching invariants of chip, aborting.

[marmarek]

Looks like some problem with accessing that device by the driver. Either DMA or config space. See xl dmesg in dom0 if you have some VT-d related errors. Also check sudo dmesg if you have Driver tried to write to a read-only configuration space field message about this device. If any of those yields anything, try permissive mode: https://www.qubes-os.org/doc/assigning-devices/#pci-passthrough-issues

[hast0011]

Locked myself out of he system by having additional pci devices assgned to sys-net. This autostarts and then… I will reinstall and then come back. Maybw tomorrow. Thank you in the meantime Von: Marek Marczykowski-Górecki Gesendet: Freitag, 15. Dezember 2017 20:56 An: QubesOS/qubes-issues Cc: hast0011; Comment Betreff: Re: [QubesOS/qubes-issues] no eth0 in sys-net // no network trafficgets routed at all to the outside. (#3349) Looks like some problem with accessing that device by the driver. Either DMA or config space. See xl dmesg in dom0 if you have some VT-d related errors. Also check sudo dmesg if you have Driver tried to write to a read-only configuration space field message about this device. If any of those yields anything, try permissive mode: https://www.qubes-os.org/doc/assigning-devices/#pci-passthrough-issues — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[hast0011]

I found the following output in xl dmesg:
intel vt-d dom0 dma passthrough not enabled

Vt-d is definitly switched on in bios settings. Maybe vt-d is more than just yes or no?
Is that the reason?

[marmarek]

This message is ok. If you didn't have VT-d enabled, VM wouldn't start at all.

Maybe vt-d is more than just yes or no?

Well, there are two versions: with and without interrupt remapping, but I believe that any not 3+ years old hardware if have VT-d at all, then it comes with interrupt remapping. You should see a message about it few lines before the one you've found.

So, you haven't found any lines clearly looking like errors (with words like "fail", "error" etc), with [VT-d] or similar tag there?

[hast0011]

Managed to successfully install RC4 and network is up:-) I needed to switch off vt-d and having sys-net in PV Mode. Additionally had to set permissive for the network card to 1 manually.

Two points came up during my struggle:

1. Settings in qubes manager always shows "default(HVM)" instead of "PV".
   However qvm-prefs --get sys-net shows "virt_mode - pv" as expected and it works.
2. There exists description about getting the parameter "class" with ist in fact named "klass"
   so qmv-prefs --get sys-net klass gives AppVM. The article is about win7 installation in https://www.qubes-os.org/doc/hvm/

[marmarek]

Settings in qubes manager always shows "default(HVM)" instead of "PV".

Already tracked here: #3517

There exists description about getting the parameter "class" with ist in fact named "klass"
so qmv-prefs --get sys-net klass gives AppVM. The article is about win7 installation in https://www.qubes-os.org/doc/hvm/

Yes, qvm-prefs list its as klass, but with qvm-create it is --class...

Additionally had to set permissive for the network card to 1 manually.

Did you have any problems with that? It should be possible to set it per device (qvm-pci attach ... -o permissive=true).

What happened if you didn't disabled vt-d?

[hast0011]

With vt-d switched on my computer hangs frequently and I thought vt-d it is also related to the network card not beeing recognized fully. Now I leave it off, maybe unrelated to the network card thing.

[DroWnThePoor]

@hast0011 I have your same network card in my system, I'm running Qubes 4 and I have no eth interface, only vif3.0.
I'm trying to figure out what our solution was exactly?

[jorgejones7711]

I wasted several hours today trying to solve this as well and I post my misadventure here in case it may save anyone else some time.

Double check in your BIOS that the ethernet NIC is enabled. Mine was disabled. Once booted, shut down sys-net, edit it's settings, and on the devices tab, look for your ethernet controller, and add it to the selected side. That's all it took for me. Now in Network Manager, rather than it saying just "Ethernet Network device not managed", it now says "Ethernet Network (controllerName) disconnected" and there's another entry for "Ethernet Network (vif3.0) device not managed".

I now know before all this mess, when I saw "Ethernet Network device not managed" it was not referring to the NIC, it was referring to vif3.0, which is a virtual interface for vm to vm communication. I always saw vif3.0 when running ifconfig, but I had no idea it was the interface to which Network Manager was referring. Now that there are two ethernet interfaces, it is labeled properly in order to differentiate.

All that said, I think the Qubes team should make Network Manager specify which interface is the Ethernet Network, even if is the only one. It should always, by default, say "Ethernet Network (vif3.0)". I wonder how many other people were chasing their tail with that misunderstanding.

Lastly, before I resolved it, I had been creating and modifying all sorts of network related config files as I found them on the net. Luckily, all that was already wiped clean when I went to clean up after myself. Good luck to all out there.

[Runnerblade84]

I have the same error in 4.0.4 i can't investigate more than already did.

root@sys-net:~# lspci -vnn | grep 577 00:06.0 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM57766 Gigabit Ethernet PCIe [14e4:1686] (rev 01) Subsystem: Broadcom Limited NetXtreme BCM57766 Gigabit Ethernet PCIe [14e4:1686]

root@sys-net:~# dmesg | grep tg3 [ 1.204346] tg3.c:v3.137 (May 11, 2014) [ 37.230477] tg3 0000:00:06.0: tg3_test_dma: Buffer write failed. err = -19 [ 37.231521] tg3 0000:00:06.0: DMA engine test failed, aborting

Someone figured out with tg3 firmware?

[jbakosi]

Switching sys-net to PV mode made it work for me, suggested by @hast0011 above in #3349 (comment). You can set that in sys-net's settings in the Qubes Manager GUI or in a dom0 terminal as

qvm-prefs sys-net virt_mode pv

[github-actions]

This issue is being closed because:

• This issue is on the "Release 4.0 updates" milestone.
• Qubes OS 4.0 reached EOL (end-of-life) over one year ago.
• There has not been any activity on this issue in over one year.

If anyone believes that this issue should be reopened and reassigned to an active milestone, please leave a brief comment.
(For example, if a bug still affects Qubes OS 4.1, then the comment "Affects 4.1" will suffice.)