Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Netlink route delete messages cannot be received when interface's VRF changed #7

Open
endoh opened this issue Dec 13, 2017 · 2 comments

Comments

@endoh
Copy link
Contributor

endoh commented Dec 13, 2017

When a VLAN interface which belongs to a VRF moves to another VRF, the static routes on the interface at the first VRF are automatically deleted, however, netlink route delete messages for those routes cannot be received.

I used ip monitor command to monitor netlink messages, but any netlink route delete messages for those routes couldn't be received, it probably means that Linux kernel doesn't send netlink route delete messages for those routes.

This problem can be appeared at both the latest version of ribd/openconfigd pair and the first commit version of them as well.

  • How to reproduce
  1. Use the config below and boot ribd
  2. Start monitoring netlink route messages using ip monitor command
    ip monitor route
  3. On openconfigd cli, chaneg the vlan100's belonging VRF from default VRF to vrf1, and commit.
    set interfaces interface vlan100 vrf vrf100
  4. At this point, two routes; 172.16.1.0/24 and 172.16.2.0/24 are properly deleted from linux kernel routing table, however, netlink route delete messages for theses routes are not received.
Deleted fe80::/64 dev vlan100  proto kernel  metric 256  pref medium
Deleted ff00::/8 dev vlan100  table local  metric 256  pref medium
Deleted local fe80:: dev lo  table local  proto none  metric 0  pref medium
Deleted local fe80::38a3:79ff:fec2:f519 dev lo  table local  proto none  metric 0  pref medium
local 172.20.0.1 dev vlan100  table 1  proto kernel  scope host  src 172.20.0.1
broadcast 172.20.0.255 dev vlan100  table 1  proto kernel  scope link  src 172.20.0.1
172.20.0.0/24 dev vlan100  table 1  proto kernel  scope link  src 172.20.0.1
broadcast 172.20.0.0 dev vlan100  table 1  proto kernel  scope link  src 172.20.0.1
ff00::/8 dev vlan100  table 1  metric 256  pref medium
fe80::/64 dev vlan100  table 1  proto kernel  metric 256  pref medium
local fe80::a2aa:aaff:feaa:aa64 dev lo  table 1  proto none  metric 0  pref medium
local fe80:: dev lo  table 1  proto none  metric 0  pref medium
  • config
vrf {
    name vrf1;
}
vlans {
    vlan 100;
    vlan 30;
    vlan 40;
}
interfaces {
    interface vlan100 {
        ipv4 {
            address 172.20.0.1/24;
        }
    }
    interface vlan30 {
        ipv4 {
            address 172.20.30.1/24;
        }
    }
    interface vlan40 {
        ipv4 {
            address 172.20.40.1/24;
        }
    }
    interface vrf1;
}
routing-options {
    ipv4 {
        route 172.16.1.0/24 {
            nexthop 172.20.0.11;
        }
        route 172.16.2.0/24 {
            nexthop 172.20.0.12;
        }
        route 172.16.3.0/24 {
            nexthop 172.20.30.13;
        }
        route 172.16.4.0/24 {
            nexthop 172.20.40.14;
        }
    }
}
@endoh
Copy link
Contributor Author

endoh commented Dec 13, 2017

The output of ribd's console when the vlan100's vrf is changed is below:

[cmd] add [interfaces interface vlan100 vrf vrf1]
[API] IfVrfBind start: vlan100 vrf1
RTPROT_KERNEL fe80::/64
Addr del: {0 21 fe80::38a3:79ff:fec2:f519/64 10 634  0}
If add: {16 vlan100 634 1500 0 69699 1 a0:aa:aa:aa:aa:64 0 0 false}
If add: {16 vlan100 634 1500 0 69699 1 a0:aa:aa:aa:aa:64 0 633 false}
NetlinkRouteAdd(): 172.16.1.0/24
NetlinkRouteAdd(): 172.16.2.0/24
NetlinkRouteAdd(): 172.16.3.0/24
NetlinkRouteAdd(): 172.16.4.0/24
RedistIPv4Add: client len 0
If add: {16 vlan100 634 1500 0 4098 1 a0:aa:aa:aa:aa:64 0 633 false}
Interface status is Up -> Down vlan100
RedistIPv4Delete: client len 0
RibWalker for all Vrf
NetlinkRouteAdd(): 172.16.1.0/24
NetlinkRouteAdd(): 172.16.2.0/24
[API] WaitAsync is called
[API] IfVrfBind end: vlan100 vrf1
NetlinkRouteAdd(): 172.16.3.0/24
NetlinkRouteAdd(): 172.16.4.0/24
RTPROT_KERNEL 172.20.0.0/24
RTPROT_KERNEL fe80::/64
If add: {16 vlan100 634 1500 0 69699 1 a0:aa:aa:aa:aa:64 0 633 false}
Interface status is Down -> Up vlan100
Re-install interface address vlan100
Re-instal addr: 172.20.0.1/24
IpAddrAdd() AddrAdd: file exists
RedistIPv4Add: client len 0
RibWalker for all Vrf
NetlinkRouteAdd(): 172.16.1.0/24
NetlinkRouteAdd(): 172.16.2.0/24
NetlinkRouteAdd(): 172.16.3.0/24
NetlinkRouteAdd(): 172.16.4.0/24
If add: {16 vlan100 634 1500 0 69699 1 a0:aa:aa:aa:aa:64 0 633 false}
Addr add: {0 20 fe80::a2aa:aaff:feaa:aa64/64 10 634  0}

@endoh
Copy link
Contributor Author

endoh commented Dec 19, 2017

I tried to do almost same thing using `ip' command and couldn't get route delete messages as well as it was done by ribd. So this could be a problem of Linux kernel.

  • ip command example
    sudo ip link add dev vrf1 type vrf table 10
    sudo ip link set vrf1 up
    sudo ip link add veth0 type veth peer name veth1
    sudo ip link set veth0 up
    sudo ip link set veth1 up
    sudo ip addr add 10.1.1.1/24 dev veth0
    sudo ip route add 172.16.1.0/24 via 10.1.1.2

    Then start monitor:
    sudo ip monitor route&
    Then move veth0 from default vrf to vrf1:
    sudo ip link set dev veth0 master vrf1

  • Result

Deleted ff00::/8 dev veth0  table local  metric 256 linkdown  pref medium
Deleted local fe80::24dd:c7ff:fe7b:ba6c dev lo  table local  proto none  metric 0  pref medium
local 10.1.1.1 dev veth0  table 10  proto kernel  scope host  src 10.1.1.1
10.1.1.0/24 dev veth0  table 10  proto kernel  scope link  src 10.1.1.1
broadcast 10.1.1.0 dev veth0  table 10  proto kernel  scope link  src 10.1.1.1
broadcast 10.1.1.255 dev veth0  table 10  proto kernel  scope link  src 10.1.1.1
ff00::/8 dev veth0  table 10  metric 256  pref medium
fe80::/64 dev veth0  table 10  proto kernel  metric 256  pref medium
local fe80::24dd:c7ff:fe7b:ba6c dev lo  table 10  proto none  metric 0  pref medium'''

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant