EVPN-VXLAN lab – IRB functionality

EVPN-VXLAN lab – IRB functionality

Comments 0 Comment

Firstly, QFX5100 series doesn’t support EVPN-VXLAN inter-VXLAN routing, so I practice all IRB related topics on vMX devices. vQFXs acts as a simple L2 EVPN gateways.
This post continues the EVPN-VXLAN lab from the previous ones.

Full vMX IRB interfaces configuration:

alex@vMX1# show interfaces irb
unit 100 {
    proxy-macip-advertisement;
    family inet {
        address 172.16.0.251/24 {
            virtual-gateway-address 172.16.0.254;
        }
    }
    family inet6 {
        address 2001:dead:beef:100::1/64 {
            virtual-gateway-address 2001:dead:beef:100::a;
        }
    }
}
unit 200 {
    proxy-macip-advertisement;
    family inet {
        address 172.16.1.251/24 {
            virtual-gateway-address 172.16.1.254;
        }
    }
    family inet6 {
        address 2001:dead:beef:200::1/64 {
            virtual-gateway-address 2001:dead:beef:200::a;
        }
    }
}


alex@vMX2# show interfaces irb
unit 100 {
    proxy-macip-advertisement;
    family inet {
        address 172.16.0.252/24 {
            virtual-gateway-address 172.16.0.254;
        }
    }
    family inet6 {
        address 2001:dead:beef:100::2/64 {
            virtual-gateway-address 2001:dead:beef:100::a;
        }
    }
}
unit 200 {
    proxy-macip-advertisement;
    }
    family inet {
        address 172.16.1.252/24 {
            virtual-gateway-address 172.16.1.254;
        }
    }
    family inet6 {
        address 2001:dead:beef:200::2/64 {
            virtual-gateway-address 2001:dead:beef:200::a;
        }
    }
}

With EVPN/VXLAN, when the proxy-macip-advertisement statement is not enabled, only the MAC routes are sent between PE devices; when enabled, both the IP and MAC host routes are installed on the receiving PE devices. In other words, you need to enable this feature to have MAC/IP routes pointing directly to vQFX devices to which this particular host is connected.

Of course you need to assign IRB interfaces to corresponding bridge-domains:

alex@vMX1# show | compare
[edit routing-instances evpn bridge-domains v100]
+     routing-interface irb.100;
[edit routing-instances evpn bridge-domains v200]
+     routing-interface irb.200;


alex@vMX2# show | compare
[edit routing-instances evpn bridge-domains v100]
+     routing-interface irb.100;
[edit routing-instances evpn bridge-domains v200]
+     routing-interface irb.200;

And one more very important part of configuration – you need to enable advertisement of IRB-specific MAC address to the Layer 2 PE device, but without the extended community option of default-gateway (default-gateway no-gateway-community):

alex@vMX1# show routing-instances evpn protocols evpn
encapsulation vxlan;
extended-vni-list [ 100 200 ];
vni-options {
    vni 100 {
        vrf-target target:65000:100;
    }
}
multicast-mode ingress-replication;
default-gateway no-gateway-community;   ### This line



That’s all configuration needed for IRB functionality, so let’s check inter-VXLAN connectivity:

alex@MX1# run ping 172.16.1.22 count 3                          
PING 172.16.1.22 (172.16.1.22): 56 data bytes
64 bytes from 172.16.1.22: icmp_seq=0 ttl=63 time=179.461 ms
64 bytes from 172.16.1.22: icmp_seq=1 ttl=63 time=56.725 ms
64 bytes from 172.16.1.22: icmp_seq=2 ttl=63 time=176.170 ms
--- 172.16.1.22 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 56.725/137.452/179.461/57.098 ms

alex@MX1# run ping 172.16.0.22 count 3 logical-system second    
PING 172.16.0.22 (172.16.0.22): 56 data bytes
64 bytes from 172.16.0.22: icmp_seq=0 ttl=63 time=676.069 ms
64 bytes from 172.16.0.22: icmp_seq=1 ttl=63 time=159.787 ms
64 bytes from 172.16.0.22: icmp_seq=2 ttl=63 time=169.944 ms
--- 172.16.0.22 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 159.787/335.267/676.069/241.019 ms

alex@MX2# run ping 172.16.1.11 count 3                          
PING 172.16.1.11 (172.16.1.11): 56 data bytes
64 bytes from 172.16.1.11: icmp_seq=0 ttl=63 time=496.691 ms
64 bytes from 172.16.1.11: icmp_seq=1 ttl=63 time=152.942 ms
64 bytes from 172.16.1.11: icmp_seq=2 ttl=63 time=249.980 ms
--- 172.16.1.11 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 152.942/299.871/496.691/144.701 ms

alex@MX2# run ping 172.16.0.11 count 3 logical-system second    
PING 172.16.0.11 (172.16.0.11): 56 data bytes
64 bytes from 172.16.0.11: icmp_seq=0 ttl=63 time=218.891 ms
64 bytes from 172.16.0.11: icmp_seq=1 ttl=63 time=121.604 ms
64 bytes from 172.16.0.11: icmp_seq=2 ttl=63 time=169.502 ms
--- 172.16.0.11 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 121.604/169.999/218.891/39.719 ms


alex@MX1# run ping 2001:dead:beef:200::22 count 3                          
PING6(56=40+8+8 bytes) 2001:dead:beef:100::11 --> 2001:dead:beef:200::22
16 bytes from 2001:dead:beef:200::22, icmp_seq=1 hlim=63 time=219.594 ms
16 bytes from 2001:dead:beef:200::22, icmp_seq=0 hlim=63 time=334.692 ms
16 bytes from 2001:dead:beef:200::22, icmp_seq=2 hlim=63 time=199.949 ms
--- 2001:dead:beef:200::22 ping6 statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/std-dev = 199.949/251.412/334.692/59.432 ms

[edit]
alex@MX1# run ping 2001:dead:beef:100::22 count 3 logical-system second    
PING6(56=40+8+8 bytes) 2001:dead:beef:200::111 --> 2001:dead:beef:100::22
16 bytes from 2001:dead:beef:100::22, icmp_seq=0 hlim=63 time=304.136 ms
16 bytes from 2001:dead:beef:100::22, icmp_seq=1 hlim=63 time=373.425 ms
16 bytes from 2001:dead:beef:100::22, icmp_seq=2 hlim=63 time=179.525 ms
--- 2001:dead:beef:100::22 ping6 statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/std-dev = 179.525/285.695/373.425/80.226 ms

alex@MX2# run ping 2001:dead:beef:200::11 count 3                          
PING6(56=40+8+8 bytes) 2001:dead:beef:100::22 --> 2001:dead:beef:200::11
16 bytes from 2001:dead:beef:200::11, icmp_seq=0 hlim=63 time=214.162 ms
16 bytes from 2001:dead:beef:200::11, icmp_seq=1 hlim=63 time=223.646 ms
16 bytes from 2001:dead:beef:200::11, icmp_seq=2 hlim=63 time=60.050 ms
--- 2001:dead:beef:200::11 ping6 statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/std-dev = 60.050/165.953/223.646/74.985 ms

[edit]
alex@MX2# run ping 2001:dead:beef:100::11 count 3 logical-system second    
PING6(56=40+8+8 bytes) 2001:dead:beef:200::22 --> 2001:dead:beef:100::11
16 bytes from 2001:dead:beef:100::11, icmp_seq=0 hlim=63 time=315.106 ms
16 bytes from 2001:dead:beef:100::11, icmp_seq=1 hlim=63 time=311.374 ms
16 bytes from 2001:dead:beef:100::11, icmp_seq=2 hlim=63 time=99.928 ms
--- 2001:dead:beef:100::11 ping6 statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/std-dev = 99.928/242.136/315.106/100.568 ms

Everything works as expected – CE have reachability between each interface in different vlans.

When you enable IRB functionality for particular EVPN domain, PE devices (vMX) start to learn IP addresses of hosts (perform ARP).
EVPN type 2 routes with IP address included:

alex@vMX1# run show route table evpn.evpn.0 evpn-mac-address 00:05:86:71:20:c0    

evpn.evpn.0: 63 destinations, 64 routes (63 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2:2.2.2.2:1::100::00:05:86:71:20:c0/304 MAC/IP        
                   *[BGP/170] 00:09:24, localpref 100, from 2.2.2.2
                      AS path: I, validation-state: unverified
                    > to 192.168.0.1 via ge-0/0/1.0
                      to 192.168.0.3 via ge-0/0/2.0
2:2.2.2.2:1::200::00:05:86:71:20:c0/304 MAC/IP        
                   *[BGP/170] 00:09:09, localpref 100, from 2.2.2.2
                      AS path: I, validation-state: unverified
                      to 192.168.0.1 via ge-0/0/1.0
                    > to 192.168.0.3 via ge-0/0/2.0
2:2.2.2.2:1::100::00:05:86:71:20:c0::172.16.0.11/304 MAC/IP        
                   *[BGP/170] 00:07:26, localpref 100, from 2.2.2.2
                      AS path: I, validation-state: unverified
                    > to 192.168.0.1 via ge-0/0/1.0
                      to 192.168.0.3 via ge-0/0/2.0
2:2.2.2.2:1::200::00:05:86:71:20:c0::172.16.1.11/304 MAC/IP        
                   *[BGP/170] 00:07:07, localpref 100, from 2.2.2.2
                      AS path: I, validation-state: unverified
                      to 192.168.0.1 via ge-0/0/1.0
                    > to 192.168.0.3 via ge-0/0/2.0
2:2.2.2.2:1::200::00:05:86:71:20:c0::2001:dead:beef:200::11/304 MAC/IP        
                   *[BGP/170] 00:09:09, localpref 100, from 2.2.2.2
                      AS path: I, validation-state: unverified
                      to 192.168.0.1 via ge-0/0/1.0
                    > to 192.168.0.3 via ge-0/0/2.0


alex@vMX2# run show route table evpn.evpn.0 evpn-mac-address 00:05:86:71:20:c0    

evpn.evpn.0: 63 destinations, 64 routes (63 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2:2.2.2.2:1::100::00:05:86:71:20:c0/304 MAC/IP        
                   *[EVPN/170] 00:09:18
                      Indirect
2:2.2.2.2:1::200::00:05:86:71:20:c0/304 MAC/IP        
                   *[EVPN/170] 00:09:02
                      Indirect
2:2.2.2.2:1::100::00:05:86:71:20:c0::172.16.0.11/304 MAC/IP        
                   *[EVPN/170] 00:07:18
                      Indirect
2:2.2.2.2:1::200::00:05:86:71:20:c0::172.16.1.11/304 MAC/IP        
                   *[EVPN/170] 00:06:59
                      Indirect
2:2.2.2.2:1::200::00:05:86:71:20:c0::2001:dead:beef:200::11/304 MAC/IP        
                   *[EVPN/170] 00:09:02
                      Indirect

As I understand this output, vMX2 learned IP addresses of hosts directly (performed ARP) and advertised this information to vMX1 via type-2 EVPN MAC/IP routes.

EVPN database:

alex@vMX1# run show evpn database
Instance: evpn
VLAN  DomainId  MAC address        Active source                  Timestamp        IP address
     100        00:00:5e:00:01:01  05:00:00:fd:e8:00:00:00:64:00  Sep 10 20:02:40  172.16.0.254
     100        00:00:5e:00:02:01  05:00:00:fd:e8:00:00:00:64:00  Sep 10 20:02:40  2001:dead:beef:100::a
     100        00:05:86:71:20:c0  00:11:22:33:44:55:66:77:88:99  Sep 10 20:04:41  172.16.0.11
     100        00:05:86:71:4c:c0  00:00:00:00:00:11:11:11:11:11  Sep 10 20:05:01  172.16.0.22
                                                                                   2001:dead:beef:100::22
     100        00:05:86:79:e2:f0  2.2.2.2                        Sep 10 20:02:40  172.16.0.252
                                                                                   2001:dead:beef:100::2
                                                                                   fe80::205:8600:6479:e2f0
     100        00:05:86:f6:e2:f0  irb.100                        Sep 10 20:00:17  172.16.0.251
                                                                                   2001:dead:beef:100::1
                                                                                   fe80::205:8600:64f6:e2f0
     200        00:00:5e:00:01:01  05:00:00:fd:e8:00:00:00:c8:00  Sep 10 20:02:40  172.16.1.254
     200        00:00:5e:00:02:01  05:00:00:fd:e8:00:00:00:c8:00  Sep 10 20:02:40  2001:dead:beef:200::a
     200        00:05:86:71:20:c0  00:11:22:33:44:55:66:77:88:99  Sep 10 20:05:01  172.16.1.11
                                                                                   2001:dead:beef:200::11
     200        00:05:86:71:4c:c0  00:00:00:00:00:11:11:11:11:11  Sep 10 20:04:41  172.16.1.22
                                                                                   2001:dead:beef:200::22
     200        00:05:86:79:e2:f0  2.2.2.2                        Sep 10 20:02:40  172.16.1.252
                                                                                   2001:dead:beef:200::2
                                                                                   fe80::205:8600:c879:e2f0
     200        00:05:86:f6:e2:f0  irb.200                        Sep 10 20:00:18  172.16.1.251
                                                                                   2001:dead:beef:200::1
                                                                                   fe80::205:8600:c8f6:e2f0
     200        00:46:d3:04:fe:06  ge-0/0/4.0                     Sep 10 20:07:07  172.16.1.111
                                                                                   2001:dead:beef:200::111



alex@vMX2# run show evpn database
Instance: evpn
VLAN  DomainId  MAC address        Active source                  Timestamp        IP address
     100        00:00:5e:00:01:01  05:00:00:fd:e8:00:00:00:64:00  Sep 10 20:02:18  172.16.0.254
     100        00:00:5e:00:02:01  05:00:00:fd:e8:00:00:00:64:00  Sep 10 20:02:18  2001:dead:beef:100::a
     100        00:05:86:71:20:c0  00:11:22:33:44:55:66:77:88:99  Sep 10 20:04:43  172.16.0.11
     100        00:05:86:71:4c:c0  00:00:00:00:00:11:11:11:11:11  Sep 10 20:05:03  172.16.0.22
                                                                                   2001:dead:beef:100::22
     100        00:05:86:79:e2:f0  irb.100                        Sep 10 20:02:09  172.16.0.252
                                                                                   2001:dead:beef:100::2
                                                                                   fe80::205:8600:6479:e2f0
     100        00:05:86:f6:e2:f0  1.1.1.1                        Sep 10 20:02:18  172.16.0.251
                                                                                   2001:dead:beef:100::1
                                                                                   fe80::205:8600:64f6:e2f0
     200        00:00:5e:00:01:01  05:00:00:fd:e8:00:00:00:c8:00  Sep 10 20:02:07  172.16.1.254
     200        00:00:5e:00:02:01  05:00:00:fd:e8:00:00:00:c8:00  Sep 10 20:02:07  2001:dead:beef:200::a
     200        00:05:86:71:20:c0  00:11:22:33:44:55:66:77:88:99  Sep 10 20:05:03  172.16.1.11
                                                                                   2001:dead:beef:200::11
     200        00:05:86:71:4c:c0  00:00:00:00:00:11:11:11:11:11  Sep 10 20:04:39  172.16.1.22
                                                                                   2001:dead:beef:200::22
     200        00:05:86:79:e2:f0  irb.200                        Sep 10 20:02:09  172.16.1.252
                                                                                   2001:dead:beef:200::2
                                                                                   fe80::205:8600:c879:e2f0
     200        00:05:86:f6:e2:f0  1.1.1.1                        Sep 10 20:02:07  172.16.1.251
                                                                                   2001:dead:beef:200::1
                                                                                   fe80::205:8600:c8f6:e2f0
     200        00:46:d3:04:fe:06  1.1.1.1                        Sep 10 20:07:10  172.16.1.111
                                                                                   2001:dead:beef:200::111

EVPN database now contains not only MAC addresses, but also IP information.
Notice IRB and virtual-gateway address IP/MAC/ESI ID.

Proxy-macip-advertisement in action – MAC/IP routes point directly to vQFX node:

alex@vMX1# ...n.0 evpn-mac-address 00:05:86:71:4c:c0 detail                

2:22.22.22.22:1::100::00:05:86:71:4c:c0::172.16.0.22/304 MAC/IP (1 entry, 1 announced)
        *EVPN   Preference: 170
                Next hop type: Indirect, Next hop index: 0
                Address: 0xd160730
                Next-hop reference count: 11
                Protocol next hop: 22.22.22.22
                Indirect next hop: 0x0 - INH Session ID: 0x0
                State: <Active Int Ext>
                Age: 3:51
                Validation State: unverified
                Task: evpn-evpn
                Announcement bits (1): 1-BGP_RT_Background
                AS path: I
                Communities: encapsulation:vxlan(0x8)
                Route Label: 100
                ESI: 00:00:00:00:00:11:11:11:11:11

Also there is a couple of new optional features for IRB interace.
You can manually assign specific ESI ID to IRB interface (don’t forget that ESI ID need to match on all IRB interfaces for this particular EVPN domain):

alex@vMX1# show interfaces irb            
unit 100 {
    virtual-gateway-esi {
        00:77:77:77:77:77:77:77:77:77;   ### New ESI ID
        all-active;
    }
unit 200 {
    virtual-gateway-esi {
        00:88:88:77:77:77:77:77:77:77;
        all-active;                    
    }
}


alex@vMX1# run show evpn database    
Instance: evpn
VLAN  DomainId  MAC address        Active source                  Timestamp        IP address
     100        00:00:5e:00:01:01  00:77:77:77:77:77:77:77:77:77  Sep 10 20:31:03  172.16.0.254      ### New ESI ID
     100        00:00:5e:00:02:01  00:77:77:77:77:77:77:77:77:77  Sep 10 20:31:03  2001:dead:beef:100::a
     100        00:05:86:71:20:c0  00:11:22:33:44:55:66:77:88:99  Sep 10 20:31:45  2001:dead:beef:100::11
     100        00:05:86:71:4c:c0  00:00:00:00:00:11:11:11:11:11  Sep 10 20:31:43  2001:dead:beef:100::22
     100        00:05:86:79:e2:f0  2.2.2.2                        Sep 10 20:31:03  172.16.0.252
                                                                                   2001:dead:beef:100::2
                                                                                   fe80::205:8600:6479:e2f0
     100        00:05:86:f6:e2:f0  irb.100                        Sep 10 20:29:43  172.16.0.251
                                                                                   2001:dead:beef:100::1
                                                                                   fe80::205:8600:64f6:e2f0
     200        00:00:5e:00:01:01  00:88:88:77:77:77:77:77:77:77  Sep 10 20:31:03  172.16.1.254
     200        00:00:5e:00:02:01  00:88:88:77:77:77:77:77:77:77  Sep 10 20:31:03  2001:dead:beef:200::a
     200        00:05:86:71:20:c0  00:11:22:33:44:55:66:77:88:99  Sep 10 20:31:20  2001:dead:beef:200::11
     200        00:05:86:71:4c:c0  00:00:00:00:00:11:11:11:11:11  Sep 10 20:32:23  2001:dead:beef:200::22
     200        00:05:86:79:e2:f0  2.2.2.2                        Sep 10 20:31:03  172.16.1.252
                                                                                   2001:dead:beef:200::2
                                                                                   fe80::205:8600:c879:e2f0
     200        00:05:86:f6:e2:f0  irb.200                        Sep 10 20:29:44  172.16.1.251
                                                                                   2001:dead:beef:200::1
                                                                                   fe80::205:8600:c8f6:e2f0
     200        00:46:d3:04:fe:06  ge-0/0/4.0                     Sep 10 20:07:07

And you can manually assign specific MAC address to virtual-gateway IP address:

alex@vMX1# show interfaces irb
unit 100 {
    virtual-gateway-v4-mac 00:44:44:44:44:44;
    virtual-gateway-v6-mac 00:66:66:66:66:66;
}
unit 200 {
    virtual-gateway-v4-mac 00:00:00:44:44:44;
    virtual-gateway-v6-mac 00:00:00:66:66:66;
}


alex@vMX1# run show evpn database    
Instance: evpn
VLAN  DomainId  MAC address        Active source                  Timestamp        IP address
     100        00:05:86:71:20:c0  00:11:22:33:44:55:66:77:88:99  Sep 10 20:39:21
     100        00:05:86:71:4c:c0  00:00:00:00:00:11:11:11:11:11  Sep 10 20:38:51
     100        00:05:86:79:e2:f0  2.2.2.2                        Sep 10 20:39:01  172.16.0.252
                                                                                   2001:dead:beef:100::2
                                                                                   fe80::205:8600:6479:e2f0
     100        00:05:86:f6:e2:f0  irb.100                        Sep 10 20:37:27  172.16.0.251
                                                                                   2001:dead:beef:100::1
                                                                                   fe80::205:8600:64f6:e2f0
     100        00:44:44:44:44:44  00:77:77:77:77:77:77:77:77:77  Sep 10 20:39:01  172.16.0.254            ### Note new MAC here
     100        00:66:66:66:66:66  00:77:77:77:77:77:77:77:77:77  Sep 10 20:39:01  2001:dead:beef:100::a   ### and here
     200        00:00:00:44:44:44  00:88:88:77:77:77:77:77:77:77  Sep 10 20:39:01  172.16.1.254
     200        00:00:00:66:66:66  00:88:88:77:77:77:77:77:77:77  Sep 10 20:39:01  2001:dead:beef:200::a
     200        00:05:86:71:20:c0  00:11:22:33:44:55:66:77:88:99  Sep 10 20:39:21
     200        00:05:86:71:4c:c0  00:00:00:00:00:11:11:11:11:11  Sep 10 20:38:51
     200        00:05:86:79:e2:f0  2.2.2.2                        Sep 10 20:39:01  172.16.1.252
                                                                                   2001:dead:beef:200::2
                                                                                   fe80::205:8600:c879:e2f0
     200        00:05:86:f6:e2:f0  irb.200                        Sep 10 20:37:27  172.16.1.251
                                                                                   2001:dead:beef:200::1
                                                                                   fe80::205:8600:c8f6:e2f0
     200        00:46:d3:04:fe:06  ge-0/0/4.0                     Sep 10 20:37:24


Remember that IRB interface is just another L3 interface for Juniper box – you do not have to terminate it in GRT, you can include it in virtual-router or VRF if you need to somehow separate EVPN domains on IP level. May be useful for DCI related tasks.


So this is all EVPN IRB functionality that I wanted to cover. There is not so much variety of IRB configuration options.
Theory of IRB functionality is definitely a lot harder to understand than just simply configure a couple of config statements. If you don’t yet understand this completly, see my previous posts on this topic to find links to usefull study materials.

Leave a Reply

Your email address will not be published. Required fields are marked *