EVPN-VXLAN lab – RT assignment methods

EVPN-VXLAN lab – RT assignment methods

This post continues the EVPN-VXLAN lab from the previous one.

For now I configured the simplest possible variant of RT assignment – one vrf-target for all ES and VNI routes (vrf-target target:65000:1):

alex@vQFX1# show switch-options
service-id 1;
vtep-source-interface lo0.0;
route-distinguisher 11.11.11.11:1;
vrf-target target:65000:1;    ### This RT applies to ALL EVPN routes


alex@vMX1# show routing-instances
evpn {
    vtep-source-interface lo0.0;
    instance-type virtual-switch;
    interface ge-0/0/4.0;
    interface ae0.0;
    route-distinguisher 1.1.1.1:1;
    vrf-target target:65000:1;   ### This RT applies to ALL EVPN routes
    protocols {
        evpn {
            encapsulation vxlan;
            extended-vni-list [ 100 200 ];
            multicast-mode ingress-replication;
            default-gateway no-gateway-community;
        }
    }
    bridge-domains {
        v100 {
            vlan-id 100;
            vxlan {
                vni 100;
                ingress-node-replication;
            }
        }
        v200 {                          
            vlan-id 200;
            vxlan {
                vni 200;
                ingress-node-replication;
            }
        }
    }
}


Let’s check some EVPN routes:

alex@vMX1# run show route table bgp.evpn.0 detail | match "entry|protocol|comm"                  
1:2.2.2.2:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 0 announced)
                Protocol next hop: 2.2.2.2
                Communities: target:65000:1 encapsulation:vxlan(0x8) esi-label:0x0:all-active (label 0)
1:2.2.2.2:1::112233445566778899::0/192 AD/EVI (1 entry, 0 announced)
                Protocol next hop: 2.2.2.2
                Communities: target:65000:1 encapsulation:vxlan(0x8)
1:11.11.11.11:0::1111111111::FFFF:FFFF/192 AD/ESI (1 entry, 0 announced)
                Protocol next hop: 11.11.11.11
                Communities: target:65000:1 encapsulation:vxlan(0x8) esi-label:0x0:all-active (label 0)
1:11.11.11.11:1::1111111111::0/192 AD/EVI (1 entry, 0 announced)
                Protocol next hop: 11.11.11.11
                Communities: target:65000:1 encapsulation:vxlan(0x8)
1:22.22.22.22:0::1111111111::FFFF:FFFF/192 AD/ESI (1 entry, 0 announced)
                Protocol next hop: 22.22.22.22
                Communities: target:65000:1 encapsulation:vxlan(0x8) esi-label:0x0:all-active (label 0)
1:22.22.22.22:1::1111111111::0/192 AD/EVI (1 entry, 0 announced)
                Protocol next hop: 22.22.22.22
                Communities: target:65000:1 encapsulation:vxlan(0x8)
2:22.22.22.22:1::100::00:05:86:71:4c:c0/304 MAC/IP (1 entry, 0 announced)
                Protocol next hop: 22.22.22.22
                Communities: target:65000:1 encapsulation:vxlan(0x8)
2:22.22.22.22:1::200::00:05:86:71:4c:c0/304 MAC/IP (1 entry, 0 announced)
                Protocol next hop: 22.22.22.22
                Communities: target:65000:1 encapsulation:vxlan(0x8)
3:2.2.2.2:1::100::2.2.2.2/248 IM (1 entry, 0 announced)
                Protocol next hop: 2.2.2.2
                Communities: target:65000:1 encapsulation:vxlan(0x8)
3:2.2.2.2:1::200::2.2.2.2/248 IM (1 entry, 0 announced)
                Protocol next hop: 2.2.2.2
                Communities: target:65000:1 encapsulation:vxlan(0x8)
3:11.11.11.11:1::100::11.11.11.11/248 IM (1 entry, 0 announced)
                Protocol next hop: 11.11.11.11
                Communities: target:65000:1 encapsulation:vxlan(0x8)
3:11.11.11.11:1::200::11.11.11.11/248 IM (1 entry, 0 announced)
                Protocol next hop: 11.11.11.11
                Communities: target:65000:1 encapsulation:vxlan(0x8)
3:22.22.22.22:1::100::22.22.22.22/248 IM (1 entry, 0 announced)
                Protocol next hop: 22.22.22.22
                Communities: target:65000:1 encapsulation:vxlan(0x8)
3:22.22.22.22:1::200::22.22.22.22/248 IM (1 entry, 0 announced)
                Protocol next hop: 22.22.22.22
                Communities: target:65000:1 encapsulation:vxlan(0x8)

Notice that all routes have RT target:65000:1 (except type 4 routes, that has special RT assignment policy).
Effectively this means that all PE routers, participating in this EVPN instance, will learn all routes from all VLANs/VXLANs.

If you need to alter this behaviour, you can configure per-VNI route target assignment for specific VNI (note that this RT applies only to EVPN type-2 and type-3 routes):

alex@vMX1# show | compare
[edit routing-instances evpn protocols evpn]
+      vni-options {
+          vni 100 {
+              vrf-target target:65000:100;
+          }
+      }


alex@vQFX1# show | compare
[edit protocols evpn]
+    vni-options {
+        vni 100 {
+            vrf-target export target:65000:100;
+        }
+    }


But unfortunately that’s not all. After applying this configuration, routes for VNI 100 are advertised with new RT:

alex@vMX1# run show route advertising-protocol bgp 2.2.2.2 detail evpn-mac-address 00:05:86:71:20:c0                

evpn.evpn.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)
* 2:1.1.1.1:1::100::00:05:86:71:20:c0/304 MAC/IP (1 entry, 1 announced)
 BGP group overlay type Internal
     Route Distinguisher: 1.1.1.1:1
     Route Label: 100
     ESI: 00:11:22:33:44:55:66:77:88:99
     Nexthop: Self
     Flags: Nexthop Change
     Localpref: 100
     AS path: [65000] I
     Communities: target:65000:100 encapsulation:vxlan(0x8)  ### Notice new RT

* 2:1.1.1.1:1::200::00:05:86:71:20:c0/304 MAC/IP (1 entry, 1 announced)
 BGP group overlay type Internal
     Route Distinguisher: 1.1.1.1:1
     Route Label: 200
     ESI: 00:11:22:33:44:55:66:77:88:99
     Nexthop: Self
     Flags: Nexthop Change
     Localpref: 100
     AS path: [65000] I
     Communities: target:65000:1 encapsulation:vxlan(0x8)   ### Old RT here

But routes for VNI 100 are not recieved on the other side:

alex@vMX2# run show route receive-protocol bgp 1.1.1.1 detail evpn-mac-address 00:05:86:71:20:c0                      

bgp.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, 0 hidden)
* 2:1.1.1.1:1::200::00:05:86:71:20:c0/304 MAC/IP (1 entry, 0 announced)
     Import Accepted
     Route Distinguisher: 1.1.1.1:1
     Route Label: 200
     ESI: 00:11:22:33:44:55:66:77:88:99
     Nexthop: 1.1.1.1
     Localpref: 100
     AS path: I
     Communities: target:65000:1 encapsulation:vxlan(0x8)

And of course connectivity for VNI 100 breaks.

If you familiar with MPLS L3VPN, you should already spot the problem: RT configuration in VNI-options stanza configures only export RT, but doesn’t create corresponding vrf-import policies. So you need to configure it manually:

alex@vMX1# show | compare
[edit policy-options]
+   policy-statement import {
+       term 1 {
+           from community evpn;
+           then accept;
+       }
+       term 2 {
+           from community vni100;
+           then accept;
+       }
+   }
[edit policy-options]
+   community evpn members target:65000:1;
+   community vni100 members target:65000:100;
[edit routing-instances evpn]
+   vrf-import import;


alex@vQFX1# show | compare
[edit policy-options]
+   policy-statement import {
+       term global {
+           from community evpn;
+           then accept;
+       }
+       term vni100 {
+           from community vni100;
+           then accept;
+       }
+   }
[edit policy-options]
+   community evpn members target:65000:1;
+   community vni100 members target:65000:100;
[edit switch-options]
+  vrf-import import;

Notice that you need to permit not only RT for this specific VNI, but also general vrf-target RT, used for EVPN type 1 routes.

After applying this config vMX starts to import routes for both VNI:

alex@vMX1# run show route receive-protocol bgp 2.2.2.2 detail evpn-mac-address 00:05:86:71:20:c0                      

bgp.evpn.0: 17 destinations, 17 routes (17 active, 0 holddown, 0 hidden)
* 2:2.2.2.2:1::100::00:05:86:71:20:c0/304 MAC/IP (1 entry, 0 announced)
     Import Accepted
     Route Distinguisher: 2.2.2.2:1
     Route Label: 100
     ESI: 00:11:22:33:44:55:66:77:88:99
     Nexthop: 2.2.2.2
     Localpref: 100
     AS path: I
     Communities: target:65000:100 encapsulation:vxlan(0x8)

* 2:2.2.2.2:1::200::00:05:86:71:20:c0/304 MAC/IP (1 entry, 0 announced)
     Import Accepted
     Route Distinguisher: 2.2.2.2:1
     Route Label: 200
     ESI: 00:11:22:33:44:55:66:77:88:99
     Nexthop: 2.2.2.2
     Localpref: 100
     AS path: I                        
     Communities: target:65000:1 encapsulation:vxlan(0x8)

And connectivity for VNI 100 restored.

So this option works, but requires manual vrf-import policy configuration.

There is another, more simple option for RT assignment – auto-RT:

alex@vMX1# show | compare
[edit routing-instances evpn vrf-target]
+    auto;


alex@vQFX1# show | compare
[edit switch-options vrf-target]
+    auto;


Routes on vMX after appying this config:

alex@vMX1# run show route receive-protocol bgp 2.2.2.2 detail evpn-mac-address 00:05:86:71:20:c0    

bgp.evpn.0: 17 destinations, 17 routes (17 active, 0 holddown, 0 hidden)
* 2:2.2.2.2:1::100::00:05:86:71:20:c0/304 MAC/IP (1 entry, 0 announced)
     Import Accepted
     Route Distinguisher: 2.2.2.2:1
     Route Label: 100
     ESI: 00:11:22:33:44:55:66:77:88:99
     Nexthop: 2.2.2.2
     Localpref: 100
     AS path: I
     Communities: target:65000:100 encapsulation:vxlan(0x8)

* 2:2.2.2.2:1::200::00:05:86:71:20:c0/304 MAC/IP (1 entry, 0 announced)
     Import Accepted
     Route Distinguisher: 2.2.2.2:1
     Route Label: 200
     ESI: 00:11:22:33:44:55:66:77:88:99
     Nexthop: 2.2.2.2
     Localpref: 100
     AS path: I                        
     Communities: target:65000:268435656 encapsulation:vxlan(0x8)   ### Auto-RT here

Notice RT for VNI 200 route – this RT is autoderived from device AS number (configured in routing options stanza) and VNI ID.

There is no need to manually configure vrf-import policies in this case – this policies configured and applied automatically:

alex@vMX1# run show policy ?
Possible completions:
  <[Enter]>            Execute this command
  <policy>             Name of policy
  __vrf-export-bd-override-100-internal__  
  __vrf-export-evpn-internal__  
  __vrf-import-__default_evpn__-internal__  
  __vrf-import-autoderive-evpn-internal__      ### This policy
  __vrf-import-evpn-internal__  
  conditions           Show conditions used by policy
  damping              Show state of route flap damping
  direct              
  fabric               Internal fabric state
  import              
  lb                  
  logical-system       Name of logical system, or 'all'
  statistics           Statistics for policy
  |                    Pipe through a command
[edit]
alex@vMX1# run show policy __vrf-import-autoderive-evpn-internal__
Policy __vrf-import-autoderive-evpn-internal__:
    Term 100:
        from community __vrf-community-evpn-100-internal__ [target:65000:268435556 ]   ### Auto-RT for VNI 100
        then accept
    Term 200:
        from community __vrf-community-evpn-200-internal__ [target:65000:268435656 ]   ### Auto-RT for VNI 200
        then accept
    Term unnamed:
        from policy import
        then accept
    Term unnamed:
        then reject

Notice that this policy allows to import Auto-RT for both VNI, but for VNI 100 this is not used because we manually configured static RT for this VNI in vni-options stanza. This configuration creates another hidden policy:

alex@vMX1# run show policy __vrf-export-bd-override-100-internal__          
Policy __vrf-export-bd-override-100-internal__:
    Term unnamed:
        then community + __vrf-community-bd-override-100-export-internal__ [target:65000:100 ] accept


So, that’s it. Not so complex after all.

Remember 3 simple rules:
– General vrf-target statement (under switch-options in QFX case and under routing-instance in MX) used for EVPN routes type 1. This RT will be used for all EVPN route-types if you don’t configure anything else.
– vrf-target auto allow you to use different autoderived per-VNI RT for EVPN type 2 and 3 routes (pay attention to AS number configured under routing options stanza – auto-RT use this number).
– protocols evpn vni-options allows you to manually configure RT for specific VNI – you can choose any VNI-to-RT mapping you want. But this configuration apply RT only for vrf-export – you need to manually define and apply vrf-import policies. Also don’t forget to include general RT in this policy.

2 thoughts on “EVPN-VXLAN lab – RT assignment methods

  1. when using auto route-target, how can import such route-targets on PEs in a different AS?

    cisco allows you to rewrite the ASN portion of the RT to your own AS of incoming advertisements, others like cumulous and sonic, ignore the ASN number all together in import so they import *:VNI

  2. Read this thoroughly and have couple of questions:

    You did use route-target export stanza in the example and it really sets only export RT on one side, that’s why you don’t see anything accepted (as import RT is not configured at all) – nothing weird here.

    You stated that “If you familiar with MPLS L3VPN, you should already spot the problem: RT configuration in VNI-options stanza configures only export RT, but doesn’t create corresponding vrf-import policies”

    AFAIK vni-options configuration is not in use in MPLS L3VPN – normally we configure only routing instances of VRF type on PEs

Leave a Reply

Your email address will not be published. Required fields are marked *