OSPF Routing Loop/Sub-Optimal Routing between Cisco IOS and NXOS for External Routes Configuration

Introduction

This document describes how the Open Shortest Path First (OSPF) protocol between Nexus and Cisco IOS^® feature is implemented in Cisco IOS and Nexus Operating System (NXOS).

Prerequisites

Requirements

Cisco recommends that you have knowledge of the OSPF protocol.

Components Used

The information in this document is based on these software and hardware versions:

• NXOS Version 6.2(6a)
• Cisco IOS Version 15.1(4)M1

Background Information

Cisco IOS devices support RFC 1583. However NXOS supports RFC 2328 and there are designs wherein this difference can create routing loops in the network when there are external OSPF routes in the network.

Important Information

The difference between RFC 1583 and RFC 2328, in regard to how to choose the best route among multiple external routes, is discussed in this section.

Abstract from RFC 1583 Section 16.4.6

In order to compare Type 1 external paths, look at the sum of the distance to the forwarding address and the advertised Type 1 metric (X+Y).  In order to compare Type 2 external paths, look at the advertised Type 2 metrics, and then if necessary the distance to the forwarding addresses.

If the new path is shorter, it replaces the present paths in the routing table entry.  If the new path is the same cost, it is added to the routing table entry's list of paths.

Abstract from RFC 2328 Section 16.4.1

Intra-area paths that use non-backbone areas are always the most preferred. The other paths, intra-area backbone paths and inter-area paths, are of equal preference.

Configure

Scenario 1

Network Diagram

R3 and R4 redistribute the same network 172.16.1.0/24 with the same metric as the OSPF external type E2 route. R6 prefers the route advertised by R3 because the forward metric to the ASBR R3 is lower than to R4 and the next-hop for 172.16.1.0/24 is R1. (As per RFC 1583, the path selection is based solely on cost.)

R6#sh ip ospf border-routers

OSPF Router with ID (192.168.6.6) (Process ID 1)

Base Topology (MTID 0)

Internal Router Routing Table
Codes: i - Intra-area route, I - Inter-area route

i 192.168.4.4 [51] via 192.168.56.5, GigabitEthernet0/0, ASBR, Area 2, SPF 17 
 >>>> Cost is 51 to reach R4 ASBR.
i 192.168.1.1 [1] via 192.168.16.1, GigabitEthernet0/1, ABR, Area 2, SPF 17
I 192.168.3.3 [42] via 192.168.16.1, GigabitEthernet0/1, ASBR, Area 2, SPF 17
 >>>> Cost is 42 to reach R3 ASBR

R6#sh ip route 172.16.1.0
Routing entry for 172.16.1.0/24
Known via "ospf 1", distance 110, metric 20, type extern 2, forward metric 42
Last update from 192.168.16.1 on GigabitEthernet0/1, 00:02:13 ago
Routing Descriptor Blocks:
* 192.168.16.1, from 192.168.3.3, 00:02:13 ago, via GigabitEthernet0/1
Route metric is 20, traffic share count is 1

R1 prefers the route advertised by R4 despite the higher cost because it is an intra-area route to ASBR. The route does not go through the backbone area and the next-hop is R6 ( as per RFC 2328).

R1-NXOS# sh ip ospf border-routers
OSPF Process ID 1 VRF default, Internal Routing Table
Codes: i - Intra-area route, I - Inter-area route

intra 192.168.2.2 [40], ABR, Area 0.0.0.0, SPF 18
via 192.168.12.2, Eth4/43
inter 192.168.3.3 [41], ASBR, Area 0.0.0.0, SPF 18 >>>> Cost is 41
via 192.168.12.2, Eth4/43
intra 192.168.4.4 [91], ASBR, Area 0.0.0.2, SPF 18 >>>> Cost is 91
via 192.168.16.6, Eth4/44

switch-R1-NXOS# sh ip route 172.16.1.0
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%' in via output denotes VRF 

172.16.1.0/24, ubest/mbest: 1/0
*via 192.168.16.6, Eth4/44, [110/20], 00:10:41, ospf-1, type-2

This causes a loop in the network as R6 sends the packets to R1 and R1 sends them back to R6.

R5#traceroute 172.16.1.1 numeric
 Type escape sequence to abort.
Tracing the route to 172.16.1.1
VRF info: (vrf in name/id, vrf out name/id)
1 192.168.56.6 4 msec 0 msec 0 msec
2 192.168.16.1 4 msec 0 msec 4 msec
3 192.168.16.6 0 msec 4 msec 0 msec
4 192.168.16.1 4 msec 0 msec 4 msec
5 192.168.16.6 0 msec 4 msec 0 msec

As you see, the packet loops between R1 and R6. In order to resolve this issue, you need to change the RFC compatibility on the NXOS.

R1-NXOS(config)# router ospf 1
R1-NXOS(config-router)# rfc1583compatibility

switch-R1-NXOS# sh ip route 172.16.1.0
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%' in via output denotes VRF 

172.16.1.0/24, ubest/mbest: 1/0
*via 192.168.12.2, Eth4/43, [110/20], 00:00:40, ospf-1, type-2

Now, R1 correctly points it to R2 and the loop is removed from the network.

R5#traceroute 172.16.1.1 numeric
Type escape sequence to abort.
Tracing the route to 172.16.1.1
VRF info: (vrf in name/id, vrf out name/id)
1 192.168.56.6 0 msec 4 msec 0 msec
2 192.168.16.1 0 msec 0 msec 0 msec
3 192.168.12.2 4 msec 0 msec 0 msec
4 192.168.23.3 4 msec 0 msec 4 msec
5 192.168.23.3 4 msec 0 msec 4 msec

Scenario 2

Network Diagram

R1 receives an NSSA-External (Type 7) route from R6 and an External (Type 5) route from R2 for the same prefix 172.16.1.0/24. R1 prefers Type 7, though normally in OSPF Type 5 is preferred over Type 7.

R1-NXOS# sh ip ospf database nssa-external 172.16.1.0 detail
OSPF Router with ID (192.168.1.1) (Process ID 1 VRF default)

Type-7 AS External Link States (Area 0.0.0.2)

   LS age: 914
   Options: 0x28 (No TOS-capability, Type 7/5 translation, DC)
   LS Type: Type-7 AS-External
   Link State ID: 172.16.1.0 (Network address)
   Advertising Router: 192.168.4.4                 >>>>> Type 7 originated by R4
 and installed in the RIB.
   LS Seq Number: 0x80000001
   Checksum: 0x3696
   Length: 36
   Network Mask: /24
         Metric Type: 2 (Larger than any link state path)
         TOS: 0
         Metric: 20
         Forward Address: 192.168.45.4
         External Route Tag: 0>


R1-NXOS# sh ip ospf database external 172.16.1.0 detail
 OSPF Router with ID (192.168.1.1) (Process ID 1 VRF default)


Type-5 AS External Link States

    LS age: 853
   Options: 0x2 (No TOS-capability, No DC)
   LS Type: Type-5 AS-External
   Link State ID: 172.16.1.0 (Network address)
   Advertising Router: 192.168.1.1         >>>>> Since Type 7 is installed
 in the RIB, it was converted to type 5
   LS Seq Number: 0x80000001
   Checksum: 0xb545
   Length: 36
   Network Mask: /24
         Metric Type: 2 (Larger than any link state path)
         TOS: 0<
         Metric: 20
         Forward Address: 192.168.45.4
         External Route Tag: 0<

   LS age: 596
   Options: 0x20 (No TOS-capability, DC)
   LS Type: Type-5 AS-External
   Link State ID: 172.16.1.0 (Network address)
   Advertising Router: 192.168.3.3              >>>>>>  Type 5 is also received from R3
   LS Seq Number: 0x80000002
   Checksum: 0x2250
   Length: 36
   Network Mask: /24
         Metric Type: 2 (Larger than any link state path)>
         TOS: 0
         Metric: 20<>
         Forward Address: 0.0.0.0
         External Route Tag: 0


R1-NXOS# sh ip route 172.16.1.0
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.1.0/24, ubest/mbest: 1/0
    *via 192.168.16.6, Eth4/44, [110/20], 00:16:54, ospf-1, nssa type-2     >>>> Type 7
 route is installed in RIB.

Since R1 does not have the rfc1583compatibility command configured under the OSPF router process and the route's Type 5 Link State Advertisement's (LSA's) adv-router-id is reachable in area 0 (backbone router), OSPF always picks up the path for the route via the non-backbone area. In this case the next-hop is chosen in area 2 (as per RFC 2328).

R1-NXOS(config)# router ospf 1
R1-NXOS(config-router)# rfc1583compatibility

R1-NXOS# sh ip route 172.16.1.0
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.1.0/24, ubest/mbest: 1/0
    *via 192.168.12.2, Eth4/43, [110/20], 00:00:04, ospf-1, type-2     >>>> Type 5
 route is installed in RIB.

Recommendation

There are other design or network scenarios wherein this compatibility issue can cause loops or suboptimal routing in the network if the network has NXOS and Cisco IOS that run together with OSPFv2.

Cisco recommends to use the RFC 1583 compatibility command in the NXOS OSPF router configuration mode if the network includes devices which only support RFC1583, that is Cisco IOS.

Verify

There is currently no verification procedure available for this configuration.

Troubleshoot

There is currently no specific troubleshooting information available for this configuration.

Configure VxLAN Flood And Learn Using Multicast Core

Introduction

This document describe how to configure and verify Virtual Extensible LAN (VXLAN) flood and learn mode over IPv4 Multicast transport. VXLAN is designed to provide the same Ethernet Layer 2 network services as VLAN. VXLAN encapsulates MAC address over UDP packet  which makes a Layer 2 packet carried over a Layer 3 network. So, it is basically a MAC-in-UDP header.

VXLAN introduces an 8 byte VXLAN header that consists of a 24 bit VNID and a few reserved bits. The VXLAN header together with the original Ethernet frame goes in the UDP payload. The 24 bit VNID is used to identify Layer 2 segments and to maintain Layer 2 isolation between th e segments. With all 24 bits in VNID, VXLAN can support 16 million LAN segments. So it resolves the issue of limitation of VLAN. Without VxLAN we can have only 4094 no of vlan, with increased demand modern networks need more vlans , and VXLAN is solution to address the issue .

Since it uses the ethernet frame to encapsulate the packet, so ethernet properties need to remain intact like broadcase , unknown unicast and multicast. To address these type of traffic multicast is being used. In this document VxLAN flood and learn is being described. As the name specify that it will flood the packet and will learn the remote end. It means that data-plane is not up all the times, as soon as traffic will flow data-plane will be built up and will expire as soon as mac address expires. 

Prerequisites

Cisco recommends that you have knowledge of basic IP multicast.

Packet Format of VXLAN

As shown in above figure original frame is being encasulated in VxLAN header which is of 8 byte and VNID is of 24 bit. That is further encapsulated in UDP header and outer header would be an IP header.

Source IP address is IP of encapsulating VTEP (Virtual Terminal End Point) and destination IP either would be a multicast or unicast one. VXLAN uses VXLAN tunnel endpoint (VTEP) devices to map tenants's end devices to VXLAN segments and to perform VXLAN encapsulation and de-encapsulation. Each VTEP has two interfaces: One is a switch interface on the local LAN segment to support local endpoint communication through bridging, and the other is an IP interface to the transport IP network.

Remote VTEP  Discovery

When the host will start sending the traffic below process will happen. At this moment VTEP does not know the mac address of the remote host .

1. End station will send ARP packet for remote end station.
2. Packet will reach to VTEP-A and since VTEP-A does not know about the VTEP-B , it will encapsulate the packet inside VxLAN header. It will put the multicast IP address as the destination IP address. Since the same multicast address will be used by all VTEPs, all will join that same multicast group.
3. This packet will reach to all VTEP and will be decapsulated, in this way remote VTEP will learn about the other VTEP. Since the decapsulated VTEP will be having the VNID, it will be forwarded in the vlan having the same VNID configured.
4. Now remote end will send the ARP reply packet and it will reach to VTEP-B, since now VTEP-B knows about VTEP-A it will again encapsulated the orginal frame but now the destination IP address will be of VTEP-B and it will be unicast IP address .
5. ARP reply will reach to VTEP-A and now VTEP-A gets to know about VTEP-B it will form the neighbor relationship with VTEP-B .

As shown in the diagram Host H1 belongs to vlan 10 and being encapsulted in VNID 10000.  As shown above SMAC with H1 and DMAC with H2 is being encapsulated inside VNI 1000 and Source IP and Destination IP could be multicast or unicast described in above section .

Components Used

This document is specific to Nexus platform.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Configure

Network Diagram

• 9396-A and 9396-B are the VPC peers considerd as VTEP-1.
• 9396-C is the VTEP-2.
• Above diagram is having two host in vlan 10 i.e. 10.10.10.1 and 10.10.10.2.
• VLAN 10 is being used with VNID as 10010.
• 230.1.1.1 is being used as multicast group.

Configurations

To enable VXLAN on Nexus we need to enable the below feature

9396-A config

!
feature vn-segment-vlan-based
feature nv overlay 
!
vlan 10
  vn-segment 10010   ------> 10010 is VNID
! 
interface nve1
  no shutdown
  source-interface loopback0
  member vni 10010 mcast-group 230.1.1.1 
! 
interface eth1/2
!
ip pim sparse-mode
! 
interface loopback0
  ip address 10.1.1.1/32
  ip address 10.1.1.10/32 secondary
  ip router ospf 9k area 0.0.0.0
  ip pim sparse-mode 
!

!
feature vpc
!
vpc domain 1
  peer-switch
  peer-keepalive destination 10.31.113.41 source 10.31.113.40
  peer-gateway
!
interface port-channel1
  vpc peer-link
!
interface port-channel112
  vpc 112
!

9396-B Config
!
vlan 10
  vn-segment 10010   ------>    10010 is VNID
! 
interface nve1
  no shutdown
  source-interface loopback0
  member vni 10010 mcast-group 230.1.1.1 
!
interface eth1/2
ip pim sparse-mode
!
interface loopback0
  ip address 10.1.1.2/32
  ip address 10.1.1.10/32 secondary
  ip router ospf 9k area 0.0.0.0
  ip pim sparse-mode 
!
feature vpc
!
vpc domain 1
  peer-switch
  peer-keepalive destination 10.31.113.40 source 10.31.113.41
  peer-gateway
!
interface port-channel1
  vpc peer-link
!
interface port-channel112
  vpc 112
!

9508-A Config
feature pim

ip pim rp-address 10.1.1.5 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8


interface loopback0
  ip pim sparse-mode

interface Ethernet5/2
  ip pim sparse-mode

interface Ethernet5/3
  ip pim sparse-mode

interface Ethernet5/4
  ip pim sparse-mode

9396-C Config
!
vlan 10
  vn-segment 10010
!
interface loopback0
  ip address 10.1.1.3/32
  ip router ospf 9k area 0.0.0.0
  ip pim sparse-mode
!
interface nve1
  no shutdown
  source-interface loopback0
  member vni 10010 mcast-group 230.1.1.1
!
int eth1/2
ip pim sparse-mode
!

Verify

As of now host has not started sending the packet stream. Since 9396-A is a VPC holding device it will originates the traffic sourcing from the secondary IP address and will act as source IP address for multicast stream.
9396-A# sh nve interface
Interface: nve1, State: Up, encapsulation: VXLAN
 VPC Capability: VPC-VIP-Only [notified]
 Local Router MAC: d8b1.9076.9053
 Host Learning Mode: Data-Plane
 Source-Interface: loopback0 (primary: 10.1.1.1, secondary: 10.1.1.10)

9396-A# sh ip mroute 230.1.1.1IP Multicast Routing Table for VRF "default"

(*, 230.1.1.1/32), uptime: 01:09:34, ip pim nve
  Incoming interface: Ethernet1/2, RPF nbr: 192.168.10.2
  Outgoing interface list: (count: 1)
    nve1, uptime: 00:11:20, nve


(10.1.1.3/32, 230.1.1.1/32), uptime: 00:12:19, ip mrib pim nve
  Incoming interface: Ethernet1/2, RPF nbr: 192.168.10.2
  Outgoing interface list: (count: 1)
    nve1, uptime: 00:11:20, nve

(10.1.1.10/32, 230.1.1.1/32), uptime: 00:11:20, nve ip mrib pim
  Incoming interface: loopback0, RPF nbr: 10.1.1.10
  Outgoing interface list: (count: 1)
    Ethernet1/2, uptime: 00:11:20, pim


In *,G entry nve interface is popultaed in OIL (Outgoing Interface List).  As shown above 10.1.1.10 is source of multiacast stream and nve interface will be the Last hop router for the multicast stream with eth1/2 facing towards core is outgoing interface.

As there is no traffic flowing from host so there are no nve peers:
9396-A# show  mac address-table vlan 10Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*   10     8c60.4f93.5ffc   dynamic  0         F      F    Po112   >> This mac is for host 10.10.10.1


9396-A# sh nve peers
Interface Peer-IP          State LearnType Uptime   Router-Mac
--------- ---------------  ----- --------- -------- -----------------

Following output shows how vPC output should look like:
9396-A# sh vpc brief
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1
Peer status                       : peer adjacency formed ok
vPC keep-alive status             : peer is alive
Configuration consistency status  : success
Per-vlan consistency status       : success
Type-2 consistency status         : success
vPC role                          : primary
Number of vPCs configured         : 1
Peer Gateway                      : Enabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Disabled
Delay-restore status              : Timer is off.(timeout = 30s)
Delay-restore SVI status          : Timer is off.(timeout = 10s)

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans
--   ----   ------ --------------------------------------------------
1    Po1    up     1-10

vPC status
----------------------------------------------------------------------
id   Port   Status Consistency Reason                     Active vlans
--   ----   ------ ----------- ------                     ------------
112  Po112  up     success     success                    1-10

9396-A# sh vpc consistency-parameters global

    Legend:
        Type 1 : vPC will be suspended in case of mismatch

Name                        Type  Local Value            Peer Value
-------------               ----  ---------------------- -----------------------Vlan to Vn-segment Map      1     1 Relevant Map(s)      1 Relevant Map(s)
STP Mode                    1     Rapid-PVST             Rapid-PVST
STP Disabled                1     None                   None
STP MST Region Name         1     ""                     ""
STP MST Region Revision     1     0                      0
STP MST Region Instance to  1
 VLAN Mapping
STP Loopguard               1     Disabled               Disabled
STP Bridge Assurance        1     Enabled                Enabled
STP Port Type, Edge         1     Normal, Disabled,      Normal, Disabled,
BPDUFilter, Edge BPDUGuard        Disabled               Disabled
STP MST Simulate PVST       1     Enabled                Enabled
Nve Admin State, Src Admin  1     Up, Up, 10.1.1.10, DP  Up, Up, 10.1.1.10, DP
 State, Secondary IP, Host
 Reach Mode
Nve Vni Configuration       1     10010                  10010
Nve encap Configuration     1     vxlan                  vxlan
Interface-vlan admin up     2
Interface-vlan routing      2     1                      1
capability
Allowed VLANs               -     1-10                   1-10
Local suspended VLANs       -     -                      -

9508-A :

Since 9508-A route is core router, it will not be aware about the VXLAN , it will be aware about the mroute entry only as shown below:
9508-A# sh ip mroute 230.1.1.1
IP Multicast Routing Table for VRF "default"

(*, 230.1.1.1/32), uptime: 01:30:06, pim ip
  Incoming interface: loopback0, RPF nbr: 10.1.1.5, uptime: 01:30:06
  Outgoing interface list: (count: 3)
    Ethernet5/3, uptime: 00:14:11, pim
    Ethernet5/2, uptime: 00:14:31, pim
    Ethernet5/4, uptime: 00:16:22, pim

(10.1.1.3/32, 230.1.1.1/32), uptime: 00:15:44, pim mrib ip
  Incoming interface: Ethernet5/4, RPF nbr: 192.168.10.10, uptime: 00:15:44, internal
  Outgoing interface list: (count: 2)
    Ethernet5/3, uptime: 00:14:11, pim
    Ethernet5/2, uptime: 00:14:31, pim

(10.1.1.10/32, 230.1.1.1/32), uptime: 00:14:31, pim mrib ip
  Incoming interface: Ethernet5/2, RPF nbr: 192.168.10.1, uptime: 00:14:31, internal
  Outgoing interface list: (count: 1)
    Ethernet5/4, uptime: 00:14:31, pim

9396-C
9396-C# show ip mroute
IP Multicast Routing Table for VRF "default"

(*, 230.1.1.1/32), uptime: 01:07:34, ip pim nve
  Incoming interface: Ethernet1/2, RPF nbr: 192.168.10.9
  Outgoing interface list: (count: 1)
    nve1, uptime: 00:10:38, nve


(10.1.1.3/32, 230.1.1.1/32), uptime: 00:10:38, nve ip mrib pim
  Incoming interface: loopback0, RPF nbr: 10.1.1.3
  Outgoing interface list: (count: 1)
    Ethernet1/2, uptime: 00:09:49, pim

(10.1.1.10/32, 230.1.1.1/32), uptime: 00:08:05, ip mrib pim nve
  Incoming interface: Ethernet1/2, RPF nbr: 192.168.10.9
  Outgoing interface list: (count: 1)
    nve1, uptime: 00:08:05, nve

Status after traffic flow startes between peers

As soon as host 1 i.e. 10.10.10.1 starts sending the traffic to 10.10.10.2 NVE peer comes up:
9396-A# sh mac address-table dynamic
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*   10     8c60.4f93.5ffc   dynamic  0         F      F    Po112
+   10     8c60.4f93.647c   dynamic  0         F      F    nve1(10.1.1.3)


9396-A# sh nve peers
Interface Peer-IP          State LearnType Uptime   Router-Mac
--------- ---------------  ----- --------- -------- -----------------
nve1      10.1.1.3         Up    DP        00:00:14 n/a


9396-A# sh nve peers detail
Details of nve Peers:
----------------------------------------
Peer-Ip: 10.1.1.3
    NVE Interface       : nve1
    Peer State          : Up
    Peer Uptime         : 00:04:49
    Router-Mac          : n/a
    Peer First VNI      : 10010
    Time since Create   : 00:04:49
    Configured VNIs     : 10010
    Provision State     : add-complete
    Route-Update        : Yes
    Peer Flags          : None
    Learnt CP VNIs      : --
    Peer-ifindex-resp   : Yes
----------------------------------------

9396-A sh nve vni 10010 detail 
VNI: 10010 
  NVE-Interface       : nve1
  Mcast-Addr          : 230.1.1.1
  VNI State           : Up
  Mode                : data-plane
  VNI Type            : L2 [10]
  VNI Flags           :   
  Provision State     : add-complete
  Vlan-BD             : 10
  SVI State           : n/a

9396-A# sh nve internal vni 10010
VNI 10010
  Ready-State         : Ready [L2-vni-flood-learn-ready]

Similarly on 9396-C  NVE peers should be up:
9396-C# show mac address-table dynamic
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*   10     8c60.4f93.5ffc   dynamic  0         F      F    nve1(10.1.1.10)
*   10     8c60.4f93.647c   dynamic  0         F      F    Eth1/13

9396-C# sh nve peers
Interface Peer-IP          State LearnType Uptime   Router-Mac
--------- ---------------  ----- --------- -------- -----------------
nve1      10.1.1.10        Up    DP        00:08:28 n/a

9396-C# sh nve peers detail
Details of nve Peers:
----------------------------------------
Peer-Ip: 10.1.1.10
    NVE Interface       : nve1
    Peer State          : Up
    Peer Uptime         : 00:08:32
    Router-Mac          : n/a
    Peer First VNI      : 10010
    Time since Create   : 00:08:32
    Configured VNIs     : 10010
    Provision State     : add-complete
    Route-Update        : Yes
    Peer Flags          : None
    Learnt CP VNIs      : --
    Peer-ifindex-resp   : Yes
----------------------------------------

9396-C sh nve vni 10010 detail 
VNI: 10010 
  NVE-Interface       : nve1
  Mcast-Addr          : 230.1.1.1
  VNI State           : Up
  Mode                : data-plane
  VNI Type            : L2 [10]
  VNI Flags           :   
  Provision State     : add-complete
  Vlan-BD             : 10
  SVI State           : n/a

9396-C# sh nve internal vni 10010
VNI 10010
  Ready-State         : Ready [L2-vni-flood-learn-ready]


As shown in above, nve peers are based upon data-plane learning and it uses flood and learn mechanism. In case mac address gets timed out nve peer will go down.