Saturday, February 12, 2011

Spanning tree in a Nexus virtualized data center

Nexus VPC reduces the reliance on Spanning Tree in a data center design, and improves link utilization and load sharing. However, spanning tree is still a necessary component of the design, often seemingly more complex, due to various options available, and connection to third party devices.

There was an earlier post about Bridge Assurance. This note provides a more complete reference model (R-PVST), to capture recommended best practices in a typical design, followed by notes explaining why some of the choices made.
Click to see larger image.

Nexus 7000 VPC peer link and non-VPC link

Enable BA with “spanning-tree port-type network” on both ends of trunk (port channel)

Why Bridge Assurance: new Cisco STP feature works in conjunction with Rapid-PVST BPDUs to protect from bridging loops (also supported by MST). BA must be supported by and configured on both switches on a point to point link, otherwise blocking will occur. BA uses bidirectional hello to prevent looping conditions caused by unidirectional links or a malfunctioning switch.  If a BA port stops receiving BPDUs, the port is moved into the blocking state.

Nexus 7000 connection to third party Load Balancer or Firewall devices

Enable port fast with “spanning-tree port-type edge trunk”

Why Port Fast: This is for third party devices which can be treated like hosts connected to access (they do not send BPDUs). Setting port-type to edge enables Port Fast which allows access port to enter the forwarding state immediately, instead of waiting for STP to converge. These ports should not receive bridge protocol data units (BPDUs), otherwise they will immediately transition to the blocking state. The trunk keyword enables edge behavior on a trunk port.

Why BPDU Guard: BPDU Guard works together with Port Fast on edge ports. In a valid design, edge ports should not receive BPDUs. Reception of a BPDU indicates an error in configuration, such as connection of an unauthorized device.  By shutting down a port that receives a BPDU, BPDU Guard protects the network, since only an administrator can put the edge port back in service.

Nexus 7000 downlink to Nexus 5000 access switches (back to back VPC)

Enable root guard on “spanning-tree port-type normal”, note this is applied on VPC

Why Root Guard: Note Nexus 7000s are SPT root. Root Guard is a feature placed on aggregation port facing access switches, preventing it from becoming a root port. In this case it prevents Nexus 5000 to become a SPT root switch, to ensure that a configuration error on an access layer switch does not cause STP disruption and instability.

SPT on VPC is a topic in itself. Cisco has a good document here with more details. Note bridge assurance is usually not needed, since VPC consistency check ensures the integrity of configurations.

Nexus 5000 uplink to Nexus 7000 (back to back VPC)

Enable loop guard on “spanning-tree port-type normal”, note this is applied on VPC

Why Loop Guard: For additional protection against loops, Loop Guard is enabled on root and alternate ports (facing root bridge). When Loop Guard on Nexus 5000 detects that BPDUs are no longer being received, the port is moved into a loop-inconsistent state instead of transitioning through SPT convergence. This will break a Layer 2 loop immediately due to misconfiguration or unidirectional link.

Nexus 5000 connection to hosts

Enable port fast with “spanning-tree port-type edge trunk”

Why Port Fast: see above, same as Nexus 7000 connection to third party.

Why BPDU Guard: see above, same as Nexus 7000 connection to third party.

A final note, Cisco Nexus 1000V does not run SPT, it does not generate BPDUs, nor does it respond to them. Nexus 1000V examines source and destination MAC address to prevent loops. This is the reason Nexus 5000 access port connected to Nexus 1000v virtual switch should be set to type edge, just like an access port connected to a physical host.

No comments:

Post a Comment