This document describes the characteristics and features of the Cisco® Nexus 7000 Series platform of specific interest to service providers, particularly software features found in the Cisco NX-OS Software, the new operating system for the Cisco Nexus solution. This guide points out how these features benefit service providers, particularly in their application rollouts, day-to-day operations, and provisioning.
Challenges
Service providers today face many challenges, from controlling expenses to rolling out new services to delivering service innovation. Many of these challenges can be addressed by the assets found in the service provider data center. Intelligent planning for the future of the data center means understanding where the challenges lie and what types of products can address them. This document summarizes the types of challenges that service providers face and the characteristics and features of a data center-class platform solution that can help meet those challenges.
Cisco Nexus 7000 Series
The Cisco Nexus 7000 Series of switches is a modular data center-class product line designed for highly scalable 10 Gigabit Ethernet networks with a fabric architecture that scales beyond 15 terabits per second (Tbps) and provides future support for 40 Gbps and 100 Gbps Ethernet. This new data center-class platform is designed for outstanding scalability, continuous system operation, operational manageability, and transport flexibility. The Cisco Nexus 7000 Series is powered by Cisco NX-OS, a state-of-the art operating system. Although this solution is not initially positioned for Internet-facing applications, the richness of the Cisco NX-OS Software and the Cisco Nexus 7000 Series hardware platform makes it well worth investigating for hosting, co-location, and other allied service provider data center applications
1.
Figure 1 is a reference diagram showing a classic service provider data center: a service delivery center; Figure 2 summarizes the Cisco data center products and provides a guide showing what the icons in Figure 1 represent.
Figure 1. Service Delivery Center (SDC) Baseline Architecture: Base Physical View
Figure 2. Cisco Data Center Portfolio
Cisco NX-OS
Cisco NX-OS is a strategic, data center-class operating system built with modularity, resilience, and serviceability at its foundation. Based on the industry-leading Cisco MDS 9000 SAN-OS Software, it sets the standard for mission-critical environments in the areas of virtualization, availability, serviceability, manageability, scalability, and security.
Purpose built for of the data center, Cisco NX-OS provides a robust and rich feature set that fulfills the routing, switching, and storage networking requirements of present and future data centers. With an Extensible Markup Language (XML) interface and a command-line interface (CLI) similar to that of the Cisco IOS® Software, Cisco NX-OS provides state-of the-art implementations of relevant networking standards as well as a variety of true data center-class Cisco innovations to benefit service providers.
Cisco Data Center Network Manage (DCNM)
DCNM is a comprehensive administration solution dedicated to data center network operations. Cisco DCNM offers such benefits as multiprotocol awareness, full network service life cycle administration with emphasis on provisioning, performance and assurance.
Table 1 lists the specific challenges service providers face in operating their data centers, the desired characteristics of potential solutions to meet these challenges, and the specific Cisco NX-OS and Cisco Nexus 7000 features that address these needs. The following section describes these features in detail and their operational benefits to service providers.
Table 1. Challenges That the Service Provider Data Center Solution Needs to Meet
Challenges
| Solution Characteristics Needed
| Cisco Nexus 7000 Series Solution Features That Address Need
|
| • Support more demanding service-level agreements (SLAs) for security and uptime • Be able to isolate system faults so they do not affect other processes • Bring a variety of services to market more quickly • Support multiple customers with significantly different response needs and limited capital expenditure (CapEx) budget for new equipment acquisition • Support complex heterogeneous topologies | Service velocity and isolation
| • Virtual device contexts (VDCs) • VDCs, with five degrees of virtualization: data plane, control plane, management plane, hardware, and software • Integrated virtualization support for Layers 2 and 3 • Future unified I/O support capability |
| • Maintain network health, control, and transparency • Efficiently and quickly understand protocol-level network problems • Build remote, automated ("lights-out") data center facility • Achieve accurate billing and SLA management • Avoid overdependence in the future on the timing of supplier MIB releases | Network and SLA control
| • Cisco DCNM holistic element and fabric provisioning, which consolidates Layer 1 to 3 LAN and storage area network (SAN) management (future) • Configuration verification and rollback • Hierarchical CLI similar to that of Cisco IOS Software • Role-based access control (RBAC) • Integrated control plane packet analyzer • Connectivity management processor for integrated lights-out management • Billing support through Cisco NetFlow Versions 5 and 9 (512,000-entry table support with true packet sampling); statistics available for use by third-party software to perform billing and are the same tools as those for Cisco CRS-1 and Gigabit switch router (GSR) solutions • Programmatic XML support |
| • Address applications heterogeneity and the rise of the empowered end user • Provide thorough support for video and other time-sensitive application distribution and updates | Thorough application delivery support, such as support for real-time video
| • Virtual output queuing (VOQ) and fabric arbitration for enhanced internal congestion management • Industry-leading multicast implementation • Support for unicast and multicast types, including Internet Group Management Protocol (IGMP), IGMPv2, Protocol Independent Multicast (PIM; Sparse and Bidirectional modes), Source Specific Multicast (SSM), and Multicast Source Discovery Protocol (MSDP) for efficient video delivery • Quality of service (QoS) • Lossless fabric, which prevents drops and optimizes handling of drop-sensitive traffic |
Comply with nonnegotiable regulatory requirements
| Regulatory support
| • Network Equipment Building Standards (NEBS) compliance • Integration of latest industry standards for routing and switching, such as graceful protocol restart |
Meet critical network and data safety needs
| Pervasive security and self-defending network
| • Cisco TrustSec • RBAC • VDCs • Link-layer encryption • Control plane policing (CoPP) • Other Cisco integrated security features (see details in "Pervasive Security" section later in this document) |
Meet customer SLA expectations for an always-on network
| Outstanding availability
| • Continuous system operation • OS modularity • Cisco In Service Software Upgrade (ISSU) • Process survivability and modularity • Process modular patching capability • Rapid and stateful supervisor failover • Reliable interprocess communication (IPC) • Redundant switched Ethernet out-of-band channels (EOBCs) • Graceful operations (future) • Network-based availability |
Efficiently build and deploy a massively scalable data center
| Hardware and software scalability
| • Designed to scale: • Distributed, multithreaded OS on symmetric multiprocessors (SMPs) • Multicore CPUs • Distributed line card processors • Control and data plane separation • Ethernet switching feature richness • IP and routing feature richness • Designed for density: • High-density 10/100/1000 Gigabit Ethernet (48-port line card- up to 768 ports per system) and 10 Gigabit Ethernet (32-port line card- up to 512 ports per system) design. • Fabric capacity of 230 Gbps per slot on day 1; built for a capacity of more than 15 Tbps • Intelligent CLI management tools to consolidate entries, designed for application control lists (ACLs) with tens of thousands of lines |
Address the needs of the data center with widely dispersed assets and fewer personnel to provide monitoring and service
| Serviceability
| • Switched Port Analyzer (SPAN) • Embedded packet analyzer • Smart Call Home • Cisco Generic Online Diagnostics (GOLD) • Cisco IOS Embedded Event Manager (EEM) |
Cisco Nexus 7000 Series Solution
The Cisco Nexus 7000 Series solution provides numerous features to address the challenges of the service provider data center.
Service Velocity and Isolation
• VDCs: VDCs enable the data center manager to partition both the software and the hardware, providing software fault containment (because each VDC runs independent processes) and the capability to maximize the port utilization of a device. Each VDC looks and feels like a separate physical device. A service provider can thus test one or more new services using a VDC in an existing Cisco Nexus platform before deployment to verify that the service is ready for deployment; the service provider can then deploy the service without risk of affecting existing services.
• Virtualization: All features are designed to be VDC or Virtual Route Forwarding (VRF) aware from the start, helping ensure the capability to provide virtualized services as they become required features in the networks of tomorrow.
• Comprehensive feature set: A fully functional and comprehensive Layer 2 and 3 feature set focuses on the requirements of the data center.
• Unified I/O: The Cisco Nexus 7000 Series is designed to support multiple interfaces (Fibre Channel over Ethernet and classic Ethernet) in one switching fabric, providing unified I/O capability and simplifying service provider architectures and design implementations.
Network and SLA Control
• Programmatic XML interface: Based on the NETCONF industry standard, the Cisco NX-OS XML interface provides a consistent API for the device, enabling rapid development and engineering of tools to enhance the network.
• Cisco DCNM: Cisco DCNM is a Cisco network management application that maximizes the overall data center infrastructure uptime and reliability, thereby enabling business continuity. Cisco DCNM automates the provisioning process, proactively surveys the SAN and LAN networks by detecting and preventing outages, secures the network, and streamlines the diagnosis of dysfunctional network elements. This Java-based software includes a fully automated, complete, and trustable discovery and resynchronization process that abstracts the network. The end user is completely shielded from infrastructure changes. The product API greatly simplifies data center operations, allowing accurate flow-through provisioning and monitoring.
• Configuration verification: The system administrator can verify the configuration and available hardware resources prior to applying the configuration, enabling the administrator to preconfigure the device and apply the configuration at a specific time, while helping ensure that the configuration is correct and that the appropriate hardware resources are available.
• CLI similar to that of Cisco IOS Software: Cisco NX-OS uses the industry-standard Cisco IOS Software CLI to minimize the amount time needed for administrators to learn the system and become operationally proficient.
• Configuration rollback: Rollback allows checkpointing of the configuration so that the system administrator can roll back operations to a known good configuration if needed.
• Connectivity management processor (CMP): The CMP provides lights-out management capabilities, in many cases eliminating the need for terminal servers.
• RBAC: With RBAC, Cisco NX-OS limits access to switch operations by assigning roles to users, thus allowing administrators to restrict and customize access to those users who require it. This feature is ideal for service providers with multiple customer sets and individual SLAs.
• Cisco NetFlow: Extensible hardware-based per-flow accounting based on Cisco NetFlow 9.0 allows service providers to use Cisco NetFlow statistics and export to third-party billing applications to prove that SLAs have been met and can be billed for.
Thorough Application Delivery Support
• VOQ and fabric arbitration: This queuing and arbitration method enable fairness in delivery of content such as video when a destination is congested (for example, uplinks and many-to-one flow) and fair sharing of resources (10 Gigabit Ethernet and Gigabit Ethernet). These features increase crossbar efficiency by avoiding blocking within the crossbar switch fabric itself- any congestion is moved to the ingress port, maximizing buffer utilization.
• Industry-leading IP Multicast feature set: The Cisco NX-OS 4.0 implementation lays the foundation for the future development of a rich portfolio of multicast-enabled network functions. As with the unicast routing protocols, Cisco NX-OS 4.0 includes state-of-the-art implementations of the following multicast protocols and functions:
– PIMv2
– SSM
– PIM Sparse mode (Any Source Multicast [ASM])
– Bidirectional Protocol Independent Multicast (Bidir-PIM)
– Anycast Rendezvous Points (RP)
– Efficient multicast replication
– Multicast Nonstop Forwarding (NSF) for IPv4 and IPv6
– RP Discovery using Bootstrap Router (BSR), Auto RP, and Static mode
– IGMPv1, v2, and v3 router roles
– IGMPv2 host mode
– IGMP snooping
– Multicast Listener Discovery (MLD) Protocol Version 2 for IPv6
– MSDP (for IPv4 only)
• Abandonment of obsolete functions such as PIM Dense mode: Cisco NX-OS is a forward-looking the operating system.
• QoS: Cisco NX-OS Software supports a rich variety of QoS mechanisms, including classification, marking, queuing, policing, and scheduling. The Modular QoS CLI (MQC) and Cisco Common Classification Policy Language (C3PL) compliant CLI are supported for all QoS features. The MQC and C3PL CLI can be used to provide uniform configurations across multiple Cisco platforms.
• Lossless fabric: The lossless fabric prevents drops and optimizes handling of drop-sensitive traffic; for example, from an SP distributing IP video streams to many customers simultaneously.
Regulatory Support
• Integration of today's latest industry standards for routing, switching, and availability with mechanisms such as graceful protocol restart for improved stability and reduced operational complexity for customers
• Rich Standards support: for a list of all IEEE, IETF, and RFC standards supported, please see the Cisco NX-OS Software Release 4.0 data sheet
• NEBS-compliant SR-3580 NEBS Level 3 (GR-63-CORE, issue 3, and GR-1089-CORE, issue 4)
• Front-to-back airflow and integrated cable management
• Grid redundancy, so a service provider can connect to dual power sources and have resilience
• Common equipment (fans, power supply units [PSUs], and fabric modules) all removed from the rear; no disruption to the cables on the user side
• Optional air filter
Pervasive Security
• Cisco TrustSec: As part of the Cisco TrustSec security suite, Cisco NX-OS provides exceptional data confidentiality and integrity, supporting industry-standard IEEE 802.1AE link-layer cryptography with 128-bit Advanced Encryption Standard (AES) cryptography. Link-layer cryptography helps ensure end-to-end data privacy while allowing the insertion of security service devices along the encrypted path. Security group ACLs (SGACLs), a new paradigm in network access control, are based on security group tags (SGTs) instead of IP addresses, enabling policies that are more concise and easier to manage because of their topology independence-resulting in simpler ACLs by decoupling of addressing from policy.
• RBAC: RBAC limits access to switch operations by assigning roles to users, allowing the administrator to restrict and customize access to those users who require it, giving the service provider flexibility in customizing per-user access.
• CoPP: CoPP limits the rate at which specific traffic can reach the CPU, helping prevent denial-of-service (DoS) attacks from affecting the CPU capacity of the network device.
• Cisco Integrated Security Solution features: Cisco offers a comprehensive suite of security features to prevent spoofing of network hosts and traffic snooping. The combination of dynamic Address Resolution Protocol (ARP) inspection and IP Source Guard mitigates distributed DoS (DDoS) attacks as well as data and voice snooping.
• Other Cisco Integrated Security Solution features of interest include the following:
– Authentication, authorization, and accounting (AAA) and TACACS+
– Protocol conformance checks
– Secure Shell (SSH) Protocol Version 2
– Simple Network Management Protocol (SNMP) Version 3 support
– Port security
– IEEE 802.1x authentication and RADIUS support
– Layer 2 Cisco Network Access Control (NAC) and LAN-port-IP
– Named ACLs: Port ACLs (PACLs), VLAN ACLs (VACLs), and router ACLs (RACLs) support policies based on MAC IPv4 and IPv6 addresses
Outstanding Availability
• Continuous system operation: Cisco NX-OS provides continuous system operation, permitting maintenance, upgrades, and software certification while providing zero service disruption. The combination of process modularity, modular patching, Cisco ISSU, and NSF minimizes the negative effects of software upgrades and other operations.
• Modularity: Cisco NX-OS, based on the industry-leading Cisco MDS 9000 SAN-OS Software, provides exceptional scalability, high availability, service isolation, and manageability, meeting the critical requirements of next-generation data centers.
• Cisco ISSU: Cisco ISSU provides the capability to perform transparent software upgrades on platforms with redundant supervisors, minimizing downtime and allowing customers to integrate the newest features and functions with little or no effect on network operation.
• Process survivability: Critical processes are run in protected memory space and independently from each other and the kernel, enabling service isolation, fault containment, modular patching and upgrades, and rapid restart. Processes can be restarted independently without loss of state information and without affecting data forwarding; processes can restart after an upgrade or failure, in microseconds, without negative effects on adjacent devices or services. Highly stateful processes such as IP routing protocols are restarted using standards-based NSF graceful restart mechanisms, and other processes use a local persistent storage service (PSS) to maintain their state.
• Process modularity: Critical processes are run in protected memory space and independently from each other and the kernel, enabling service isolation, fault containment, modular patching and upgrades, and rapid restart.
• Process modular patching: The modular architecture allows modular patching and upgrading of process-specific code, enabling faster implementation of features.
• Stateful supervisor failover: Redundant supervisors are kept synchronized at all times to enable stateful supervisor failover in less than a second. Sophisticated checks help ensure that the state is consistent and reliable throughout the entire distributed architecture after a failover.
• Reliable IPC: IPC facilitates reliable communication between processes to help ensure that all messages are delivered and properly acted on during failures and other adverse conditions.
• Redundant switched EOBCs: Cisco NX-OS can make full use of redundant switched EOBCs for communications between control plane and line card processors.
• Graceful system operations (future): Cisco NX-OS allows graceful insertion and removal of network elements (line cards, ports, processes, entire devices, etc.).
• Always-on operation: Cisco NX-OS is built on top of the industry-leading Cisco MDS 9000 SAN-OS Software operating system, which is designed to run data center SAN fabrics that run operations twenty-four hours a day.
• Network-based availability: Network convergence is optimized through tools and functions that make failover and fallback transparent and fast. Functions provided in Cisco NX-OS include Spanning Tree Protocol enhancements such as PortFast Bridge Protocol Data Unit (BPDU) Guard, Loop Guard, Root Guard, BPDU filters, and bridge assurance to help ensure the health of the Spanning Tree Protocol control plane; Unidirectional Link Detection (UDLD) Protocol; NSF graceful restart of routing protocols; millisecond timers for routing and first-hop resiliency protocols; shortest-path first (SPF) optimizations such as link-state advertisement (LSA) pacing and incremental SPF; and IEEE 802.3ad link aggregation with adjustable timers.
Hardware and Software Scalability
• Cisco NX-OS is a multithreaded, distributed OS based on an SMP control plane, allowing the OS to provide optimal performance with minimal wasted CPU cycles.
• Cisco NX-OS modular processes are instantiated on demand, with each in a separate protected memory space; thus, processes are started and system resources allocated only when a feature is enabled. The modular processes are governed by a real-time preemptive scheduler, which helps ensure the timely processing of critical functions. This modularity allows the control plane to scale and support very large system configurations and rich topologies.
• The control and data planes are separated, adding flexibility and helping ensure that there are no unnecessary co-dependencies and loss of performance on control plane and data plane processes.
• Distributed line card processors enable computationally intensive tasks, such as hardware table programming, to be offloaded to dedicated processors distributed across the line cards, further speeding processing.
• Rich Ethernet switching features are supported. Cisco NX-OS is built to support high-density, high-performance Ethernet systems and provides a complete data center-class Ethernet switching feature set. The feature set includes IEEE 802.1D-2004 Rapid Spanning Tree (RST) and Multiple Instance Spanning Tree (MST) Protocols, IEEE 802.1Q VLANs and trunks, support for 16,000 VLANs, IEEE 802.3ad link aggregation, private VLANs, cross-chassis private-VLANs, UDLD Protocol in aggressive and standard modes, traffic suppression (unicast, multicast, and broadcast), transparent ISSU in Spanning Tree Protocol environments, BPDU Guard, Loop Guard, Root Guard, BPDU filters, bridge assurance, and jumbo frame support.
• Rich IP and routing features are supported. Cisco NX-OS supports a wide range of IPv4 and v6 services and routing protocols. State-of-the-art implementations of the following routing protocols are provided in Cisco NX-OS Release 4.0:
– Open Shortest Path First (OSPF) Protocol Versions 2 and 3
– Intermediate System-to-Intermediate System (IS-IS) Protocol
– Border Gateway Protocol (BGP)
– Enhanced Interior Gateway Protocol (EIGRP)
– Routing Information Protocol (RIP) Version 2 and RIP next generation (RIPng)
The implementations of these protocols are fully compliant with the latest standards, providing modern enhancements and parameters such as 4-byte autonomous system numbers and incremental SPF, while eliminating unused older functions to provide a lean implementation that enables quick implementation of new features. All interface types are supported by all protocols. Among the available interface types are Ethernet interfaces, switch virtual interfaces (SVIs) and subinterfaces, PortChannels, tunnel interfaces, and loopback interfaces.
• The rich variety of routing protocols and functions is complemented by several salient services, including the following:
– VRF
– Dynamic Host Configuration Protocol (DHCP) Helper
– Unicast Reverse Path Forwarding (uRPF)
– Hot-Standby Routing Protocol (HSRP)
– Virtual Router Redundancy Protocol (VRRP)
– Gateway Load Balancing Protocol (GLBP)
– Enhanced object tracking
– Policy-based routing (PBR)
– Generic routing encapsulation (GRE) tunneling
Support for High Density
• Both hardware and software are specifically designed to support ACLs with tens of thousands of entries.
• The switching fabric is built to support high-density multiport 10 Gigabit Ethernet interfaces.
Serviceability
• Switched Port Analyzer (SPAN, RSPAN, ERSPAN): The SPAN feature allows the administrator to analyze all traffic between ports (called the SPAN source ports) by nonintrusively directing the SPAN session traffic to a SPAN destination port that has an external analyzer attached to it. RSPAN (remote SPAN) and ERSPAN (Encapsulated Remote SPAN) add the capability to analyze cross multiple switches from a central switch, thus avoiding the need to move the external analyzer multiple times.
• Embedded packet analyzer: Cisco NX-OS includes a built-in packet analyzer to help monitor and troubleshoot control plane traffic. The packet analyzer is based on the popular Wireshark open source network protocol analyzer.
• Smart Call Home: Smart Call Home provides e-mail-based notification of critical system events. A versatile range of message formats is available for optimal compatibility with pager services, standard e-mail, and XML-based automated parsing applications. Common uses of this feature include direct paging of a network support engineer, e-mail notification to a network operations center, and use of Cisco AutoNotify services for direct case generation with the Cisco Technical Assistance Center (TAC). This step toward autonomous system operation allows the networking devices to inform the administrator when a problem occurs to help ensure that it is acted on quickly, reducing time to resolution and helping ensure maximum system uptime.
• Cisco GOLD (Generic Online Diagnostics): Cisco GOLD enables the end user to run scheduled tasks to verify that hardware and internal data paths are operating as designed. This industry-leading diagnostics subsystem allows rapid fault isolation and continuous system monitoring, which are crucial in today's 24-hours-a-day operating environments.
• Cisco IOS EEM (Embedded Event Manager): Cisco IOS EEM is a powerful device and system management technology integrated into Cisco NX-OS. Cisco IOS EEM helps customers harness the network intelligence intrinsic to the Cisco software and gives them the capability to customize behavior based on network events as they happen.
Intelligent Networking
The Cisco Nexus 7000 Series is another important step in the ongoing process of moving intelligence onto the network. The Cisco Data Center 3.0 launch in late 2007 established a vision of how the data center progressed from consolidation, to virtualization of important data center assets, and now automation and orchestration of functions across formerly independent disciplines. With its unified fabric and rich software support, the Cisco Nexus 7000 Series is a crucial step in this continuing data center evolution.
Why Cisco?
Among competitors, only Cisco provides product breadth in crucial data center functions: Ethernet and Fiber Channel switching, InfiniBand switching, virtualization technology for LAN and SAN assets, application networking technologies, and service orchestration and management through Cisco VFrame to link disparate, independent technologies in the data center for faster time to provisioning and service rollout. These features in combination with its comprehensive services and channel and partner engagement with major storage and server vendors enable Cisco to deliver the industry-leading data center solution.
