Foreword xxvi
Introduction xxvii
Part I Introduction to Troubleshooting Nexus Switches
Chapter 1 Introduction to Nexus Operating System (NX-OS) 1
Nexus Platforms Overview 2
Nexus 2000 Series 2
Nexus 3000 Series 3
Nexus 5000 Series 4
Nexus 6000 Series 4
Nexus 7000 Series 5
Nexus 9000 Series 6
NX-OS Architecture 8
The Kernel 9
System Manager (sysmgr) 9
Messages and Transactional Services 11
Persistent Storage Services 13
Feature Manager 14
NX-OS Line Card Microcode 17
File Systems 19
Understanding NX-OS Software Releases and Packaging 25
Software Maintenance Upgrades 27
Licensing 28
NX-OS High-Availability Infrastructure 28
Supervisor Redundancy 29
ISSU 34
NX-OS Virtualization Features 35
Virtual Device Contexts 35
Virtual Routing and Forwarding 37
Virtual Port Channel 37
Management and Operations Capabilities 39
NX-OS Advanced CLI 39
Technical Support Files 44
Accounting Log 45
Feature Event-History 46
Debug Options: Log File and Filters 47
Configuration Checkpoint and Rollback 48
Consistency Checkers 49
Feature Scheduler, EEM, and Python 50
Bash Shell 51
Summary 51
References 51
Chapter 2 NX-OS Troubleshooting Tools 53
Packet Capture: Network Sniffer 53
Encapsulated Remote SPAN 57
SPAN on Latency and Drop 60
Nexus Platform Tools 63
Ethanalyzer 63
Packet Tracer 71
NetFlow 72
NetFlow Configuration 73
NetFlow Sampling 77
sFlow 78
Network Time Protocol 81
Embedded Event Manager 83
Logging 87
Debug Logfiles 90
Accounting Log 91
Event-History 92
Summary 93
References 93
Chapter 3 Troubleshooting Nexus Platform Issues 95
Troubleshooting Hardware Issues 95
Generic Online Diagnostic Tests 98
Nexus Device Health Checks 108
Nexus Fabric Extenders 124
Virtual Device Context 130
VDC Resource Template 131
Configuring VDC 133
VDC Initialization 134
Out-of-Band and In-Band Management 137
VDC Management 137
Troubleshooting NX-OS System Components 142
Message and Transaction Services 144
Netstack and Packet Manager 148
ARP and Adjacency Manager 160
EthPM and Port-Client 175
HWRL, CoPP, and System QoS 179
MTU Settings 192
Summary 195
References 196
Part II Troubleshooting Layer 2 Forwarding
Chapter 4 Nexus Switching 197
Network Layer 2 Communication Overview 197
Virtual LANs 200
VLAN Creation 201
Access Ports 203
Trunk Ports 204
Private VLANS 207
Spanning Tree Protocol Fundamentals 218
IEEE 802.1D Spanning Tree Protocol 219
Rapid Spanning Tree Protocol 220
Multiple Spanning-Tree Protocol (MST) 236
Detecting and Remediating Forwarding Loops 241
MAC Address Notifications 242
BPDU Guard 243
BPDU Filter 244
Problems with Unidirectional Links 245
Summary 252
References 254
Chapter 5 Port-Channels, Virtual Port-Channels, and FabricPath 255
Port-Channels 255
Basic Port-Channel Configuration 259
Verifying Port-Channel Status 260
Verifying LACP Packets 262
Advanced LACP Configuration Options 265
LACP System Priority 268
Port-Channel Member Interface Consistency 271
Troubleshooting LACP Interface Establishment 272
Troubleshooting Traffic Load-Balancing 272
Virtual Port-Channel 274
vPC Fundamentals 275
vPC Configuration 278
vPC Verification 280
Advanced vPC Features 288
FabricPath 294
FabricPath Terminologies and Components 296
FabricPath Packet Flow 297
FabricPath Configuration 300
FabricPath Verification and Troubleshooting 303
FabricPath Devices 310
Emulated Switch and vPC+ 310
vPC+ Configuration 311
vPC+ Verification and Troubleshooting 314
Summary 320
References 320
Part III Troubleshooting Layer 3 Routing
Chapter 6 Troubleshooting IP and IPv6 Services 321
IP SLA 321
ICMP Echo Probe 322
UDP Echo Probe 324
UDP Jitter Probe 325
TCP Connect Probe 328
Object Tracking 329
Object Tracking for the Interface 330
Object Tracking for Route State 330
Object Tracking for Track-List State 332
Using Track Objects with Static Routes 334
IPv4 Services 335
DHCP Relay 335
DHCP Snooping 341
Dynamic ARP Inspection 345
IP Source Guard 349
Unicast RPF 351
IPv6 Services 352
Neighbor Discovery 352
IPv6 Address Assignment 357
IPv6 First-Hop Security 362
First-Hop Redundancy Protocol 370
HSRP 370
VRRP 380
GLBP 385
Summary 391
Chapter 7 Troubleshooting Enhanced Interior Gateway Routing Protocol (EIGRP) 393
EIGRP Fundamentals 393
Topology Table 395
Path Metric Calculation 396
EIGRP Communication 399
Baseline EIGRP Configuration 399
Troubleshooting EIGRP Neighbor Adjacency 401
Verification of Active Interfaces 402
Passive Interface 403
Verification of EIGRP Packets 405
Connectivity Must Exist Using the Primary Subnet 409
EIGRP ASN Mismatch 412
Mismatch K Values 413
Problems with Hello and Hold Timers 414
EIGRP Authentication Issues 416
Troubleshooting Path Selection and Missing Routes 419
Load Balancing 421
Stub 421
Maximum-Hops 424
Distribute List 426
Offset Lists 427
Interface-Based Settings 430
Redistribution 430
Classic Metrics vs. Wide Metrics 433
Problems with Convergence 439
Active Query 441
Stuck in Active 443
Summary 446
References 447
Chapter 8 Troubleshooting Open Shortest Path First (OSPF) 449
OSPF Fundamentals 449
Inter-Router Communication 450
OSPF Hello Packets 450
Neighbor States 451
Designated Routers 452
Areas 453
Link State Advertisements 453
Troubleshooting OSPF Neighbor Adjacency 456
Baseline OSPF Configuration 456
OSPF Neighbor Verification 458
Confirmation of OSPF Interfaces 460
Passive Interface 461
Verification of OSPF Packets 463
Connectivity Must Exist Using the Primary Subnet 468
MTU Requirements 469
Unique Router-ID 471
Interface Area Numbers Must Match 471
OSPF Stub (Area Flags) Settings Must Match 473
DR Requirements 474
Timers 476
Authentication 478
Troubleshooting Missing Routes 482
Discontiguous Network 482
Duplicate Router ID 485
Filtering Routes 487
Redistribution 487
OSPF Forwarding Address 488
Troubleshooting OSPF Path Selection 494
Intra-Area Routes 494
Inter-Area Routes 495
External Route Selection 495
E1 and N1 External Routes 496
E2 and N2 External Routes 497
Problems with Intermixed RFC 1583 and RFC 2328 Devices 499
Interface Link Costs 500
Summary 504
References 505
Chapter 9 Troubleshooting Intermediate System-Intermediate System (IS-IS) 507
IS-IS Fundamentals 507
Areas 508
NET Addressing 509
Inter-Router Communication 511
IS Protocol Header 511
TLVs 512
IS PDU Addressing 512
IS-IS Hello (IIH) Packets 513
Link-State Packets 515
Designated Intermediate System 516
Path Selection 517
Troubleshooting IS-IS Neighbor Adjacency 518
Baseline IS-IS Configuration 518
IS-IS Neighbor Verification 520
Confirmation of IS-IS Interfaces 523
Passive Interface 526
Verification of IS-IS Packets 528
Connectivity Must Exist Using the Primary Subnet 535
MTU Requirements 537
Unique System-ID 539
Area Must Match Between L1 Adjacencies 539
Checking IS-IS Adjacency Capabilities 541
DIS Requirements 543
IIH Authentication 544
Troubleshooting Missing Routes 546
Duplicate System ID 546
Interface Link Costs 549
Mismatch of Metric Modes 553
L1 to L2 Route Propagations 556
Suboptimal Routing 562
Redistribution 566
Summary 567
References 568
Chapter 10 Troubleshooting Nexus Route-Maps 569
Conditional Matching 569
Access Control Lists 569
ACLs and ACL Manager Component 570
Prefix Matching and Prefix-Lists 577
Route-Maps 581
Conditional Matching 582
Optional Actions 586
Incomplete Configuration of Routing Policies 586
Diagnosing Route Policy Manger 586
Policy-Based Routing 591
Summary 594
References 595
Chapter 11 Troubleshooting BGP 597
BGP Fundamentals 597
Address Families 598
Path Attributes 599
Loop Prevention 599
BGP Sessions 600
BGP Identifier 601
BGP Messages 601
BGP Neighbor States 602
BGP Configuration and Verification 605
Troubleshooting BGP Peering Issues 609
Troubleshooting BGP Peering Down Issues 609
Demystifying BGP Notifications 619
Troubleshooting IPv6 Peers 621
BGP Peer Flapping Issues 622
BGP Route Processing and Route Propagation 630
BGP Route Advertisement 631
BGP Best Path Calculation 636
BGP Multipath 640
BGP Update Generation Process 643
BGP Convergence 646
Scaling BGP 649
Tuning BGP Memory 650
Soft Reconfiguration Inbound Versus Route Refresh 654
Scaling BGP with Route-Reflectors 657
Maximum Prefixes 659
BGP Max AS 662
BGP Route Filtering and Route Policies 662
Prefix-List-Based Filtering 663
Filter-Lists 669
BGP Route-Maps 673
Regular Expressions (RegEx) 676
AS-Path Access List 684
BGP Communities 684
Looking Glass and Route Servers 687
Logs Collection 687
Summary 687
Further Reading 688
References 688
Part IV Troubleshooting High Availability
Chapter 12 High Availability 689
Bidirectional Forwarding Detection 689
Asynchronous Mode 691
Asynchronous Mode with Echo Function 693
Configuring and Verifying BFD Sessions 693
Nexus High Availability 707
Stateful Switchover 707
ISSU 713
Graceful Insertion and Removal 719
Custom Maintenance Profile 727
Summary 731
References 732
Part V Multicast Network Traffic
Chapter 13 Troubleshooting Multicast 733
Multicast Fundamentals 734
Multicast Terminology 735
Layer 2 Multicast Addresses 738
Layer 3 Multicast Addresses 739
NX-OS Multicast Architecture 741
Replication 744
Protecting the Central Processing Unit 745
NX-OS Multicast Implementation 747
IGMP 750
IGMPv2 751
IGMPv3 752
IGMP Snooping 756
IGMP Verification 761
PIM Multicast 771
PIM Protocol State and Trees 772
PIM Message Types 773
PIM Interface and Neighbor Verification 780
PIM Any Source Multicast 785
PIM Bidirectional 799
PIM RP Configuration 811
PIM Source Specific Multicast 841
Multicast and Virtual Port-Channel 848
vPC-Connected Source 849
vPC-Connected Receiver 861
vPC Considerations for Multicast Traffic 870
Ethanalyzer Examples 871
Summary 871
References 872
Part VI Troubleshooting Nexus Tunneling
Chapter 14 Troubleshooting Overlay Transport Virtualization (OTV) 875
OTV Fundamentals 875
Flood Control and Broadcast Optimization 877
Supported OTV Platforms 878
OTV Terminology 878
Deploying OTV 881
Understanding and Verifying the OTV Control Plane 885
OTV Multicast Mode 887
OTV IS-IS Adjacency Verification 888
OTV IS-IS Topology Table 898
OTV IS-IS Authentication 905
Adjacency Server Mode 907
OTV Control Plane Policing (CoPP) 912
Understanding and Verifying the OTV Data Plane 913
OTV ARP Resolution and ARP-ND-Cache 915
Broadcasts 917
Unknown Unicast Frames 918
OTV Unicast Traffic with a Multicast Enabled Transport 919
OTV Multicast Traffic with a Multicast Enabled Transport 924
OTV Multicast Traffic with a Unicast Transport (Adjacency Server Mode) 932
Advanced OTV Features 937
First Hop Routing Protocol Localization 938
Multihoming 939
Ingress Routing Optimization 940
VLAN Translation 941
OTV Tunnel Depolarization 942
OTV Fast Failure Detection 944
Summary 946
References 947
Part VII Network Programmability
Chapter 15 Programmability and Automation 949
Introduction to Automation and Programmability 949
Introduction to Open NX-OS 950
Shells and Scripting 951
NX-SDK 964
NX-API 968
Summary 975
References 975
9781587145056 TOC 4/24/2018
