Cisco IOS-XE / NX-OS Device Health Check

Structured triage procedure for assessing Cisco device health across IOS-XE and
NX-OS platforms. Produces a prioritized findings report with severity
classifications and recommended actions.

Commands are labeled [IOS-XE] or [NX-OS] where they diverge. Unlabeled
commands work on both platforms.

When to Use

• - Device reported as slow, dropping traffic, or unresponsive
• Scheduled health audit of IOS-XE routers/switches or Nexus platforms
• Post-change verification after upgrades, patches, or configuration changes
• Capacity planning data collection for CPU, memory, and link utilization
• Incident response when a Cisco device is suspected as the fault domain
• NX-OS VDC isolation troubleshooting — health check scoped to a specific VDC

Prerequisites

• - SSH or console access to the device (privilege 1 minimum for IOS-XE, show role for NX-OS)
• IOS-XE 16.x/17.x or NX-OS 9.x/10.x (commands validated against IOS-XE 17.3+ and NX-OS 10.2+)
• Network reachability to management interface confirmed
• Awareness of the device's normal baseline (CPU, memory, traffic patterns)
• For NX-OS: identify which VDC you are operating in (show vdc current)

Procedure

Follow this sequence. Each step produces data for the final report. If the device
is unresponsive, skip to Step 6 for crash recovery.

Step 1: Establish Baseline Context

Collect identity, uptime, and platform context.

[IOS-XE]
CODEBLOCK0

[NX-OS]
CODEBLOCK1

Record: hostname, OS version, uptime, hardware model, current time.
Short uptime means a recent reload or crash — investigate immediately.
On NX-OS, confirm VDC context: all subsequent commands apply to this VDC only.

Step 2: CPU Utilization

IOS-XE separates Route Processor (RP) and QuantumFlow Processor (QFP). Triage
both planes independently. NX-OS uses a unified supervisor model — show system resources is the primary view.

[IOS-XE]
CODEBLOCK2

[NX-OS]
CODEBLOCK3

IOS-XE interpretation: Compare 5-second, 1-minute, and 5-minute RP averages.
If RP CPU is normal but QFP drops are climbing, the data plane is overloaded while
control plane is healthy — different remediation than RP overload. Key RP processes:
IP Input (traffic storms), BGP Router (route churn), Crypto IKMP (VPN storms),
IOSD (control plane congestion).

NX-OS interpretation: show system resources reports CPU as kernel + user
percentages. Normal idle should be above 60%. NX-OS control plane CPU runs hotter
than IOS-XE RP because the supervisor handles both planes. Key processes: netstack
(forwarding), bgp (route processing), stp (spanning tree), vpc (vPC keepalive).

Step 3: Memory Utilization

[IOS-XE]
CODEBLOCK4

[NX-OS]
CODEBLOCK5

IOS-XE: Calculate used percentage from processor pool: (Total - Free) / Total * 100.
Check fragmentation: if largest free block < 10% of total free, memory is fragmented.

NX-OS: show system resources reports memory as total/used/free in KB. NX-OS
reserves a larger kernel memory footprint than IOS-XE — used percentages above 60%
are typical. Watch for MemFree dropping below 15% of total.

Step 4: Interface Health

CODEBLOCK6

[IOS-XE]
CODEBLOCK7

[NX-OS]
CODEBLOCK8

For each interface with errors:

• - Calculate error rate: INLINECODE11
• CRC errors → Layer 1 issue (cabling, optics, SFP seating)
• Input errors without CRC → buffer overruns
• Output drops → congestion; check QoS policy or link capacity
• On NX-OS, show hardware internal errors reveals ASIC-level drops not visible

Step 5: Routing Protocol Health

CODEBLOCK9

[IOS-XE]
CODEBLOCK10

[NX-OS]
CODEBLOCK11

Verify: expected route count, all neighbors in established/full state, no unexpected
withdrawals. On NX-OS, vPC peer-link state is critical — a failed peer-link orphans
traffic. BGP prefix counts that deviate >10% from baseline indicate route churn.

Step 6: Environment and Platform

[IOS-XE]
CODEBLOCK12

[NX-OS]
CODEBLOCK13

Check: power supply status, fan health, temperature readings. Any environmental
alarm escalates immediately. On IOS-XE, review crashinfo: for recent crash dumps.
On NX-OS, show cores lists supervisor and module core dumps — presence indicates
a process crash requiring investigation.

Threshold Tables

Reference: references/threshold-tables.md for detailed per-parameter thresholds.

Decision Trees

Primary Triage

CODEBLOCK14

Escalation Criteria

Escalate to senior engineer or TAC when:

• - CPU sustained > 90% for 15+ minutes with no identifiable cause
• Memory below 15% free with no recent change to explain consumption
• Traceback, CPUHOG, or MALLOCFAIL in logs (IOS-XE); core dump present (NX-OS)
• Any environmental alarm (power, fan, temperature)
• More than 3 routing neighbor state changes in the last hour
• NX-OS module in non-OK state or vPC peer-link failure
• QFP drops exceeding 10,000/sec with no traffic anomaly explanation (IOS-XE)

Report Template

CODEBLOCK15

Troubleshooting

Device Unresponsive to SSH

Try console access. If console also unresponsive, check power and environment
via out-of-band management (CIMC for UCS-based, smart PDU, or console server).
On IOS-XE: collect dir crashinfo: after recovery. On NX-OS: show cores and
show system reset-reason.

NX-OS VDC Context Confusion

Health data is VDC-scoped. If metrics look wrong, verify VDC: show vdc current.
Switch VDC with switchto vdc [name]. The default VDC contains system-level
resources; admin VDC shows aggregate hardware.

CPU Spikes During Health Check

Show commands and SNMP polling can briefly spike CPU. Wait 30 seconds after
connecting before collecting CPU data. Use terminal length 0 (IOS-XE) or
terminal length 0 (NX-OS) to avoid paging pauses.

IOS-XE QFP Drops vs RP CPU

High QFP drops with normal RP CPU means the data plane is stressed but control
plane is healthy. Do not restart the device — the RP is functioning normally.
Investigate traffic patterns, ACL complexity, and NAT session counts. Use
show platform hardware qfp active feature to identify which QFP feature is
consuming resources.

NX-OS Module Issues

If show module reports a module not in "ok" state, collect show module internal exception-log module [n] before taking action. Powercycling a linecard
(poweroff module [n] / no poweroff module [n]) can resolve transient failures
but requires change control.

Inconsistent Memory Between Platforms

IOS-XE and NX-OS report memory differently. IOS-XE show memory statistics shows
IOS memory pools; NX-OS show system resources shows Linux-level memory. NX-OS
baseline memory usage is higher (55–65% is normal) because the Linux kernel and
system daemons consume more than IOS-XE's monolithic process model.