Juniper JunOS Device Health Check

Structured triage procedure for assessing Juniper device health across MX, SRX,
EX, QFX, and PTX platforms. Produces a prioritized findings report with severity
classifications and recommended actions.

JunOS separates Routing Engine (RE) and Packet Forwarding Engine (PFE). These
are independent health domains — a healthy RE does not guarantee a healthy PFE,
and vice versa. This procedure assesses both explicitly.

When to Use

• - Device reported as slow, dropping traffic, or unresponsive
• Scheduled health audit of Juniper routers, switches, or firewalls
• Post-change verification after commits, upgrades, or ISSU
• Capacity planning data collection for RE CPU, memory, and link utilization
• Incident response when a Juniper device is suspected as the fault domain
• RE failover event — verify mastership and standby RE state
• Chassis alarm triggered — severity triage and root cause identification

Prerequisites

• - SSH or console access to the device (login class with view permissions minimum)
• JunOS 21.x or later (commands validated against JunOS 23.2+)
• Network reachability to management interface or fxp0 confirmed
• Awareness of the device's normal baseline (CPU, memory, traffic patterns)
• For dual-RE systems: know which RE should be master under normal operations
• Knowledge of recent commit history if correlating symptoms with changes

Procedure

Follow this sequence. Each step produces data for the final report. RE mastership
verification is mandatory first — all subsequent data is RE-scoped.

Step 1: Verify RE Mastership (Mandatory)

On dual-RE systems, health data comes from the RE you are logged into. If you
are on the backup RE, all metrics reflect the standby engine — not the active
forwarding path. This step is non-negotiable.

CODEBLOCK0

Verify: your session is on the master RE. If Current state shows Backup,
switch to master: request routing-engine login other-routing-engine.

On single-RE platforms, confirm RE is Master (not in a degraded state).
Record: hostname, RE slot, mastership state, uptime, last reboot reason.
Short uptime after an unexpected reboot — investigate immediately.

Step 2: Alarm Analysis

JunOS surfaces alarms as first-class status indicators. Check chassis and
system alarms before deeper investigation — alarms may already identify the
problem.

CODEBLOCK1

Alarm severities:

• - Major — service-affecting condition, requires immediate attention
• Minor — degraded but service continues, investigate promptly

If alarms are present, record each alarm's class, description, and time.
Major alarms take priority over all other triage — address them first.
Common alarm sources: FPC offline, power supply failure, rescue config
not set, license expiry, FRU removal.

No alarms → proceed with systematic health assessment.

Step 3: Routing Engine Health

RE handles control plane: routing protocols, management, commit operations.

CODEBLOCK2

Key fields from show chassis routing-engine:

• - CPU utilization — temperature, idle percentage (idle below 30% is warning)
• Memory utilization — total and used; watch for used > 80%
• Temperature — compare to platform-specific thresholds
• Start time — recent RE restart indicates crash or failover
• Load averages — 1min/5min/15min; sustained > 1.0 per core is elevated

High RE CPU with top process identification:

• - rpd — routing protocol daemon: route churn, table size, peer instability
• INLINECODE7 — chassis management: sensor polling issues, FPC communication
• INLINECODE8 — SNMP polling storms
• INLINECODE9 — management: large config, slow commit, CLI session overload
• INLINECODE10 — key management: IKE/IPsec negotiation storms (SRX)

RE CPU spikes during commit operations are normal (can hit 80–90% briefly).
Compare against commit history: show system commit.

Step 4: PFE Health

PFE handles data plane forwarding independently from RE. A healthy RE with
a degraded PFE means traffic is being dropped even though the control plane
looks fine.

CODEBLOCK3

INLINECODE13:

• - State must be Online. Any other state (Present, Offline, Empty)

indicates a hardware issue or intentional deactivation.

• - CPU Total — PFE CPU utilization; above 80% is warning, above 90% critical
• Memory heap utilization — above 80% indicates PFE memory pressure

INLINECODE18:

• - Compare input vs output packet counts — large delta indicates drops
• Check fabric input drops and local input drops for discard sources

INLINECODE21:

• - Any non-zero error counters warrant investigation
• Sustained incrementing errors (check twice 30 seconds apart) indicate active issues

On MX platforms with multiple FPCs, check each FPC individually. A single
degraded FPC affects only interfaces on that linecard.

Step 5: System Resources

CODEBLOCK4

Storage: JunOS partitions can fill from logs, core dumps, or failed upgrades.
Any partition above 85% used is warning. /var filling above 90% can prevent
commits and logging.

Memory: show system memory gives kernel-level view. Compare to RE memory
from Step 3 for consistency. Sustained growth without corresponding config
changes suggests a memory leak.

Core dumps: Presence of recent core files (within last 7 days) indicates
process crashes. Record the process name and timestamp — this is JTAC-relevant
data.

Commit history: Recent commits correlate with symptoms. A device that was
healthy before a commit and unhealthy after has an obvious investigation path.

Step 6: Interface and Routing Health

CODEBLOCK5

For each interface with errors:

• - CRC errors → Layer 1 (cabling, optics, SFP)
• Input errors without CRC → buffer overruns, MTU mismatch
• Output drops → congestion or policer drops
• Carrier transitions → link flap, check SFP DOM: INLINECODE24

Routing: verify expected neighbor count, all adjacencies in Established/Full state.
BGP prefix counts deviating > 10% from baseline indicate route churn.

Step 7: Environment

CODEBLOCK6

Check: all temperature sensors within thresholds, all power supplies OK, all
fans operational. Any environmental alarm maps directly to Major alarm severity.

On platforms with redundant RE: check both RE temperatures. A standby RE running
hot may indicate cooling issues even if master RE temperature is normal.

Threshold Tables

Reference: references/threshold-tables.md for detailed per-parameter thresholds.

Parameter	Normal	Warning	Critical	Notes
RE CPU idle	> 40%	20–40%	< 20%	Spikes during commit are normal
RE memory used

< 75% | 75–85% | > 85% | |
| RE load avg (1min) | < 0.7/core | 0.7–1.5/core | > 1.5/core | Scale by RE core count |
| PFE CPU | < 60% | 60–80% | > 80% | Per-FPC |
| PFE heap used | < 70% | 70–85% | > 85% | Per-FPC |
| Storage partition | < 80% | 80–90% | > 90% | /var critical for commits |
| Interface error rate | < 0.01% | 0.01–0.1% | > 0.1% | |
| Output drops/hr | < 100 | 100–1000 | > 1000 | |
| Chassis alarm | None | Minor present | Major present | |
| Temperature | Within spec | 5°C of max | At/above max | Per-sensor |

Decision Trees

Primary Triage

CODEBLOCK7

Alarm Severity Triage

CODEBLOCK8

Escalation Criteria

Escalate to senior engineer or JTAC when:

• - RE CPU sustained above 90% for 15+ minutes with no identifiable cause
• RE memory above 90% used with no recent config change
• PFE offline or in non-Online state after power cycle attempt
• Core dumps present from critical processes (rpd, chassisd, pfed)
• Major chassis alarm with no clear remediation
• Multiple FPC failures or fabric errors
• RE failover loop (multiple failovers in short period)
• Any environmental alarm (power, fan, temperature)
• More than 3 routing neighbor state changes in the last hour

Report Template

CODEBLOCK9

Troubleshooting

Device Unresponsive to SSH

Try console access. If console is also unresponsive, check power and environment
via out-of-band management (craft interface, console server). After recovery:
show system core-dumps, show chassis routing-engine for reboot reason,
show log messages | match "kernel|panic|watchdog".

Logged Into Backup RE

If show chassis routing-engine shows your RE as Backup, you are collecting
standby metrics. Switch to master: request routing-engine login other-routing-engine.
If master RE is unreachable from backup, this indicates master RE failure — check
show chassis routing-engine from backup for master's last known state.

RE CPU Spikes During Commit

JunOS RE CPU can spike to 80–90% during commit operations. This is expected
behavior — the config daemon and rpd both consume CPU during commit processing.
Verify: show system commit to confirm a recent commit, then wait 2–3 minutes
and re-check. Sustained high CPU after commit settles indicates a real problem.

PFE Drops With Healthy RE

The RE (control plane) and PFE (data plane) are independent. High PFE drops with
a normal RE means traffic is being discarded at the forwarding level. Check:
show pfe statistics traffic for drop categories, show chassis fpc detail
for PFE CPU and memory. Common causes: filter/policer drops (may be expected),
next-hop resolution failures, PFE memory exhaustion from large tables.

Storage Full Preventing Commits

If /var is above 95%, commits will fail. Clear space:
request system storage cleanup — removes old logs, core dumps, and temporary
files. If that is insufficient: show system storage to identify the largest
consumers, then selectively remove old software images or rotated log files.

Dual-RE Failover Investigation

After an RE failover: verify new master is healthy (Steps 1–3), then investigate
the old master. From the new master: show chassis routing-engine shows both
REs' state. Check show log messages | match "mastership|failover|switchover"
for the event trigger. Common causes: RE crash (core dump present), watchdog
timeout, manual switchover, GRES/NSR failure.