Change Verification

Event-driven change verification skill for structured change windows. Guides
baseline capture before a change, provides change execution safety patterns,
performs post-change diff analysis, and supports rollback decision-making when
unexpected deviations are detected.

This skill covers a single change event lifecycle (before → during →
after). For ongoing configuration drift detection and compliance auditing, use
the config-management skill instead.

Commands are labeled [Cisco], [JunOS], or [EOS] where syntax
diverges. Unlabeled statements apply to all three vendors.

Safety Note — Read-Write Operations: This skill includes procedures that
modify device state during change execution and rollback phases. Steps that
write to devices are marked with ⚠️ WRITE. Always confirm authorization,
change ticket approval, and maintenance window status before executing write
operations. Baseline capture and post-change verification steps are read-only
and safe to run at any time.

When to Use

• - Planned maintenance window requiring structured pre/post verification
• Configuration change (routing policy, ACLs, interface config) with rollback plan
• Software upgrade or patch requiring before/after state comparison
• Hardware replacement (linecard, SFP, PSU) with service validation
• Circuit turn-up or decommission with adjacency and traffic verification
• Emergency change requiring rapid baseline capture and rollback readiness
• Post-change soak period with periodic re-verification against baselines

Prerequisites

• - SSH or console access to all devices in the change scope (read-only for

baselines; enable/configure privilege for change execution and rollback)

• - Approved change ticket with documented scope, expected impact, and rollback

plan including timing criteria

• - Pre-identified list of devices and interfaces in the change scope
• Knowledge of expected state changes: which routes will move, which interfaces

will bounce, which adjacencies will flap

• - Access to a file store (flash, SCP server, or local disk) for baseline

archival

• - Contact information for escalation if rollback criteria are met
• See references/checklist-templates.md for per-change-type prerequisites

Procedure

Follow these steps sequentially for each change event. Steps 1–2 are
always read-only. Steps 3–4 include write operations. Steps 5–6 are
analytical and drive the rollback decision.

Step 1: Pre-Change Baseline Capture

Capture device state snapshots before any changes. Store outputs with
timestamps for post-change comparison.

Routing state:

[Cisco]
CODEBLOCK0

[JunOS]
CODEBLOCK1

[EOS]
CODEBLOCK2

Interface and adjacency state:

All vendors — capture interface status, error counters, and neighbor tables:

[Cisco]
CODEBLOCK3

[JunOS]
CODEBLOCK4

[EOS]
CODEBLOCK5

Configuration and hardware:

[Cisco]
CODEBLOCK6

[JunOS]
CODEBLOCK7

[EOS]
CODEBLOCK8

⚠️ WRITE — Archive baseline config to persistent storage:

[Cisco] copy running-config flash:pre-change-[ticket]-[date].cfg
[JunOS] request system configuration save /var/tmp/pre-change-[ticket].conf
[EOS] INLINECODE4

Record baseline metrics for comparison: total route count, BGP peer count
(Established), OSPF neighbor count (Full), interface error counters, and
hardware sensor readings.

Step 2: Change Scope Documentation

Before executing any changes, document:

1. 1. Change description — what configuration lines are being added, modified,

or removed

1. 2. Expected impact — which peers will flap, which routes will shift, which

interfaces will bounce, expected duration of disruption

1. 3. Rollback trigger criteria — specific thresholds that mandate rollback

(see Threshold Tables below)

1. 4. Rollback procedure — exact commands to revert (see

references/cli-reference.md for vendor-specific rollback commands)

1. 5. Success criteria — what "done" looks like: all baselines restored,

intended changes visible, no unexpected deviations

1. 6. Soak period — how long to monitor after change before declaring success

Step 3: Change Execution

⚠️ WRITE — Apply changes using commit-confirm patterns when available.

[Cisco] — No native commit-confirm. Apply changes in config mode and
immediately verify. For bulk changes, use configure replace with a prepared
config file.

[JunOS] — Use commit confirmed [minutes] to auto-rollback if not
confirmed within the timer window. Confirm with commit after verification.
CODEBLOCK9

[EOS] — Use configure session for atomic staged changes. Review before
committing.
CODEBLOCK10

Staged rollout for multi-device changes: Apply to one device first, verify
post-change state (Step 4), then proceed to remaining devices only after the
first device passes all checks.

Step 4: Post-Change Verification

Re-capture all baseline metrics from Step 1 using identical commands. Perform
a structured diff against pre-change baselines.

Key comparisons:

Metric	Compare Against	Expected Outcome
Route count	Pre-change summary	Within deviation threshold
BGP peers Established

Pre-change peer list | All peers restored (or changed per plan) |
| OSPF neighbors Full | Pre-change neighbor list | All adjacencies restored |
| Interface errors | Pre-change counters | No new sustained errors |
| Config diff | Archived pre-change config | Only intended lines changed |

Config diff verification:

[Cisco]
CODEBLOCK11

[JunOS]
CODEBLOCK12

[EOS]
CODEBLOCK13

Review every line in the diff output. Classify each changed line as expected
(directly part of the change plan) or unexpected (not in the change scope).

Step 5: Impact Assessment

Classify all deviations from baseline into categories:

1. 1. Expected — Intended: Changes that are the direct goal of the change

window (e.g., new BGP peer appearing, old ACL removed). No action needed.

1. 2. Expected — Side Effect: Changes caused by the intended change but not

the primary goal (e.g., route count increase because a new peer is now advertising). Verify they are benign.

1. 3. Unexpected — Minor: Deviations not related to the change scope but low

severity (e.g., a single interface counter increment). Investigate but do not necessarily roll back.

1. 4. Unexpected — Critical: Deviations indicating collateral damage (e.g.,

adjacency loss on an unrelated interface, route withdrawal not in change scope). Evaluate rollback immediately.

If any deviation is classified as Unexpected — Critical, proceed directly
to Step 6 rollback evaluation.

Step 6: Rollback Decision

Evaluate whether to accept the change or roll back using the criteria below.

Rollback if ANY of these conditions are true:

• - Service-affecting outage on interfaces/peers outside the change scope
• Route count deviation exceeds threshold AND routes are not accounted for in

the change plan

• - Adjacency loss persists beyond the expected convergence window
• Hardware errors (PSU, fan, temperature) emerged that were not present in

baseline

• - The change did not achieve its intended effect (success criteria from Step 2

not met)

Accept if ALL of these conditions are true:

• - All success criteria from Step 2 are met
• Config diff contains only expected change lines
• All baseline metrics are within acceptable deviation thresholds
• No unexpected critical deviations detected
• Soak period has elapsed without regression

⚠️ WRITE — If rolling back:

[Cisco] configure replace flash:pre-change-[ticket]-[date].cfg force
[JunOS] rollback 1 then commit
[EOS] INLINECODE13

After rollback, re-run Step 4 post-change verification to confirm the device
has returned to its pre-change state.

Threshold Tables

Acceptable Deviation Thresholds

Metric	Normal Deviation	Warning	Rollback Trigger
IPv4 route count	±2% of baseline	±5% of baseline	>10% or unplanned loss
IPv6 route count

±2% of baseline | ±5% of baseline | >10% or unplanned loss | | BGP Established peers | 0 lost (unless planned) | 1 lost (if in scope) | ≥1 lost outside scope | | OSPF Full adjacencies | 0 lost (unless planned) | Flap then recover <2 min | Lost >2 min | | Interface errors (new) | 0 new CRC/input errors | <10 in first 5 min | Sustained >10/min | | Interface flaps | 0 (unless planned bounce) | 1 flap on in-scope intf | Any flap outside scope |

Rollback Timing Thresholds

Phase	Maximum Duration	Action if Exceeded
Change execution	Per change ticket	Pause and escalate
Post-change convergence

5 minutes | Begin rollback assessment | | Adjacency re-establishment | 2 minutes per peer | Escalate if critical peer | | Route table stabilization | 3 minutes | Check for route oscillation | | Soak period (minor change) | 15 minutes | Declare success or investigate | | Soak period (major change) | 60 minutes | Declare success or investigate | | Rollback execution | 5 minutes | Escalate to senior engineer |

Decision Trees

Post-Change Diff Contains Unexpected Lines

CODEBLOCK14

Adjacency Loss Detected Post-Change

CODEBLOCK15

Route Count Deviation Outside Normal Threshold

CODEBLOCK16

Report Template

CODEBLOCK17

Troubleshooting

Baseline capture commands fail or return incomplete output

• - Verify SSH session stability — long command outputs may be truncated by

terminal buffer limits. Use terminal length 0 [Cisco/EOS] or set cli screen-length 0 [JunOS] before capture.

• - Check device CPU — high CPU may cause CLI timeouts. Run INLINECODE16

[Cisco] / show system processes extensive [JunOS] / show processes top [EOS] to verify.

• - If archival to remote storage fails, save to local flash as fallback and

note the location for later retrieval.

Config diff shows excessive noise

• - Filter out timestamp and comment lines that change on every config display

(e.g., ! Last configuration change at ...).

• - On [JunOS], show | compare rollback 1 gives clean structured diffs. On

[Cisco], show archive config differences may include line-order differences that are not real changes — focus on substantive config lines.

• - Use references/checklist-templates.md checklists to focus verification on

change-relevant sections rather than full config comparison.

Adjacency does not recover after expected bounce

• - Check interface state: show interfaces [intf] — look for down/down vs

up/down to distinguish physical vs protocol issues.

• - Verify the peer device accepted the change — a mismatched configuration on

both sides of a link (e.g., mismatched OSPF area, BGP ASN) will prevent adjacency formation.

• - Check for hold-timer expiry: OSPF default dead interval is 40s; BGP default

hold time is 180s. Wait at least one full timer cycle before escalating.

Rollback command fails or produces unexpected state

• - [Cisco] configure replace requires the IOS archive feature to be

enabled. If unavailable, manually reverse the config changes line by line.

• - [JunOS] rollback N may fail if the commit history has been cleared or

the device has rebooted since the baseline commit. Use show system commit to verify available rollback points.

• - [EOS] configure replace requires the replacement file to be a complete

config, not a partial fragment. Verify the archived file is a full show running-config capture.

• - After any rollback, re-run the full post-change verification (Step 4) to

confirm the device has returned to its pre-change state.

Change window time exceeded before verification completes

• - Prioritize critical services: check routing adjacencies and interface states

first, defer detailed config diff analysis to after the window if services are healthy.

• - If the soak period must be shortened, document the reduced observation window

and schedule a follow-up verification at the next opportunity.

• - Escalate if service impact is detected and the change window has closed —

do not delay rollback due to window constraints if there is active service degradation.