The Circuit Nobody Could Find: How Exact-Match Searching Nearly Cost Me the Audit

If you run infrastructure you didn't build, you eventually inherit a problem like this one: the carrier sends over a circuit inventory spreadsheet, finance asks "are we actually using all of these," and nobody in the building can answer the question. The circuits predate the current staff. The labels on the patch panel predate the current firewall. The institutional knowledge left in someone's head three jobs ago.

That was my week. A carrier Ethernet services inventory landed in my inbox with eight active circuits billed to us, and I could not find a single one of them reflected anywhere in our environment. Not in the firewall. Not in the switch stack. Not in any documentation. Before opening a dispute with the carrier, I needed to prove a negative: that these circuits genuinely have no footprint in our gear, or find the footprint I was missing.

Proving a negative across two large configs by hand is miserable work. So I made the AI do it, and the way that played out taught me something about how much the question you ask shapes the answer you get.

The setup

Two configuration exports and one carrier spreadsheet:

• A firewall running config, roughly 15,000 lines, five virtual routers, a few hundred address objects, and years of accumulated rules

• A five-member switch stack config with every port labeled by hand over a decade

• A carrier inventory: 16 rows, 59 columns, covering 8 Ethernet circuits with service IDs, circuit IDs, NTE management addresses, gateway IPs, VPLS instance numbers, S-VLAN tags, CLLI codes, and aggregation router details

The workflow was simple in concept. Feed the model both configs, have it fully ingest them, then hand it the spreadsheet and ask one question: does anything in this inventory exist anywhere in our equipment?

That question, it turns out, was the problem. But I didn't know that yet.

Round one: the exact-match pass

The first pass -- I was running on Opus at this point -- did what I would call competent senior engineer work. It parsed the spreadsheet, extracted every IP, and ran exact-match comparisons against both configs. Result: zero verbatim matches. Then it went one step further with a /24 comparison and caught something real. Four of the carrier's gateway addresses, call them 192.168.4.x and 192.168.15.x, fell inside two subnets the firewall actually routed.

It traced the path cleanly: a static route on the primary virtual router handing off to a transit VR, which forwarded to a next-hop out the DIA interface. It even flagged a labeling inconsistency where the interface name claimed one carrier while the routed address space matched the other. Genuinely useful finding.

I asked for a double-check of the static routes. It re-extracted all 57 routes on the primary VR, ran proper subnet containment math instead of eyeballing octets, and was honest about noise: most "matches" were just the default routes catching everything, which is not a relationship. Conclusion: two routes, four gateway IPs, nothing else. It checked the other virtual routers and the rest of the config for the remaining carrier ranges. Zero occurrences anywhere.

At this point I had what looked like an exhaustive answer. Two routes. Four IPs. Done. If I had stopped there, I would have sent the carrier a report that was accurate but incomplete, and I would not have known it.

Round two: asking a broader question

Before writing the final report, I did one more pass with a deliberately broader instruction. Stop checking whether the spreadsheet values exist, and start checking every way the carrier's address space could touch this firewall -- broader than, adjacent to, or historical relative to the literal values in the sheet.

I'll be upfront about the setup, because it matters for what you should take from this. I changed two things going into that pass, not one: the question got wider, and I also switched models, since Claude Fable 5 had just become available and I ran the pass there. On top of that, it was a third pass over ground two earlier passes had already mapped. Three variables moved at once. So I can't cleanly hand the credit to the model -- I didn't re-run Opus with this same broad instruction to see if it would have caught the same things, and without that control I won't pretend the delta is a model story. What I can tell you is what the broader question found. And it was a lot.

First, the broad pass immediately surfaced an address family the exact-match passes had skipped entirely. The carrier spreadsheet contained IPv6 NTE addresses -- six of them, all under the carrier's 2001-prefix space -- that no prior pass had checked. That is the kind of omission that quietly invalidates an "exhaustive" claim. The sweep confirmed neither config contained a single address in that range, which closed the gap properly instead of leaving it unexamined.

Second, the broader instruction expanded the search surface in ways the exact-match passes never attempted. Instead of only checking routes and raw IP tokens, this pass systematically swept every configuration section type: address objects, address groups, NAT rules, security rules, VPN gateway definitions, DHCP, DNS proxy, SNMP, syslog targets, and external lists. Critically, it added supernet overlap analysis -- testing whether any configured object was broader than the carrier subnets rather than only equal to or smaller than them.

That last technique is what cracked the case open.

The supernet sweep found four entirely new traces that every previous pass had missed:

1. Two address objects defining a 192.168.0.0/16 block, labeled for a separate internal network, that quietly contained all six of the carrier's gateway addresses, including two gateways that had no specific route at all.

2. That /16 object was a member of an address group referenced by three active security rules, meaning the firewall's policy still permits this carrier address space today even where routing does not deliver it.

3. An address object literally named after the carrier's gateway subnet, prefixed "Remove_", with its value deliberately altered to a bogus range. Someone, years ago, had neutered this object instead of deleting it. That is direct archaeological evidence the subnet was once actively configured.

4. A group description reading "From [previous firewall vendor]: (Interface was [other carrier]-DIA)", which explained the carrier labeling mismatch in one line. The objects had been migrated from the prior firewall, labels and all.

It also surfaced an operational finding with real consequences: two of the six gateways, including the one for our highest-bandwidth circuit, were permitted by policy but unrouted. Traffic toward them would follow the default route to the internet and die. If those circuits are supposed to work, our routing is incomplete. If they are not supposed to work, we may be paying for dead circuits. Either way, that finding is the reason we requested a utilization report from the carrier.

And for completeness, the pass ran the carrier hostnames, CLLI codes, port AIDs, equipment model numbers, raw service ID numbers, and billing account numbers as strings against both configs. It even investigated apparent matches for the string "ASE" and demonstrated they were substrings of crypto profile names, not real hits. Then it generated the formal report.

The scoreboard

To put numbers on it, here is what each phase of the investigation produced:

One case study, one environment, so calibrate accordingly. But within this investigation, the broad-spectrum pass found three times the traces, surfaced a verification gap the earlier passes had left open, and turned a flat "nothing else exists" into an actionable picture: which circuits are wired, which are policy-only ghosts, and which question to put to the carrier next.

Why the difference, practically speaking

My read after watching the same problem worked two different ways:

The exact-match pass answered the question I asked. I asked whether the spreadsheet values existed in the configs, and it checked whether the spreadsheet values existed in the configs. Thoroughly, honestly, with good subnet math.

The broad-spectrum pass answered the question I needed answered. "Does this exist" became "what is every possible way this carrier's address space could touch this firewall, including ways broader than, adjacent to, or historical relative to the literal values in the sheet." Supernet containment, policy reference tracing, and deprecated-object archaeology are all things an experienced firewall engineer would think to do on a third pass. The broader instruction is what put them on the table. Same data, same configs -- a different question.

The IPv6 catch is the part that stuck with me. An entire address family had gone unexamined across two passes, and the only thing that surfaced it was widening the question from "do these specific values appear" to "what have we not looked at yet." That is the whole lesson compressed into one finding: the exhaustive-sounding answer was a function of the narrow question, not of the configs being clean.

Takeaways for your own config archaeology

1. Exact-match searching proves almost nothing. Carrier inventories describe the carrier's side of the demarc. Your configs describe yours. The overlap lives in subnet math, not string matching.

2. Always run supernet overlap, not just containment. The dangerous objects are the broad ones somebody created fifteen years ago. A /16 will never string-match a /24 from a spreadsheet.

3. Trace objects into policy. An address object that exists is trivia. An address object referenced by three active allow rules is an attack surface and an audit finding.

4. Deprecated objects are evidence. That "Remove_" prefixed object told us more about circuit history than any document we possess.

5. Re-run with a broader question before you trust "nothing." I would have shipped an incomplete report with full confidence if I had stopped after the exact-match pass. The fix was not a smarter tool -- it was a wider question on a fresh pass, and the marginal cost of that pass was a few minutes. The marginal value was four findings and a carrier dispute armed with actual evidence. A different model on that pass doesn't hurt, but the load-bearing move is changing the question, not the tooling.

We closed the investigation with a formal report to the carrier requesting per-circuit utilization data, because the configs can only tell you what should flow, never what does. But we walked into that conversation knowing exactly which circuits our equipment can reach, which it merely tolerates, and which appear to exist only on an invoice.

That is a much stronger position than "we couldn't find anything," which is where I started the week.

All names, addresses, and identifiers in this article have been altered. The methodology is real; the data is not.