NetInspector: Measuring and Improving LLM Capabilities for Reliable Intent-Based Networking Policy Generation
NetInspector grounds LLM network-policy decisions and cuts security false negatives by over 30%.
The paper finds fine-tuned LLMs can translate operator intent into network configuration but often fail to flag intents that violate existing security policy. The authors attribute those false negatives to a lack of persistent grounding in topology and group hierarchy, not weak logical reasoning. NetInspector is a three-layer agent that retrieves verifiable facts from a live environment before approval. On the 2,224-sample NetInspector-Bench, covering campus, enterprise, and WAN topologies, it cuts false-negative rate by more than 30 percent versus ungrounded baselines and stays robust under phrasing shifts.
- Fine-tuned LLMs translate network intents well but miss policy violations.
- Errors come from missing topology and group-hierarchy grounding, not weak logic.
- NetInspector retrieves live network facts before any policy approval.
- On a 2,224-sample benchmark, relative false negatives fall by over 30 percent.
Full article235 words · extracted from arxiv.org · click to collapse
Modern networks are large in scale and heterogeneous in configuration, making manual policy management increasingly impractical. Intent-Based Networking (IBN) addresses this by automating the translation of high-level operator goals into low-level network configurations. Yet existing IBN systems rely on static heuristics and fixed-feature classifiers that generalize poorly to distribution shifts such as new service definitions or evolving phrasing in operator requests. Large Language Models (LLMs), with strong reasoning and translation capabilities demonstrated across many domains, are a natural candidate for IBN policy generation. However, it is unclear whether LLMs can be reliably applied to this task, nor whether their use mitigates or worsens the underlying security risk. In this work, we show that while fine-tuned LLMs excel at intent translation, they exhibit false negative rates when checking whether a proposed intent violates an existing security policy. The root cause is not a lack of logical reasoning capability, but LLMs lack of persistent grounding in network topology and group hierarchy. Motivated by this finding, we introduce NetInspector, a three-layer agentic framework that enforces a verify-then-act protocol, decoupling information retrieval from reasoning so that the LLM focuses on symbolic reasoning while every policy decision is grounded in verifiable network facts retrieved from a live Environment Layer before approval. On NetInspector-Bench, a 2,224-sample synthetic benchmark spanning campus, enterprise, and WAN topologies, NetInspector reduces FNR by over 30\% relative to ungrounded baselines and remains robust under linguistic distribution shifts.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.21103