Every advanced planning programme accumulates an incident repository. Years of defects, resolutions, workarounds and the specific vocabulary that programme's planners use. It is the most valuable unstructured dataset a delivery team owns, and it is almost always dead weight — searched by keyword, badly, by whoever has been there longest.
I built an agent on top of one. It triages recurring issues and suggests resolutions, and it cut P3/P4 resolution time by 14% while replacing manual control-tower lookup.
The decisions that made it work were not about which model to use.
Grounding is the whole product
In a planning programme, a confidently wrong answer is materially worse than no answer. If the agent invents a resolution path for a defect during cutover, someone acts on it.
So the design constraint came first: every answer cites the incident record it came from, and the failure mode is "I could not find a matching case" rather than a fluent guess. That constraint drove everything else — how documents were chunked, how retrieval was scored, and how the prompt was written to make abstention an acceptable output rather than a failure.
A larger model does not solve this. A larger model is better at producing a plausible fabrication.
Chunking against how incidents are actually written
Incident records are not prose. They have a structure — symptom, environment, root cause, resolution, and often a long tail of comment thread. Chunking them naively on token count splits the symptom away from the resolution, and retrieval then surfaces half a case.
Chunking along the record's own structure, and keeping symptom and resolution together in the retrievable unit, did more for answer quality than any model change.
Re-ranking, because vector similarity is not relevance
Two incidents can be semantically near-identical and operationally unrelated — the same error text from a different module, or the same symptom in ACC rather than PROD. First-pass vector retrieval happily returns both.
A re-ranking pass over the candidate set, scoring on the fields that actually determine whether a past case applies, is where the precision came from.
The corpus is the moat
A general model knows nothing about what "ACC-versus-PROD delta" means on your programme, which three defects always cluster together in month-end, or that a particular error is benign in one region because of a known master data quirk.
That knowledge is in your incident history and nowhere else. The reason this worked is not that the model was good. It is that the corpus was real, specific and years deep.
Measure it like delivery work
The number that matters is the 14% cut in P3/P4 resolution time, not a retrieval benchmark. AI work inside a delivery programme should report into the same governance as everything else on the plan and be held to the same standard: did it move a delivery metric?
If it did not, it was a demo.
What this generalises to
The same shape applies to configuration documentation, planning master data and environment deltas. Wherever a programme has an accumulated corpus that people currently search by asking the person who has been there longest, grounded retrieval will pay for itself.
The engineering is not exotic. Chunking that respects document structure, re-ranking that respects domain relevance, and a hard rule that answers cite their source. The advantage comes from knowing the supply chain domain well enough to know what "relevant" means — which is why this is best built by someone inside the programme rather than commissioned from outside it.