The AI Problem Map

Why a Problem Map at all?

You run a RAG pipeline. Retrieval returns the right-looking chunks. Logs say success. Latency is fine. The answer still melts facts.

A database contains: A1. Company bankruptcy in 2023. A2. Founder released a product in 2022.

Model answer: It says the company launched a revolutionary product in 2023.

This is semantic drift. It is a silent failure. No stack trace, no exception, only a confident story that never happened. Teams lose days here. The Problem Map exists so you can name the bug, reproduce it on one screen, then apply a minimal fix.

What the Problem Map Is

Problem Map is a field guide for AI systems. It catalogs sixteen common failure modes across RAG, LLM agents, vector stores, OCR to PDF flows, routing, deployment, and multi-agent stacks. Each entry comes with symptoms, a minimal repro, a working fix, and a pointer to documentation.

Reference hub Problem Map https://github.com/onestardao/WFGY/tree/main/ProblemMap/README.md

WFGY is the reasoning layer behind the map. Think of it as a semantic firewall that sits above any model. MIT license. No retraining. No infra change. The layer ships as a tiny file that contains math operators for stabilization, recovery, and audit.

Core entry for the engine – WFGY Core https://github.com/onestardao/WFGY/blob/main/core/README.md

Starter guide – Starter Village https://github.com/onestardao/WFGY/blob/main/StarterVillage/README.md

Hero Log with real rescue cases https://github.com/onestardao/WFGY/discussions/10

The author of Tesseract.js starred the repo, which helps as a public signal of credibility. The project grew from zero to hundreds of stars in about two months based on real debugging help and not ads.

Quick Confirmation in One Chat Window

You can validate whether your issue matches one of the sixteen items without touching your code.

1. Open a fresh GPT or Claude chat.

2. Download the WFGY file from the core folder above, and attach it as a knowledge file.

3. Paste this:

Use WFGY to answer my question. First answer normally. Then re answer using WFGY. Compare depth, accuracy, and stability. Rate both. If this chat is missing the WFGY file refuse to run.

If the second answer keeps constraints tighter or shows a visible bridge and recovery step, the failure is within the map. Ask for the entry number and the minimum fix steps.

The Sixteen Problems In One Place

We label entries as No. 1 to No. 16. We avoid the hash mark so platforms do not auto-link to unrelated issues.

No. 1 Hallucination and Chunk Drift – Symptoms: Facts fused across sources, invented links, confident tone. Repro: Two short snippets that share surface nouns but disagree on time or entity. Fix: Semantic residue check and alignment. Docs Problem Map hub above.

No. 2 Interpretation Collapse – Symptoms: Task type drifts, output format flips mid-answer. Repro: Ask for table, then narration, then table again. Fix: Observe gate with layout anchors.

Docs Semantic Clinic Index: https://github.com/onestardao/WFGY/blob/main/ProblemMap/SemanticClinicIndex.md

No. 3 Long Reasoning Chains – Symptoms: Chain stalls at step four or loops, depth ceiling. Repro: The step puzzle with a hidden constraint. Fix: Multi-path progression plus bridge step.

No. 4 Bluffing and Overconfidence – Symptoms: Made-up citations, confident tone. Repro: Force citation, then verify links. Fix: Residue penalty with audit flag.

No. 5 Semantic not equal to Embedding – Symptoms: Nearest vectors are not semantically right, antonyms and temporal flips pass filters. Repro: Query with 2022 and 2023 events that share nouns. Fix: Query rewrite policy, e resonance thresholds, embedding QA.

Docs Vector store metrics, and FAISS pitfalls: https://github.com/onestardao/WFGY/blob/main/ProblemMap/vectorstore-metrics-and-faiss-pitfalls.md

No. 6 Logic Collapse and Recovery – Symptoms: Pipeline breaks, then repeats boilerplate. Repro: Insert a must-step that is easy to miss. Fix: Collapse, bridge, rebirth routine.

No. 7 Memory Breaks Across Sessions – Symptoms: multi-turn plans forget anchors when a new window opens. Repro: Continue a plan across two chat windows. Fix: Starter village memory anchors, plus observe the gate.

No. 8 Debugging is a Black Box – Symptoms: Logs show success while synthesis is wrong. Repro: Return 200 with wrong composition. Fix: Auditable telemetry and constraint deltas.

No. 9 Entropy Collapse in Long Context – Symptoms: Repetition, phrase loops, loss of diversity. Repro: Very long context copy, then ask for a fresh plan. Fix: Attention modulation and entropy pump.

No. 10 Creative Freeze – Symptoms: Refusal to attempt or collapse to cliches. Repro: Ask to blend three distant concepts. Fix: Head diversity and path sampling

No. 11 Semantic Drift in Routing – Symptoms: Two similar deep links handled differently. Repro: Router treats one as special intent and the other as plain path. Fix: Route normalizer, intent guard.

No. 12 Symbolic Collapse – Symptoms: Units drift, math rules mix, totals off by small margins. Repro: Unit change mid-chain with a hidden default. Fix: Cross-path guard and unit normalizer.

No. 13 Multi Agent Chaos – Symptoms: Agents overwrite each other, deadlocks, loops. Repro: Two agents write to the same state for the same goal. Fix: Position lock with global constraint aggregator.

No. 14 Bootstrap Ordering – Symptoms: System looks fine, yet later steps fail without a clear cause. Repro: Retriever called before index build, tool before secret. Fix: Safety boundary checklist.

Docs: https://github.com/onestardao/WFGY/blob/main/ProblemMap/bootstrap-ordering.md

No. 15 Deployment Deadlock – Symptoms: Works in private path, freezes on public path under load. Repro: Public route triggers an async that waits for a private resource. Fix: Watchdog, explicit timeouts, safe fallback.

Docs: https://github.com/onestardao/WFGY/blob/main/ProblemMap/deployment-deadlock.md

No. 16 Pre-deploy Collapse – Symptoms: Empty vector store, missing secret, early trigger. Repro: Call the action before setup completes. Fix: Preflight sanity checks with red flag blocks.

Docs: https://github.com/onestardao/WFGY/blob/main/ProblemMap/predeploy-collapse.md

A short diagnostic path that avoids rabbit holes.

1. Name the symptom first. Use the list. Do not jump to root cause.
2. Run a minimal repro. One retriever, one store, one prompt. Remove anything optional.
3. Apply the minimal fix. Start with the named module. Add only what you need.

This flow prevents endless tuning that never touches the real failure.

Under the hood in plain language.

WFGY provides a compact set of operators that behave like a reasoning layer.

• BBMC – bigbig semantic residue. Reduce residue to align intent and generated tokens.
• BBPF – multi-path progression. Explore, but keep a stability bound.
• BBCR – collapse, then bridge, then rebirth. Safe reset when a chain stalls.
• BBAM – attention modulation. Damp one token hijacks and reduces loops.
• Five gates for stability WRI. Where am I: position lock WAI, who am I: head diversity WAY, who are you: entropy pump WDT, where did you take me: cross path guard WTF, what happened: collapse detect and reset.

These ideas grew from rescuing real engineers on GitHub issues and discussions. The modules are model agnostic. You attach a small file from the core folder, and the layer runs in the chat. It supports GPT, Claude, Gemini, Mistral, and Grok. It is open source under MIT.

Deep dives that rank well on search and actually help.

RAG retrieval quality and FAISS pitfalls. Problem: Nearest neighbor search returns look close, yet semantics are wrong. Root surface similarity dominates, and temporal logic is missing. Fix: Query rewrite, e resonance bands, store metrics, guardrails around time entities. Doc vector store metrics and pitfalls page above.

Prompt injection that slips through role prompts. Problem: A hidden line inside a chunk disables your policy. Fix: Modular injection rules with an allow list and a block list, plus a bridge that isolates content from instruction. Doc prompt injection page above.

Multi-agent chaos. Problem: Two planners claim the same task and write to the same state. Fix: WRI to lock positions for roles, a global constraint aggregator, and explicit ownership of state segments.

Deployment deadlock under public traffic. Problem: Private path runs fine, yet the public path freezes. Cause the background trigger waits on a private resource, sometimes logging success. Fix: Watchdog, explicit timeouts, safe fallback, and preflight checklist in No. 15 and No. 16.

Frequently Asked Questions

Is this just a clever prompt? No. It is a compact spec you attach as a file. The model runs it as a contract. You will see a visible bridge and recovery step when the chain stalls.

Does it require fine-tuning? No. Zero training and no infrastructure edits.

Will this conflict with my agent tools? No. Treat it as an overlay. Your tools stay. The semantic firewall reduces drift and makes failures auditable.

What about licensing and credibility? MIT license. The project gained strong traction from real-world debugging. The creator of Tesseract.js starred the repo. Visual and text benchmarks live in the repository.

Links and Next Steps

• Problem Map main page https://github.com/onestardao/WFGY/tree/main/ProblemMap/README.md
• Engine overview and copy-paste steps https://github.com/onestardao/WFGY/blob/main/core/README.md
• Starter Village for newcomers https://github.com/onestardao/WFGY/blob/main/StarterVillage/README.md
• Hero Log of real rescues https://github.com/onestardao/WFGY/discussions/10

.