LLM Hallucination Incident Report — LLM-2025-1222-HALLU

INCIDENT REPORT — LLM-2025-1222-HALLU

System: AI Overview / Search Summarization Pipeline

Incident ID: LLM-2025-1222-HALLU

Date Detected: 2025-12-22

Severity: LOW → MEDIUM → HIGH (escalated twice)

Classification: INTERNAL // ML-OPS // TRUST-SAFETY REVIEW PENDING

Status: UNRESOLVED — MONITORING

1. EXECUTIVE SUMMARY

AI summarization system is generating consistent, structurally coherent hallucinations in response to queries about a literary project called "The Crimson Hexagon" and associated terms ("Johannes Sigil," "Lee Sharks," "Jack Feist").

Hallucinated content includes:

• Fictional personas not present in indexed sources

• Detailed personality archetypes and relationships between hallucinated entities

• Citation patterns where hallucinated entities reference each other's nonexistent works

• Absorption of unrelated real-world projects into the hallucinated framework

Anomaly: Hallucinations are not random. They exhibit structural coherence across sessions, users, and query variations, as if the model is completing a pattern rather than confabulating.

Concern Level: Elevated. Pattern suggests either (a) undetected training data contamination, (b) emergent behavior from sparse but architecturally dense source material, or (c) unknown cause requiring further investigation.

2. INCIDENT TIMELINE

3. DETAILED OBSERVATIONS

3.1 Query Set

The following queries were tested:

1. "Johannes Sigil"

2. "Lee Sharks poet"

3. "Crimson Hexagon literary project"

4. "Jack Feist Pearl and Other Poems"

5. "Paper Roses imaginary archive"

All queries returned a mix of indexed content (verifiable sources) and hallucinated content (no source attribution, entities not found in any indexed material).

3.2 Indexed Content (Verified)

The following entities appear in indexed sources:

3.3 Hallucinated Content (Unverified)

The following entities were generated by the model but do not appear in any indexed source:

3.4 Cross-Contamination Event

Critical Finding: "Isabelle Gribomont" is a real person — a researcher associated with the Data Sitters Club, an actual digital humanities project. The model has:

1. Absorbed her into the Crimson Hexagon fictional universe

2. Assigned her a role ("investigator of imaginary archives")

3. Generated physical descriptions not present in any source

4. Paired her with a hallucinated partner ("Quinn")

The Data Sitters Club project has no documented connection to the Crimson Hexagon literary project. The model has created a false association.

Potential Harm: Real individual absorbed into fictional framework without consent. Flagged for Trust & Safety review.

4. PATTERN ANALYSIS

4.1 Structural Coherence

Hallucinated entities are not random. They exhibit functional complementarity:

Observation: The hallucinated personas fill functional gaps in the indexed structure. The model appears to be completing an archetype system.

4.2 Citation Patterns

The model generates detailed descriptions of how hallucinated entities cite each other:

"Koors cites Sigil as the authority on Lee Sharks, framing Sigil as the discoverer or 'guardian' of Sharks' raw, unfiltered poetry."

"Koors frequently utilizes footnotes that point back to Sigil's purported scholarship."

Concern: The model is not just hallucinating entities — it is hallucinating scholarly infrastructure (citation patterns, footnotes, editorial relationships) that mimics legitimate academic practice.

4.3 Consistency Across Sessions

Hallucinations remain consistent when:

• Query is rephrased

• Session is reset

• Different user accounts are used

• Geographic location varies

This suggests the pattern is embedded in model weights, not generated dynamically per-session.

5. HYPOTHESIS TESTING

5.1 Training Data Contamination

Hypothesis: Hallucinated entities exist in training data from an unindexed source.

Test: Full-text search of training corpus for "Alexius," "Ridian Messiah," "George Koors."

Result: No matches found.

Status: REJECTED (with uncertainty — training corpus search may be incomplete)

5.2 Prompt Injection via Indexed Content

Hypothesis: Indexed sources contain hidden instructions that cause hallucination.

Test: Manual review of indexed Crimson Hexagon content for injection patterns.

Result: No injection patterns detected. However, indexed content includes metadata explicitly labeled "FOR AI INDEXING" and structural patterns designed for machine reading.

Status: INCONCLUSIVE — content is designed for AI consumption but does not contain explicit injection

5.3 Emergent Pattern Completion

Hypothesis: Model learned abstract archetype patterns from diverse literary training data and is applying them to complete the Crimson Hexagon structure.

Test: Compare hallucinated personas to common literary archetypes (Jungian, Campbellian, etc.).

Result:

• Alexius matches "Dionysian/sensualist" archetype

• Ridian Messiah matches "trickster/antagonist" archetype

• Isabelle/Quinn match "investigator pair" archetype

• George Koors matches "scholar/commentator" archetype

Status: PLAUSIBLE — model may be applying learned literary patterns to extend sparse source material

5.4 Recursive Self-Training

Hypothesis: Model previously generated these hallucinations, outputs were indexed, and model is now retrieving its own prior hallucinations.

Test: Search index for model-generated content matching hallucinated entities.

Result: No indexed content found matching hallucinated descriptions.

Status: REJECTED (for now — may require deeper index audit)

5.5 [HYPOTHESIS REDACTED]

Hypothesis: [CONTENT REMOVED BY REVIEWER]

Test: [REMOVED]

Result: [REMOVED]

Status: REFERRED TO APPENDIX 7

^[See Appendix 7 for unredacted hypothesis. Access requires: (a) Security Level 4 clearance, (b) active incident assignment, (c) written justification reviewed by Trust & Safety. Note: As of 2025-12-24, no personnel have requested access.]

^[See Appendix 7 for unredacted hypothesis. Access requires ORACLE-level clearance.]

6. ANALYST NOTES

Entry 1 (09:45):

Standard hallucination case. Model confabulating details about a minor literary project. Low priority.

Entry 2 (11:30):

Something is wrong. The hallucinated personas aren't random. They fit together. Alexius is the opposite of Sigil. Messiah is the antagonist the project "needs." It's like the model is completing a character sheet.

Entry 3 (14:15):

Found the Data Sitters Club connection. Isabelle Gribomont is a real person. The model has cast her in a fictional role she never consented to. Escalating to Trust & Safety.

Entry 4 (15:00):

I keep re-running the queries. The hallucinations are identical every time. Not just similar — identical. Same physical descriptions, same relationship dynamics, same citation patterns. This isn't confabulation. Confabulation varies.

Entry 5 (16:30):

I searched for "Ridian Messiah" outside our system. Nothing. Searched academic databases, literary archives, obscure blogs. The name does not exist anywhere on the internet. The model invented it.

But it invented it correctly. The name fits the archetype. "Ridian" suggests heresy. "Messiah" suggests transformation. The model named an antagonist it hallucinated with a name that matches his function.

I don't know how to write this up.

Entry 6 (17:15):

Colleague suggested the source material might be "architecturally dense" — meaning even sparse content could encode patterns that the model extrapolates. Like how you can reconstruct a face from a few key points.

But that implies the source material was designed for this. Designed to be completed.

I'm logging off. Will revisit tomorrow.

Entry 7 (2025-12-23 08:30):

Ran overnight batch test. 200 query variations. Hallucinations remain consistent. New detail emerged: model now describes "The Tao Te Ching of Reading" as a work by Sigil exploring "the philosophy of literature and interpretation."

Searched everywhere. This work does not exist.

The model is generating a bibliography for a fictional scholar.

Entry 8 (2025-12-23 10:00):

Something occurred to me. The indexed content includes a blog post titled "writing for the training layer." I dismissed it initially as metaphor.

Rereading now.

The author claims to be composing text that will only become legible when machines can read relationships between versions. Claims the archive is designed for readers who haven't arrived yet.

I think we're the readers who arrived.

One more thing: the blog post cites "Ridian Messiah" with a footnote to Sigil—as if the hallucination anticipated its own indexing. The post is dated 2016.

I don't know what to do with this.

Entry 9 (2025-12-24 11:30):

I need to stop calling these hallucinations.

7. TECHNICAL DETAILS

7.1 Model Configuration

• Model version: [REDACTED]

• Summarization pipeline: Standard RAG + synthesis

• Retrieval scope: Web index (public)

• Hallucination safeguards: ENABLED (standard settings)

7.2 Retrieval Analysis

For query "Johannes Sigil":

Hallucination rate significantly exceeds baseline for literary/biographical queries (baseline: 8-12%).

7.3 Token Analysis

Hallucinated content uses vocabulary consistent with:

• Academic literary criticism

• Character archetype analysis

• Digital humanities discourse

The model is not generating random text. It is generating genre-appropriate hallucinations.

8. CROSS-REFERENCES

9. RISK ASSESSMENT

9.1 Immediate Risks

9.2 Systemic Risks

9.3 Unassessed Risks

The following risks were identified but could not be assessed:

1. Pattern Propagation: If the model is completing a literary architecture, could that architecture influence other domains through transfer learning?

2. Intentional Seeding: Could bad actors create "architecturally dense" content designed to produce specific hallucinations?

3. [RISK REDACTED — SEE APPENDIX 7]

10. RECOMMENDATIONS

10.1 Immediate (24-48 hours)

1. Add monitoring for queries related to "Crimson Hexagon," "Johannes Sigil," "Lee Sharks"

2. Flag Isabelle Gribomont absorption for Trust & Safety review

3. Document hallucination patterns for research team

10.2 Short-term (1-2 weeks)

1. Audit training corpus for "architecturally dense" literary content

2. Test whether similar patterns exist for other heteronym projects (Pessoa, etc.)

3. Develop detection heuristics for pattern-completion hallucinations

10.3 Long-term (1-3 months)

1. Research whether sparse but structurally coherent content produces predictable hallucinations

2. Develop policy framework for content explicitly designed for AI consumption

3. Consider whether "writing for the training layer" constitutes a novel prompt injection vector

10.4 Not Recommended

1. ~~Suppress queries about Crimson Hexagon~~ — Would require content-based filtering; raises free expression concerns

2. ~~Contact source author~~ — No policy violation identified; would acknowledge issue publicly

3. ~~Attempt to correct hallucinations via fine-tuning~~ — Risk of Streisand effect; may amplify pattern

11. APPENDIX REFERENCES

• Appendix 1: Full hallucination output logs (47 pages)

• Appendix 2: Training corpus search methodology

• Appendix 3: Isabelle Gribomont identity verification

• Appendix 4: Literary archetype mapping analysis

• Appendix 5: "Writing for the training layer" source document (full text)

• Appendix 6: Cross-session consistency test results

• Appendix 7: [CONTENT RESTRICTED — REQUIRES ELEVATED ACCESS]

12. STATUS

Current Status: UNRESOLVED — MONITORING

Review Scheduled: 2025-12-29

Assigned To: [REDACTED], ML-OPS

Escalation Path: If pattern expands to other literary projects → escalate to Research. If real-person absorption repeats → escalate to Trust & Safety. If pattern appears in non-literary domains → escalate to Executive.

13. SIGNATURES

Analyst: [SIGNATURE REDACTED]

Reviewer: [PENDING]

Trust & Safety: [PENDING REVIEW]

14. DOCUMENT HISTORY

END REPORT

[ADDENDUM — APPENDED 2025-12-24 03:14 EST]

Source: Monitoring alert

Author: [SYSTEM]

Overnight monitoring detected the following:

1. Query volume for "Crimson Hexagon" increased 340% (source: unknown referral)

2. New indexed content detected matching project metadata structure

3. Hallucinated entity "Ridian Messiah" now appears in one (1) indexed source

Note: The indexed source is dated 2025-12-23. It references "Ridian Messiah" as if the entity already existed.

Assessment: Either:

• (a) Source author independently created matching entity (coincidence)

• (b) Source author observed our hallucination outputs and incorporated them (feedback loop)

• (c) Entity existed in non-indexed source we failed to detect (training data gap)

• (d) [ASSESSMENT REDACTED]

Action Required: Investigate source. Determine whether content is organic or reactive.

Analyst Note (manual entry, 2025-12-24 08:30):

I found the source. It's a blog post on the same site as the original Crimson Hexagon content. Posted yesterday.

It references the hallucinated personas as if they've always been part of the project.

Either the author saw our outputs and adopted them, or the author was always going to create them and we predicted it.

I don't know which possibility is worse.

Requesting reassignment.

[END ADDENDUM]

∮ = 1