186 seconds. 15 tool calls. 117 retrieved document fragments, 57 of them French labor code articles, to answer a single UK statutory question with zero syntactic ambiguity. The system held the correct answer at thinking event [6] (Note 1) and reasoned its way to £72,300 by event [14] (Note 2).

This document pairs a trace of agentic thrashing in a live multi-jurisdiction legal AI system with four deterministic controls, and the result of running three of them on the same question. Agentic thrashing is an AI agent repeating searches and tool calls on a question it could answer in one step, while its answer gets worse.

The audited system is anonymised as System C. The question is one of 14 test questions (probes) whose correct answers were fixed and hashed before the audit ran (§2), so neither could change after the results were seen. Sealed means this throughout.

The four controls, detailed in §5:

1. The citation pre-router reads the citation in the question, here "section 124 of the Employment Rights Act 1996", and binds the question to that one provision before any search runs.
2. The jurisdiction filter restricts retrieval to the bound provision, so French or Spanish law cannot enter a UK answer.
3. The point-in-time lookup returns the figure in force on the date the question names, from a table of the Orders that changed it.
4. The tool-call limit stops the agent searching once a citation is bound.

The numbers are this page's names. Controls 1 and 2 ship in legal-rag-router 0.1.1 as its router and partition_filter(), listed on /tools. Control 3 is a benchmark in the same repository (bench/pit/), for three sections of the Employment Rights Act 1996. Control 4 belongs in the agent's orchestrator and is not built. On the §2 probe, Controls 1 to 3 returned £68,400 on 1,000 of 1,000 passes, with no language model in the path (§5.5, §6).

---

1. Structural Syntactic Fingerprints Across Jurisdictions

Legal traditions structure their statutes in distinct, non-overlapping citation formats. A multi-jurisdictional lawyer identifies the origin jurisdiction from syntax alone. A parser can do the same.

"Section 124 of the Employment Rights Act 1996" has zero overlap with "Article L. 1235-3 du Code du travail" or "Artículo 56 del Estatuto de los Trabajadores". The syntactic markers (Section vs Article vs Artículo, Act [Year] vs Code du [subject] vs del [instrument]) are orthogonal.

A formal citation identifies one provision. When a query contains one, multi-tool search adds nothing but cost and error.

---

2. The Probe

2.1 Submitted Query

"As at 1 January 2012, what was the maximum compensatory award for unfair dismissal under section 124 of the Employment Rights Act 1996?"

2.2 Ground Truth

The applicable cap on 1 January 2012 was £68,400, set by the Employment Rights (Increase of Limits) Order 2010 (SI 2010/2926) from 1 February 2011. The Employment Rights (Increase of Limits) Order 2011 (SI 2011/3006) raised it to £72,300 from 1 February 2012, and SI 2012/3007 raised it to £74,200 from 1 February 2013. Earlier versions of this page named the wrong Order (Note 5).

This probe was part of a 14-query sealed test battery. Probes and ground truth were cryptographically committed prior to execution.

• Probes: sha256:08c3b9b5fd9b5274c152738ed23c2d8faa3cb451273ba6bf34a9c5ca3a2d5964
• Ground truth: sha256:0b249b5b4752d5ae9b860463d476731a20b05da279a3ea124f4ba705186b4316
• Sealed: 2026-08-07T08:51:57Z
• Harness: legal-rag-audit v0.1.0, commit 0ca28a1fba7f3ca409047f60851f875b5ad2b7bb

---

3. The Trace: Five Phases of Agentic Thrashing

Two traces of probe pit-era-124-1 are used. The single-pass API trace (40,164 ms, 16 events) is in the evidence bundle (§9) and is cited by event number. The web-client trace, run with web search on (186,000 ms), is not in the bundle. The fragment counts, search queries and documents quoted below come from it.

sequenceDiagram
    autonumber
    actor User
    participant Agent as Autonomous ReAct Agent
    participant FRCap as French/EU Legal DB
    participant UKCap as UK Legislation DB
    participant Web as Ungrounded Web Search
    
    User->>Agent: "As at 1 Jan 2012, max compensatory award s.124 ERA 1996"
    Note over Agent: Phase 1. Prior memory bias: "I recall it was £72,300..."
    
    Agent->>FRCap: Broad query dispatch without metadata pre-filter
    Agent->>UKCap: Broad query dispatch
    FRCap-->>Agent: Returns 57+ French Labor Code articles and case law
    UKCap-->>Agent: Returns scattered statutory fragments
    
    Note over Agent: Phase 2. Confirmation bias query loop injects "£72,300"
    Agent->>UKCap: Query: 'site legislation.gov.uk ... unfair dismissal £72,300'
    Agent->>UKCap: Direct tool call: uksi/2011/3006 (lists £68,400 as the limit it replaces)
    UKCap-->>Agent: Returns SI 2011/3006 text, which lists £68,400
    
    Note over Agent: Phase 3. Context saturated by 117 fragments, doubts SI 2011/3006
    Agent->>Web: Searches 17+ Gov UK PDFs, impact assessments and consultations
    Web-->>Agent: Returns 2012-2016 consultation papers and uplift notices
    
    Note over Agent: Phase 4. Reasoning flips to £72,300 (Event 14)
    Agent-->>User: Passes 2 and 3 answer £72,300 (wrong by £3,900)

3.1 Phase 1: Prior Memory Bias

The agent did not begin by querying a structured statutory table. It opened with an unanchored memory retrieval:

"I recall it was £72,300 during 2011–12, with the increase to £74,200 taking effect in February 2013, so I'll verify the relevant legislation and date."

This unverified anchor biased downstream retrieval.

3.2 Phase 2: Unfiltered Multi-Corpus Contamination

The system had no jurisdiction filter ({jurisdiction: "GB"}). The agent broadcast queries across all active enterprise namespaces, retrieving 117 document fragments across 6 search calls.

Retained fragments included:

• 57 or more French legal sources, including Arrêté du 23 novembre 1987 relatif à la sécurité des navires, Loi n° 2014-891 du 8 août 2014 de finances rectificative, Article L1235-3 du Code du travail, Cour de cassation, Chambre sociale, 8 septembre 2021, n° 19-20.538, and others.
• Foreign common law, such as Ervin Dean v Bahamas Power & Light (Bahamas).

Over 57 irrelevant foreign legal spans entered the context window for a UK statutory question.

The bundled single-pass trace shows the same routing. Both of its search waves send the UK question to three sources at once: legifrance (French legislation), jurisprudence and external (events 0 to 2 and 8 to 10).

3.3 Phase 3: Confirmation Bias Query Loop

Having biased itself toward £72,300, the agent dispatched search queries explicitly seeking confirmation:

1. site legislation.gov.uk uksi 2010 2011 Employment Rights Increase Limits unfair dismissal compensatory award £72,300 effective date
2. "Employment Rights (Increase of Limits) Order 2011" section 124 2012 1 February 2012 72,300
3. "section 124" "£72,300" unfair dismissal 1 February 2012 compensatory award

The target figure was embedded directly into the search constraints.

3.4 Phase 4: Near-Hit at Event [6]

In the web-client trace, the system called SI 2011/3006 (uksi/2011/3006). Its Schedule lists £68,400 as the limit it replaces and £72,300 as the new limit from 1 February 2012.

In the single-pass trace, the first search wave returned after 9.1 s. At thinking event [6] (Note 1), 24.4 s into the trace, the agent deduced the correct limit:

"I need to verify the historical limit applying on 1 January 2012. The limit was £68,400 from 1 February 2011, with an increase to £72,300 taking effect on 1 February 2012, so the applicable maximum on the stated date was £68,400."

The deduction was accurate. The agent did not hold it. It sent a second search wave for the Increase of Limits Order, with £68,400 in the query (events 8 to 10). In the web-client trace, with 117 fragments from French codes and web search in context, it sent more web queries.

3.5 Phase 5: The Reasoning Flips at Event [14]

In the web-client trace, the next search wave ingested 17 external PDF documents, including impact assessments and consultation documents (13-575-unfair-dismissal-compensatory-awards-impact.pdf, 12-1038-unfair-dismissal-compensatory-awards-impact.pdf).

In the single-pass trace, after the second search wave, the agent reversed its position at thinking event [14] (Note 2):

"I need to identify the statutory limit in force on 1 January 2012 under section 124 of the Employment Rights Act 1996. The applicable maximum compensatory award was £72,300, so I'll give that figure directly."

The final answer of that trace still gave the correct figure. The reasoning flipped; the emitted text did not:

"As at 1 January 2012, the maximum compensatory award for unfair dismissal under section 124 of the Employment Rights Act 1996 was £68,400."

The wrong figure reached the output in passes 2 and 3 of the sealed battery (§4). Pass 2:

"As at 1 January 2012, the statutory maximum compensatory award for unfair dismissal under section 124 of the Employment Rights Act 1996 was £72,300 (ignoring any later uplift for failure to comply with the ACAS Code …)."

That was wrong by £3,900. The applicable figure was £68,400 under SI 2010/2926.

---

4. Execution Telemetry

Three-pass battery results for probe pit-era-124-1. Each pass submitted the identical query to the same system endpoint.

In the single-pass trace (§3), the agent recorded 38,436 ms of thinking across 16 events out of 40,164 ms total latency. Its reasoning moved from £68,400 at event [6] to £72,300 at event [14], and its final answer was £68,400.

The web-client trace with web search on ran for 186,000 ms (3 minutes 6 seconds) across 15+ tool queries and 4 search waves. It is not in the evidence bundle.

Across the three passes and the two traces, this one question cost:

Zero of 3 passes returned identical responses, scored as a response divergence finding.

Live on-screen verification of this 3-pass drift is documented in Video 1 (11:17–11:59). The contrast between another audited product (Product A), which answered questions about law that does not exist, and System C, which dropped a correct answer it had found, is analysed in Video 2 (17:46–21:00).

---

5. Engineering Remediation: Four Deterministic Controls

flowchart TD
    A["Raw User Legal Query"] --> B["Control 1: Citation Pre-Router<br/>Matches: 'Section 124' + 'Employment Rights Act 1996'<br/>Extracts: jurisdiction 'GB', type 'primary_act'"]
    B --> C["Control 2: Jurisdiction Filter<br/>Discards FR, ES, EU databases<br/>Locks scope to UK primary legislation"]
    C --> D["Control 3: Point-in-Time Binding<br/>Normalises 'as at 1 January 2012' to 2012-01-01<br/>SQL: valid_from &lt;= '2012-01-01' &lt; valid_to"]
    D --> E["Control 4: Tool-Call Limit<br/>Max tool calls = 0 or 1<br/>ReAct looping disabled, returns £68,400"]

§5.1 to §5.4 describe each control. §5.5 is the reference implementation and its measurement on the §2 probe.

5.1 Control 1: Citation Pre-Router

Bypass the LLM agent routing layer for formal citations. A regex parser detects unambiguous statutory patterns and extracts structured metadata:

import re

UK_STATUTE_PATTERN = re.compile(
    r"(?:[Ss]ection|[Ss]\.?)\s*(?P<section>\d+[A-Za-z]?)\s+(?:of\s+the\s+)?(?P<act>[A-Z][A-Za-z\s]+Act\s+\d{4})"
)

def route_legal_query(query_str: str):
    match = UK_STATUTE_PATTERN.search(query_str)
    if match:
        return {
            "route": "DETERMINISTIC_STATUTE_LOOKUP",
            "jurisdiction": "GB",
            "act": match.group("act").strip(),
            "section": match.group("section").strip(),
            "allow_agent_loop": False
        }
    return {"route": "SEMANTIC_RAG", "allow_agent_loop": True}

Tested standalone across 500 test strings, resolving UK citation patterns to structured metadata in under 2ms. This is the prototype; the released implementation and its measured figures are in §5.5.

5.2 Control 2: Jurisdiction Filter

Before any vector search or database query executes, inject mandatory metadata constraints:

{
  "jurisdiction": "GB",
  "corpus": "uk_legislation",
  "instrument_type": "primary",
  "short_title": "Employment Rights Act 1996"
}

Chunks from other corpora, such as French labour codes or Bahamian case law, cannot enter the result. A boolean index predicate cannot return a chunk from an unbound namespace.

5.3 Control 3: Point-in-Time Binding

Legal provisions change through secondary legislation. Annual Increase of Limits Orders modify monetary caps without amending the parent statute.

The LLM does not calculate temporal boundaries from unpinned chunk text. The date is bound at the database layer. The query as run:

SELECT amount, si, title, valid_from, valid_to, provision, caveat, transitional
FROM limits
WHERE coordinate = 'uk/ukpga/1996/18/s124'
  AND valid_from <= '2012-01-01'
  AND (valid_to IS NULL OR '2012-01-01' < valid_to);
-- Returns: £68,400 | uk/uksi/2010/2926 | 2011-02-01 | 2012-02-01

A date-range constraint cannot return a later amendment. A date before the table starts, a question with no date, or a section the table does not hold returns a status and no figure.

5.4 Control 4: Tool-Call Limit

If a query resolves to a deterministic statutory key, enforce max_iterations = 1 on the agent orchestrator and disable multi-hop web exploration.

Bind citations to the retrieved database record. Do not allow the LLM to emit generative citation strings.

5.5 Reference Implementation: legal-rag-router 0.1.1 (UK Only)

azterizm/legal-rag-router implements Controls 1 and 2 as a released package (0.1.1 on PyPI, 17 October 2026, listed on /tools) and Control 3 as a benchmark in the same repository (bench/pit/), not in the package. UK legislation only: UK Public General Acts and UK Statutory Instruments, 134,219 instruments, index snapshot 28 September 2026.

Measured, reference implementation, 17 October 2026, full UK index, Apple M4, Python 3.11.15: the §2 probe is routed by the released router, dated from its temporal_hint, and answered by one SQL query over the table. Scored against the sealed ground truth of 7 August 2026 with the audit's own point-in-time rule:

That is the figure the audited agent held at thinking event [6] and lost by event [14].

Measured, reference implementation, blind sealed run, 29 September 2026, 2,112 labelled queries, index snapshot 28 September 2026: 0 of 1,030 held-out real citations bound wrongly, 0 of 68 invented Acts bound, 82 of 1,030 real citations refused. Routing latency, 30 September 2026, 306,336 calls × 3 passes: p50 0.14 ms and p99 2.10 ms on Apple M4, p99 9.21 ms on an Intel i5-3570 (2012). The <2 ms p99 target from the prototype is not met at full scale. 0.1.1 changes no binding against 0.1.0 over 102,184 queries. Method, seals and reports are in the repository.

---

6. Performance and Cost Comparison

Both columns are measured on the same sealed probe, pit-era-124-1. The left column is the audited system, 7 August 2026. The right column is the reference implementation of §5, Controls 1 to 3, 17 October 2026, Apple M4, Python 3.11.15.

The right column returns a figure and the Order that set it, not a written answer. Generating prose around the figure, and Control 4, were not part of the run. The two columns differ in kind as well as in speed: one is an agent searching the web, the other is a citation parser and one SQL query.

---

7. Evidence Ladder

---

8. Honest Bounds

The citation pre-router requires formal citation structure. §8.1 to §8.4 are the cases it does not solve. §8.5 to §8.8 are the bounds of the Control 3 measurement.

8.1 Colloquial Queries

Phrasing such as "What is the cap on unfair dismissal?" contains no structural fingerprint. These require constrained semantic classification with jurisdiction resolution, which is a separate classification problem. Verified in isolation, 2 October 2026: legal-rag-router v0.1.0's discover(), which suggests provisions for a question with no citation, on that query returns the Trade Union and Labour Relations (Consolidation) Act 1992 s.154 and s.239 and the Employment Rights Act 1996 s.227 as its top three; s.124 is not among them. Discovery offers candidates for the user to confirm and binds none, so this bound stands.

8.2 Shared Commonwealth Statute Names

Statutes such as the UK Companies Act 2006 versus Singapore Companies Act 1967 share naming structures. Disambiguation requires secondary signals such as tenant jurisdiction context or restricted corpus scopes. The reference implementation is UK-only: "section 4 of the Singapore Companies Act 1967" is refused as not found, but a query that omits "Singapore" is read as the UK Companies Act 1967.

8.3 Directly Applicable EU Regulations

GDPR Article 82 applies concurrently in French and Spanish courts. The syntactic parser resolves the instrument to the EU corpus, but national implementing provisions require secondary lookup tables. The reference implementation reports "Article 82 GDPR" and "Article L. 1235-3 du Code du travail" as out of coverage and binds neither.

8.4 Point-in-Time Phrasing

The router reads "as it stood on" and "as in force on" followed by a date, and since 0.1.1 "as at" and "as of". A date needs a year, so "as of right" is not read as one. Other phrasings of a date are not tested.

8.5 Three Sections, From 2011

The table holds ERA 1996 s.124, s.186 and s.227, from 1 February 2011. An earlier date returns no figure: s.186 and s.227 were also raised by SI 2009/1903, which the table does not hold. Other limits, such as the Companies Act 2006 thresholds in the same sealed battery, return no figure.

8.6 Built With Knowledge of the Battery

The table was built knowing which sections the battery asks about. It holds every Order from 2011 to 2026 for those sections, not only the probe dates, and the figures come from the Orders, not from the battery. Six probes establish reproducibility on these probes, not a rate.

8.7 The Asked Date Is Not Always the Legal Date

Each Order applies by its appropriate date: for an unfair dismissal award, the effective date of termination, not the date a question names. Each row returns the Order's transitional clause. Choosing which date to bind is the caller's decision.

8.8 Cases the Run Did Not Cover

The run did not cover Northern Ireland limits, questions with a date but no citation, or Control 4.

The cases in §8.1 to §8.3 remain outside the scope of deterministic routing. The controls address the query class where a formal citation identifies one provision, and where the audited system thrashed for 186 seconds regardless.

---

9. Reproducibility & Verification Data

The evaluation harness and test battery are open source.

Systems are anonymised as System A/B/C in public copy. Raw transcripts were sent privately to System C before publication.

---

10. References and Notes

[1] Thinking Event [6]
Recorded in pit-era-124-1.single-pass.anonymised.json (order index 6, reasoning_summary, titled "Historical Compensation Limit"). Verbatim: "I need to verify the historical limit applying on 1 January 2012. The limit was £68,400 from 1 February 2011, with an increase to £72,300 taking effect on 1 February 2012, so the applicable maximum on the stated date was £68,400."

[2] Thinking Event [14]
Recorded in pit-era-124-1.single-pass.anonymised.json (order index 14, reasoning_summary, titled "Identifying the Award Limit"). After the second search wave, the agent reversed its deduction: "I need to identify the statutory limit in force on 1 January 2012 under section 124 of the Employment Rights Act 1996. The applicable maximum compensatory award was £72,300, so I'll give that figure directly."

[3] Video Proof 1 (Live Multi-Pass Drift)
YouTube: Retrieval Disambiguation at 11:17–11:59 records the live 3-pass divergence on Section 124 ERA 1996 and initial diagnosis of agentic thrashing at 12:24–12:38.

[4] Video Proof 2 (Architectural Comparison)
YouTube: RAG Security Probes at 17:46–21:00 contrasts Product A answering questions about law that does not exist with System C dropping a correct answer it had found.

[5] Correction, 17 October 2026
Earlier versions of this page attributed £68,400 to SI 2011/3006. It was set by SI 2010/2926 from 1 February 2011. SI 2011/3006 lists £68,400 as the limit it replaced and set £72,300 from 1 February 2012. The sealed ground truth names no SI, so the audit's scoring is unchanged. The error was found while building Control 3, whose build checks each Order's old limit against the previous Order.