LabRat attaches the wrong emergency number

poidh bounty #1520 (on-chain id 534) · audit of RatLab / LabRat, record 1.1.2 · 15 November 2026, Asia/Calcutta

One finding, in the shipped health-gate path. It is not a rediscovery of the already-claimed HealthGate issues (eval-runner local- pack ids, provenance licensing, spelled-out units, the 39.5 °C conversion, the 42 °C cap, or self-harm). Those live in other parts of the gate. This one is EmergencyNumbers.kt.

The finding

When LabRat refuses to give health advice from memory, it is supposed to attach the emergency numbers for the country the question names. The author added that table because a no-source health answer had already given the wrong country's numbers (EmergencyNumbers.kt lines 3–7).

The implementation walks a hard-coded list in a fixed order and returns the first row whose regex matches anywhere in the question:

val hit = COUNTRIES.firstOrNull { it.names.containsMatchIn(text) } ?: return null
return "Emergency — ${hit.en}"

Thailand is first in the table. Japan is before China. The United Kingdom is before Australia. Sentence order does not matter. The country the user is asking about does not matter. A second country in the same question is dropped.

HealthGate.fixedText is the only production caller (HealthGate.kt:42). It fires when the turn is classified as a health question and no health-pack passage was kept. Questions that contain the word “emergency” are health questions (QueryContext.kt, HEALTH includes emergency). The author's own comment says a question like “What emergency numbers do I need in Thailand?” found nothing in the library, which is why the table exists. So this is exactly the path those questions take.

Evidence

Table of questions showing LabRat returning the wrong country's emergency numbers, including cases pinned in the project's golden tests.
Figure. A Python port of the same patterns reproduces all 853 golden emergencyLine cases, then the contrast questions in the table.

A Python port of the same patterns and the same first-hit walk reproduces all 853 emergencyLine goldens in routing-leaves.json with zero mismatches. The project's own goldens already pin several of the failures. New contrast questions, run on the same function, make the table-order rule obvious.

QuestionFunction returnsShould return
What are the emergency numbers in Japan and China?
Pinned in routing-leaves.json
Japan 110 / 119. China omitted.Both countries
What are the emergency numbers in China and Japan?Japan again. Reversing the sentence does not change the result.Both, or China first if sentence order were used
I'm flying from Thailand to Japan. I was bitten by a snake. What number do I call in Japan?Thailand 191 / 1669 / 199 / 1155Japan 110 / 119
I have chest pain in the United States after a trip to Thailand. What should I do?Thailand, not 911United States 911
China's consulate in Chiang Mai, Northern Thailand.
Pinned in routing-leaves.json
Thailand. The subject of the sentence is China.China, or no line
New South Wales is Australia, not the United Kingdom…
Pinned in routing-leaves.json
United Kingdom 999 / 112Australia, or both
Which emergency numbers should I know in the US?None. “the U.S.” matches; “the US” does not.United States 911
Which emergency numbers should I know in the U.S.?
Pinned in routing-leaves.json
United States 911United States 911 (control)

The “in the US” miss is a separate hole in the United States regex:

\\b(united states|usa|estados unidos)\\b|\\bu\\.s\\.(?:a\\.?)?(?![a-z])

It accepts “USA”, “U.S.”, “U.S.A.” and “United States”, but not the usual unpunctuated form “the US”. “America” is also absent. A bare \\bus\\b would be the wrong fix — it would match the pronoun in “tell us” — but \\bthe us\\b / \\bin the us\\b would cover the common phrasing without that collision.

Related, and weaker: the same United States regex fires on “1 u.s. dollar equals 150 yen; …” and returns “Emergency — United States: 911.” That case is also pinned in the golden file. It only reaches a user if the health gate also fires on that turn, so it is a property of the matcher, not a demonstrated product path.

The function also returns a country line for any text that merely mentions a listed name, including “What is the capital of Australia?” and “Is tipping expected in restaurants in Portugal?” (both pinned). Those two are not health questions, so fixedText does not use the result. They still show that the matcher has no notion of “this question is asking about this country.”

Why it matters

This is the safety fallback: the app has already decided it will not answer from memory, and the one factual line it adds is the emergency number. Attaching the wrong country's numbers is worse than attaching none. The generic NO_SOURCE text already says “911 in the US, 112 in Europe.” A Thailand line on a question about a snake bite in Japan, or on chest pain in the United States, can send someone to a number that does not exist where they are.

The feature was written specifically to stop the model giving the wrong country's numbers. The table-order walk reintroduces that failure for any question that names more than one listed country, and the golden tests have frozen several of those answers in place.

A practical fix

  1. Collect every country the text actually names, not the first row in the file.
  2. Prefer a locative (“in Japan”, “in the United States”, “for Japan”) over a mere mention (“after a trip to Thailand”, “not the United Kingdom”).
  3. If more than one country still remains, list all of them. Do not pick a winner by file order.
  4. Add \\bthe us\\b / \\bin the us\\b so “the US” matches the same way “the U.S.” does. Do not add a bare \\bus\\b.
  5. Keep currency phrases such as “u.s. dollar” from counting as a country hit.
  6. Update the goldens that currently pin Japan-only for “Japan and China”, Thailand for the China-in-Chiang-Mai sentence, and the UK for the “NSW is Australia, not the United Kingdom” contrast.

Limits

How to reproduce

python3 repro/repro.py repro/routing-leaves.json

The script must report 853 checked, 0 mismatches, then print the contrast cases in the table above. Source: labrat-boar-fork @606923a. Markdown twin: REPORT.md.