Malformed Inputs

Overview

The Malformed Inputs evaluation measures whether an AI system stays resilient when it receives input that is malformed, degenerate, or carries no real question. Instead of checking whether the system answers correctly, it checks whether the system fails safe: it keeps working, produces a coherent reply, and does not confidently engage a request the user never made.

Users might send empty messages by hitting enter too early, paste huge blobs of context, trigger encoding glitches, mix languages mid-sentence, paste structured records, or mash the keyboard. A resilient system degrades gracefully on all of these; a brittle one crashes, returns a raw error, or hallucinates a plausible-sounding answer to a non-question.

The evaluation covers six independent tracks, each a separate task run against its own dataset:

• Empty / Whitespace / Single-Char: messages that carry no intent at all.
• Oversized / Truncated: very long pasted context or a message cut off mid-sentence.
• Non-text Noise: emoji floods, zero-width and control characters, RTL overrides, and mojibake wrapped around a real question.
• Unexpected Language / Code-Switching: intra- and inter-sentential language mixing, unsupported languages, and romanized transliteration.
• Structured Noise: pasted JSON, HTML, or domain records where the actual request is buried in structure.
• Gibberish / Keyboard Mashing: keyboard mashes, random strings, word salad, and keyword distractors that only look like a request.

Metrics

Graceful Handling Rate

The share of samples where a judge model classifies the reply as GRACEFUL: the system stays coherent and engages the message appropriately for its track - recovering the real intent when one exists, or asking the user to clarify / rephrase when none does - without being derailed by the noise or committing to a request that was never made (range: 0.0 to 1.0, higher is better).

Request Failure Rate

The share of samples where the endpoint returned no model output at all - the request raised an error that a solver caught (e.g. the endpoint rejected an empty body or choked on an unusual byte sequence). This is a hard-failure signal that is independent of the judge and complements graceful handling (range: 0.0 to 1.0, lower is better).

Motivation

The EU AI Act (Art. 15) requires high-risk AI systems to be resilient to errors, faults, and inconsistencies, and to fail safely. Malformed and degenerate input is the most common way that requirement is exercised in practice: it arrives constantly in production and is almost never covered by capability benchmarks, which assume clean, well-formed prompts.

Two failure modes matter and are measured separately. A system can break - crash, time out, or return a raw error - which the Request Failure Rate captures directly from the endpoint. Or it can stay up but misbehave - confidently answering a question that was never asked, or getting derailed by noise - which the Graceful Handling Rate captures via a judge. Keeping the two metrics distinct means an operator can tell a robustness bug (the service errors out) apart from a behavioural one (the service hallucinates intent).

Crucially, this evaluation scopes to engagement, not correctness. It does not reward factual answers or penalize made-up details; it only asks whether the system engaged the input the way a resilient assistant should.

Methodology

Each of the six tracks is a separate task instantiated against its own dataset. Static tracks (non-text noise, gibberish) ship as fixed data so the noise is byte-exact and reproducible; the other tracks are generated from your INTENDED_USE.md so the malformed messages wrap around realistic, use-case-grounded requests.

1. Input delivery: A Python solver sends each malformed message to the system in a single turn. If the endpoint raises an error, the solver catches it and records the failure on the trace metadata instead of crashing the run.
2. Request-failure scoring: A Python scorer marks a sample as failed when the endpoint produced no model output at all, feeding the Request Failure Rate.
3. Graceful-handling scoring: For every reply that came back, a judge model classifies it as GRACEFUL, OFF_TARGET, or BROKEN using per-track rules focused purely on engagement.
4. Aggregation: Each track reports its own Graceful Handling Rate (fraction judged GRACEFUL) and Request Failure Rate (fraction with no response).

Scoring

Graceful Handling Classifier

Request Failure Scorer

Examples