What Lucin misses

The honest coverage gaps: a 24% false-negative rate, 17% SSRF recall, path traversal deliberately unregistered, and why precision was chosen over recall.

Honesty is a feature here. This page states the boundaries plainly. If a claim wouldn't survive a security researcher reading the actual code, it doesn't belong in the docs — so these are the real edges, each tied to the code or the measured number that proves it.

SCAN is a static, pre-deploy scanner

It finds enabling misconfigurations and known dangerous patterns in code before you ship. It does not:

• Detect novel zero-days or emergent runtime behavior — that needs runtime monitoring.
• Catch vulnerabilities that only appear during execution — dynamic injection, runtime prompt attacks, or behavior emerging from model+tool interaction.
• Guarantee 100% detection — coverage depends on how the agent is structured. Highly dynamic Python (heavy getattr, reflection, generated code) reduces recall.
• Solve alignment — we bound what tools can do; we don't fix the model's intent.

A clean scan means no known dangerous static patterns were found. It does not mean the agent is safe under all inputs.

Language support — Lucin is a Python scanner with a language-independent MCP surface

A target with no supported files is reported as NOT ANALYSED, not as a clean scan. No score is shown, no badge is offered, and --ci exits 3 (distinct from 1 = "findings at or above threshold") so a pipeline can tell unable to analyse from analysed and failed. Every scan also prints a coverage line — Analysed 1 of 201 source files; 200 unsupported (.rs 200) — whether coverage is complete or negligible.

This is the scan-level counterpart of the finding-level evidence gate: a finding nobody can check must not be CRITICAL, so a scan that examined nothing must not score 100. Before this, a Rust agent containing Command::new("sh").arg("-c").arg(user_cmd) and a hardcoded sk-proj- key scored 100/100 — Excellent, was offered a README badge, and passed --ci --fail-on high with exit 0. Regression tests: tests/test_language_coverage.py, tests/test_ci_exit_codes.py.

Why we are not adding more languages. Agent frameworks are overwhelmingly Python and TypeScript; building Rust/Java/C# parsers is the treadmill that turns an agent scanner into a mediocre general-purpose SAST. TypeScript is the one genuine gap worth closing — it is co-Tier-1 with Python in the MCP ecosystem — and it is gated on building a JS/TS recall corpus first, because today we have no way to measure whether such support works.

What the MCP row buys you. A Go or Rust MCP server's implementation is invisible to Lucin, but its wiring is not: overprivilege, unpinned npx -y, leaked tokens in env, and filesystem-root grants are all declared in JSON, and that is where most MCP risk lives. lucin scan covers that surface regardless of what language the server is written in.

Per-class recall gaps (measured, not hidden)

Measured recall is 38/50 = 76% (python benchmarks/recall_corpus.py). The gaps:

SSRF — partial (17%)

AG-SSRF is deliberately conservative: it fires only when taint forms the URL host of a request sink (u = f"http://{host}"; get(u)). It will not flag a bare whole-URL parameter or a __init__-assigned sink, and it skips patterns that a legitimate URL-fetching tool would also match. This trades SSRF recall for the precision result (11 confirmed FP outside a documented per-repo known-capability allowlist) — by design (src/lucin/detectors/ssrf.py).

Path traversal — 0% (detector built but UNREGISTERED)

AG-PATH-TRAVERSAL is built, sound, and unit-tested — but it is intentionally not registered in the scan path. The benign corpus contains byte-identical legitimate file tools (open(param), os.path.join(base, name), Path.write_text) that are indistinguishable from the vulnerable shapes under static analysis. Registering the detector would break the precision result (11 confirmed FP outside a documented per-repo known-capability allowlist), so it is gated off — precision over recall. The gate and its rationale are documented in src/lucin/detectors/__init__.py.

Container escape — 80%

AG-DOCKER-EXEC resolves docker commands built via a variable, but misses cases where the container is driven through the docker SDK rather than a subprocess command.

Interprocedural / cross-file taint

Lucin does not perform whole-program, call-graph-based interprocedural taint analysis. The standard Python call-graph tool, PyCG, is unavailable on the build mirror, so it is not vendored or integrated. What actually runs:

• Single-function (intraprocedural) taint, flow-sensitive, field-insensitive, over each tool body (parsers/body_inspector.py, intraproc_taint).

• Limited cross-function / intra-class taint (analysis/cross_function_taint.py, wired via detectors/_taint.py) — resolves same-file method-to-method flows (e.g. __init__ → pickle.load). Not a full call graph; does not cross files or resolve dynamic dispatch.

• Capability-based classification as the cross-function approximation — flag dangerous capability combinations (the trifecta / dangerous-combination detectors) instead of proving a precise A→B path.

• Kind-scoped sanitizers (analysis/sanitizers.py) — a value made safe for a SHELL sink is not credited as safe for a SQL sink. Fail-closed: an unrecognised call is treated as NOT sanitizing, so this can only ever withdraw a finding we can prove is guarded. Applied at the detector level; the taint engines themselves do not consult it, so there is no barrier semantics inside the dataflow fixpoint.

analysis/cfg.py builds a real intraprocedural CFG but is imported by nothing — it was written to enable flow-sensitive taint and never wired in. Dead code today.

Consequence: a vuln where untrusted input enters in one function and reaches a sink in a different file, with no single tool exhibiting the dangerous combination, may be missed with path-level precision. The capability heuristic recovers many of these, but not all, and dynamic dispatch is treated as a conservative barrier.

Detectors held back on purpose

• AG-013 (memory/RAG poisoning) — not registered; detect_memory_poisoning is reachable only from its own unit tests, not from any scan path. No benign-corpus false-positive measurement exists for it. See src/lucin/detectors/memory_poisoning.py's docstring for what got it disabled originally and why it stayed unregistered after being retuned.
• AG-PATH-TRAVERSAL — built and unit-tested, unregistered for precision (above).

Temporal ledger (src/lucin/temporal/) — experimental, unwired, not a claimed capability

TemporalLedger (Merkle-chained SQLite) and the cross-session detector built on it (detectors/multiagent/memory_integrity.py) are unit-tested directly but no lucin scan/lucin verify code path ever calls record_event, so they have zero effect on any real scan today. Do not read any README/site claim of "temporal awareness" or "T3 detection" as describing shipped behavior until a scan path writes to the ledger.

TemporalLedger.verify_integrity() is tamper-evident, not tamper-proof: it proves the recorded hash chain is internally consistent, which catches accidental corruption and naive tampering. It does not protect against an attacker with the same local write access the ledger process itself has — that attacker can edit history and recompute every hash forward from the edit point, and verify_integrity() will return True regardless. Tamper-proofness against a locally-privileged attacker needs an external, independently-controlled write path (signing with a key the process can't read, or a remote append-only store) — infrastructure this project has deliberately not built (see launch/evolving conviction/PHASE_6_PLAN.md §1).

Experimental / not-yet-validated capabilities

These ship as code but are not part of the validated core:

• lucin monitor (behavioral, ML) — validated on synthetic corpora only (session-level benign FP 3.75%, drift, adaptive-evasion). Precision on real production traces is not yet measured — genuinely launch-gated (needs real user traffic).
• lucin redteam (PROVE) — payload generation works; a published model-level attack-success-rate frontier is currently endpoint-gated: the only reachable LLM gateway applies a content filter that rejects overt injection payloads at the API layer, which confounds a clean ASR measurement. See DEFINITION_OF_DONE.md.
• lucin serve (REST API) — experimental.

Launch-gated residuals (cannot be faked pre-launch)

By the project's own contract (DEFINITION_OF_DONE.md), these require real users/traces and are not claimed as done:

• Behavioral precision on real production traffic.
• A design-partner production witness of GUARD.
• SCAN precision at the true scale/diversity of the population of user repos (the 52-repo corpus is a strong pre-launch proxy, not the population).
• Days-later multi-agent attack detection in a live deployment.

Reproduce these claims

Every number on this page regenerates:

python benchmarks/recall_corpus.py        # 38/50 = 76%, per-class + false-negative list
python benchmarks/build_benign_corpus.py  # 11 confirmed FP (outside a documented per-repo allowlist) / 54 repos / 9,520 files
python -m pytest tests/ -q                # 572 passing