Docs

What it is

fixhunt tests the automatic fixes of a linter. A fix that changes what the program does is a finding; everything else is noise. It is written for the people who maintain those fixes: a rule's fix can be hunted with generated programs, or tried on a real repository under that repository's own tests. Findings come with the smallest reproducer the tool could reach, the exact tool version, both outputs, an advisory grade against the rule documentation, an upstream search, and an evidence sheet. Nothing is posted anywhere.

Hunt a rule

Scan a repository

• One rule at a time, its fix is applied to a copy of the project.
• The project's command (tests, a build, a script) runs before and after in a sandbox, network on.
• The command runs twice on the unmodified project first; a non-deterministic command stops the scan.
• A finding is a fix that changed the command's normalized output or exit status.
• Delta debugging over the fix's hunks reduces it to a 1-minimal set that still reproduces the difference, within a run budget.
• The reduced patch is applied with git apply and run once more before it is called confirmed.
• The changed files are linted again with the rule: sites the fix left behind are counted.
• Then judge, upstream search and evidence sheet, as for a hunt finding.
• Fixes with no difference are applied together, run once more and published as a patch.
• Default rules: the fixes the tool labels safe among the rules the repository enables; clang-tidy takes rule ids.
• From the dashboard the repository is cloned at depth 1 from an allow-listed host and discarded afterwards.

What a finding is not

• A finding is an observation of one deterministic run, not a proof that the rewrite is wrong for every input, and not a reading of the rule author's intent. The judge's grade is a model's reading of the documentation; it is labelled advisory.
• Several generated programs can share one root cause. The findings page groups them by rule; the counts are programs, not bugs.
• A scan observes one command. "No difference" means the normalized output and exit status matched for that command; code it does not exercise is not observed.
• Hunts observe stdout, the exit code and the first stderr line; scans compare stdout and the last stderr line after scrubbing timings, paths and positions. "Fix breaks the build" is a heuristic on that output (a parse, import or compile error at the start of a line).
• Fixers run on the host with the repository's own tool configuration disabled; programs and project commands run in disposable sandboxes.
• Only GitHub issue and pull-request URLs count as upstream matches. A "fixed in" note comes from a snippet and is advisory.

CLI

bun src/cli.ts hunt --tool <id> --rule <rule> [--n 8] [--rounds 1] [--unsafe]
bun src/cli.ts scan --tool <id> --dir <checkout> --run "<command>" [--rules A,B] [--limit k] [--no-net] [--no-minimize] [--no-judge] [--no-upstream]
bun src/cli.ts apply --tool <id> --dir <checkout> --run "<command>"     # scan, then apply the fixes that made no difference
bun src/cli.ts risk --tool <id> --dir <checkout>                        # enabled rules with a finding in the bank (no sandbox, no model)
bun src/cli.ts sweep --tool <id> [--n 8] [--rounds 2] [--limit k]       # hunt every rule with a safe fix
bun src/cli.ts check --tool <id> --rule <rule> --file prog.<ext>        # one program through the oracles
bun src/cli.ts serve [--port 8731] · render [--out runs/site] · tools · catalog --tool <id>

Tool specs

A tool is a JSON file in tools/ (or a directory in FIXHUNT_TOOLS, or --tool-file): how to print the version, look up a rule and its documentation, lint one file for one rule, apply the fix, check syntax, run a program in a sandbox image, read which rules a repository enables, and build the rule catalog. Biome, ruff, ESLint and clang-tidy ship as specs; no code in the pipeline names a tool. See docs/tool-spec.md.

Models and spend 2217 sandbox runs · $2.68

fast

nvidia/Nemotron-3_5-Lightning

gen

nvidia/nemotron-3-super-120b-a12b

mid

nvidia/nemotron-3-super-120b-a12b

judge

nvidia/Nemotron-3-Ultra-550b-a55b