One pass, every language: moving ik's analysis in-process

ik runs 20 checks over a repository and turns them into a single score. Two of those checks — cyclomatic complexity and semantic duplication — used to shell out to external command-line tools: radon for Python, eslint for JavaScript/TypeScript, pmd for Java. If the tool wasn’t installed, the check didn’t run. It just quietly skipped.

We replaced all of that with a single in-process library: chamele, a pure-Go port of the lizard complexity analyzer. This post is about what that bought us, the performance trap it opened, and how we closed it.

The problem with shelling out

Shelling out to radon/eslint/pmd had three costs:

1. It skipped silently. No radon on the box? Python complexity scored nothing — and a repo with zero complexity findings looks clean. The most common outcome of “scan a fresh machine” was a check that didn’t run rather than a check that found nothing.
2. It needed installs. Every external analyzer is one more thing a user (or our scanner image) has to have on PATH, at a compatible version.
3. It was per-language. Three tools covered three languages. C#, Rust, Kotlin, Swift, Scala — no complexity signal at all.

In-process analysis fixes all three at once. The analyzer is compiled into the ik binary, so it is always available, never skips, and there is nothing to install.

What going in-process unlocked

chamele understands 27 languages. The day we switched, complexity went from “Go, Python, JS/TS, Java (if the tool is installed)” to first-class coverage for C#, Rust, Kotlin, Swift, Scala, PHP, Ruby and more — for free.

The concrete difference, scanning Django (a 2,918-file Python codebase):

A check that contributed nothing now contributes 414 real findings. On a fresh machine with no Python toolchain, that gap was invisible — the score looked better because a whole dimension of risk was missing.

And because chamele already computes per-function metrics during its parse, we got three brand-new checks almost for free — each just reads a field off the same FunctionInfo:

• Function Length — methods that should have been split (Django: 166).
• Parameter Count — signatures that grew one argument at a time (Django: 333).
• Nesting Depth — the if/for/try pyramids that bury the one branch that matters (Django: 758).

Semantic duplication got broader too. We deleted four hand-written regex extractors and let chamele find function spans instead — so duplicate detection now covers every language it knows, including the ones the regex extractors never handled.

(If you want the per-language breakdown of what fires where, the scanner pages spell it out.)

The catch: five walks of the same tree

Here’s the trap. Each check was independent, and each one walked and parsed the entire repository itself. After the migration, that meant:

• complexity (non-Go) — one full parse
• function-length — another
• parameter-count — another
• nesting-depth — another
• semantic-duplication — another

Up to five passes over the same files, every scan. On small repos, concurrency hides it. On large ones, it doesn’t — five tokenizers competing for the same cores is just slower. We measured it: on a 521K-line Python codebase the new checks added ~15 seconds of wall time, almost all of it redundant re-parsing.

Broader, never-skipping analysis was the right call. Paying for it five times over was not.

The fix: analyze once, share it

The five chamele-backed checks now share one analysis per run, behind a small concurrency-safe cache. The first check to need it triggers a single tree walk — with every metric computed in that one pass — and the other four read straight from the result instead of re-parsing.

One subtlety made this clean: the metrics aren’t all “core.” Nesting depth comes from an analyzer extension. So the shared pass always runs with that extension on, producing a superset — cyclomatic complexity, function length, parameter count, and nesting depth — that every consumer can read from. The checks that don’t care about nesting simply ignore that field.

We re-ran the same four repositories, comparing the binary before the cache (five passes) against the binary after it (one pass). End-to-end ik run wall time, best of two runs on a 4-core machine:

Small repos are a wash — there was never much to save. Large repos get most of the regression back: Django’s scan dropped by 9.2 seconds, landing close to where it was before the new checks existed. We added three checks and a whole new dimension of coverage, and the scan is barely slower than it was with none of them.

The win scales with repo size precisely because the waste did — the cache removes work proportional to (number of chamele checks − 1) × repo size.

What’s next: only re-scan what changed

The cache is per-run today: it makes one scan walk the tree once. The natural next step is to make that knowledge survive between scans.

chamele can already analyze an explicit list of files, and even raw source buffers — not just a directory. So for a large repo, an incremental ik run doesn’t have to re-parse everything: hash each file, keep the analysis for the ones that didn’t change, and feed only the changed files back through. We structured the cache’s storage around per-file results specifically so that layer slots in on top. That’s the next thing we’re building.

The theme here is boring on purpose: do the expensive thing once, in-process, for every language, and never skip it. That’s most of what makes a scanner you can trust on a machine you didn’t set up.

Want to see what it finds in your codebase? Install the CLI — one line, about a minute, no account required.