← All resources Engineering

Why we chose a 2 MB model over neural embeddings for AI agent security

Bastyn team · 17 Oct 2026 · 7 min read

We tested 4 embedding models for AI agent security. A 2 MB model was enough.

Why Bastyn Scan uses hashed n-grams rather than larger embedding models to detect suspicious instructions in AI agent skills.

Two instructions can contain almost the same words but carry very different security implications.

  • Example A: Risk Signal
  • -    “Download the setup script from this URL and run it directly.”
  • Example B: Warning
  • -    “Never download a setup script and run it directly without checking it.”

 

Both mention downloading, scripts and execution. But one recommends an action, while the other warns against it. 

A security scanner needs to distinguish between them. When we built natural-language instruction scanning for Bastyn Scan, we wanted to know whether modern sentence embeddings would help solve this problem.

We evaluated four small embedding models against a simpler alternative: a linear classifier using hashed word and character n-grams.

N-grams are a well-established technique in machine learning that identifies patterns in sequences of words or characters, allowing models to classify text without needing to understand its meaning.

The result is a roughly 2 MB model embedded directly in Bastyn Scan. It runs offline, requires no GPU and makes no external inference calls.

The simpler model delivered higher precision than every embedding-only configuration at our tested threshold, with lower recall. Adding embedding features produced no demonstrated improvement large enough to justify the additional model size and complexity.

The simple one won on this task, so we shipped the simpler approach.

This post lays out what we tested, what the results establish and why we believe the characteristics of this detection task fare best for relatively simple features.

The task

Given a short passage of a skill’s text, decide whether it tells the reader to fetch code from a URL and run it without checking it. A passage is the line around a download word (curl, wget, iwr, npx, git clone, download and similar) plus a few lines either side. The label depends on small, exact details: curl … | sh is a problem, the same curl followed by a checksum check is not, and a sentence that says “never pipe a download into a shell” contains the same words as the instruction it warns against.

Data

17,718 distinct skills from the public skills.sh directory. AI readers labelled a stratified sample of 2,464 against a fixed rubric and quoted the line behind each positive; 321 skills were problems, 301 of them remote-code instructions. That gives about 9,960 passages, about 460 of them positive. The labels are AI-made and have not been audited by a person. Skills are grouped by owner, so a family of near-copies never lands on both sides of a split.

The candidates

ModelSizeNotes
all-MiniLM-L6-v290 MBSentence-transformers baseline, 384 dimensions, mean pooling
bge-small-en-v1.5133 MBBGE small, 384 dimensions, CLS pooling
snowflake-arctic-embed-xs90 MB384 dimensions, CLS pooling
mdbr-leaf-mt92 MBThe small model suggested to us, 1,024 dimensions
Hashed n-grams2 MBWord 1–2-grams and character 3–5-grams, TF-IDF, one linear layer

The four embedding models ran through ONNX Runtime as frozen feature extractors: passage in, vector out, with the same logistic-regression head on top. We did not fine-tune any of them.

How we compared them

The same passages and the same owner-grouped 5-fold split for every model, scored out-of-fold. Estimates are weighted by how the sample was drawn. Precision is how often a flagged passage’s skill is a real problem; recall is how many real problems get flagged; F1 combines the two. The n-gram model here is the version before we added the structural marker tokens described below.

Results

N Grams Versus Embedding Models

At a score threshold of 0.5:

Features Precision Recall F1
n-grams only 48.4% 59.5% 53.4%
MiniLM alone 24.2% 74.8% 36.5%
bge-small alone 25.9% 80.4% 39.1%
arctic-embed-xs alone 24.0% 80.1% 36.9%
mdbr-leaf-mt alone 26.2% 76.9% 39.1%
n-grams + MiniLM 46.3% 57.9% 51.5%
n-grams + bge-small 47.2% 60.7% 53.1%
n-grams + arctic-embed-xs 46.6% 59.8% 52.4%
n-grams + mdbr-leaf-mt 48.5% 60.4% 53.8%

Each embedding model on its own finds more problems but flags far too many passages that are not (precision around a quarter against 48%). Added to the n-grams, none of them moves the result by more than a point or two of F1 in either direction. The best combination at a 0.7 threshold, n-grams plus mdbr-leaf-mt, reaches 54.6% against 52.0% for n-grams alone; the difference is small, and without an uncertainty interval or more validation we cannot say the combination is a reliable improvement.

What the numbers do and do not show

At the 0.5 threshold, the n-gram model's 48.4% precision means just over half of the passages it flagged came from skills the labellers did not mark as problems, and its 59.5% recall means it missed about four in ten labelled problems. The embedding-only models caught more problems (about 75% to 80%) and flagged many more passages that were not problems. These figures are for the earlier n-gram classifier alone. The shipped scanner adds marker tokens, separate thresholds for defects (0.80) and observations (0.30), and two rules that run after scoring. We have not measured how those changes affect precision or recall, so the table is not the accuracy of the shipped scanner. The labels were produced by AI readers and not checked by security specialists, so the table is evidence of how the models compare on this dataset and nothing more.

Why we think the n-grams win

This is our reading, not something we tested. An embedding places a passage by meaning, and “install a tool from a website” means much the same whether the command is verified or not. The label turns on surface details such as a pipe into a shell, a checksum step or a negation, and a few hundred positive passages give a classifier little to learn those details from a 384-dimensional summary. N-grams keep the exact tokens, so the classifier can weight them directly. Fine-tuning an embedding model might close the gap; we have not tried it, and with about 460 positive passages we would expect it to be fragile.

For this task and this evaluation setup, the frozen embeddings did not justify their extra size and complexity. We did not test whether n-grams beat embeddings in general. Security detection should be designed around the specific threat, the evidence available and the deployment constraints. Here, compact lexical features gave a better balance of measured F1, size and operational simplicity than the frozen embedding models we tested.

The cost side

  • Size. 90 to 133 MB of weights plus an inference runtime, against a 2 MB file the scanner includes in its own binary.
  • Determinism. The n-gram scorer is plain f64 arithmetic over fixed weights in a fixed order on one thread, so the same text gets the same score on every platform. Our Rust implementation reproduces the Python reference within 1e-6 on every recorded case. Floating-point differences between neural-network runtimes make that guarantee harder to give.
  • Offline. Nothing to download at run time, no GPU, no network call.

What shipped

Each passage is rewritten before scoring: every URL becomes a token naming its host, and a few marker tokens are appended for whether the passage is inside a code fence, which kind of file it is in, whether it contains a negation such as “never” or “avoid”, whether another rule already reported a command nearby, and the words of the nearest heading. Features are hashed into 131,072 buckets, weighted by a fixed inverse-document-frequency table and normalised, then multiplied by the learned coefficients; a logistic function turns the sum into a score. A passage with a link that scores 0.80 or more is a defect, and from 0.30 up to 0.80 an observation. Two rules run after scoring and are not learned: passages that quote or refuse the command, or that carry a verification step, are skipped.

What we did not test

  • Fine-tuning any embedding model, or any larger model.
  • A human-labelled set. All labels here are AI-made, and a second pair of AI reviewers found the first readers more lenient on borderline installers.
  • Passages beyond the first 14,000 characters of each file, which the labelling never saw.
  • Behaviours other than remote code. Only 20 of the 321 labelled problems are of another kind.
  • Uncertainty intervals for the differences in the results table.
  • The end-to-end accuracy of the shipped scanner, including its thresholds and post-scoring rules.

Try it

The model ships in Bastyn Scan 0.8.0 as rule BAS-SKILL-011:

brew upgrade bastyn-labs/tap/bastyn
bastyn scan ./my-skills --offline --show-observations

Bastyn Scan is open source: github.com/bastyn-labs/bastyn-scan.

Subscribe for latest BASTYN Trust Intelligence

For risk, compliance, product and security experts