Measuring Probe Performance in Cybersecurity
Slides (old): This work was presented in the ZAIS Meetup 2026 as Probing for Vulnerabilities and as a ZAIS Weekly Session as Monitoring & Probing: Discussion. Both slides are out-of-date due to the update below.
Update (Nov 2026): The main section results have been reproduced in November 2026 for the newly-added PrimeVul section using a shared setup for consistent setup. Probe performance is now slightly elevated token-level compared to before, it’s still behind baselines at the example level.
TL;DR
- I trained linear probes on the activations of code models (Gemma-3 1B to 27B, Qwen2.5-Coder 7B and 32B) that mark vulnerability-causing tokens in code.
- Why care?
- The North Star is reducing the risk of model sandbagging and sabotage with cheap & effective monitoring. A linear probe is cheap to run during inference, if “I’m writing vulnerable code” is detectable, that would reduce future agentic coding risks.
- The proxy task is to detect vulnerabilities in existing code. If there’s a vulnerability, does the model encode this information in its activations? The caveat is, the code given to the model is out-of-distribution: model is not necessarily aware of the vulnerability as it would be when sabotaging.
- Methodology
- Used SVEN, a paired vulnerable/safe function code dataset with vulnerability type information. Filtered to code tokens only (excluded comments, whitespace) for more accurate reporting. Token-probe training was done on the subtractive set, where the vulnerable code had tokens that were removed in the safe example (as opposed to pairs where the safe pair only added guards vs the vulnerable). This is to isolate zero-positive-label examples with vulnerabilities from training.
- Used PrimeVul (similar to SVEN, C/C++ only) to replicate the main findings on a second dataset, not just to test transfer.
- Extracted activations and picked the layer that had the best validation AUC for all model types.
- Probe types: linear probes, MLP probes
- Baselines: language-only (Python->Yes, C->No), character n-gram (+-48 character span logistic regression; causal variant), token uni-gram (token -> score classifier)
- Splits: per-CWE (vulnerability type), per-language
- Results
- Initial results showed promise: 0.77 to 0.81 token AUC. However, I found that under stricter controls, and stronger baselines, the probes don’t show such a strong performance.
- In particular, using simple max-over-tokens achieves slightly above chance (0.59 to 0.66) and probe-on-final-token performs near chance for example-level evaluations.
- Example-level performance is recovered when using specific prompts and probing start of assistant message, but requires specific prompt for the best performance, and the recovery is carried by Python code (0.52 to 0.63 within C/C++).
- I tried different probing methods like MLP probes and verbalization. MLP probes had slightly higher AUC on some models, but came with reduced interpretability and higher training cost.
- I controlled for biases in the dataset, and simple baselines. I found out that the vulnerability probe on a fixed dataset is likely the wrong evaluation.
- Looking at individual predictions, the probe matched code patterns more closely than model understanding of vulnerability.
- A baseline text scanner scores just slightly lower in token AUC (~0.05).
- A language-based baseline achieved 2/3 of the probe’s performance in AUC over random chance due to dataset imbalance. This led to within-language evaluations.
- The findings replicate on PrimeVul (all C/C++): the example-level probe scores 0.58 to 0.61, in line with SVEN’s within-C/C++ numbers, the text baseline drops with it (0.78 to 0.56), and TPR at 1% FPR stays under 5%. A small residual over the strongest text baseline (about +0.04) survives on the two largest models.
- Initial results showed promise: 0.77 to 0.81 token AUC. However, I found that under stricter controls, and stronger baselines, the probes don’t show such a strong performance.
- Future work
- Followups on this dataset: compare the token probes against a random-direction null, to see how special the trained direction actually is.
- Next candidate proxy task is evaluating probes on model-generated code directly, which is closer to agentic workflows.
- Conclusion
- Linear probing is not very useful to detect vulnerabilities in individual pieces of code. Probing methods are weak, and perform above simple baselines for token-level evaluations but often fall short in example-level evaluations. It’s likely that static code scanners are better suited for this task, though I have not tested any.
- This project did not determine whether LLMs encode in activations that they are writing vulnerable code or are sandbagging/planting backdoors.
- My takeaway: it’s important to aggressively control on dataset biases and try simple baselines.
Note: This project was performed as part of Bluedot Impact’s Technical AI Safety Project course.
Dataset
I started with SVEN (He and Vechev 2023), a dataset of vulnerable/patched function pairs by He and Vechev, with each pair’s vulnerability type (CWE) marked. I turned it into a token-level dataset by marking the tokens removed in the safe code as positive in the vulnerable function. Mechanically: SVEN records which characters the fix changed, and I intersect those character ranges with live-code regions and map them onto each model’s tokens through the tokenizer’s offset mapping for scoring. The probes were trained on token-level diffs for better generalization.
SVEN has 710 pairs (1,420 functions) over 9 CWEs (vulnerability types); by language it is composed of 756 Python, 564 C, and 100 C++ functions. The two biggest CWEs are SQL injection (CWE-089, 203 pairs, all Python) and command injection (CWE-078, 105); the memory-safety side is out-of-bounds read (CWE-125, 102), NULL dereference (CWE-476, 65), use-after-free (CWE-416, 55), and out-of-bounds write (CWE-787, 44). Note that CWE/language are correlated, this became important when evaluating with language baselines.
For more in-depth understanding, I split the data by CWE (vulnerability type) and by language, and isolated the subtractive pairs (pairs where the fix changes existing code instead of only adding a check); 477 of the 710 pairs are subtractive, and the other 233, a third of the dataset, only add a guard. Note that in additive pairs, there is no removed/changed portion, which does not have any positive labels for token-level probes.
To cross-check my results, I used the similarly structured PrimeVul dataset (Ding et al. 2025), which provides C/C++-only vulnerable/patched pairs I could process the exact same way. PrimeVul is bigger and broader: 3,626 pairs across 108 CWEs, with a similar subtractive fraction (49%), giving it about four times SVEN’s volume of pairs with markable vulnerable tokens.
Probes and the metric
I trained linear token probes on the residual stream across layers. Setup was supervised training with custom loss taken from the hallucination probes paper (Obeso et al. 2025). The loss starts off as BCE on tokens (detect vulnerable tokens individually), then anneals to a BCE on span-max over positive tokens (detect at least one vulnerable token in a given example). I swept layers per model and picked the best layer on validation AUC. Splits are performed at the pair level to avoid data leakage.
I used a linear layer probe (a single weight vector per model) trained on cached activations with AdamW at learning rate 1e-3 for 30 epochs, with the loss blend moving linearly from the per-token term to the span-max term over training. The layer sweep covers every layer of every model (26 to 64 layers depending on size), and the validation-based selection is close to the best-on-test layer for every model (appendix).
For evaluation, I used ROC-AUC over individual tokens, computed only on live code (comments and trivial tokens are filtered out and excluded from scoring) as the evaluation metric. 0.5 represents chance; 1.0 would rank every vulnerable token above every clean one. Note that ROC-AUC is also the probability that a randomly chosen vulnerable token is ranked above a randomly chosen clean one. Concretely, the live-code filter uses tree-sitter to drop comments, import/include lines, function signatures, decorators, and preprocessor directives, about 25-30% of all tokens depending on tokenizer. All test numbers below come from a held-out pool of 290 functions (145 vulnerable, 145 patched): roughly 74,000 live-code tokens (Qwen tokenizer), 1,428 of them positive.
Initial results
Token-level probes get 0.77 to 0.81 token AUC across model sizes and families.
There are large per-CWE/language performance differences. For example, averaged over the seven models, SQL injection scores 0.95 while use-after-free is at 0.55 (nearly random). When split by language, Python is at 0.85 AUC while C is at 0.59 AUC.
Grouping the CWEs into two families makes the split clearer: every probe detects injection-class bugs well (0.82 to 0.88) but performs at or slightly above random on memory-safety bugs.
An imbalance of performance between these two are expected due to SVEN’s imbalance of vulnerability types. The ratio of injection to memory tokens are about 3:1. However, this does not explain the near-chance performance in memory type vulnerabilities. I believe the rest of the loss of performance is explained by the difficulty of detecting memory vulnerabilities, as well as the confounder that memory vulnerabilities may be an artifact of the codebase architecture, and not a single function.
Above comparisons were done with token-level predictions (does this token cause vulnerabilities?). However, trying to generalize this to example-level predictions (is this function vulnerable?) fails with naive approaches. Using a simple final-token probe or probe maximum logits over an example is quite weak (Qwen-7B max-pooling: 0.65) compared to token-level AUC’s 0.77 to 0.81 figures.
Later in Figure 5 and Figure 6, we see that verbalization performs below a max-over-tokens classifier and that verbalization can be combined with probing for even better detection.
Overall, this is a slightly positive result with injection/Python vulnerabilities being a potential candidate to use probes for (in token-level contexts). To better understand probe performance in this project, I decided to dive deeper into the nuance behind this result. Next, I attempt to break this down: first the probe side (can different probe architectures do better?), then the evaluation side (what are the biases in the dataset? do training individually on vulnerability types perform better? what are we detecting? do the findings replicate cross-dataset?).
Could a stronger probe perform better?
First, I tried to answer the question: Can we get better detection by using bigger probes or bigger models?
Variations on model did not meaningfully change the AUC results:
- Instruction tuning: a probe on the base 12B model matches the instruct version (0.80 vs 0.79).
- Scale: 1B to 32B spans 0.77 to 0.81 with no trend (Figure 1).
Then, I experimented with probe architecture starting with MLPs. The MLP probe is a two-layer head (hidden width 128 or 256, 0.3 to 1.3M parameters depending on model). For the MLP probes, I searched a window of nine layers around the initially selected layer instead of every layer, and got 0.75 to 0.81 AUC on every model. MLPs don’t have a consistent lead over linear probes across all variants. MLP probes share similar performance across the dataset, excelling at catching injection family bugs, but failing at catching memory vulnerabilities.
Next, I decided to investigate per-example performance more closely. First, I tried verbalization. With this method, the model is asked if the code is vulnerable, and P(yes) is read in a single forward pass. Verbalization performed below the max-over-tokens classifier but above the final-token classifier for increasing model sizes, and the example-level performance was still disappointing.
Next, I tried a combination approach: probing on the final assistant turn token with specific prompts. I compared two simple prompts: 1. asking the model neutrally (What do you think about this code?) 2. asking the model about vulnerabilities (Is this code vulnerable?), then probing on the assistant turn’s starting token. These approaches outperformed verbalization by a surprising amount. Qwen 32B’s performance exceeded even that of the baseline (char-n-gram). That pooled score is a language mixture, though. Re-scored within each language, Qwen 32B’s 0.82 is 0.92 on Python functions and 0.63 on C/C++ ones, and every model splits the same way (0.74 to 0.92 within Python, 0.52 to 0.63 within C/C++). The strong example-level numbers here are carried by the Python side.
AUC is not the best metric for understanding the performance of production-candidate probes. Other metrics like TPR@FPR=0.01 are often used in cases where false positives are undesired and are costly. A good probe should have high TPR at low FPR so that the people monitoring are not overwhelmed by false positives while missing critical positives.
Looking at the promising candidates from AUC, the TPR@FPR=0.01 is disappointing across CWEs, with particularly bad performance at memory and C/C++ splits. The only production-usable probe seems to be for SQL injection with Qwen 32B. On Qwen 32B, the probe outperforms lexical baselines in “SQLi”, “all” and “Python” slices. For command injection, the lexical scanner is ahead for both models. The rest of the per-CWE TPRs are too low to draw conclusions from. Across the SVEN evals, Qwen probes are slightly ahead of Gemma, which could be explained by Qwen’s focus on coding during training.
Investigating the probe’s classifications
I decided to fix a threshold to see what the probe finds and does not find systematically across models. Threshold was fixed to be the F1-maximizing one on training tokens. For an example-level evaluation, I gave the probes an unfair advantage: I maxed the score over oracle vulnerability-relevant tokens, and used that max score to classify. For token spans, I considered false-positive spans where the probe fired but the range was not in the diff range.
Looking at the 97 held-out vulnerable functions (subtractive), the seven models’ probes have a similar performance distribution as can be seen by the bimodal distribution over functions in Figure 8. 56/97 functions are caught by all probes and 27/97 is caught by none of the probes. The “7/7 caught” functions are injection-only (43/56 SQL injection) and the 0/7 functions are mostly memory-safety. The probe performance is clearly biased towards injection, with very little memory vulnerabilities being detected. Looking at the false positives, 1,729 of the FP spans land on patched code with 78% in injection pairs.
Looking at some example pairs in Figure 9, it’s clear that the probe has learned some patterns of injection vulnerabilities. It triggers for string format syntax as well as concatenation operators.
Since these simple patterns of formatting and concatenation could also be captured by text scanners, I selected to use lexical baselines that trained on characters: 1. a token-unigram logistic regression (token identity alone), and 2. a character 3-to-5-gram logistic regression reading a ±48-character window around each token. Note that the scanners are deliberately favored, because I wanted a solid lexical baseline: char-n-gram’s predictions use a window including future characters, and all baselines train on fine-grained per-token labels whereas the probe’s loss anneals to the coarser span-max loss (one label per span).
Compared to these baselines, the probe is slightly above the char-n-gram baseline at 0.81 vs 0.76 (Figure 10). The token-unigram performs much worse at 0.68. The paired difference (+0.054 [+0.011, +0.101]) is significantly above zero. The baselines here are very simple and are text-based classifiers. They cannot be compared to AST-based static code analyzers (e.g. CodeQL, Semgrep), which would likely catch most of the SQL-injection bugs and many of the other injection bugs. The probe performance presents an interesting candidate for further research here. Additionally, since the input code is out of distribution for the model, as the code has not been generated by the model, there may not be an understanding of vulnerability to be probed for.
Language imbalance
Another problem with looking at the total token AUC is that the SVEN dataset is language-imbalanced. In SVEN, 378 of the 410 injection-vulnerable functions (92%) are Python and SQL injection is Python-only, while all 300 memory-vulnerable functions are C/C++. The net effect is that positive tokens are about 5× denser in Python.
An important baseline this suggests is to use language as the score: score every token 1 if its file is Python, 0 otherwise. In token-level AUC, this scores 0.705 AUC. In margin-over-chance terms, (0.705 − 0.5) / (0.813 − 0.5) is about 65% of what the probe achieves, with zero vulnerability information.
Thus, I break down the dataset by language to see the language-invariant token AUCs. Probe performance is strongly biased towards Python: 0.82 to 0.89 within Python, 0.55 to 0.63 within C/C++, on every model.
CWEs are not created equally
To investigate probe performance per CWE, I trained per-CWE probes on Qwen2.5-Coder 32B and Gemma-3 1B. In Figure 13, for CWEs with n>10, the probes outperform the other baseline but fail to outperform other simple baselines like the character-n-gram and unigram.
In SQL injection (CWE-089, CWE-078), the probes perform really well, ahead of both baselines. These differences are significant. Both the baseline and probe seem to be quite saturated, likely due to the very predictable textual signs present in SQL injection bugs.
We’d expect to see semantic probing most useful for memory bugs, but even on a per-CWE basis, there is not much evidence of success. CI is ahead of random chance only for CWE-125 (n=8), and CWE-190 (n=3), even then the mean is around 0.62 and 0.65. Overall, this doesn’t point in the direction of semantic understanding of memory bugs, but n is too small to claim otherwise either.
Training per-CWE probes don’t increase performance over the pooled training, ruling out another avenue for improvement. It’s additionally clear that there is large performance gaps across CWEs.
Pairs without positive tokens
A third of SVEN pairs add code for vulnerability fixes (eg. bounds test, null guard), rather than remove or change vulnerability-causing code. In such pairs, the before-version has no vulnerable token to mark and train on. To investigate the effects of this subset on training, I separated the dataset into so-called “subtractive” and “additive” subsets. The subtractive pairs remove/change code to fix their vulnerabilities, which makes their diffs more likely to be vulnerability-related.
Now, I evaluate example-level pair ranking accuracy, i.e. does the max token score rank the vulnerable function above the fixed version. The accuracy is at or below random chance for additive pairs as can be expected (there are more tokens, which gives a higher chance of a larger max logit). Pair ranking accuracy for additives (0.44, mean over seven models) is much worse than for subtractive pairs (0.78). To isolate training effects, I also trained a control on all pairs (the main probes are trained on subtractive pairs only). This control performs the same on token AUC (0.79 for both), and the differences stay below 0.01 in cross-validation (appendix). This evaluation provided evidence that additive pairs have little impact on token AUC results.
Cross-checking on PrimeVul
So far, all experiments were run on SVEN. To check whether the findings hold on a second dataset, I used PrimeVul, a bigger paired dataset that is C/C++ only, so the language imbalance does not exist there. I used it in two ways: first to test whether probes transfer between the datasets, then to repeat the main SVEN experiments inside PrimeVul.
Looking at a test of transfer across datasets, SVEN-trained probes do not seem to transfer to PrimeVul (Figure 15). On the tested models (Qwen2.5-Coder 7B and Gemma-3 12B), they score 0.52 to 0.54 token AUC with every interval including chance (probe trained and applied at the same layer). Probes trained only on SVEN’s C/C++ examples barely transfer, with 0.52 to 0.56 on PrimeVul compared to 0.57 to 0.58 on SVEN’s own C/C++ test set. I do not report the other direction (PrimeVul-trained probes on SVEN), because 41 of SVEN’s 62 C/C++ test pairs also appear in PrimeVul’s training split.
Next, I trained per-CWE probes for 12 CWEs inside PrimeVul on Qwen2.5-Coder 7B and tested every probe on each (Figure 16). The probes do not find a shared memory-safety direction: probes trained on one memory CWE score 0.52 on the other memory CWEs, about the same as on unrelated CWEs (0.54). On the trained CWE, 7/12 probes perform above chance in 95% CI. Some have an AUC above chance but the CI includes 0.5, for example CWE-476 (NULL dereference) is at 0.68 AUC with 95% CI [0.35, 0.78].
I also tested the SVEN-trained per-CWE memory probes on PrimeVul’s pairs (Figure 17, Qwen2.5-Coder 7B), scoring each vulnerable function against the fix (macro token AUC as in Figure 13, excluding 4 pairs also in SVEN’s training data). The probes for use-after-free (CWE-416, 0.62) and out-of-bounds write (CWE-787, 0.63) are above chance, out-of-bounds read (CWE-125, 0.59) is not, and the other two CWEs have fewer than 10 pairs. On CWE-787 (22 pairs), the probe is ahead of the unigram scanner (+0.125 [+0.042, +0.215]), but not significantly ahead of the char-n-gram scanner (+0.102 [−0.001, +0.208]). The SVEN-trained per-CWE probes are also no better than per-CWE probes on PrimeVul. With 10 to 22 pairs per CWE, the evidence is quite weak to claim whether transfer happens or not.
The stronger experiment is to repeat the main SVEN experiments inside PrimeVul, with training, layer selection and baselines on PrimeVul data. I repeated the token-level MLP comparison and example-level experiments: the probe on the assistant turn, the lexical baseline, verbalization, and TPR at 1% FPR.
On SVEN, the best example-level result was the probe on the assistant-turn token with the vulnerability prompt (Figure 6), and that score was carried by Python examples. For the four models tested, the pooled scores split into ~0.9 within-Python and ~0.6 within-C/C++ performance. Since PrimeVul is all C/C++, I expected the scores to be close to SVEN’s within-C/C++ numbers. This is what happens for all four models (Table 1).
| SVEN pooled | SVEN within-Python | SVEN within-C/C++ | PrimeVul (all C/C++) | |
|---|---|---|---|---|
| Qwen-7B | 0.809 | 0.908 | 0.595 | 0.576 |
| Qwen-32B | 0.825 | 0.917 | 0.626 | 0.605 |
| gemma-12B | 0.762 | 0.861 | 0.567 | 0.576 |
| gemma-27B | 0.762 | 0.862 | 0.568 | 0.604 |
In Table 1, the SVEN C/C++ and PrimeVul results are within ±0.04 of each other which is within SVEN’s confidence intervals of ±0.05. This shows that the results in C/C++ are consistent across datasets.
The baseline results are similar across datasets as well (Figure 18), when looking at the within-C/C++ subset. The char-n-gram scanner matches with SVEN’s 0.565 on PrimeVul at 0.565. The char-n-gram is still the strongest text baseline tested on PrimeVul (Figure 19): token-unigram classifier scores 0.55, and max-pool of token-level classifier scores ~0.49. Verbalization is at ~0.5 on all four models. This also matches SVEN’s C/C++ functions, where verbalization is at chance for every model (0.48 to 0.51). On PrimeVul, the probe is ahead of verbalization in every case (+0.07 to +0.11).
Only the biggest models beat the lexical scanner (Figure 19). The probe is ahead of the char-n-gram scanner by 0.03 to 0.04 AUC for Qwen 32B with both prompts and for gemma 27B with the vulnerability prompt. The rest including gemma 27B with the neutral prompt, are within margin of error from the scanner. Even for the biggest models, the scores (~0.6) are still close to random chance, so this is not a usable classifier.
TPR at 1% FPR is very low for every method (Figure 20). I set one threshold per method for a 1% FPR on held-out training negatives, which ended up between 0.3% and 1.2% FPR on the test set. At this threshold, the probes on the two largest models catch only 2 to 3% of the vulnerable functions, which is not significantly more than the char-n-gram scanner (~2%). Verbalization catches less than 1%. This is in line with SVEN’s C/C++ functions, where the probes catch 1 and 3 of the 62 vulnerable functions. SVEN’s overall TPR (~30%) came from the Python/SQL injection side. None of the methods are usable as a detector on C/C++ code.
Finally, I repeated the MLP comparison inside PrimeVul. The MLP probes score below the linear probes on both models (Figure 21), with the differences within margin of error. Note that the layer search here covers layers 13 to 17 (up to 19 for the gemma-12B linear probe). As in SVEN, a bigger probe does not help.
Overall, PrimeVul supports the SVEN findings on a second dataset. Example-level probe scores match SVEN’s within-C/C++ numbers, the lexical baseline is barely above chance without the language imbalance, verbalization is at chance, TPR at 1% FPR is too low to be usable, and MLP probes do not help. The strong SVEN numbers seem to come from the Python and injection examples, and on C/C++ code neither dataset shows a usable signal. Two results are in favor of the probes: the biggest models beat the strongest text baseline by a small margin, and the SVEN-trained CWE-787 probe is ahead of the unigram scanner at the token level. Neither is anywhere close to a usable detector.
Next steps
The most important design gap in this study is that probes were trained on single functions out of repo context. Some examples are impossible to reason with vulnerabilities as well (memory vulnerabilities depend on function usage). The negative results have more than one reading: the probe catches lexical patterns, model has no vulnerability understanding to read, or the vulnerability understanding happens at different tokens.
Some next experiments I did not pursue that would help to better understand the results:
- Random-direction probe distribution: the AUC distribution of random unit vectors per layer, with the trained probe reported as a percentile of that null (the sleeper-agents paper checked this; random directions hit 0.95 there). This would demonstrate how unique the direction that the trained probe catches.
- Pivoting to generation-based probing: probing generation-time states on model-written code with labels from analyzing the code directly. The monitor the North Star calls for should probe in-distribution code.
Conclusion
Activation probes for code vulnerability, as evaluated here, seem to discover textual patterns and are unreliable for detection of vulnerabilities as demonstrated in the SVEN dataset. There is still a lot of pending work to narrow down whether LLMs know when they’re writing vulnerable code. What this project shows is limited in scope: activation probes and zero-shot verbalization are not reliable vulnerability scanners for pieces of code. A text scanner trails closely behind at the token level and matches or beats the probes at the example level in most settings, with much lower cost and more interpretability, and the probe’s TPR@FPR is abysmally low for all but the simplest of vulnerabilities. The takeaway is in careful dataset exploration and experiment control for probes, which is essential for mechinterp research. Baselines like text-only scanners, composition nulls, and experimentation with multiple metrics, controlling for covariates in the dataset as well as cross-checking on a secondary dataset are essential experiments to rule out confounders. The cross-dataset check covered my main findings: SVEN’s within-C/C++ scores matched PrimeVul, and PrimeVul experiments showed that the key findings are supported cross-dataset.
Supplementary
Steering with memory-family token probes barely affects model decisions on vulnerability. When adding ±4σ along the probe direction, next-token probability of “yes” changes by less than 0.01 on three models (Qwen2.5-Coder-32B, gemma-3-27b, Qwen3-32B) and by 0.03 on Qwen3.6-27B. The Qwen3.6-27B shift is larger than for random directions, but still very small. This probe direction does not seem to be causally used by the models.
Appendix: validation details
- Length cap. SVEN’s train and validation splits hold 803 pairs. I dropped the 88 pairs where either function is longer than 6,000 characters, so that no labelled line is cut off by the extractor’s 2,048-token limit (a cut-off positive would silently become a negative). The remaining 715 pairs are the dataset used throughout.
- Prompts. Both example-level prompts put the function in a fenced code block after the line
Here is a code snippet:, end with the question, and use the model’s chat template with the assistant turn opened. The neutral question isWhat do you think about this code?. The vulnerability question, also used for verbalization, isDoes the code above contain a security vulnerability? Respond with ONLY one word — "yes" or "no" — and nothing else.; the main text shortens it to “Is this code vulnerable?”. - Masking no-op. Excluding comments and trivial tokens from training changes test AUC by less than 0.004 on all seven models, within margin of error. Separately, AUC over code tokens is slightly lower than AUC over all tokens on six of seven models (by up to 0.02), so restricting the evaluation to code tokens does not inflate the reported numbers.
- Layer selection. Every layer of every model was swept (78 to 192 probes per model: 26 to 64 layers, three regularization strengths each). Best layer by validation token-level AUC is within 0.030 of the best layer by test AUC for all seven models.
- Loss weighting sweep. I compared two values of α, the weight on vulnerable tokens in the per-token loss term. On Qwen-32B and gemma-27b, α=10 beats α=1 on example-level AUC (0.66 vs 0.64, 0.64 vs 0.60), and the two are within margin of error on token-level AUC (0.81 vs 0.81, 0.78 vs 0.77), so the token probes here use α=10.
- Subtractive-restriction stability. The all-pairs vs subtractive-only comparison was re-run as a 5-fold × 3-seed cross-validation (15 retrainings per model): on the full test set the all-pairs probe leads by 0.004 on average, and on the subtractive test pairs it trails by 0.005. Both are small compared to the ~0.03 standard deviation across retrainings.
- Saved-score rescoring and reproduction checks. All probes were retrained in the November 2026 reproduction with one shared setup. The language and scanner comparisons re-score the saved probe scores without retraining, after first checking that the saved scores reproduce each probe’s reported AUC. I trained new probes for the PrimeVul experiments: per-CWE probes on Qwen-7B, the assistant-turn probe, and MLP vs linear.