Auto-Context Prompt Injection: Measuring Code Slop and Helper Duplication Across 36 Controlled Sessions

Injecting repository concordance hints reduced helper re-implementation, but the overall reduction in mechanical slop (paired McNemar test p = 0.062) did not cross the 0.05 significance threshold.

Core Finding:We evaluated whether automatically injecting symbol concordance hints into agent prompts reduces duplicate helper implementation and code slop across 36 controlled coding sessions. While on one target helper, duplication went from 4 of 4 runs without injection to 0 of 2 with it, too small a sample to generalize, the reduction in mechanical code slop from 6 events to 1 event across 18 task pairs yielded an exact McNemar test p = 0.062, which is not statistically significant at the standard 0.05 alpha level.

Firing Reliability

17 / 18 Sessions

Context injection fired on 17/18 ON sessions with active symbol hits (0/18 OFF); single miss returned empty in 0.27s on cold startup.

Hook Latency

0.27 - 1.16 s

Hook execution latency ranged from 0.27s to 1.16s, safely within the 2.0-second safety timeout.

Target Helper Duplication

4/4 -> 0/2

On one target helper, duplication went from 4 of 4 runs without injection to 0 of 2 with it, too small a sample to generalize.

Mechanical Code Slop

6 -> 1 Events (p = 0.062)

Mechanical slop dropped from 6 events to 1 event across 18 pairs (paired McNemar p = 0.062, sign test p = 0.062, not significant at alpha = 0.05).

Empirical Findings Table

Automated context injection reduced utility function re-implementation and showed a directional drop in code slop (6 events to 1 event across 18 task pairs), but this difference (exact McNemar p = 0.062) did not achieve statistical significance at the alpha = 0.05 level.

Telemetry Metric	Measured Value	Context & Observability
Firing Reliability	17 / 18 Sessions	Context injection fired on 17/18 ON sessions with active symbol hits (0/18 OFF); single miss returned empty in 0.27s on cold startup.
Hook Latency	0.27 - 1.16 s	Hook execution latency ranged from 0.27s to 1.16s, safely within the 2.0-second safety timeout.
Target Helper Duplication	4/4 -> 0/2	On one target helper, duplication went from 4 of 4 runs without injection to 0 of 2 with it, too small a sample to generalize.
Mechanical Code Slop	6 -> 1 Events (p = 0.062)	Mechanical slop dropped from 6 events to 1 event across 18 pairs (paired McNemar p = 0.062, sign test p = 0.062, not significant at alpha = 0.05).
Blind Judge Score	9 -> 6 Events (p = 0.51)	Independent Opus judge recorded 6 slop events ON vs 9 OFF (paired McNemar test p = 0.51, not statistically significant).
Context Overhead	1.06x - 1.29x	Injected context added 6% tokens on compact tooling tasks and 29% tokens on multi-service tasks.

Engineering Action Directives

Providing models with focused, pre-computed symbol concordance hints at the start of a turn can guide them toward existing utility reuse rather than writing duplicate code.
Injected context introduces a tangible token overhead (+6% to +29%), meaning injection should be reserved for build and implementation prompts rather than general conversational turns.
Prompt hooks must execute against an already warm index service. Cold startup races during the first prompt can cause the hook to fail open, as seen in the single missed session.
While on one target helper, duplication went from 4 of 4 runs without injection to 0 of 2 with it, too small a sample to generalize, establishing statistical significance across broad categories of code slop will require a larger sample size.

Methodology Boundaries & Limits

Statistical Insignificance: The overall reduction in mechanical slop yielded a p-value of 0.062, which exceeds the standard 0.05 threshold for statistical significance. The findings represent a directional trend, not a confirmed effect.
Blind Judge Disparity: The independent blind judge identified qualitative inconsistencies (such as manual catalog parsing) that resulted in a p-value of 0.51, reflecting substantial variance in subjective code quality evaluations.
Token and Latency Overhead: Context injection added between 6% and 29% additional tokens to session context and introduced up to 1.16 seconds of pre-turn hook latency.
Sample Size: The evaluation was limited to 18 matched task pairs (36 total sessions), which constrained statistical power for rare slop events.