Published by Sean Carr
Working Is Not Organized: Measuring How Data Access Spread in an AI-Built Application
In an AI-built application, operations on one settings table were named in 29 files through 14 differently named calls. We measured the spread and what a structural rule changed.
At a Glance
Settings table
53 sites, 29 files, 14 call names
53 call sites in the application code name a settings operation. 29 code files name a settings read; 7 of 11 write mentions sit in the owner file.
Spread across record types
77 of 145
Record types named by literal operation that are referenced from two or more files. 13 are referenced from five or more.
One rule, different errors
Two implementations
The rule that no-service visits cannot be paid lives in a route helper and in a rules file, with a third related check in payroll.
After one consolidation lane
29 files to 1
Settings references after a single unmerged, unvalidated lane moved them into a table owner module. User profiles went from 19 files to 2.
| Telemetry Metric | Measured Value | Context & Observability |
|---|---|---|
| Application code measured | 843 Python files | Including 159 route files and 58 files in the rules directory (counting its initializer), at one integration commit on October 23, 2026. |
| Direct calls to the older data entry | 404 in 104 files | 258 reads and 146 writes. Route files alone hold 151 reads in 53 files and 93 writes in 35 files. |
| Record types referenced from 2+ files | 77 of 145 | Across 752 call sites that pass a literal operation name. These are operation-literal references, not verified access paths. 13 types are referenced from 5 or more files and 4 from 10 or more. 87 are referenced through 2 or more differently named calls. |
| Settings table | 53 sites, 29 files, 14 call names | Old and new data entries are both in use for the same table. Settings write operations are named in 5 code files, with 7 of the 11 mentions in the owner file. Settings reads are named in 29 code files. These are text matches for operation names, not confirmed accesses. |
| User profile table | 60 sites, 19 files, 19 call names | The second most widely referenced record type by file spread. |
| Uppercase literal sets copied across files | 32 of 186 | Distinct literal sets of two or more uppercase constant strings that appear in 2 or more files. They include statuses as well as roles, error codes, channels and document types. 9 appear in 3 or more. One set accepts two spellings of the same error in 4 files. |
| After one consolidation lane | Settings 29 to 1 file; profiles 19 to 2 | Record types referenced from 5 or more files fell from 13 to 8. The lane changed 127 application files, 3,470 lines added and 3,073 removed. It is unmerged; its one recorded scoped test run had 349 passed, 95 failed and 7 errors, and its later changes are unvalidated. |
Count access paths per table and copies per business rule across the codebase.
Assume a clean diff review means the system is organized.
Send every table through one shared data picker with per-table configuration, keep each rule in one file, and lock each table with a guard test once it is clean.
Expect per-diff review alone to catch spread that is easiest to see as a count.
Guard reads as well as writes.
Stop at write owners: here the remaining spread was in reads, which the write guards allowed.
Published by Sean Carr
View & print executive brief version →Executive Summary & Core Finding
AI coding agents are usually judged task by task: does the change work, do the tests pass, is the code clean? We looked at what those changes add up to in one application our agents built over several months. Operations on one settings table are named in 29 files through 14 differently named calls. A rule that a visit with no service delivered cannot be paid exists as two separate implementations that return different errors. Of 145 record types named by literal operation strings in the code, 77 are referenced from more than one file. This is a different problem from code slop. Slop is dead code, swallowed errors, invented functions and fake tests, and each instance is visible in the diff that adds it. Scatter is hard to see in any one diff: a new read path can look reasonable where it is written. It shows up when you count across the codebase. The application's slop gate has 42 numbered checks, and none of them counts how many files reach the same table. We then measured one consolidation lane that moved data access into per-table owner modules: settings references went from 29 files to 1, and user profile references from 19 files to 2. That lane is unmerged and its validation is blocked, so it describes what the rule does to the code, not a released outcome. Data from October 22 and 23, 2026 (code as of those commits), analyzed October 23, 2026.
Method
The subject is the application code of one staffing application built by our AI coding agents under human direction. We measured its integration branch read-only at a single commit on October 23, 2026, plus two consolidation lanes and their shared starting commit from late October 22. All counts come from scripts and text searches that read the code at a named commit without checking it out. We recorded every command and commit identifier in our internal evidence notes.
We used three counts. First, direct calls to the platform core's older raw data entry, split into reads and writes and by folder. Second, every call that passes a literal operation name such as a settings read. We grouped those by record type and counted call sites, distinct files and distinct calling function names. These are operation-literal references, not verified runtime access paths: a diagnostic helper that names an operation counts too. Third, literal sets of two or more uppercase constant strings that appear in more than one file. These include status values but also roles, error codes, channels and document types. A copied set can mean a decision was restated instead of imported; we did not classify each set.
We used a code-search tool we built to locate rule definitions by meaning rather than by name, and read the history of one duplicated rule commit by commit. For the slop comparison we read the slop gate's check list and its scoped baseline runs recorded by two consolidation lanes. Our own attempt to run that gate on the 30 settings files did not finish within five minutes, so we report the lane-recorded runs.
This is a static census. We did not measure user-facing bugs caused by the spread, and we did not compare with code written by people.
What We Measured
Most record types named by literal operation were referenced from more than one file. The most-used tables were referenced from dozens of call sites, through many differently named calls.
Application code measured
843 Python files
Including 159 route files and 58 files in the rules directory (counting its initializer), at one integration commit on October 23, 2026.
Direct calls to the older data entry
404 in 104 files
258 reads and 146 writes. Route files alone hold 151 reads in 53 files and 93 writes in 35 files.
Record types referenced from 2+ files
77 of 145
Across 752 call sites that pass a literal operation name. These are operation-literal references, not verified access paths. 13 types are referenced from 5 or more files and 4 from 10 or more. 87 are referenced through 2 or more differently named calls.
Settings table
53 sites, 29 files, 14 call names
Old and new data entries are both in use for the same table. Settings write operations are named in 5 code files, with 7 of the 11 mentions in the owner file. Settings reads are named in 29 code files. These are text matches for operation names, not confirmed accesses.
User profile table
60 sites, 19 files, 19 call names
The second most widely referenced record type by file spread.
Uppercase literal sets copied across files
32 of 186
Distinct literal sets of two or more uppercase constant strings that appear in 2 or more files. They include statuses as well as roles, error codes, channels and document types. 9 appear in 3 or more. One set accepts two spellings of the same error in 4 files.
After one consolidation lane
Settings 29 to 1 file; profiles 19 to 2
Record types referenced from 5 or more files fell from 13 to 8. The lane changed 127 application files, 3,470 lines added and 3,073 removed. It is unmerged; its one recorded scoped test run had 349 passed, 95 failed and 7 errors, and its later changes are unvalidated.
Experimental Notes & Observability Details
- Writes had already been given one owner per main record type in an earlier refactor, with nine writer guard tests added on October 18, 2026. Those guards deliberately allow reads from anywhere. In this code, most of the spread is in reads: settings has a writer guard, yet settings write operations are still named in 5 code files and reads in 29.
- The duplicated rule has a traceable history. In July 2026 the 'no-service visits cannot be paid or exported' check was first added as two pasted copies in two route files. In September it was extracted into a shared route helper. In October a separate fix added a second implementation on the writer path, which a later refactor moved into the rules file. Two implementations are live. One returns a validation error and the other a conflict error, so the same rule can return a different error depending on which path a request takes. We did not measure how often users reach either path. A payroll step applies a related check a third time.
- A code-search query in plain words for that rule returned both implementations as its top two results. The duplication was easy to find once someone asked the question.
- A ratchet test added on October 22 stops direct data calls from growing. Its committed baseline went from 708 to 704 direct calls in its first day. Counted our way, the ratchet's first two commits had 706 and 702 such calls (it also counts two probe helpers). The same counter read 404 at the integration commit, so the drop happened in later commits rather than in a different counting method; we did not attribute it to specific changes. The baseline still read 704 at the integration commit. A ratchet stops new spread; it neither records nor requires the drop.
What We Think It Suggests
- **Working and organized are separate properties, and a task-by-task check measures only the first.** An agent's task is typically judged done when the behavior works and the tests pass. Reading a table the nearest convenient way, or restating a rule close to where it is needed, can meet that bar. Our hypothesis is that, over hundreds of tasks, local choices like these produce the spread measured here. We did not audit individual tasks, diffs or briefs to test it.
- **Slop review does not measure this.** Slop checks look for bad patterns inside a change. Spread is a count across the codebase, and in any one diff it can look like good code. Many of the slop gate's checks cite the specific bug that prompted them. Its baseline runs on the code before consolidation reported only informational pattern findings, none about how many places reach a table. We think stricter per-diff review is unlikely to surface spread, because it shows up as a count across files, but we did not test that.
- **The change that moved the numbers was a structural rule with a mechanical check.** The lane we measured applied an earlier version of our data access standard, which gave each table its own owner module. The current standard, revised on October 23, replaces those per-table modules with one shared data picker: a single implementation of reads and writes for every table, with each table described by a configuration entry naming its operations, allowed filters, sort and page size, plus a small hook only where a table has special rules. Routes call the picker, business rules live in rules files, and a guard test fails the build if any other file calls the older data entry or names a table, with each table locked as soon as it is clean. The lane took settings references from 29 files to 1. The per-table guard, which is meant to keep that number down, had not yet been switched on in this snapshot.
- **Guards need to cover reads.** For settings, which has a writer guard, write operations are named in 5 files and most of those mentions sit in the owner. Reads were left open, and settings reads are named in 29 code files. In this snapshot most of the spread sits on the side the guards did not cover.
Limits
- **One application, one snapshot.** This is a static census of a single AI-built codebase at named commits. It does not estimate how common the pattern is, and it has no comparison with human-written code of similar age.
- **The counts see only what is literal in the code.** Calls that build the operation name at runtime, or reach a table through an older read module by function name, are invisible to the per-type count. On the scheduling consolidation lane our metric moved only from 5 files to 4, while the lane itself reported moving 198 call sites. The scanner also counts any call that names an operation, including diagnostic helpers, so it can overcount as well. The per-type figures are syntactic reference counts, not a bound on real access paths in either direction.
- **The after numbers are not a released result.** The consolidation lane we measured is unmerged, and the per-table guard was held back. Its one recorded scoped test run had 349 passed, 95 failed and 7 errors; two setup fixes followed, and revalidation is blocked by an unrelated database setup failure, so its later changes are unvalidated. We report what the change does to the code, not whether it held up in use.
- **We did not measure harm.** We did not count user-facing bugs caused by the spread. The team's stated reason for the standard is that the same data read two ways lets pages disagree and lets fixes miss copies. The differing errors for one rule are an example, but we did not measure how often this happens.
Detailed Analysis & Field Observations
| Record type | Call sites | Files | Differently named calls | After one consolidation lane |
|---|---|---|---|---|
| Settings | 53 | 29 | 14 | 1 file |
| User profiles | 60 | 19 | 19 | 2 files |
| All types referenced from 5+ files | 13 types | 8 types |
A practical test for any AI-built codebase: pick your three most-used tables and count the files that read each one, and the different functions they use to do it. Then pick one business rule, search for it by meaning rather than by name, and count the implementations. In this codebase, both counts were well above one. None of the slop gate's 42 numbered checks counts how many files reach a table; whether per-change review would surface these counts is a hypothesis we did not test.
Related: Patching on Top covers a repair series that was merged and then reverted, and The AI Proposes, Plain Code Decides covers keeping rules in plain code. Two companion studies cover the same application in the same week: Find the Bug Class, Not the Bug counts its flaw classes with a code census, and Are Your Tests Honest? runs its test suite against a fake and the real platform. This article counts how spread out that structure had become.