Dev Tool Experiences
All articles

How to Use AI for Coding Without Losing Track of What the Code Does

By R. H.

  • artificial intelligence
  • software development
  • code review
  • testing
  • security
How to Use AI for Coding Without Losing Track of What the Code Does

AI coding tools are most useful when they reduce the amount of typing and searching you have to do without taking over the engineering decisions. The failure mode people call “slop” is not merely untidy code. It is code that appears plausible, passes a superficial demo, and leaves nobody able to explain its assumptions, failure cases, dependencies, or effect on the rest of the system. The practical defense is simple: make the AI work in small, inspectable units, and never accept a change you could not maintain after the tool is gone.

Start with a contract, not a vague request

Before asking for code, write a compact contract for the change. State the input, output, constraints, and acceptance tests. If the work touches an existing application, include the relevant interface, data model, style conventions, and files the tool may edit. “Add user search” invites the model to invent product behavior. “Add a case-insensitive search endpoint for display_name; return at most 20 active users; preserve the existing pagination format; do not change authorization; add tests for empty queries and punctuation” gives it boundaries.

This matters because a language model predicts a likely continuation from the context it receives. It does not automatically know which parts of your repository encode business rules, which backward-compatibility promises matter, or which edge cases have cost your team money before. Context is an input to manage, not a guarantee of understanding. Supply the sources of truth and say which ones win if they conflict.

Ask for a plan before an implementation when the change is larger than a small function. A useful plan names the files likely to change, the existing code paths it will reuse, the decisions that remain uncertain, and the tests it intends to add. Read that plan as an engineer, not as a formality. If the plan is wrong, generating more code only makes the wrong direction more expensive to unwind.

Keep the change small enough to review

A diff is the set of lines changed between two versions of code. Treat it as the primary product of an AI-assisted session. Ask for one behavior change at a time: extract a function, add one endpoint, migrate one call site, or write tests for one bug. Avoid prompts that authorize a tool to “clean up the project,” “modernize everything,” or “fix all failing tests.” Those requests combine many decisions and make it difficult to determine whether a changed line is necessary.

Set an explicit editing boundary. Tell the tool which directories it may modify, whether it may add packages, whether it may alter configuration, and whether it may run commands. An agent that can edit files and execute shell commands has more power than autocomplete. Give it the least access needed for the task, particularly in repositories containing deployment credentials, customer data, production configuration, or release automation.

Do not let an agent silently solve failures by weakening the evidence. A deleted test, a skipped test, a broad exception handler, a disabled lint rule, or a timeout increased tenfold should trigger a question: what behavior is being concealed? Require a written explanation for any change to an existing test or security control.

Make the model explain the code in terms you can check

After generation, switch roles. Ask the tool to explain the control flow: what calls what, which inputs are trusted or untrusted, where state changes, what errors can occur, and what happens on a retry. Then verify the explanation against the code itself. This is faster than reading every line cold, but it is not a substitute for reading the critical paths.

Ask adversarial questions rather than open-ended ones. For example: “What inputs make this function return the wrong result?” “Which invariant does this migration rely on?” “What happens if the network request succeeds but saving the result fails?” “Which permissions are checked before this record is returned?” “What would make this cache serve stale data?” These questions expose assumptions that polished-looking code often hides.

An invariant is a condition that must remain true while a program runs. For a bank-transfer feature, an invariant might be that money cannot be created or lost when a request is retried. For a permissions system, it might be that every data fetch is scoped to the authenticated tenant. Write down the invariants for consequential code, then inspect whether the implementation and tests actually enforce them.

Use tests as evidence, not decoration

A passing test suite means only that the assertions in the suite passed. It does not prove the intended behavior was tested. AI tools can produce tests that repeat the implementation’s mistake, mock away the meaningful part of a system, or assert a trivial fact. The common misconception is that more generated tests automatically mean more confidence. Confidence comes from tests that would fail if an important requirement were broken.

For each change, identify a normal case, a boundary case, and a failure case before generating tests. A normal case exercises expected input. A boundary case tests a limit such as an empty collection, a maximum length, or the first and last page of results. A failure case checks rejected input, unavailable dependencies, expired credentials, or conflicting updates. For a bug fix, add a test that fails on the old code and passes on the new code; otherwise you have not demonstrated that the reported defect was fixed.

Run the tests yourself, and read the new tests. For high-risk behavior such as authentication, authorization, payment handling, data deletion, input validation, or cryptography, write or independently review the core tests yourself. Do not have the same model both implement the safeguard and certify that safeguard without another source of scrutiny.

Check every new dependency and every external claim

AI tools sometimes suggest packages, APIs, flags, or configuration settings that do not exist, are outdated, or are unsuitable for your license and deployment environment. Verify a proposed dependency in its official registry and documentation. Check who maintains it, whether it is actively released, what license it uses, and whether the exact package name is correct. A package name that looks close enough is not evidence that it is safe.

Apply the same discipline to generated explanations. When the tool says a framework behaves a certain way, find the authoritative documentation or reproduce the claim in a minimal test. Ask it to distinguish facts it can point to in the repository from assumptions it is making. That separation is valuable even when the implementation is ultimately correct.

Build a workflow that makes unsafe changes hard to merge

Good individual habits need mechanical backstops. Put formatting, type checking, unit tests, integration tests, dependency checks, secret scanning, and static analysis in continuous integration, the automated checks that run on a proposed change. Require a reviewable pull request. Keep generated changes attributable to a human owner who is responsible for the code after merge.

Use AI twice if it helps: once to draft a narrowly scoped change, and again to review that diff against a checklist. The second pass should be given a different job: identify missing tests, changed assumptions, authorization mistakes, concurrency risks, unnecessary complexity, and new dependencies. Treat its review as a lead for investigation, not an approval stamp.

  • Define the behavior and constraints before code is generated.
  • Ask for a plan first when the task affects multiple files or systems.
  • Limit the tool’s file access, command execution, and ability to add dependencies.
  • Review a small diff; reject unrelated cleanup bundled into a feature.
  • Explain the control flow, assumptions, and invariants back to yourself.
  • Create tests that can disprove the change, then run and read them.
  • Verify packages, APIs, and framework behavior from authoritative sources.
  • Use automated checks and human review as merge gates, especially for sensitive code.

The right standard is not whether the AI wrote the code or whether the code looks elegant. It is whether a named engineer can explain the change, demonstrate its behavior, identify its limits, and safely modify it six months later. If that standard makes an AI-generated patch feel too expensive to review, the patch is probably too large.

Sources & citations

  1. [1]GitHub Docs: Review AI-generated code
  2. [2]OWASP Cheat Sheet Series: Secure Coding with AI
  3. [3]NIST: Secure Software Development Framework
  4. [4]NIST IR 8397: Guidelines on Minimum Standards for Developer Verification of Software
How to Use AI for Coding Without Losing Track of What the Code Does | Dev Tool Experiences