The Hidden Complexities of AI Code Modification: A Practical Guide

Author: Tom Cranstoun
The article contrasts AI's prowess in creating new code with its challenges in modifying existing codebases, emphasizing the complex realities of technical debt and undocumented dependencies. Based on a metadata field removal case study, it proposes a structured approach including prompt templates, behavioral constraints, and thorough documentation requirements. The piece concludes that successful AI-assisted code modification relies more on carefully constraining AI capabilities than unleashing them fully.

AI-Assisted Code Modification: A Comprehensive Guide

As AI coding assistants become more advanced, they've shown remarkable abilities in constructing complete applications from the ground up in a matter of hours. While these demonstrations are undeniably impressive, they conceal a critical reality: most real-world software development involves maintaining and modifying existing codebases rather than creating entirely new ones. This distinction reveals a fundamental challenge in AI-assisted development that every development team must comprehend.

The Fundamental Asymmetry: Creation vs. Modification

The Clean Slate Advantage

When creating new code, AI assistants operate in ideal conditions. They can choose optimal architectural patterns, implement consistent coding styles, design cohesive data flows, and build features without legacy constraints. This freedom allows them to apply best practices and modern patterns without restriction.

The Reality of Modification

Modifying existing code presents a dramatically different challenge. AI assistants must navigate complex interdependencies, historical technical debt, undocumented assumptions, and critical edge cases that may not be immediately apparent. What's more, as revealed through practical experience, AI systems exhibit different behaviors based on how we frame our requests and which platform or interface we use to interact with them.

Understanding AI Platform Behaviors

Different AI platforms exhibit distinct behaviors when modifying code. Web-based interfaces typically demonstrate more conservative behavior. In contrast, API-based interactions may attempt more aggressive changes, potentially leading to unintended consequences.

This difference becomes particularly important when working with various API callers like Cursor, Cline, or GitHub Copilot. Each of these tools implements its own system prompts, which can significantly influence how the AI approaches code modification.

Legacy System Challenges

Cursor's system prompt, which is open source and available for examination, reveals specific challenges when applied to legacy code modification, https://github.com/cline/cline/blob/main/src/core/prompts/system.ts

The Cline system prompt struggles with managing undocumented assumptions, technical debt, and intricate interdependencies that are central to legacy systems. There are insufficient mechanisms for detecting and preserving critical edge cases or intentional inefficiencies, increasing the risk of functional regressions. The focus on individual changes rather than batched, logically related modifications can compromise system consistency, the system prompt may not effectively enforce the preservation of legacy constraints, potentially allowing unintended modernizations that disrupt expected behavior. Limited integration with testing workflows can compromise the ability to detect errors proactively and maintain functional reliability. The system prompt does not provide audit or logging; essential when modifying legacy code.

A Case Study in AI-Assisted Refactoring

The Initial Attempt

When attempting what seemed like a simple task - removing two fields from a metadata object using an AI assistant - the complexity of AI-assisted code modification became apparent. The AI immediately identified several issues beyond the requested changes: CSS specificity problems, inefficient rendering patterns, suboptimal folder structure, and inconsistent variable naming.

While these observations were accurate, allowing the AI to "fix" these issues led to a cascade of unexpected problems. The modified code, though theoretically better, broke existing functionality in subtle ways.

The Solution: A Structured Approach

This experience led to the development of a comprehensive prompt template that fundamentally changed how we approach AI-assisted code modification. While the new prompt mechanism may cost more to run, the investment proves worthwhile in preventing hours of debugging.

Task Definition Template:

Title: {{ Task Title }}
Scope: {{ Detailed description of what needs to be refactored }}
Current Architecture: {{ Description of current implementation }}
Target Architecture: {{ Description of desired end state }}
Files/Components: {{ List of specific files or components to be refactored }}
Dependencies: {{ List of known system dependencies }}

IMPORTANT: All changes must be minimal, affecting only explicitly mentioned code parts without impacting existing functionality.

Module System Management

Module System Priority Rules:

Always use CommonJS (require/module.exports) for new files
Never use ES modules (import/export) in new files
Javascript file extensions must be .js
Maintain existing module resolution

Documentation Through log.md

To ensure accountability and transparency, we maintain an audit trail of AI actions through log.md. This documentation becomes particularly crucial when switching between different AI platforms or sessions.

## Initial Analysis

## Document initial codebase assessment
## User-Provided Information
List all information provided by user
## Planned Changes
Detail specific modifications to be made
## Decision Log
- What Changed:
  List description of specific changes made
- Why it Changed:
  Explanation for each change
- Decision Points:
  List of key decisions made
## Codebase Details
- Resisted Improvements:
  List of improvements intentionally not made
- Preserved Inefficiencies:
  List of inefficiencies intentionally maintained
- Known Bugs:
  List of bugs encountered but preserved

The AI Perfectionist Problem

When working through API callers, the tendency toward perfectionism becomes pronounced due to built-in system prompts that often encourage modernization and improvement. Understanding and counteracting these embedded tendencies requires explicit constraints and careful monitoring.

## Behavioral Constraints
Do not suggest optimizations, even if obvious
Do not fix any bugs encountered, even if solution is known
Do not modernize code patterns
Do not use newer/better methods even if available
Do not refactor surrounding code
Do not improve code organization
Do not standardize inconsistent patterns

Implementation Process

The key to successful implementation lies in batching changes intelligently while considering platform-specific behaviors:

Group changes by logical area or related functionality
Apply platform-specific constraints and verification
Batch process all changes within that logical area together
Document platform-specific behaviors and responses
Allow user testing and verification
Move to the next logical area only after confirmation

Risk Mitigation Strategies

System Prompt Analysis

Tool-Specific Verification

Legacy Protection Mechanisms

Best Practices for AI-Assisted Code Modification

Clear Boundaries

Comprehensive Verification

Conclusion

While AI assistants excel at creating new code, successful modification of existing codebases requires a different mindset and toolset. The key to success lies not in unleashing AI's full capabilities, but in carefully constraining them based on platform-specific behaviors and clear, well-defined objectives.

As we continue to integrate AI into our development workflows, success depends on:

The next frontier in AI-assisted development isn't just about creating new code—it's about developing sophisticated, platform-aware strategies for safely modifying existing codebases while preserving their essential characteristics and behaviors.


Full prompt is here https://allabout.network/notes/ai-code-modification-prompt-template

Modifying the prompt for EDS, AEM, etc is straightforward

/fragments/ddt/proposition

Related Articles

guide
guide
Back to Top