Taming AI Code Modifications: A Case Study in Structured Prompting

The Challenge with AI Code Modifications

We've all heard the joke in the AI development community: "Tell AI there's a bug in the code, and it'll fix it... creating two new bugs. Ask it to fix those, and you'll end up with three." While humorous, this reflects a real challenge in using AI for code modifications, especially as projects grow in complexity.

Through extensive experimentation, I've developed an advanced prompting system that significantly improves reliability by forcing AI to think systematically and document its decision-making process. Let me walk you through a real-world example.

The Structured Prompt Approach

Here's a case study of modifying a header component in a web application. The task seems simple: show marketing links only on the index page and add a toolbar for page actions. However, the complexity lies in maintaining consistency across the entire application while avoiding unintended side effects.

Key Components of the Prompt

Task Definition and Context

Task Definition

- Taskname: heading-task

- Title: reorder heading design

- Scope: Modify header.html to show marketing links only on index.html

Planning Rules

- Ask for clarification on ambiguity

- Optimize for minimal code and overhead

- Attempt reuse of existing code

- Split long code into sections

- Keep responses brief but complete

Project Requirements

Core Technology Stack:

- Modern JavaScript (ES modules)

- Pure CSS without preprocessors

- No TypeScript

- No build-heavy frameworks

The Power of Introspection

The most crucial aspect of this approach is forcing the AI to think through its actions

Based on my analysis, I need to:

1. Modify header.html to have two variants

2. Add toolbar section under header

3. Update components.js to handle conditional loading

Detailed Change Logging

Every modification is meticulously logged with timestamps and rationale:

Change Batch #1: Header Component Modification

Timestamp: 2025-01-28 13:03

Location: src/renderer/components/header.html

Changes:

- Split header into two variants using data attributes

- Maintain existing HTML structure

- Add conditional display logic

The Results

This structured approach led to:

• Clean, focused code modifications
• No unintended side effects
• Clear documentation of all changes
• Maintainable and consistent codebase

Cost vs. Benefit Analysis

While this approach requires more tokens and potentially higher costs compared to simple "fix this bug" prompts, the benefits are substantial:

• Reduced need for multiple iterations
• Fewer introduced bugs
• Clear audit trail of changes
• Better maintainability

With the emergence of more cost-effective AI models like DeepSeek, this comprehensive approach becomes increasingly practical for everyday development tasks.

Key Takeaways

1. Structure Matters: A well-structured prompt forces AI to think systematically about changes.
2. Document Everything: Explicit logging of decisions and changes helps track and understand modifications.
3. Think Before Coding: Essure the AI considers implications before making changes.
4. Test Thoroughly: Built-in testing steps verify changes work as intended.

Conclusion

While this approach might seem overly formal for simple tasks, it proves invaluable for complex code modifications. By forcing AI to work within a structured framework, we can harness its capabilities while minimizing the "fix one bug, create two more" syndrome that often plagues AI-assisted development.

The extra upfront cost in tokens and complexity pays off in reduced iterations and more reliable code modifications. As AI models become more cost-effective, this methodical approach becomes increasingly viable for everyday development tasks.