Why Your AI Agents Write Bad Code: Orchestration Patterns and Prompt Engineering Pitfalls in Production

Denis Shokhirev

12 Jun 2026 — 3 min read

I’m Denis Shokhirev, Enterprise AI architect based in Freiburg (DennisCraft AI Studio, DACH B2B stack: Claude, Supabase, n8n, Doppler, Postgres). Last month, three separate AI agents shipped PRs with vulnerable code—SQL injection, hardcoded credentials, and missing input sanitization—caught only because I run semgrep and bandit before merge. This isn’t a rare edge case, but a systemic flaw in how orchestration and prompt engineering are done in production.

Where Orchestration Patterns Break Down

In a typical production AI agent, orchestration looks like this:

n8n triggers a workflow, assembles a prompt, and calls Claude Code via API
Claude outputs Python/SQL code
n8n merges the code to a repo or runs it live via Supabase/Postgres

The problem: this pipeline often skips real-time code quality gates. LLM output gets passed straight through without static analysis or sandboxing.

Common Orchestration Mistakes

No automated post-generation static analysis: LLM output is not scanned by bandit, semgrep, or gitleaks
Errors caught “in prod”: bugs are only discovered after user impact
No rollback/atomicity: partial bad changes break production state

In one recent fintech project (Germany, 2024), an agent generated a logging function that exposed logs to a public S3 bucket—gitleaks would have caught this instantly, but the orchestration passed it straight through.

Prompt Engineering Pitfalls in Production

Certain prompt patterns keep causing vulnerabilities in shipped code:

Overly Broad Instructions

Prompts like “Write a function to fetch users from the database” typically yield unsafe SQL (risk of injection, no sanitation). Without context or constraints, LLMs default to bad patterns.

Missing Explicit Constraints

If you don’t specifically instruct the LLM to use prepared statements, it won’t. Here’s a real bad prompt:

# Bad prompt
"""
Write a function to find a user by email in Postgres.
"""

And the LLM generates:

def find_user(email):
    query = f"SELECT * FROM users WHERE email = '{email}'"
    cur.execute(query)
    return cur.fetchone()

This is trivially vulnerable to SQL injection.

No Feedback Loop with the LLM

Without automated static analysis (e.g., semgrep + Claude), the agent keeps repeating the same mistakes. Vulnerable patterns reappear in every deployment.

Production-Oriented Orchestration Patterns for LLMs

Pattern	Benefit	Tools
Post-generation static analysis	Filters unsafe code before runtime	bandit, semgrep, gitleaks
Prompt constraint injection	Forces LLM to use secure patterns	Hand-crafted prompts, system messages
Runtime sandbox	Limits blast radius of LLM bugs	Docker, Firejail
Rollback/Atomic ops	Prevents partial changes from breaking prod	Supabase transactional APIs, git revert

Example: Claude Code + n8n + Supabase Stack

My pipeline inserts a static analysis step between generation and execution:

import subprocess

def safe_exec(generated_code: str):
    with open("temp.py", "w") as f:
        f.write(generated_code)
    result = subprocess.run(
        ["bandit", "-r", "temp.py"],
        capture_output=True, text=True
    )
    if "No issues identified." not in result.stdout:
        raise Exception("Unsafe code blocked")
    # Only safe code is executed or merged

This way, even if the LLM generates insecure code, the orchestration pipeline blocks it before it hits production.

Human-in-the-Loop: When to Intervene

In 5 out of my last 14 production agents, I added manual PR review whenever static analysis flags issues. Yes, it slows things down, but it’s the only way to catch edge-case vulnerabilities that static analysis misses.

Automating Human-in-the-Loop

n8n can route flagged PRs for human review automatically:

// n8n custom node
if (staticAnalysis.findings.length > 0) {
  sendToHumanReview(prId, staticAnalysis.findings);
}

In a logistics client project (2024), an AI agent leaked hardcoded credentials; gitleaks flagged it, and the PR was sent straight to human review instead of merge.

External Data on LLM Code Risks

A Stanford CodeLM study (2023) found 38% of LLM-generated Python code samples exhibited at least one OWASP Top 10 vulnerability, even when prompts requested “secure code.” In my experience, these numbers align with what I see in DACH regulated B2B deployments.

FAQ

Can LLM code quality gates be fully automated?

No. Even the best automated tools miss edge cases; manual review is still required for security-sensitive code, especially in regulated industries.

What’s the minimum toolset for LLM orchestration?

At least bandit, semgrep, and gitleaks. If handling credentials, use an OWASP checker. For SQL, always enforce a sanitation layer with prepared statements (e.g., psycopg2).

How do you write prompts that force secure code?

Add explicit constraints: “Only use prepared statements,” “Do not use eval,” “Log only via the standard logging module.”

How do you catch recurring LLM code bugs?

Aggregate findings from static analysis, track error types, and adjust both prompt phrasing and orchestration. Don’t rely solely on tests.

What if your LLM keeps generating insecure code?

Rewrite prompts, use stricter system instructions, fine-tune if possible, and never skip human review for critical flows.

Which stage in your LLM pipeline catches the most issues in prod—static analysis, runtime sandbox, or human review? I’d genuinely like to know. I run a free 30-min stack audit for DACH founders building AI in regulated markets. DM me on LinkedIn or write to @ger_dennis_ai.