Working with an AI on a big Total War mod

The brain and the execution

I've got the brain, and you're the execution.

That split isn't a slogan. I set the vision, made the decisions, and above all judged in game, with my Warhammer I and Warhammer III screenshots side by side.

Claude did the rest. What it produced, and why a frontier agent rather than a free AI, I covered in A whole map in four days; here, I show how we worked together.

My decisions are written in the file the AI reads first, marked “not to reopen”. So an AI picking the work up the next day doesn't pitch an option I already turned down, like carving up the Immortal Empires map. We don't replay the same debates every session.

The error log

On day one, I asked Claude to “take careful note of every mistake you've made and make sure it doesn't happen again”. The result is a file, the error log: about 146 entries on 23 September, over 260 two days later.

Each entry has four parts: what was done, the symptom, the cause, the rule. Here's one, as it stands in the file (translated from French, lines re-cut for phones):

21. [avoidable] Guessing the name of a file
    a tool produces.
For the map.hex round trip, I passed
Attritions.hex_layer to import-layer while
export-layer had written
layer_attritions.hex_layer and printed it;
the import failed ("Layer file not found")
and the byte comparison concluded
"identical" on a file never rewritten.
→ Rule: reuse the path the tool printed;
a test that compares two files first checks
that the write actually happened (timestamp
or "Saved" message).
Coded into the round-trip script.

Note the symptom: a test that says “identical” when nothing was written. That's the kind of false success an AI reports in good faith, and only a precise rule keeps it from coming back.

Each entry also gets a tag, settled by a single question: had anyone written it down somewhere before? If so, the mistake was avoidable, and the entry gives the rule that prevents it. If not, it was a discovery: the entry gives the fact and its proof, and the fact joins our guide's list of pitfalls, which counted 135 on 23 September (150 on the 25th).

A written rule gets forgotten: code it

The log's clearest lesson is that a rule that's only written down gets forgotten again. Whenever possible, we turn it into a guard in a script. A guard doesn't get tired, and it doesn't rely on memory.

• The pack build refuses to run if a file declared in a table is missing.
• It checks the syntax of every Lua script before shipping it.
• The text injection step rejects any text containing a control character.
• A check makes sure the startpos save counter is 1.
• Any pack that touches a database table goes through a scripted startup test before it's announced.

The opposite is telling. One very simple rule, never write a script through a “heredoc” (a block of text pasted into a terminal command), was broken more than ten times in three days, by several sessions. Usually with no harm done, once with 102 French texts stripped of their punctuation, repaired afterwards.

The guard, for real

On the evening of 24 September, after the ninth relapse of the day, we went to the safest level: a Claude Code hook. It's a small script Claude Code runs before each of the agent's commands; if the command has a forbidden shape, it never goes out, and the agent gets the advice instead.

# .claude/hooks/garde_commandes.py (excerpt)
# Claude Code runs it before every command.
def motif_refus(tool, cmd):
    if PYTHON.search(cmd):
        if HEREDOC_BASH.search(cmd):
            return ("Python passed "
                    "via heredoc (<<)")
    # … compound python -c, sed with backslashes
    return None

motif = motif_refus(tool, cmd)
if motif:
    sys.stderr.write(
        "Command refused by the workshop "
        f"guard: {motif}. {ADVICE}")
    return 2  # refused: the agent reads why
return 0      # otherwise the command runs

It's declared in the project's .claude/settings.json file, on Bash and PowerShell commands. Before plugging it in, we tried it on fifteen commands (seven to refuse, eight to let through). And if the guard itself breaks, the command goes through: a guard must never block the work through its own fault.

The five rules that cost the most

The log's first five entries are method rules. They're the ones that cost the most time whenever they were forgotten.

• Never throw away a validator's messages: group them and count them. Filtered too hard, they hide the cause.
• Check the data before calling it a bug: a tool message that cites a table gets checked in the table.
• Have a reference document reviewed against its sources before delivering it: the first version of our guide had nine errors.
• Ask what the screen would teach you that the disk or the command line can't, before driving a window.
• Never launch a tool with guessed options: BOB's help option opens a dialog box that blocks the program.

Control case, always a control case

A good share of our breakdowns got solved the same way: by comparing with something that works. A control case is a known-good example you run next to your own. ChaosRobie's terrain project, copied under another name, proved in twenty minutes that BOB had nothing against our files.

You still have to replay the control case exactly. One night, Claude reran it without one flag from the original command: the game quit in nine seconds without doing anything, and six tests were lost hunting a regression that didn't exist. Another time, it announced progress to me for twenty minutes by comparing two measurements taken on two different campaigns.

The trick that makes the second rule easy: our test logs have a “What to redo to replay this state” section, with the exact command, ready to copy. Before replaying a control case, you reread it; you never retype it from memory.

Several sessions in parallel

Claude worked in several sessions at once: several conversations open on the same folder, each with its own area, and the only one writing there. One session built the pack, the startpos and the terrain, and kept the documentation; another handled game data, scripts and text.

Others were opened later: map rendering, the expansion to all of Bretonnia, the 9.0 update and the Red Duke, then the illustrations.

These rules were born from slip-ups. On day one, two sessions worked on the same files: half an hour of reconciliation. Later, one session overwrote another's script, restored byte for byte. And one session wrote to the kit before its notice ran out, four times in the same day.

For that last one, the answer was a script again: the time announced to the other sessions and the end of the timer are now the same value, computed once, instead of an “about five minutes” set by hand.

The human at the screen

The weak point wasn't the AI alone: it was the AI and me on the same computer. One night, Claude was driving the screen to place Terry's camera while I was working. Afterwards, Windows thought the AltGr key was held down, and I couldn't type any more.

Since then, Claude asks before taking over the screen and checks the keyboard state afterwards. The other way round, I clicked in test games that looked frozen, and we took it for crashes. Now it warns me before every game launch, and reads my clicks in the game's log before drawing a conclusion.

I'm the judge

My role as the judge, on the other hand, proved irreplaceable. Claude's audits saw 45 floating objects when I saw them everywhere, and its numbers looked fine when I no longer recognised the map.

Warhammer I

Warhammer III, 24.09

An algorithm only measures what it thinks it's doing; my eyes, next to the Warhammer I screenshots, measured what the player will see. The rule is written down now: what shows in game gets checked in game, and the AI reports the measured result, never its algorithm's own report.

What worked

• The debugger. A crash dump read in a debugger tells you which data was missing. That's what unlocked the startpos, the loading and the end of turn.
• CA's examples first. Before inventing a convention, look at how CA does it on a map that works. Our port settlements were fixed by exporting one of CA's maps.
• Side-by-side screenshots. Warhammer I on one side, Warhammer III on the other: that's the reference, not the numbers.
• Timestamped backups. A copy before every write to the kit's tables. One breakdown on the first evening was only solved because we could compare with the last state that worked.
• Automated games. Games played without me, one lord picked and the AI running the other factions: that's how a visual effect crash was caught for three other lords, before any player ran into it.
• A snapshot before every game update. Before 9.0, we recorded the state of the game so we can compare afterwards.

An automated game starts from one line and returns a report you read like a match sheet. Here's the one from a ten-turn game with Morghur, on 25 September (the keys are in French: seigneur = lord, tours = turns, issue = outcome, erreurs = errors, blocages = hangs):

# the command (on one line):
python essai_tours_auto.py
    --seigneur morghur --tours 10

# the bilan.json it returns (excerpt):
{
 "seigneur": "morghur",
 "tours_demandes": 10,
 "all_players_ai": true,
 "issue": "fin",
 "dernier_tour": 10,
 "compteurs": {
  "erreurs": 0,
  "blocages": 0,
  "chargements_rates": 0,
  "executions_ratees": 0
 }
}

Zero errors, zero hangs, the game ran to the end. One limit worth knowing: in that mode, the player's faction doesn't play. These games prove the campaign holds; the lords' mechanics get checked by hand.

Understanding it, not just running it

A question I got asked this week while showing the mod around: do you end up with code nobody understands anymore? Fair question. My answer: if I can't sum up what a script does in one sentence, I don't accept it, even if it runs.

For us that means a few simple habits: clear script and variable names, in French like the rest of the project; and at the top of every file, what it does and why, with the number of the error that gave birth to it. For instance, the startpos check (the campaign's starting file), translated:

"""
verifier_compteur_startpos.py - does the
startpos carry a new game's script values?

Why (23.09.2026, error 112):
cm:is_new_game() means "__save_counter == 1".
The counter is only written if the
campaign's scripts run while the startpos
is generated. A startpos older than the
scripts leaves it at 0: every new game
passes for a saved one (no intro, no
missions).
"""

When a script comes back to me across sessions, I reread that header, even without mastering all the code: it's my memory of the project too, not just the AI's.

The other worry people raise a lot: am I not aiming too big? The Season of Revelation is finished first at the scale it had in Warhammer I, before even thinking about all of Bretonnia; and on the Workshop, two separate versions are planned, the original map and an expanded one. The smallest project that stands on its own and actually ships always beats a big one that stays in a drawer.

My advice

• Write your decisions down, marked “not to reopen” once they're settled.
• Keep an error log from day one: what, symptom, cause, rule.
• Code the rules that can be coded; reread the others every session.
• Demand a control case and an exact command before any conclusion.
• Judge by eye, in game, against a reference, not on a numbers report.
• Split the work into areas, with a single writer per area.
• Don't share the screen without warning, in either direction.
• Keep everything: backups before writing, tidy archives, no hasty deletion.
• Never trust a result you can't sum up in one sentence: have the script explained to you before accepting it.
• Aim small first: ship the smallest version that stands on its own before thinking about expanding it.

Monday morning: where to start

If you're starting from scratch, here's the full sequence, from installing the agent to the first task judged in game. Don't skip the seventh move: a brief that changes nothing. Judge the answer, and above all where the agent says each piece of information comes from, before you let it write anything.

The full story of those four days is here, and the nine biggest obstacles there. And if you want to see this method applied to what comes next, the map of all Bretonnia is being drawn right now, with your ideas.