COBOL Isn't a Code Problem — It's a Memory Problem

Ask a chief information officer what keeps them awake and you will rarely hear about model benchmarks. You will hear about a payroll run, a claims batch, or a reservation system that has worked for thirty years and that nobody on staff fully understands any more.

That is the backdrop for the latest wave of enthusiasm about pointing large language models at COBOL. Reporting suggests OpenAI used an October 19 announcement to tout what it called "substantial improvements" in its newest models' ability to work with legacy code, framing it as a route out of a chronic shortage of engineers who can safely touch these systems. The problem is real. The attention is welcome. But the framing is off in a way that will cost organisations money if they adopt it uncritically.

The lock-in is institutional memory, not syntax

COBOL has been in production since 1959. The language itself is verbose but not difficult — a competent developer can learn the grammar in a fortnight. What makes these systems immovable is that the programs encode business rules that were never written down anywhere else: a rounding convention agreed during a merger in the late eighties, an exception carved out for one class of accounts, a regulatory formula that changed and then changed back. The code is the only surviving specification.

When people say nobody dares touch it, they do not mean the code is unreadable. They mean nobody can prove a change is safe, because the test oracle is the production system itself. Migration, then, is not translation. It is recovering intent from an artefact and then deciding, rule by rule, whether that intent is still what the business wants.

What AI genuinely changes

There are three real gains, and they are substantial.

Comprehension at scale. A model can summarise a four-thousand-line program, name its inputs and outputs, and produce a plausible narrative of what it does — in minutes, for a system where a senior engineer would bill weeks. Even when the narrative is imperfect, it gives a reviewer somewhere to start.

Characterisation testing. Generating test harnesses and fixture data to capture current behaviour is exactly the kind of tedious, high-volume work models are good at. Freezing behaviour before you change it is the single most valuable engineering act in any migration.

Drafting and mapping. Dependency graphs, field-level lineage, first-draft translations, migration proposals, data dictionary reconstruction. The unglamorous connective tissue that nobody has budgeted for in twenty years.

Figure: Editorial context image from the source page (preview only; rights remain with the publisher) · Credit / further reading: www.theverge.com

The counterpoint: confident translation of misunderstood rules

The failure mode is not that the model produces broken code. It is that the model produces idiomatic, compiling, plausible code that quietly changes behaviour on the path that handles three accounts out of a thousand — the path that also happens to carry the largest balances.

The circularity is the trap. If you ask a model to explain a program and then ask it to generate tests based on that explanation, the tests validate the model's reading, not the business's intent. You have built a very fast, very confident echo. The generated suite goes green, the migration ships, and the discrepancy surfaces in an audit eighteen months later.

There is a second, structural objection. Regulated institutions must be able to explain why a change was safe — to an auditor, a supervisor, a board. "The model said so" is not a control. Any AI-assisted migration needs a human-readable evidence trail: which rules were extracted, who confirmed each one, what test proves it. That trail is expensive, and it is precisely the part vendors do not sell.

A third issue is incentive. It is far easier to market a tool than to market five years of patient organisational archaeology. Expect a wave of products that promise to "modernise your mainframe" and deliver a translation layer that moves the risk rather than removing it — often into a runtime abstraction nobody on your team can debug either.

Where the leverage actually is

Narrow the ambition deliberately. Point AI at the boring majority: documentation, dependency mapping, test scaffolding, field lineage, report program conversion, migration plan drafts. Keep humans on the money paths — interest accrual, eligibility determination, settlement, tax calculation, anything with a regulator attached to it.

The highest-return use is one almost nobody budgets for: using models to build the inventory that was never built. A machine-readable map of program to business rule to human owner to risk rating. That artefact has value whether or not you ever migrate a single line, because it converts an undocumented liability into a managed one.

Figure: HEXOPS original illustration inspired by the topic (not a copy of the source article imagery) · Credit / further reading: www.theverge.com

A 30-day action plan

Week 1 — Inventory and triage

Pick one domain, not the whole estate. List every program in it, its interfaces, its batch schedule, and its last change date. Rank by blast radius: what breaks, and who notices, if this fails silently. You are looking for the two or three modules where an error is survivable, because that is where you will pilot.

Week 2 — Establish ground truth

Have a model produce plain-language summaries of the shortlisted programs. Then put them in front of the one person who actually knows the system — usually someone close to retirement and heavily over-committed. Their corrections are the most valuable dataset you will generate all year. Record them.

Week 3 — One deliberately boring pilot

Choose a module with low business risk and real complexity. Freeze its behaviour with characterisation tests, produce an AI-assisted translation, and require line-by-line human sign-off against those tests. Measure two things only: hours of senior-engineer time consumed, and defects found in review. Not lines converted.

Week 4 — Decide the economics honestly

Compare three numbers: cost of the pilot per module, cost of doing nothing including key-person risk, and cost of the vendor alternative. If the pilot needed more senior review than you expected, that is a finding, not a failure — it is the number that tells you whether the programme scales.

Key takeaways

• COBOL's barrier is lost intent and unprovable safety, not unfamiliar syntax; AI addresses the syntax, not the intent.
• The most dangerous output is fluent, compiling code that passes tests generated from the same model's own reading of the original.
• Highest-return applications are comprehension, characterisation testing, and building the inventory nobody ever built.
• Regulated work needs a human-readable evidence trail; "the model said so" fails audit.
• Pilot on boring, survivable modules and measure senior-engineer hours, not lines converted.

Next steps

• Commission a one-domain inventory before you commission a migration.
• Stand up a behaviour-freezing test harness on your most-changed legacy module this quarter.
• Add an AI-change-evidence requirement to your change management policy now, before the first AI-assisted release.
• Identify your single-point-of-failure engineers by name and start capturing their knowledge through structured model-assisted review sessions.

Sources

• Can AI help solve the COBOL code crisis? — The Verge AI