Developer journal · Building With LLMs

It’s Not Really Artificial Intelligence. At Least, Not the Kind You’re Thinking Of.

On large language models, the AI buzzword, and why the distinction matters.

A three-part series exploring the technology, the engineering, and the experience of building Far Reach Industries as an independent developer.

AI is everywhere lately, and suddenly everything is powered by artificial intelligence.

Your phone has AI. Your search engine has AI. Your email client has AI. Your refrigerator might have AI. Every software company seems determined to put those two letters somewhere in its marketing material.

And then there’s ChatGPT, Claude, Gemini, and the growing collection of tools that promise to think, reason, create, and sometimes replace the people using them.

But there’s a question worth asking:

Is any of this actually artificial intelligence?

The answer depends on what you mean by intelligence. And that’s precisely where the marketing gets ahead of the technology.

AI Is an Umbrella Term, Not a Description of Intelligence

First, a technical distinction.

Large language models, or LLMs, are a legitimate branch of artificial intelligence. That’s how the field has classified machine learning systems for decades.

But the term artificial intelligence covers an enormous range of technologies, from relatively simple decision-making algorithms to sophisticated neural networks.

A computer system that identifies objects in an image is AI. So is a system that recommends what movie you should watch next. Neither necessarily understands what it’s looking at or why you might enjoy the movie.

The problem isn’t that calling an LLM “AI” is technically incorrect.

The problem is what people imagine when they hear the word intelligence.

For most of us, intelligence suggests comprehension, awareness, reasoning, and an understanding of the world. We think of something that knows what it’s doing.

Today’s LLMs don’t establish that kind of intelligence simply by producing convincing language.

They are extraordinary computational systems. But extraordinary computation and human-like understanding aren’t automatically the same thing.

What an LLM Actually Does

At its foundation, a large language model is trained to predict tokens—the pieces of text that make up language.

During training, it processes enormous quantities of information and adjusts billions or even trillions of numerical parameters to identify patterns and relationships.

Given a sequence of tokens, it estimates what tokens should come next.

Repeat that process, and you can produce an essay, a conversation, a computer program, or an explanation of quantum physics.

Of course, describing an LLM as merely predicting the next word undersells the complexity of what’s happening. The patterns these models learn can support surprisingly sophisticated behaviors, including problem-solving, code generation, translation, and forms of reasoning.

But the mechanism matters.

An LLM doesn’t consult an internal encyclopedia of verified truths every time it responds. It generates output from learned relationships, the context it’s given, and, where available, external tools and retrieved information.

That’s one reason it can confidently produce something completely wrong.

The model can generate an answer that looks and sounds exactly like a correct answer without that answer actually being correct.

In software development, this becomes especially obvious.

An LLM can generate a beautifully structured function with sensible variable names, explanatory comments, and convincing logic.

And the function can be completely broken.

It doesn’t necessarily know that it’s broken. It may recognize the problem when given test results, additional context, or an opportunity to review its work. But the original confidence of its explanation tells us very little about the correctness of the code.

Fluent output is not proof of understanding, and confidence is not proof of accuracy.

The Illusion of Understanding

One of the most fascinating things about LLMs is how naturally they communicate.

Ask a model to explain a difficult technical concept, and it can respond with analogies, examples, and careful explanations.

Ask it to rewrite that explanation for a twelve-year-old, and it can do that too.

It’s easy to start treating the system as though you’re talking to another person.

We naturally associate language with thought. When something communicates like an intelligent person, we instinctively assume an intelligence resembling our own is behind the words.

But language generation and human cognition aren’t identical processes.

Humans develop knowledge through perception, physical experience, memory, social interaction, and constant feedback from the world. LLMs learn statistical and conceptual relationships from their training and can be supplemented with tools, memory, and other systems.

Whether sufficiently advanced AI systems genuinely understand anything is still a subject of scientific and philosophical debate. We don’t have a universally accepted test for machine understanding or consciousness.

But we don’t need to settle that debate to recognize a practical limitation.

An LLM sounding like it understands something does not mean we should trust it as though it does.

That distinction becomes extremely important when we’re using these systems to do real work.

Why Companies Love the Word AI

Because “AI” sells.

It’s short, recognizable, exciting, and vague enough to encompass almost anything involving machine learning or automation.

Calling a feature “advanced statistical pattern recognition” doesn’t generate much excitement.

Calling it “AI-powered intelligence” sounds like the future.

And companies have every incentive to associate their products with that future.

The result is that genuinely impressive technical breakthroughs get mixed together with ordinary automation, predictive analytics, simple recommendation algorithms, and features that existed long before the current AI boom.

Everything becomes AI.

The word loses much of its descriptive value.

This isn’t unique to artificial intelligence. Technology has always attracted fashionable terminology. Cloud computing, big data, blockchain, Web3—each became a marketing label applied far beyond its most useful technical meaning.

AI is simply the latest, and potentially the most consequential, example.

When every feature is intelligent, the label tells us very little about what the feature actually does.

So Why Am I Using LLMs to Build a Game?

Here’s where my skepticism might sound contradictory.

I’m developing Far Reach Industries, a colony simulation game, and large language models have become an important part of my development workflow.

Not because I believe I’ve acquired an artificial software engineer that understands my entire project.

Quite the opposite.

I use LLMs because I have a reasonably clear understanding of what they’re good at, what they’re bad at, and how much work is still required from me.

They can generate implementation ideas, help explore architectural approaches, write code, suggest tests, explain unfamiliar APIs, and accelerate repetitive programming tasks.

They can also introduce subtle bugs, misunderstand requirements, invent APIs, and confidently recommend approaches that aren’t appropriate for the project.

Those limitations aren’t theoretical. They’re the kinds of problems you encounter when you actually use these tools to build substantial software.

And Far Reach Industries isn’t a small project.

I’m attempting to create a simulation with interconnected systems, procedural worlds, infrastructure networks, colony development, and enough underlying complexity to make the simulation interesting.

It’s the kind of project that would have been difficult for me to justify attempting alone.

LLMs have changed that calculation.

They don’t magically eliminate the work. They let me attempt more of it.

I’m still responsible for deciding what the game should be, how the systems fit together, whether the architecture makes sense, and whether the resulting code actually works.

The LLM is a tool. A remarkably capable tool, but a tool nonetheless.

I don’t need it to be genuinely intelligent in the human sense for it to be extraordinarily useful.

The Difference Between Generating Code and Engineering Software

This is where I think a lot of the discussion about AI-assisted development goes wrong.

Generating code is not the same thing as engineering software.

Software engineering involves understanding requirements, selecting appropriate abstractions, managing complexity, evaluating trade-offs, testing assumptions, debugging failures, and maintaining systems over time.

An LLM can contribute to many of those activities.

But producing 500 lines of code in a few seconds doesn’t mean you’ve solved the engineering problem.

Sometimes you’ve simply created 500 lines of code that now need to be understood, tested, and maintained.

The important questions aren’t whether the model produced convincing code or whether it completed the task quickly.

The questions are:

• Does the implementation actually work?
• Does it fit the architecture?
• Is it correct under conditions that weren’t included in the prompt?
• Can it be tested and maintained?
• Do I understand the decisions being made well enough to recognize when they’re wrong?

When I use an LLM effectively, it speeds up an engineering process that I’m still directing.

When I use one carelessly, it can accelerate the creation of technical debt.

That’s a distinction worth keeping in mind whenever someone claims AI can build an entire application from a few sentences.

It might generate something that looks like an application. Whether the result is good software is another question entirely.

We Don’t Need to Pretend It’s Something Else

I think one of the unfortunate consequences of AI marketing is that it creates two opposing reactions.

On one side, people treat LLMs as nearly magical systems approaching human intelligence, capable of replacing entire professions.

On the other, people dismiss them as glorified autocomplete, implying there’s nothing particularly impressive happening.

Neither position is especially useful.

Modern language models are more capable than simple autocomplete. Their ability to generalize across tasks, work with code, synthesize information, and solve certain complex problems is genuinely remarkable.

At the same time, those abilities don’t justify assuming they possess human-style understanding, reliable judgment, or autonomous expertise.

There is a vast amount of useful technology between a basic text predictor and a conscious artificial mind.

That’s where today’s LLMs belong.

We can appreciate what they accomplish without exaggerating what they are.

And we can recognize their limitations without pretending they aren’t transforming parts of software development and other fields.

Intelligence Isn’t the Point. Capability Is.

For me, the most interesting question isn’t whether an LLM is really intelligent.

It’s what the technology allows me to accomplish.

A few years ago, building a game with the scope I’m envisioning for Far Reach Industries would have seemed impractical as a hobby project.

Today, I can explore implementations, develop systems, investigate problems, and iterate on ideas at a pace that changes what’s feasible for one developer.

That’s significant.

But it’s not significant because I’ve found an artificial mind that can build a game for me.

It’s significant because I’ve gained access to a new category of development tool that substantially expands what I can attempt.

The distinction matters.

When we treat these systems as magical intelligence, we overestimate them, trust them too easily, and misunderstand their failures.

When we understand them as powerful but imperfect computational tools, we can use them more effectively.

And nowhere is that distinction more important than software development.

Because there’s a fundamental difference between a tool that can generate code and one that can reliably engineer an entire software system.

LLMs have become remarkably good at the first. The second is much more complicated.

That’s what I want to explore next.

Up next: AI Can Write Code. Can It Actually Engineer Software?

What happens when generating an implementation becomes almost trivial, but maintaining the software is as difficult as ever?

I’m documenting the development of Far Reach Industries, including the game design, the engineering challenges, and how I use LLMs throughout the programming process. You can follow the project through Crown & Crabapple, the small independent studio I created to give this game a home.

Follow the development of Far Reach Industries, a science-fiction colony and industrial simulation game from Crown & Crabapple.