Agentic AI Development

Before you can build any Ai agent that plans, remembers, or takes action on your behalf, you need a working mental model of what's actually running underneath it: a large language model (LLM). So that's where we are starting — not with agents, not with frameworks, but with the agent's engine itself. This chapter builds that mental model from the ground up: what generative AI actually is, how a model turns your sentence into a prediction, and, this is the part that matters most, exactly where it runs out of road. Hold onto that last part, because every limitation covered here is the specific reason a later chapter exists. By the time you finish, you won't just know what a large language model is — you'll know precisely why it isn't enough on its own, which is the question this entire series spends the rest of its pages answering.

What you'll learn

Prerequisites

This chapter assumes the practitioner baseline for the whole series (see the series overview): general programming experience, comfort with HTTP/REST and JSON, and basic Git — but zero prior exposure to AI or machine learning. If you've never trained a model, read a research paper, or called an LLM API, this chapter is written for exactly that starting point.

No environment setup, SDK, or API key is needed here — there's no code to run yet. Chapter 2 is where hands-on code against a real model begins; this chapter builds the vocabulary you'll need once you get there.

What generative AI actually is

Let's start with the basic distinction, because everything else in this chapter builds on it. Traditional software is a long chain of if-this-then-that: a developer enumerates the rules, and the program can only do what's explicitly written down. Generative AI works differently. Instead of following hand-written rules, a generative model learns statistical patterns from an enormous amount of existing data. It then leverages those learned patterns to generate novel content that is consistent with what it has learned, a sentence, a software function, an image, or another form of content, even when that exact output has never appeared in its training data.

Here's the plain-terms version, and it's worth sitting with for a second: a generative model doesn't "look up" the answer to your prompt the way a search engine or a database would. It computes, one small piece at a time, what's statistically likely to come next given everything it has seen so far including everything you've typed in the current conversation. That single fact, new content computed rather than retrieved, is the seed of almost everything interesting and everything frustrating about these systems. It's also why the same prompt can produce a slightly different answer twice: the model isn't recalling a fixed record, it's recomputing a plausible one.

Generative AI isn't limited to text, either. Depending on how a model was trained, it can generate natural language and source code, images, audio and video, structured data such as JSON or SQL, even design prototypes. This series is about text- and code-generating models specifically, the kind that power AI agents, but the "learn patterns, generate new output" definition holds across all of them.

Now, here's a question worth pausing on before moving forward: if a model just learns patterns from data, where does that data come from, and what kind of model do you actually end up with? That's exactly what the next idea, the foundation model, answers — and it's the concept that makes almost everything else in this series possible.

Foundation models: one model, many applications

Before approximately 2020, machine learning systems were typically developed for narrowly defined, task-specific objectives. A team might train one model to classify spam, another to perform machine translation, and yet another to detect fraudulent transactions. Each model was optimized for a particular problem and dataset, with little ability to generalize beyond its intended purpose. If a fundamentally different task emerged, the conventional approach was to develop and train a new model, often from the ground up. In essence, the model’s capabilities were largely confined to the specific task for which it had been trained.

A foundation model breaks that pattern, and it's worth understanding why, because it's the entire reason you can build an agent without training anything yourself. It's trained once, on a huge and deliberately broad dataset (books, code, articles, conversations), using self-supervised learning — a process where the model teaches itself by predicting missing or upcoming pieces of its own training data, rather than a human labeling every example by hand. We'll get precise about what "trained" actually means for a model in a moment; for now, think of it as the one-time process that sets up everything the model knows how to do. What comes out the other end isn't a spam classifier or a translator. It's a general-purpose model whose learned patterns can be adapted, through prompting, fine-tuning, retrieval, or tool use, to a wide range of downstream jobs, without retraining from scratch for each one.

That's also where the name comes from: a building's foundation supports many different rooms built on top of it, and a foundation model supports many different applications built the same way.

flowchart TD
    FM[Foundation Model] --> LLM["Large Language Models
text + code"]
    FM --> VMod[Vision Models
image understanding]
    FM --> IG[Image Generation Models]
    FM --> AMod[Audio Models]
    FM --> VID[Video Models]
    FM --> MM["Multimodal Models
text + image + audio + video"]

Diagram: a foundation model as the parent category, branching into large language models, vision models, image generation models, audio models, video models, and multimodal models.

GPT, Gemini, Llama, and Claude are all foundation models. A large language model (LLM), the type this entire series is about, is a foundation model specialized in understanding and generating language and code. Every LLM is a foundation model; not every foundation model is an LLM — some are trained for images, audio, or video instead, or in addition.