How do large language models work? They break text into small pieces called tokens and use a huge neural network, trained on vast amounts of text, to predict the most likely next token, one at a time. Training sets billions of internal numbers (parameters) so those predictions become accurate, and later fine-tuning teaches the model to follow instructions and answer helpfully. Tools like ChatGPT, Claude and Gemini are built on this approach.

Ask Claude or ChatGPT to rewrite an email, explain a tax form or summarize a 40-page report, and the answer arrives in seconds, often in better shape than a first draft from a human. It feels like talking to something that knows things. Under the hood, the mechanism is both simpler and stranger than that.

Understanding the basics helps you use these tools well, and it explains their quirks: why they sometimes state wrong facts confidently, why they’re great at drafting and weaker at arithmetic, and why “just ask the AI” isn’t always safe for money decisions.

Key takeaways

  • LLMs generate text by repeatedly predicting the next token based on everything before it.
  • Pretraining on enormous text datasets teaches language patterns and a lot of world knowledge.
  • Fine-tuning and human feedback turn a raw text predictor into a helpful assistant.
  • LLMs can “hallucinate” plausible but false information, so verify facts and numbers.
  • Newer systems add tools like web search and code execution to cover their blind spots.

What is a large language model?

A large language model (LLM) is an AI system trained on massive amounts of text to predict and generate language, which lets it answer questions, write, summarize and translate. “Large” refers to both the training data and the number of parameters, which can run into the hundreds of billions.

Almost every major LLM today is built on the transformer architecture, introduced by Google researchers in the 2017 paper “Attention Is All You Need”. The transformer’s key innovation, called attention, lets the model weigh how relevant every word in a passage is to every other word, which is how it keeps track of context across long documents.

How do large language models work, step by step?

Step 1: Text becomes tokens

A token is a chunk of text, often a word or part of a word. “Budgeting” might be one token or split into “budget” and “ing.” Each token is mapped to a number, and then to a long list of numbers (a vector) that represents its meaning in relation to other tokens. Usage limits and pricing for AI tools are often measured in tokens for this reason.

Step 2: The model predicts the next token

Given your prompt, the model calculates a probability for every possible next token. “The Federal Reserve raised interest…” makes “rates” very likely. It picks one, appends it, and repeats, token by token, until the answer is complete. A setting called temperature controls how adventurous those picks are.

Step 3: Context is everything

Each prediction takes into account all the text in its “context window,” which includes your conversation so far and any documents you’ve pasted. Context windows have grown dramatically, so many models can now take in entire reports or codebases at once.

How are LLMs trained?

Stage What happens What the model learns
Pretraining The model predicts missing or next tokens across trillions of tokens of text, adjusting its parameters to reduce errors Grammar, facts, reasoning patterns, coding, writing styles
Fine-tuning Training on curated examples of good instructions and answers To follow instructions and act like an assistant
Feedback training Humans or AI systems rate answers; the model is trained to prefer better ones Helpfulness, honesty, refusing harmful requests
Reasoning training Practice on problems with checkable answers, such as math and code Working through multi-step problems before answering

Pretraining is the expensive part. It runs for weeks or months on thousands of specialized chips, which is a big reason AI companies are spending heavily on data centers. Our explainer on AI data centers and energy covers the power side of that build-out.

Developers publish details about their approaches. Anthropic, which makes Claude, shares research on training methods and model behavior at anthropic.com/research.

Do LLMs actually understand what they say?

This is a genuine debate. LLMs don’t look up answers in a database, and they don’t have experiences. But to predict text well across so many subjects, they develop internal representations of concepts and relationships that go well beyond memorized phrases. Interpretability research has found features inside models that correspond to recognizable ideas.

The practical answer for users: LLMs are very good at producing language that fits the context and are often correct, but they don’t have a built-in sense of whether a specific fact is true. That’s the root of hallucination.

Why do LLMs make mistakes?

  • Hallucination. When a model lacks reliable information, it may still produce a fluent, confident answer. Fake citations and invented statistics are classic examples.
  • Knowledge cutoff. A model’s built-in knowledge stops at its training date. Current interest rates, this year’s contribution limits or yesterday’s market move may be wrong unless the tool searches the web.
  • Arithmetic and precise counting. Token-by-token generation isn’t a calculator. Many assistants now run code for math, which helps.
  • Prompt sensitivity. Vague questions get vague answers. Our guide to prompt engineering tips shows how to ask better.

ChatGPT vs Claude vs Gemini: are they the same thing?

They’re built on the same core idea but differ in training data, fine-tuning, safety approach, context window size and the tools they connect to. OpenAI makes ChatGPT, Anthropic makes Claude, and Google makes Gemini. Each company releases new model versions frequently, and their relative strengths shift with each release, so independent comparisons go stale quickly. Some companies also release models with publicly downloadable weights; see our comparison of open-source vs closed AI models.

What this means for you

Treat an LLM like a brilliant, fast, occasionally overconfident assistant. It’s excellent for drafting, explaining, summarizing, brainstorming and organizing, and it can save hours. It’s a poor sole source for facts that carry consequences: tax rules, medical dosages, legal deadlines, or the current rate on a CD.

Our rules for money tasks:

  1. Use AI to understand, not to decide. Ask it to explain how a Roth conversion works, then confirm the numbers on irs.gov.
  2. Ask for sources, then click them. If a link doesn’t exist or doesn’t say what the model claimed, discard the claim.
  3. Don’t paste sensitive data like full account numbers or your Social Security number into any chatbot.
  4. Use it for the boring parts. Categorizing transactions and building a budget template are great uses. Our guide to AI budgeting has step-by-step prompts.

For a broader framework on AI risk, the US National Institute of Standards and Technology publishes its AI Risk Management Framework.

Frequently asked questions

What is a token in AI?

A token is a small unit of text, usually a word or part of a word, that a language model reads and generates. Models process and produce text token by token, and AI services often price usage per million tokens.

Do LLMs learn from my conversations?

It depends on the provider and your settings. Some services may use conversations to improve models unless you opt out, while business and API plans often exclude your data from training by default. Check the privacy settings of the tool you use.

Can LLMs access the internet?

The model itself only knows what was in its training data. Many chat products add a web search tool, letting the model look up current information and cite sources, which reduces but doesn’t eliminate errors.

What does “parameters” mean in AI?

Parameters are the internal numbers a neural network adjusts during training. More parameters generally allow a model to capture more patterns, though training quality and data matter as much as size.

Is agentic AI the same as an LLM?

No. Agentic AI uses an LLM as its “brain” but adds the ability to plan, use tools and take actions over multiple steps. Our explainer on what agentic AI is covers the difference.