How AI actually works.

The short version

Modern AI assistants are powered by large language models (LLMs). An LLM has learned patterns from an enormous amount of text, and it generates answers by predicting, one small piece at a time, what text is most likely to come next given everything it has been told.

That simple idea — very good prediction at massive scale — turns out to produce systems that can write, summarize, translate, reason through problems, and write code. It also explains their main weakness: a prediction can sound perfectly plausible and still be wrong.

Tokens: how AI reads

LLMs don't read words the way we do. Text is broken into tokens — chunks that are often a word or part of a word. Pricing, speed, and limits for AI tools are usually measured in tokens. As a rough rule of thumb, 1,000 tokens is about 750 English words.

# Input text
"Our volunteers are amazing!"

# Roughly how it is split into tokens
["Our", " volunteers", " are", " amazing", "!"]

# The model predicts the next token, again and again
next: " Thank" → " you" → " for" → …

Training vs. using a model

Training is the expensive, one-time (per version) process where a lab teaches a model using huge datasets and specialized hardware. Afterwards, models are refined with human feedback so they follow instructions and behave more safely.

Using a model — called inference — is what happens when you type a prompt. The model doesn't permanently learn from your conversation. Whether your inputs are ever used to train future models depends on the product and plan you use, which is why business plans with "no training on your data" terms matter.

Because training happens at a point in time, every model has a knowledge cutoff. Many assistants now add web search or connect to your documents to work with current information.

The context window: AI's working memory

Everything the model considers when answering — your instructions, the conversation so far, uploaded files, search results — must fit in its context window. Modern context windows are large enough to hold long reports or even several books, but the principle stays the same: the model can only use what's in front of it.

This is why context is the single biggest lever for quality. Telling the AI who the audience is, what good looks like, and what to avoid usually improves results more than any clever trick. Our prompting guide shows a simple structure.

Why AI makes things up

A hallucination is when AI produces information that sounds right but isn't — a made-up statistic, a fake citation, a policy that doesn't exist. It happens because the model is generating plausible text, not looking facts up in a database.

• Verify facts, figures, names, quotes, and links before you rely on them.
• Ground the AI in trusted sources — paste the document, or use tools that search your files and cite them (see RAG).
• Ask the AI to flag uncertainty and list its assumptions.
• Keep a human responsible for anything that affects people, money, or reputation.

Reasoning models

Many leading models now offer a reasoning or "thinking" mode. Before answering, the model works through the problem step by step internally. This substantially improves results on math, logic, planning, analysis, and complex coding — at the cost of slower and more expensive responses.

A good rule: use fast, standard responses for everyday writing and summarizing; switch to reasoning for multi-step analysis, tricky questions, or anything where you'd want a careful colleague to "think it over."

Beyond text: multimodal AI

Today's frontier models are multimodal: they can read images, charts, screenshots, and PDFs; listen and speak in real time; and in some cases generate images, audio, and video. That opens up practical uses like extracting data from scanned forms, describing images for accessibility, or translating a live conversation.

It also raises new responsibilities — especially around deepfakes and voice cloning. Organizations should have a clear rule for verifying unusual requests (for example, a "CEO" voice message asking for an urgent wire transfer) through a second channel.

What this means for your organization

• AI is a powerful first-draft and thinking partner, not an oracle.
• The tool and plan you choose determine how your data is handled — use organization accounts on business plans.
• Quality depends on context in and review out.
• A short AI use policy and basic training prevent most problems.

Further reading

• Elements of AI (free course) — a gentle, non-technical introduction
• Anthropic Academy — AI fluency courses
• OpenAI Academy — recorded sessions on using AI at work