Generative AI & Large Language Models
Tools like ChatGPT work by predicting the next word, over and over, extremely well.
What you will learn
- Explain how an LLM generates text
- Define tokens, prompts and hallucination
- Write clearer prompts
What “generative” means
Generative AI creates new content — text, images, code, audio. The most famous kind is the Large Language Model (LLM) behind tools like ChatGPT, Gemini and Claude.
The one trick: predict the next word
An LLM is, at heart, a gigantic next-word predictor. Trained on a huge amount of text, it learned which word is likely to come next. To write a sentence, it predicts one word, adds it, and predicts again — over and over.
Here is the exact loop the model repeats to turn your prompt into an answer:
- Read the text so far. It starts with your prompt, e.g.
The sky is. - Rank the likely next words. From everything it learned, it scores each possible next word —
bluehigh,greenlow. - Pick one of the likely words (here,
blue). - Append it to the text, giving
The sky is blue. - Repeat steps 1–4 with the new, longer text — predicting the next word each time.
- Stop when it predicts a special “end” signal or hits a length limit, then returns the finished sentence.
# A cartoon of how an LLM generates text (the real model is far bigger)
prompt = "The sky is"
# The model ranks likely next words from what it learned:
# "blue" -> very likely
# "clear" -> likely
# "green" -> unlikely
# It picks a likely word, appends it, and repeats:
# "The sky is" -> "blue" -> "blue today" -> "blue today and" -> ...Note: Output: The sky is blue today and clear. The model never “looked up” the weather. It produced text that is statistically likely from its training — which is usually sensible, and sometimes confidently wrong.
Words you should know
| Term | Meaning |
|---|---|
| Token | A chunk of text (≈ a word or word-piece) the model reads and writes |
| Prompt | The instruction or question you give the model |
| Hallucination | When the model states something false but sounds confident |
| Fine-tuning | Extra training to specialise a model for a task |
Watch out: LLMs hallucinate: because they predict plausible text rather than look up facts, they can invent names, dates or sources. Always verify anything important.
Writing better prompts
- Be specific: say the role, the format, and the length you want.
- Give an example of a good answer when you can.
- Ask it to show its steps for reasoning tasks.
Tip: This connects right back to Lesson 1: an LLM is dazzling pattern-matching on text, not understanding. That single idea explains both its magic and its mistakes.
Q. At its core, how does an LLM like ChatGPT produce a sentence?
✍️ Practice
- Rewrite a vague prompt (“tell me about dogs”) into a specific one (role, format, length).
- Explain in one sentence why an LLM can give a confident but wrong answer.
🏠 Homework
- Try the same question on an AI chatbot with a vague prompt and a specific prompt. Note how the answers differ.