The whole article on one screen
Read time 9 min · Scan time 40 sec1. Training
Terabytes of unlabelled internet text. Millions of fill-in-the-blank exercises. Output is a foundation model.
Millions of dollars · weeks2. Tuning
Narrow the generalist to your task, with labelled data (fine tuning) or human scoring (RLHF).
Labour, not compute3. Generate & retune
A loop, not a finish line. Add RAG to reach live sources the training data never saw.
Retuned weeklyVAEs
Compress, then decode many variations. Anomaly detection.
GANs
Two networks compete. Style transfer, data augmentation.
Diffusion
Add noise, learn to remove it. DALL-E runs on this.
Transformers
Attention across a whole sequence. ChatGPT, Copilot, Gemini.
Creates
Six named risks
What this covers
What generative AI is
The definition IBM settles on, and the two adoption figures it uses to size the shift.
- Generative AI, shortened to gen AI, creates original content on request: text, images, video, audio or software code.
- It runs on deep learning models, described as algorithms that simulate the learning and decision-making processes of the human brain.
- Those models encode the patterns and relationships sitting inside enormous datasets, then use that encoding to interpret a natural-language request and produce fresh output.
- ChatGPT's arrival in 2022 is treated as the moment the technology reached worldwide headlines and adoption accelerated.
- McKinsey found one third of organisations already using generative AI regularly in at least one business function, in its August 2023 state of AI report.
- Gartner projected more than 80% of organisations would deploy gen AI applications or use gen AI APIs by 2026.
The three phases
This is the part most explainers skip. A working application is not one step, it is three, and each one runs on a different clock.
Training
A deep learning algorithm runs millions of fill-in-the-blank exercises over terabytes of raw, unlabelled data pulled from the internet, predicting the next word, the next element of an image or the next command in code, adjusting each time to close the gap.
Tuning
The foundation model is a generalist that knows a lot about many things and cannot hit the accuracy a specific task needs. Tuning narrows it, either with labelled application-specific data or with human feedback.
Generation, evaluation, retuning
An ongoing loop rather than a finish line. Applications get retuned as often as once a week. The foundation model underneath updates perhaps every year or 18 months.
- Training is expensive. Thousands of clustered GPUs, weeks of processing, and costs running into millions of dollars.
- Open-source models such as Meta's Llama 2 let developers skip that step entirely, which is why most teams never train anything.
- Fine tuning means feeding the model labelled data for your task. The example given is a customer service team building hundreds or thousands of documents pairing labelled questions with correct answers. It is labour-intensive, and often outsourced to firms with large data-labelling workforces.
- RLHF, reinforcement learning with human feedback, has people score competing outputs to the same prompt, or correct a chatbot conversationally.
- RAG, retrieval augmented generation, extends the model to sources outside its training data so the application stays current without retraining.
- RAG carries a second benefit worth noting: those additional sources are visible to the user in a way the foundation model's own training data never is.
Foundation models and LLMs
- A foundation model is a deep learning model that serves as the base for many different applications rather than one.
- Large language models are the most common kind today, built for generating text.
- Foundation models also exist for images, video, sound and music.
- Multimodal models support several content types at once.
The four architectures
Four designs, arriving across four years, each still in use for different jobs. The one everybody talks about is the last.
| Architecture | How it works | What it is good at |
|---|---|---|
| VAEs2013 | Two networks paired. One compresses unlabelled data into parameters, the other decodes those parameters back into content. A variational autoencoder decodes multiple new variations, so it can close in on higher fidelity over time. | Anomaly detection, including medical image analysis, and natural language generation. Credited with breakthroughs in image recognition and NLP. |
| GANs2014 | Two networks compete. A generator produces new content and a discriminator judges its accuracy and quality. | Image and video generation. Particularly strong at style transfer, such as turning a photo into a pencil sketch, and at data augmentation to widen a training set. |
| Diffusion2014 | Noise is added to training data until it is unrecognisable, then the algorithm learns to diffuse that noise back toward a desired output, step by step. | Slower to train than VAEs or GANs, and repaid with finer control and higher-quality images. DALL-E runs on a diffusion model. |
| Transformers2017 | Documented in the paper Attention is All You Need by Ashish Vaswani and colleagues. Attention identifies what matters most inside a sequence, so whole sentences process at once rather than word by word. | Most of the tools you know: ChatGPT, GPT-4, Copilot, BERT, Bard and Midjourney. Best at longer output such as articles and papers, and can be tuned to use tools like spreadsheets or HTML. |
They capture context across a whole sequence instead of a word at a time, they train faster because they process in parallel, and they encode training data into embeddings that carry both the data and its context. That combination is what made long, coherent output possible.
What it can create
- Text. Documentation, brochures, emails, website copy, blogs, articles, reports and papers, plus repetitive work such as document summaries and page meta descriptions.
- Images and video. DALL-E, Midjourney and Stable Diffusion for realistic images or original art, style transfer and image-to-image translation. Newer video tools animate from a text prompt or apply effects to existing footage.
- Sound, speech and music. Natural-sounding speech for voice chatbots, digital assistants and audiobook narration, plus original music that mimics professional composition.
- Software code. Writing original code, autocompleting snippets, translating between languages, summarising what a function does, prototyping, refactoring and debugging.
- Design and art. Graphic design assistance, and dynamic generation of environments, characters, avatars and effects for simulations and games.
- Simulations and synthetic data. Including drug discovery, where models generate molecular structures with the properties a compound needs.
The benefits claimed
Five, with efficiency sitting above the rest.
- Greater efficiency. On-demand generation accelerates or automates labour-intensive work, cuts cost, and moves staff onto higher-value tasks.
- Enhanced creativity. Automated brainstorming produces several novel versions at once, which serve as starting points when a creator is stuck.
- Faster decisions. Analysing large datasets, spotting patterns, pulling out insight, then generating hypotheses and recommendations for the people who decide.
- Dynamic personalisation. Reading a user's preferences and history to generate tailored content in real time.
- Constant availability. Running continuously without fatigue, which is what makes round-the-clock support answerable.
Use cases by business area
- Customer experience. Marketing teams draft blog copy, web pages, collateral and emails, and produce personalised copy and visuals in real time based on when, where and to whom an ad is delivered.
- Next-generation chatbots and virtual agents give personalised responses and can start actions on a customer's behalf, which is a step beyond earlier conversational AI trained narrowly for one task.
- Software development and modernisation. Automating new code, and speeding up legacy application modernisation for hybrid cloud by taking on the repetitive coding.
- Digital labour. Drafting and revising contracts, invoices, bills and other paperwork across HR, legal, procurement and finance.
- Science, engineering and research. Proposing novel solutions to complex problems. In healthcare, synthesising medical images to train and test imaging systems.
Agents and agentic AI
- An AI agent is an autonomous program that performs tasks and pursues goals without human intervention, designing its own workflow and reaching for whatever tools it has.
- Agentic AI orchestrates several agents toward a goal that no single agent could reach alone.
- The difference from a chatbot is autonomy. Chatbots operate inside predefined constraints. Agents show goal-driven behaviour and adapt.
- Agents are framed as the natural next step: generative AI creates content from learned patterns, and agents use that content to interact with tools and with each other to decide and finish tasks.
- The example given is worth keeping. A gen AI application identifies the best time to climb Everest around your work schedule. An agent also books the flight and reserves the hotel room in Nepal.
Risks and limitations
Six, named plainly. This is the section to read twice before you put anything in front of a customer.
- Hallucinations. Output that is nonsensical or flatly wrong while reading as entirely plausible. The case cited is the lawyer whose research tool produced several fictional court cases complete with quotes and attributions. Some see this as an unavoidable trade-off against creative capability. Guardrails restricting a model to trusted sources help, as does continual evaluation.
- Inconsistent output. These models are probabilistic, so the same input can return a different answer. That matters where consistency is expected, such as customer service. Prompt engineering, refining and compounding prompts, is the practical fix.
- Bias. Models absorb societal bias from training data, labelled data, external sources or human evaluators. Mitigations named: diverse training data, explicit guidelines during training and tuning, and ongoing evaluation of what comes out.
- No explainability. Many models are black boxes whose reasoning cannot be traced. Even the engineers and data scientists who built them may struggle to explain a specific result.
- No agreed metrics. Judging generated content is hard, because traditional measures miss creativity, coherence and relevance. Building sound evaluation methods is still an open research problem.
- Security, privacy and IP. Models can be pushed into producing convincing phishing emails and fake identities. Separately, what you type during tuning or in a prompt may expose your own intellectual property or someone else's.
- Deepfakes. Images, video or audio manipulated to convince people someone did or said something they never did. Beyond reputation damage, criminals now use them in voice phishing and financial fraud. Detection models are being researched, and in the meantime user education and not resharing unverified material limit the harm.
A black-box model that cannot show its reasoning is a problem long before it is wrong. If nobody can explain how a result was reached, nobody can tell a legitimate decision from a biased one.
The timeline
Generative AI did not start in 2022. IBM traces it back more than a century.
- 1906 The Markov Chain is developed, the mathematical model that later sits underneath Google autocomplete.
- 1964 Joseph Weizenbaum builds ELIZA at MIT, essentially the first chatbot, answering typed input with empathetic responses from pattern-matching scripts.
- 1999 Nvidia ships the first GPU, built for video game graphics. It becomes the default platform for AI model development.
- 2004 Google autocomplete launches, suggesting the next word as you type.
- 2013 The first variational autoencoders appear.
- 2014 The first GANs and the first diffusion models appear.
- 2017 Attention is All You Need documents the transformer, from Ashish Vaswani, a Google Brain team and a University of Toronto group.
- 2019 to 2020 OpenAI releases GPT-2 and GPT-3.
- 2022 ChatGPT launches as a front end to GPT-3, producing coherent, contextual long-form output.
- After 2022 Releases accelerate: Google Bard, now Gemini, Microsoft Copilot, IBM watsonx.ai and Meta's open-source Llama 2.
Watch it explained
Eight lectures, ordered easiest first, chosen because they explain this without assuming you write code. Press Summary on any card to read what it covers before you spend the time. Those bullet points were written from each video's own captions, not from its title.
Start here2 min
Generative AI Explained in 2 Minutes
The shortest honest explanation on this list. If a colleague has two minutes, this is the one.
Easy10 min
AI, Machine Learning, Deep Learning and Generative AI Explained
Sorts out the four words everyone uses interchangeably. Watch this before anything else if the vocabulary is the problem.
Easy, visual8 min
Large Language Models, Explained Briefly
The clearest mental model on this page, and the two scale comparisons are the ones your audience will repeat afterwards.
Easy9 min
What Are Generative AI Models?
The best explanation of foundation models, and unusually candid about the two disadvantages.
Easy6 min
How Large Language Models Work
Turns the numbers into units you can picture, and ends on what businesses do with it in practice.
Easy22 min
Introduction to Generative AI
Carries the single most useful test on this page for telling generative AI from ordinary AI.
Deeper, still non-technicalFull course
Non-Technical Intro to Generative AI
The one to send someone who wants the whole picture, including the ethics, without any maths.
Go deeper27 min
Transformers, the Tech Behind LLMs
Watch when the shorter ones leave you wanting to see inside. Entirely visual, no code.
Watch the two-minute KI-Campus explainer for the shape of it, then the IBM nesting-doll video for the vocabulary. That pair covers enough to follow any conversation about this at work.
Read the original
This summary condenses a long reference page into what a working professional needs to hold in their head. The full article carries more detail on each architecture, the product links, and IBM's own tooling.