How AI Video Generation Works in 2026: The 120-Frame Pipeline That Cost OpenAI Sora

On 24 March 2026, OpenAI told developers that Sora was finished. The consumer app closed on 26 April. The API goes dark on 24 September. OpenAI's own help centre states both dates plainly, and no successor video product has been announced.

Six months before that notice, Sora had been the most talked-about consumer AI product on the planet.

Nothing had broken. The model still generated video. What ended Sora was arithmetic, and that arithmetic happens to be the clearest available route into understanding how AI video generation actually works. Almost every design decision inside a modern video model exists to make one number smaller. Once you can see which number and why it grows so fast, the technology stops looking like magic and starts looking like a series of expensive engineering compromises.

What follows tracks a single clip from the instant you press enter to the moment it appears on screen, then tests that understanding against the current public rankings and the current price sheets.

The model has never seen a camera

A generative video model does not film anything. It has no scene, no lens, no lighting setup. It begins with random noise and removes that noise in steps until what remains resembles a video matching the words you typed.

This is worth separating from two neighbouring things that also get sold as AI video. Template editors stitch together stock clips and captions, which is automation rather than generation. Avatar platforms such as Synthesia or HeyGen animate a presenter reading your script, a narrower problem of matching one face and one voice to text. Neither invents pixels from scratch.

Hold onto that last row. Every glitch examined later in this article traces back to it.

What happens between pressing enter and seeing a clip

The pipeline runs in six stages. Modern systems overlap and optimise them heavily, but the sequence holds across effectively every major model shipping today.

1.   Your words become numbers. A text encoder converts the prompt into a numerical representation capturing subject, setting, lighting, motion and mood. The model never reads your sentence. It reads this encoding, which is why two phrasings of the same idea can produce different clips.

2.   The canvas starts as static. Generation begins with pure random noise across every frame at once, closer to an untuned television than to any recognisable scene.

3.   The model subtracts noise, over and over. At each pass it predicts a slightly cleaner version, checking against the prompt encoding every single time. Rough shapes appear first, then structure, then detail. Fewer passes generate faster. More passes look better.

4.   Attention holds the frames together. Transformer layers track relationships across space and across time, so an object keeps its identity as the camera moves. This is the stage that decides whether a clip is usable at all.

5.   Audio is generated in the same pass. On newer models, dialogue and ambient sound are produced jointly with the picture rather than added afterwards. Joint generation is what gets lip movement to match speech.

6.   The result is decoded. Everything above happened in a compressed mathematical space. A decoder expands it back into ordinary frames, and upscalers may raise resolution or smooth motion.

What a diffusion model actually learns

A diffusion model is trained by doing the reverse of its eventual job. Researchers take real video, add noise in escalating amounts until nothing recognisable survives, and train the network to predict exactly how much noise was added at each step.

Run that process backwards, starting from noise and repeatedly subtracting whatever the model predicts, and you have generation. Nothing was memorised and replayed. The model learned a statistical shape of what video looks like.

Why transformers are the other half of the answer

Diffusion answers a question about a single frame: what should this look like. Transformers answer a question about relationships: how does this frame connect to the ones around it.

Neither solves the problem alone. Diffusion with no sense of time would happily render a beautiful frame ten with no connection to frame nine. Transformers with no diffusion underneath would have relationships to track and no way to paint convincing pixels for any of them. That pairing is why technical papers say "latent diffusion transformer" as one phrase rather than three separate techniques.

Latent space, and why it saves the whole enterprise

Running all of this on full-resolution pixels would be prohibitive, so the work happens in a compressed representation instead. The scale of that compression is startling. Lightricks' LTX-Video operates at roughly a 1:192 compression ratio, encoding video at about 32 by 32 by 8 pixels per token.

Compression is the cost lever. It is also, as the next section shows, nowhere near enough.

The branch that works differently

A smaller family of models rejects the approach described above. Instead of denoising an entire clip at once, autoregressive models generate frames or short chunks conditioned on whatever came before, closer to how a language model produces one word after another.

The trade is real. Extending a clip becomes cheap, because the model only ever holds a short window in memory, and long-form behaviour tends to hold together better. The cost is that errors compound forward. A mistake in chunk three propagates into chunk four with nothing to correct it, whereas a diffusion model revisits the whole clip on every pass. Most flagship models today take the diffusion route, and the leaderboard in the next section is dominated by them.

Why any of this changes how you write a prompt

The pipeline explains several habits that experienced users arrive at by trial and error.

•   Describe motion explicitly. The encoding carries motion information, and a prompt silent about movement leaves the model to guess.

•   Specificity compounds. The prompt is consulted at every denoising pass, so a tighter description gives the model a firmer target dozens of times over rather than once at the start.

•   Reach for image-to-video when identity matters. No amount of prompt detail locks a face as reliably as a reference frame does.

•   Keep shots short and simple. Complexity multiplies the number of relationships the attention layers have to track, and each one is a chance for the clip to come apart.

Anatomy of five seconds

Here is where the arithmetic that killed Sora comes into view.

A five-second clip at 24 frames per second is 120 frames. Every one of those frames has to look correct on its own, and all 120 have to agree with each other about where objects sit, what colour they were a moment ago, which direction the light falls from, and how fast anything is moving. Generating a still image means getting one thing right. Generating five seconds of video means getting 120 things right simultaneously, plus the relationships between them.

Generating an image is one shot that has to look right. Generating video is 120 shots that all have to look right and agree with each other.

MIT Technology Review measured this directly in 2025. An open-source model producing five-second clips at 16 frames per second consumed roughly 3.4 million joules per video, more than 700 times the energy of a single high-quality image. An earlier, grainier version of the same model at 8 frames per second needed about 109,000 joules. Doubling the frame rate and raising quality multiplied energy use more than thirtyfold.

3.4 million joules is about 944 watt-hours. Roughly an hour of microwave time, for five seconds of footage.

OpenAI's stated reason for winding down Sora was a shift of engineering capacity toward next-generation models and world simulation research. Public reporting has consistently pointed at the same underlying pressure: video inference costs far more per request than text, and a consumer subscription never covered sustained per-user generation. The GPUs earned more serving other workloads.

Three doors into the same pipeline

Everything above describes one route in, text. There are two others, and the choice between them is a trade between creative range and control.

The pattern is consistent. The more you hand the model up front, the less it invents and the more predictable the output becomes. Text-to-video gives the model the widest latitude and gives you the least say in what comes back.

The 2026 leaderboard, and the names you probably expected

Understanding the pipeline is one thing. Knowing which models are winning with it is another, and the current answer surprises most people.

Artificial Analysis runs a blind video arena where users compare two clips generated from the same prompt without seeing which model made either one. Votes produce Elo ratings. As of August 2026, the top of the text-to-video leaderboard for models with audio looks like this.

Alibaba's Wan 3.0 leads at 1,240. Google's Gemini Omni Flash sits three points behind. MiniMax H3, which ships with open weights on Hugging Face, holds third at 1,226. ByteDance's Seedance 2.0 follows.

Veo 3.1, the model most Western coverage still calls the quality leader, ranks twelfth at 1,090.

Sora does not appear on the leaderboard at all.

Two structural facts explain most of this table. Chinese labs have shipped faster and priced lower through 2026, and open-weight models have closed most of the gap to closed ones. MiniMax H3 ranks third overall while being downloadable.

What a minute of video costs

Artificial Analysis also publishes API pricing for one minute of 1080p output at each model's default settings. Plotting price against quality produces a chart with almost no diagonal to it.

Veo 3.1 is the most expensive model on the board at $24.00 per minute and ranks twelfth. Gemini Omni Flash costs $6.00 and ranks second. Wan 3.0 leads the entire leaderboard at $12.00, half the price of the model it beats by 150 Elo points.

Price tracks compute allocation and commercial positioning. It does not track how much a stranger prefers your clip.

Where the illusion slips

The failure row from the first table now pays off.

Runway ran a perception study it calls The Turing Reel. 1,043 participants each judged 20 five-second clips, half real footage and half generated by Gen-4.5, viewing one clip at a time. Overall detection accuracy landed at 57.1%, barely above a coin flip. Only around 9.5% of participants scored well enough to count as reliable detectors.

The breakdown by subject is the more interesting result.

On animals and architecture, accuracy fell below 50%. Participants were more likely to call a real clip fake than the reverse. Human faces and hands remained the categories people read most reliably, which fits what we know about the pipeline: hands contain many small fast-moving parts, and human faces are the subject viewers have spent their entire lives learning to scrutinise.

Every visible artifact maps to a specific stage of the pipeline described earlier.

Long-range coherence and the quadratic cost of attention are shared weaknesses of the whole diffusion transformer class, which is why clip lengths across the field cluster in the same range instead of scaling freely.

Who owns the clip once it exists

Two separate questions hide inside this one, and conflating them causes most of the confusion.

•   The tool's terms. Commercial rights vary by platform and by subscription tier. The terms attached to the specific model matter, not a general impression of the company.

•   The rights on your inputs. Reference footage, images, audio and any person's likeness carry whatever rights they arrived with. Feeding protected material into a model does not launder it.

Provenance infrastructure has moved faster than the law. Several major providers now embed machine-readable C2PA Content Credentials into their outputs, which travel with the file and record how it was made. Runway states publicly that it does this across its model outputs.

This also draws the line between generation and deepfakery. A deepfake replaces a real person's likeness, usually without consent. Generation is the broader underlying capability, and most of its output involves no real person at all. Provider usage policies generally prohibit the former.

None of the above is legal advice, and rules differ sharply by jurisdiction.

What the next twelve months change

Resolution has stopped being the frontier. Every serious model on the leaderboard delivers 1080p, and several offer more.

Four things are moving instead.

•   Duration and multi-shot continuity. Holding a character consistent across cuts, rather than only across adjacent frames, is the consistency problem restated at a larger scale.

•   Native audio as a default rather than a feature. The separate leaderboards Artificial Analysis maintains for models with and without audio will likely collapse into one.

•   Open weights at the frontier. MiniMax H3 sitting third on a blind leaderboard while being freely downloadable changes who can build on this technology.

•   Real-time generation. Video that responds and updates while it generates, instead of rendering once and stopping. Runway has published research in this direction.

Sora's shutdown is the clearest available reading of where the constraint actually sits. The model was not beaten on quality. It was retired because 3.4 million joules per five seconds does not survive contact with consumer pricing, and because the compute was worth more elsewhere.

The 120 frames are the whole story. Every architectural choice in this article exists to make them cheaper to agree with each other, and the labs currently winning are the ones making that trade at the lowest price per minute.