What is Sora? New OpenAI product stuns users with its ultra-realistic videos

The simulation hypothesis argues that our reality is not what it seems. Instead of inhabiting a real, physical universe, it proposes we’re actually living in a sophisticated computer simulation. Think of it like the movie The Matrix, where the world humans perceive is just a hyper-realistic digital construct.

The theory goes: if technology reaches a level where we can create lifelike simulations, then couldn’t a more advanced civilization have already done this? Well, we may have just hit that point.

OpenAI has unveiled a new artificial intelligence system called Sora, which can make stunningly realistic videos from text descriptions. The internet is going nuts for it, with people claiming that they can’t tell the fake results apart from the real thing.

If you’re getting a sense of déjà-vu, that’s because the wild reaction echoes the newfound sense of wonder elicited by ChatGPT, the AI chatbot also from OpenAI. The digital helper is now used by more than 100 million users per month, including more than half of UK undergrads.

Will Sora be as big? And, when can you use it to wow your mates? Here’s what you need to know about the new AI product everyone is talking about.

What is Sora?

Sora is a new AI diffusion model from OpenAI, the company that’s become a poster child for the modern field of artificial intelligence.

These types of software can create new content that closely resembles the data they were trained on. In the case of Sora, the AI can produce hyper-realistic videos up to a minute in length based on the instructions you type. It can also extend videos to make them longer.

AI diffusion models are another form of generative AI. They work by “destroying” an image by turning it into indistinguishable pixels, and then reversing the process to create a coherent image. One way to look at it is the AI is crumpling and smoothing out a piece of paper with a doodle on it.

Once trained, a diffusion model can be guided by users to craft highly detailed visuals from clumps of data.

What can Sora do?

In OpenAI’s words: “Sora is able to generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background.

“The model understands not only what the user has asked for in the prompt, but also how those things exist in the physical world.”

The results are jaw-dropping, to say the least. To show what Sora is capable of, OpenAI shared a series of realistic demo videos including scenes of humans and animals in motion, a tracking shot of a vehicle on a dirt road, period footage of California during the gold rush, CGI red pandas in a petri dish, and an interior of an art gallery.

As with other AI images, look closely and you may spot some errors, like a cat with an extra paw or a floating chair that makes you feel like you’re watching a glitched-out video game.

OpenAI says Sora is trained on data with variable durations, resolutions, and aspect ratios, which it claims can result in more accurate representations of reality.

Meanwhile, Al boffs reckon it may have been hooked up to Unreal Engine 5, a video game engine designed to build interactive 3D experiences. The giant toolbox provides the core elements, from physics to lifelike lighting and sound design, for developers to create more realistic-looking games.

More broadly, OpenAI believes that video generation models like Sora offer “a path towards building general purpose simulators of the physical world”.

We’re waiting for a call from Morpheus any minute now.

Is Sora out now?

Sora isn’t available to the public yet. Maybe because we’re in an election year and OpenAI doesn’t want to open a can of worms with a realistic video generator that could be deployed to create harmful deepfakes.

As such, the AI model is going through safety checks and testing by select users and experts, including creatives such as visual artists, designers, and filmmakers. The goal is to gather feedback and ensure it can’t be used to create malicious content and sow discord.

As with its other AI tools, OpenAI says it will ban clips that fall foul of its content guidelines, including videos that contain extreme violence, sexual content, hateful imagery, celebrity likeness, or the intellectual property of others.

For now, OpenAI is previewing its research to give the public a taste of things to come. Or, it’s looking to reclaim some of the hype it has recently lost to rival AI products, including ChatGPT competitor Google Gemini and fellow video generator Pika.

When Sora does eventually come out, we assume access could be limited, much like it is for OpenAI’s image generator Dall-E. So, you’ll be able to create a few videos for free and then have to pay for additional clips.

Then again, seeing as power-hungry AI comes with a massive energy bill, OpenAI could choose to make Sora a premium feature for ChatGPT Plus subscribers.