On February 8th, researchers at Apple published a paper, revealing the company’s ongoing work on an AI tool that can animate images with text prompts.
Now, OpenAI, the maker of ChatGPT and Dall-E, has introduced Sora, its text-to-video model.
According to OpenAI, the new model can generate up to a minute-long videos from scratch, “while maintaining visual quality and adherence to the user’s prompt.” Hyperrealistic examples shown on OpenAI’s page show prompts like “A movie trailer featuring the adventures of the 30 year old space man wearing a red wool knitted motorcycle helmet, blue sky, salt desert, cinematic style, shot on 35mm film, vivid colors.”
Through such detailed prompts, Sora is able to generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background. “The model understands not only what the user has asked for in the prompt, but also how those things exist in the physical world.”
Sora isn’t perfect, though. In a shared example of archaeologists excavating, a plastic chair appears out of nowhere and then continues to hover around. In a different example, where wolf pups are frolicking and chasing each other, one baby wolf seems to run at another, and the two appear to merge. According to OpenAI, Sora might “struggle with accurately simulating the physics of a complex scene, and may not understand specific instances of cause and effect. For example, a person might take a bite out of a cookie, but afterward, the cookie may not have a bite mark.”
However, it shows how far we’ve come in just a year, going from the haunting ‘Will Smith-eating Spaghetti’ video to what Sora can do now. If you haven’t watched the latter, look below, though you might not be able to get the image out of your head.
It’s also worth noting that before making the tool available to users via its products, OpenAI is working with red teamers (experts that try to find and report vulnerabilities and potential for abuse) to put the model through its paces. Further, it is also working on tools to detect if a video was generated by Sora or not.
It is currently unclear when Sora might be available to OpenAI users.
You can learn more about Sora, and check out some video examples shared by OpenAI, here.
Image credit: OpenAI
Source: OpenAI
