Since late last year, the AI space has been exploding with interest. New tools are emerging that can create images, correct audio, and even write software for us. Some people are predicting that we’ll all be out of a job soon, but I believe the reality will be quite different. The impact of AI will vary across society, but let’s focus on the video industry for now and explore what’s possible.
Where is the tech today?
In the latest releases of Premiere Pro and DaVinci Resolve, text-based editing has made waves. However, this technology has actually been around for years. It’s more appealing now with the advancements in AI transcription, but using a transcript for rough-cutting hasn’t revolutionized the editing process yet. It’s useful, but it doesn’t make editors obsolete.
Recently, there has been a lot of excitement around image and text generation tools. These tools, such as DALL·E, Midjourney, Stable Diffusion, and ChatGPT, have paved the way for new instances of AI technology. We now have tools that can clean up audio, synthesize voiceovers, and remove objects or people from videos. While image generation is more commonly used for tasks like creating video thumbnails, video generation is also possible.
Runway has recently introduced their Gen1 generative tools, which have augmented their AI-powered video utilities. Their new video-to-video transformer allows creators to transform their own videos based on the look of a still image. However, the level of realism is still not up to par for professional work.
Overall, AI can succeed in contexts where perfect realism isn’t required. It’s especially effective if you’ve trained your own model to create the specific style of work you need. Special effects will also become easier to create, and the more you move away from realism, the better AI performs.
Concept art is something that AI can do well and much faster than humans. AI-generated music may not be as good as what a talented human can produce, but it can still work in a pinch. ChatGPT’s writing may not be inspired, but it can spark fresh ideas or effectively massage existing ones. The common theme here is that AI excels at remixing rather than pure creativity, making it more suited for non-creative tasks like summarizing or interpreting client feedback.
Here’s a great example of an AI remix that combines still images with animation:
At the moment, I would grade the creative output from most AI video tools as a B- on average. While competent, true imagination and flair still come from humans. There is plenty of poor AI content out there, and it remains to be seen if AI can consistently produce reliable, repeatable A-class results. To be considered good enough, it has to look and sound real, and full reality simulation is still out of reach.
What is coming soon?
Runway has just introduced their Gen 2 update, which includes text-to-video synthesis. This update will undoubtedly improve the quality, although it may never reach “real world” quality. However, it’s another step forward for the pre-viz process and for creators who don’t require realism. If you need a temporary clip of “dude surfing at sunset,” you can quickly generate one, but it won’t be photorealistic. It’s still compelling, and Runway isn’t the only one working on advancements. Adobe’s entry into the AI space with Firefly technology is creating waves, and it’s currently in beta.
While Adobe has been using AI techniques for Content-Aware Fill and other features for a while now, modern image generation techniques promise to do even more. It makes sense for Adobe to stay on top of the best “inpainting” methods and to utilize ChatGPT’s power to drive software features with human-written instructions. Runway also incorporates this approach, but integrating it into existing software will be a significant advantage.
I believe that the true potential of AI lies in letting models like ChatGPT control our software for us. This has the potential to revolutionize all industries. Imagine having a super-powered Siri that knows how all your software works and can assist you in ways we can’t even imagine yet.
, understand the intricacies of their software, and utilize AI as a tool to enhance their work. While AI can bring new techniques and capabilities to the table, it’s important to integrate them seamlessly into existing workflows. The goal is not to rely solely on flashy gimmicks, but to make complex tasks more accessible and efficient.
Adobe’s future-looking demo showcases various AI-powered features for video editing. From music generation to sound effect placement, text-based color correction, and face correction, these tools have the potential to streamline the editing process. Transcription capabilities and automatic B-roll placement also offer promising advancements. However, it’s crucial for these features to be controllable and adaptable to real-world footage.
One standout feature is the script-to-storyboard-to-animatic tool. This innovation allows filmmakers to provide clients with a rough version of the final video, improving the collaborative process and enhancing the overall filmmaking experience. Combined with voice synthesis technology, this tool revolutionizes the previewing process and ultimately leads to better films.
The true power of AI lies in its ability to integrate new techniques into existing workflows. By harnessing the capabilities of AI while leveraging the expertise of professionals, complex tasks can become simpler and more efficient. This may also result in a shift in the skillsets required for certain jobs.
The impact of AI in video production will be significant. AI-based plug-ins and apps will automate routine tasks, such as face replacement and voice synthesis. Keying and background replacement will become easier, thanks to advancements in technology. Animation and special effects will also become more accessible and cost-effective. However, this progress will raise client expectations, pushing professionals to deliver even better work.
Unfortunately, the rise of AI will also lead to job displacement in certain areas. Tasks that involve producing temporary work that can be replaced by AI-generated content are at risk. For example, a 3D artist who used to spend weeks creating models for games now relies on generative AI for faster output. Those in non-creative roles or those whose work can be reduced to curating AI-generated content should consider exploring new areas of expertise.
Despite these breakthroughs, AI still has limitations. Progress in areas like self-driving cars and image segmentation has shown that there are boundaries to what AI can achieve. While image segmentation technology has improved, it may not yet be reliable enough for professionals to abandon greenscreens entirely.
Creating an app that consistently produces great output is far more challenging than creating one that produces decent output most of the time. As the novelty of AI-based generative tools wears off, funding for advanced AI projects may dwindle. Actors will continue to act, writers will continue to write, and post-production professionals will still play a vital role in editing, audio correction, and visual effects. AI will serve as a valuable tool, raising the standard of output expected from professionals.
In conclusion, AI has the potential to revolutionize the video production industry by making complex tasks more accessible and efficient. By integrating AI into existing workflows and leveraging its capabilities, professionals can enhance their work and deliver higher-quality results. However, it’s important to recognize the limitations of AI and understand that human expertise and creativity will always be essential in the filmmaking process.Recognizing a problem is crucial in order to effectively utilize AI to solve the right issues. It’s pointless to have an AI that can fix complex technical problems if you can’t identify the problems and articulate them correctly. Identifying and resolving a problem can be just as challenging as actually fixing the problem itself, and it requires a solid understanding of the task at hand to ask the right questions. While AI can assist in achieving high-quality work, humans will continue to play a vital role in bridging the gap between a client’s vision and the final product.
In conclusion, it’s important to stay vigilant and embrace new workflows without fear. If you sense a wave of change approaching, either strive for excellence to stay ahead or find alternative paths to avoid it. Progress is not linear and will not be evenly distributed, so the key is to keep an open mind.
Despite the inevitable changes, AI will bring revolutionary improvements. By utilizing AI as a tool to assist us, we can make the process of creating great work easier. As standards continue to rise, let’s enjoy the journey to the top.
Other Articles
7 Tips to Safeguard Your Gear from Overheating
Creating a Vintage Aesthetic in the Digital Era
