Mike explains how a simple AI video project turns into a machine produced AI historical manifesto, and why working with AI isn't as easy as it seems.

Negotiating with GPT Models

I use the term “negotiate” throughout this piece to discuss the act of prompting the GPT Models to produce the necessary materials to make a video project. These materials include script, footage, music, edits, and voiceover. Even though the generally used term is “prompting,” the sheer amount of back and forth was usually closer to negotiating with (and sometimes begging) the GPT Models to do what I wanted. 

The “State of the (AI) Union”

We exist in a world divided between awe, frustration, and fear of the rise of AI. Despite my best efforts to produce something else, it seems the ChatGPT overlord insisted that the final product be more akin to an AI Manifesto or an AI historical text than that I make anything else “meta.” Despite my best negotiations, if ChatGPT was going to discuss the state of AI, it would happen on its own terms.

Fine, I thought, audibly sighing as I started the tedious process of vetting the quotes that the model had included—none of which were accurate and, of the three, I could not find any mention of two of them on Google, Bing, or Duck Duck Go. The third was attributed to Dr. Max Tegmark of MIT but should have been attributed to Dr. Stephen Hawking. After spending considerable time chasing three quotes, I asked ChatGPT if it could provide me with links to the quotes. At that point, it informed me that it had knowingly misattributed the quotes because they were common knowledge (apparently, the “quotee” most likely said the things anyway?), and its data model was outdated. Inevitably, I had ChatGPT strip the script of the quotes and moved on to Invideo AI (to produce my video) and Runway ML (to make some generated shots).

Invideo and Runway both required negotiations of their own. In particular, Invideo required a lengthy voice recording to build a model of my voice and then insisted on re-writing the script based on its available video clips to produce the actual film. Runway, the only half-decent “text-to-video” generation model I could find that is publicly available, produced questionable results, especially with the human form. OpenAI has a new model called Sora that is extremely, even scarily, accurate. Still, it’s in closed testing and unavailable for this project. As a result, I was left with footage that sometimes looked like it came out of a Rob Zombie film.

But what about the ethics?

ETHICS BE DAMNED! (Just kidding). Interestingly, most ethical considerations pertain to the use of unlicensed training data. This training data is particularly concerning for artists who unknowingly have their footage or artwork used without consent. One perk of Invideo AI is that they only use licensed stock footage—our filmmakers got paid! Admittedly, Runway ML is closer to a typical diffusion model. Unfortunately, they do not state where their training data came from, so it’s likely not ethically sourced.

So? What’d you learn? Are we screwed?

I don’t think we’re screwed. Times are just changing. Some of the technology is quite impressive—especially Sora. However, my “custom” voice model was monotone at best and struggled with words like “era,” so it still has a way to go. Will AI displace jobs? Yes, inevitably, but it will likely become another tool to increase production in its current iteration. We are in a situation today where these models are a giant technological leap forward. Still, we have seen many examples of these leaps displacing even highly educated individuals in the past. Humans are exceedingly adaptable, and our skills will always have value or use somewhere in ways that a machine just can’t. Our society will need to catch up. Ultimately, it is more important that we learn to use and work alongside these tools rather than fight their existence and usage. The industry doesn’t care that it can write your essay for you. They care that you don’t have to spend a half hour carefully crafting an email.

The weirdest thing about this project was having a platform use me as a platform. These models used me for their own promotion in a way that allowed these AI models to tell their history and wax poetic about their future. Even though I know that these models aren’t thinking, how they seemed to capitalize on my assistance instead of the other way around was creepy. Regardless, it was an interesting adventure.