Meet Frida, the robot who paints AI-driven art in real life

Do you remember the old days? When did AI tools like Dall-E and Stable Diffusion turn your short text prompts into digital art? That’s right 2022…

Meet Frida, an AI-driven robot from Carnegie Mellon University that transforms your prompts into physical paintings, complete with bold brushstrokes in a variety of techniques. Perhaps most amazingly, the bot can change course while painting to mimic the iterative nature of making art.

“He will work with his failures and change his goals,” Peter Schaldenbrand, Ph.D. student in CMU’s School of Computer Science and one of the robot’s creators, said in a video describing the project.

Frida aims to explore the intersection of robots and creativity, says the team, which is presenting its research paper in May at the IEEE International Conference on Robotics and Automation in London. Robots have created art before and even exhibited it. But Frida is designed expressly to collaborate with humans, using the same kind of generative intelligence that powers experimental tools like the AI ​​chatbot ChatGPT.

Frida stands for Framework and Robotics Initiative for Developing Arts, but also shares a name with famous artist Frida Kahlo. But it doesn’t look like Kahlo or any other person. For now, it’s just a robotic arm with a paintbrush attached, a configuration that underscores the team’s insistence that Frida is “a robotic painting system, not an artist.”

“Frida doesn’t generate ideas for communication,” Schaldenbrand said. This is where people come in, communicating goals for Frida with text input. They can also show the bot images in a style they like, or even photos they want to see presented as a picture. Frida suggests appropriate paint colors on the screen, then humans mix them into the robot’s palette.

Frida will probably never reach the fame of its namesake, but some impressive skills set it apart from other artistic robots whose input images generally match their end goal. Making art is a dynamic, ever-evolving process, and after planning its trajectory in a simulated environment, Frida uses machine learning to evaluate and progress in real time. The robot does its planning, as CMU robotics professor James McCann notes in the video, “in a meaning space instead of an outcome space.”

Frida does not value precision like most robots and can, for example, incorporate a “mistake” such as the wrong spot of paint into her final product. Each painting takes hours to complete, and the results are often whimsical and brightly colored.

“There’s a painting of a ballerina frog that I think turned out really well,” Schaldenbrand said in a statement. “It’s really silly and fun, and I think the surprise of what Frida generated based on my input was really fun to see.”

To create their AI datasets, the team fed their models with current news headlines and further trained them on images and text representative of different cultures to avoid American or Western bias.

AI’s role in generating visual art, composing songs, and even writing poetry and film scripts is generating excitement, but it’s also raising ethical and copyright concerns among artists and even lawyers. AI art is not created in a vacuum. It works by absorbing and reconstructing existing human-made art. As machine-generated art improves, will these people—actual graphic designers, illustrators, composers, and photographers—find themselves out of a job?

Some artists I’ve spoken to describe feeling nervous about the complex questions raised by AI art. Others — like Steve Coulson, an avid comic book fan who wrote a comic book series drawn entirely by Midjourney – accept what they see as inevitable change. The comic book artists Coulson has long loved “have an eye for dramatic composition and dynamic narrative that I highly doubt machine learning will be able to match,” Coulson says. “But as a visualization tool for non-artists like me, it’s a hell of a lot of fun.”

Frida’s inventors share a similar point of view.

The armed bot, the research paper says, “is a robotics initiative to foster human creativity rather than replace it by providing intuitive ways for humans to express their ideas using natural language or sample images.”

Editor’s note: CNET uses an AI engine to create some personal finance explanations that are edited and fact-checked by our editors. For more see this post.