All about AI Art

From machine learning with fathomless depths to hands that clearly aren’t hands – this is a deep-dive into exactly how AI art generators work, who’s suing who, and how this affects the future of art and artists.

If you told me that once I finished university, I’d still be writing essay-length pieces of writing… I’d probably believe you (I genuinely researching stuff!).

And this is a topic that is near and dear to my heart: AI art.

Videos and articles about AI art have been coming up in my social media feeds for a little while now, but I wanted to put my head in the proverbial sand. I didn’t want to know about it. Not interested. The more I thought about it, the more annoyed I got. How could machines doing maths replace flesh and blood human beings making art?

That’s a very simplified view, because as an artist who wants to make a living out of selling their art, I felt like I was watching my career path disintegrate before my eyes. Who would want to buy an expensive piece of art when you could put a few words into a text box and have AI generate something for you?

Like this:

Castle ruins in animal crossing.

Or this:

Glowing fairy flying in mysterious forest.

These images took literal seconds to generate using NightCafe, an AI art generator (and look nothing like the janky images I generated a few months ago when I first started researching this topic). The more I look and think about AI art (if you can call it that – I’ll stop now and save my opinions for last), the more I think how does this work?

And how did we get to this point?

I just want to put in a quick disclaimer here: this is an extremely complex topic, and the more I’ve researched, the more I’ve realised I’m barely scratching the surface of what’s going on. All the information I’ve used to make this post will be linked throughout.

So how does it work?

The process sounds deceptively simple (according to this article):

Step 1: Find an existing dataset.

Step 2: Train the machine learning algorithm on said dataset.

Step 3: The algorithm learns how to produce images.

Step 4: The algorithm continues to get better at producing said images.

Easy, right?

Right.

That’s the surface level of how all this works. But look below the surface, that’s when you start to see the potential issues. Let’s look at those steps again.

Step 1: Where is the data coming from?

The images are gathered in an automatic process called “scraping,” which sounds kind of dodgy, but is technically only illegal when non-publicly available information is scraped (like people’s personal details). The Wayback Machine is a good example of internet scraping, using the information gathered to archive webpages.

Stability AI’s art generator, Stable Diffusion, is one of the only AI art generators that I could find solid information about. Unfortunately the excellent article I read on the Lawfare blog is no longer available (I can’t find any trace of the article, really), but there is a Reddit thread where users discuss the main points of the article.

The reason I bring up Stable Diffusion is that this AI art generator is trained using the LAION-5b dataset (LAION stands for Large-scale Artificial Intelligence Open Network). This is a “large-scale multi-modal dataset” that contains references to over 5.85 billion images that have been scraped from the web.

First of all, over 5.85 billion images. That’s a lot of images.

Second of all, notice how I said references to the images? That’s because the LAION-5b dataset catalogues information about the image (like the URL of the website where it was found, the image dimensions and file size, the subject matter of the image, stuff like that), then discards the original image. By doing this, no copyright laws are violated (technically – but put a pin in this, because it comes up later when artists start suing companies).

And third: when you scrape pictures from the internet, you get everything. You get the good stuff, like pictures of puppies and kittens, classic works of art, beautiful natural landscapes, the list goes on. But you also get the bad stuff. I’m talking disturbing dark web stuff, the kind stuff that gives Facebook content moderators nightmares. There’s even a content warning disclaimer on the LAION website, here’s a few sentences:

“Be aware that this large-scale dataset is uncurated. Keep in mind that the uncurated nature of the dataset means that collected links may lead to strongly discomforting and disturbing content for a human viewer.”

Lovely.

Step 2: Machines can learn?

AI art generators use a type of machine learning called “deep neural networks” to be able to process all of the data required to turn a text prompt into art.

To my understanding, a deep neural network is a machine learning model that has more than one layer of hidden decision-making to turn the input into the desired output. For example, using an AI art generator, the user puts in the input (which is a text prompt, maybe something like, an ocean made of Pringles), which then goes through several hidden iterations, before arriving at the final output (which is an image of an ocean made of Pringles, duh!).

According to this article, using deep neural networks increases the accuracy of the output, but reduces the explainability of how it arrived at the result. Meaning, scientists literally have no idea what’s going on in those hidden layers.

Step 3: It’s actually image into text, into image again?

So AI is using the images scraped from the web, which have then been put into the LAION-5b database. But I said earlier that it’s only references of these scraped images that are being kept and used.

Which means what, exactly?

According to the Lawfare article (the good one that I can’t find anymore, the one with the Reddit thread), this reference information is called CLIP data (which stands for Contrastive Language-Image Pre-training). It’s essentially associating certain words with pictures. The example used in the article was an image of a banana. It might have the tags banana, yellow, and fruit attached to it for the AI art generator to process. But it would also have more abstract tags, like monkey, skin, joke, or eat. Stuff that’s indirectly connected to the concept of bananas.

AI uses what’s called a diffusion model to turn this CLIP data into an image. Diffusion models are an essential part of how AI art generators work; the learning model is trained to remove “noise” (like random pixels) from images, filling in the gaps until it becomes so good that it can do the process in reverse – make an image from something that is one hundred percent noise.

This is where everything else comes into play. AI takes a text prompt and uses it to grab the relevant CLIP data attached to images. Now that AI has been trained using this diffusion model, it can start with a completely “noisy” image, and can make something out of nothing.

If I had to write a very simple maths equation explaining AI art generators, it would go something like this:

Initial text prompt + (scraped images + LAION-5b database + CLIP data) + (deep neural network + diffusion model) = AI art

But this is where AI art generators start to go wrong. It’s through this CLIP data and the labelling of images that AI art first learns and then perpetuates human biases. The source data has been taken from the internet, which was made by humans in the first place! (Remember earlier, scraping the internet for images harvests everything). Journalist and author Tracey Spicer worked with visual designer Meng Koach to create a cover for Spicer’s upcoming book, Man-Made: How the bias of the past is being built into the future, using an AI art generator (the irony, right?).

The front cover of Tracey Spicer’s book, *Man-Made: How the bias of the past is being built into the future*.

Spicer wrote about how she and Koach, through the process of “creating” the book cover, began to discover trends in the images that would be generated. She writes, “if you ask for images of a CEO, it’s generally an older white male. Nurses? Almost all female. And if you don’t specify skin colour, the bots default to white people.”

If you’re not specific with the wording of your text prompts, gender, age and racial bias is prolific. But the existence of this bias has been apparent since before AI art generators even existed (just look into ImageNet Roulette).

Step 4: How far can AI art go?

Despite all the hype, generative learning models have been used in other fields like statistics for much longer than the average AI art user realises. In 2014, computer scientist and engineer Ian Goodfellow (who was just a university student at the time), started making GANs (generative adversarial models), which were basically two neural networks pitted against each other, each network pushing the other to get better at generating content.

2016 marked the early beginnings of GANs being used to generate images from text prompts, although it was very clear that this early “art” was computer generated, and definitely not aesthetically pleasing. In 2017, a project called CycleGAN showed that these models could modify parts of images; then in 2019, software company Nvidia proved that they could generate photorealistic human faces using GANs (although they still don’t look quite right).

It was in January 2021 that OpenAI announced DALL-E, one of the most well-known AI art generators (we’re now up to the new and improved DALLE-2, which was released in 2022). And not long after that, other AI art generators like Muse AI, Deep Dream Generator, Craiyon, and many others began to pop up all over the internet in late 2021 and early 2022.

Some people say that AI art generation is pushing the boundaries of art, and “creating new forms of expression that are impossible with traditional techniques” while others say that AI art generators are devaluing the skills of humans artists, and in a world where art can be “created” with the click of a mouse, it will lead to “a steady decline into the monoculture, where everything looks and feels the same.”

When will art created by AI become indistinguishable from art created by humans?

Well, that’s already happened.

AI art is all around us

Yes, I repeat, we have already mistaken AI art for human art!

(How it hurts me to type those words.)

Look at this photo. Beautiful, right?

Photo entered by Jan Van Eyck (later revealed to be a fake name).

Earlier this year, it won a competition run by camera and photography company digiDirect. Except it’s not actually a photo. It’s an AI generated image; a publicity stunt by Australian company Absolutely AI.

Go on, scroll back up. Have another look. You can’t even tell that it was made by AI. Or at least the judges of this competition couldn’t (they thought it was drone photography).

There is even AI generated art hanging in art museums.

Earlier this year while Johannes Vermeer’s Girl with a Pearl Earring was loaned from the Mauritshuis to the Rijksmuseum in the Netherlands, an artwork made using AI generator Midjourney was hung in its place.

*A Girl With Glowing Earrings*, made by Julian van Dieken using Midjourney.

Late last year, Jason Allen won first place in Colorado State Fair’s fine arts competition with his piece Théâtre D’opéra Spatial.

*Théâtre D’opéra Spatial*, made by Jason Allen using Midjourney.

It was made using AI art generator Midjourney. In one article, Allen said that spent approximately eighty hours curating nine hundred iterations of works using different prompts, before deciding on the final three works that he entered into the Colorado State Fair (it was one of these that won first prize, beating the other twenty-one entrants in the “digitally manipulated photography” category).

Allen says that he doesn’t consider himself an artist (he actually runs a company that makes tabletop games).

But it’s not about that. Yes, he put time and effort into curating the images that AI spat out. But did he actually create anything? You could argue that he created the word prompt used, but they’re just words, not images. Not art.

Allen would like to say different. “The AI is a tool like a paintbrush is a tool, and there is a creative force and mind behind it. There is an imagination and author behind the prompts,” he stated.

(It’s further quotes from Allen that make my skin crawl. His reply to the backlash he received online? “This isn't going to stop. Art is dead, dude. It's over. A.I. won. Humans lost.”)

Despite this, Allen has failed to claim copyright for his work, with the Review Board of the United States Copyright Office declaring it has a substantial amount of content generated by artificial intelligence.

And many other users of AI generators are experiencing the same problem.

Stephen Thaler’s copyright claim for his AI generated image A Recent Entrance to Paradise has been rejected multiple times. The United States Copyright Office deemed A Recent Entrance to Paradise ineligible for copyright protection, as Thaler did not contribute any form of authorship to the creation of the work.

*A Recent Entrant to Paradise*, made by Stephen Thaler’s *Creativity Machine*.

Similarly, Kris Kashtanova’s graphic novel Zarya of the Dawn has only been granted partial copyright by the United States Copyright Office, as Kashtanova wrote the words themselves but used Midjourney to generate the images.

Front cover of *Zarya of the Dawn*, made by Kris Kashtanova using Midjourney.

But what do the people think?

Depends on what corner of the internet you inhabit.

The defenders of AI art say that the rapid progression of AI technology is “prompting a paradigm shift” and has the “potential to open the gates for new perceptions of image-making” just like photography did when it was first invented. And similar to inception of photography, AI generated art is also creating a serious uproar in the art community.

Why wouldn’t it? Some artistic establishments have a notoriously “deep-rooted dislike of things that challenge the status quo” – new art movements were initially challenged, but eventually the ideas stuck around. Is this what’s happening with AI art now?

One AI art generator, NightCafe, claims it’s on a “mission to democratise art creation.” The about page states:

“Creating art is satisfying… it makes you feel better. But most methods of art creation require skill. They must be learned and practiced, and without the skill, you don’t get to experience that satisfaction.”

Art doesn’t need to be democratised. It’s already available to everyone! You don’t need to be extremely skilled to gain satisfaction from making art. And being able to create “art” with the click of a mouse devalues the skills human artists worked so hard to gain.

Some argue that machines are just drawing inspiration from observing art around them, like any other artist would – machines are just making the resulting observational art faster. But the “initial thrill at seeing an image appear” will only last so long. Sooner or later, everything will look bland and homogeneous.

The #notoaiart protest

ArtStation is a website used by millions of artists to showcase their portfolios to their peers, write blogs and search for job opportunities – and in December 2022, it became the site of the #notoaiart protest.

The first no to AI generated images post was on Instagram by illustrator Alexander Nanitchkov, who blamed ArtStation for allowing artists’ works from being scraped, and for allowing AI art uploaded to ArtStation itself. The protest began to gain traction as other artists began to post, and soon ArtStation’s main explore page was spammed with the iconic no to AI art image.

ArtStation began to remove the images, claiming they violated the Terms of Service. (Whereas corporate giants like Shutterstock and Getty Images have taken steps to ban AI generated images from their sites altogether). ArtStation also released a new FAQ page, defending its choice to have AI art remain on the site, stating “ArtStation’s content guidelines do not prohibit the use of AI in the process of artwork being posted… the works on your portfolio should be work that you created and we encourage you to be transparent in the process.”

Illustrator Nicholas Kole called this response “inadequate and evasive”, stating that AI art generators only create “meaningless, regurgitative cardboard cutouts that remind us of real art.”

(And quite frankly, I agree).

One of the major issues that’s arising (and will continue to arise, until it’s deal with) is copyright.

Who can claim copyright on an AI generated image? The person who input the text prompt? The company that created the AI? Or the artists whose images were used in the datasets? It seems like the technology is progressing faster than the legal system can keep up, and artists are chomping at the bit to know what laws are going to be put in place in the future to protect their life’s work.

Where there’s smoke, there’s fire: the lawsuits

At the time of writing this blog post, there are two main ongoing lawsuits.

January 13, 2023 – Sarah Andersen, Kelly McKernan, and Karla Ortiz sue Midjourney Inc, DeviantArt Inc, and Stability AI Ltd

The suit was filed in the Northern District of California on behalf of the three illustrators, by Matthew Butterick from Joseph Saveri Law Firm. The suit claims that these companies have used the illustrators’ works in their AI training datasets “without consent or compensation.” The AI companies assert that they are covered by Fair Use insisting that no artist can claim a certain artistic look or style; whereas the illustrators’ suit claims that AI generated artworks do not change enough elements of the existing images to be covered by Fair Use at all (essentially the AI stuff looks the same as the originals – and now the original illustrators’ works are being undervalued amongst the flood of AI generated content).

January 17, 2023 – Getty Images sue Stability AI

A suit filed in the District Court in Delaware from media company Getty Images followed not long after, alleging that Stability AI “unlawfully copied and processed millions of images protected by copyright.”

An independent study found that a large portion of the images used in the LAION-5b dataset are from the Getty Images site – at least 12 million of them, anyway.

(The Getty Images watermark is even showing up in AI generated content. Like, come on.)

stable diffusion accidentally generates getty images watermark.jpg

The CEO of Getty Images has said the company isn’t interested in monetary reparations – the point of the suit is to create new laws surrounding copyright and AI that are favourable for the individuals and companies whose images are being used in the AI training datasets. Getty Images has disclosed that the company is in collaboration with “ethical” visual generative AI company BRIA allowing its images to be used in training BRIA’s AI model.

But once again, Stability AI is hoping to claim Fair Use.

Remember earlier in the blog post, how I said that the AI doesn’t keep the original images, but references of the image?

That’s what they’re going with.

(Never mind that no one consented to having their content scraped from the internet in the first place.)

But this isn’t the first time

Yes, this has happened before!

Like in November 2022, when Microsoft (and its subsidiaries GitHub and Open AI) had a lawsuit filed against them by Matthew Butterick (yes, the same lawyer as before) on behalf GitHub users. Allegedly, Copilot (which is GitHub’s “AI-powered coding assistant”) had been found to be using long copyrighted sections of code without giving credit to the original creators.

Or again, in February 2023, when a group of Illinois residents filed a class-action lawsuit against Prisma Labs, the company responsible for creating Lensa AI. This AI creates custom illustrated portraits of users – and in the process, illegally collected users’ facial geometry data through Lensa, and violated Illinois’s 2008 Biometric Information and Privacy Act.

(Plus, mangled artists’ signatures are showing up in these portraits as well.)

So we can see from past experiences that people don’t like having their data / art / content / information / insert-thing-here collected and used without their permission.

(Even authors like Jodi Picoult and George R.R. Martin are now suing Open AI, because ChatGPT is using their books as training material without their permission.)

At the time of writing this blog post, both the Andersen, McKernan, and Ortiz suit and the Getty Images suit are still ongoing. It gives me hope that as of August 18 2023, the U.S. Copyright Office declared that art created AI is not eligible for copyright protection. The judge who delivered the ruling stated that copyright “protects only works of human creation” and not works “generated by new forms of technology operating absent any guiding human hand.”

That’s a big relief to me, anyway.

(If you want a more in-depth explanation of what’s going on and the potential outcome of the suits, I recommend this fantastic YouTube video by LegalEagle.)

What does the future hold?

AI learning models (and AI art generators) will keep getting smarter. There’s no doubt about that, it’s practically inevitable. And hopefully copyright laws will match the growth of technology, providing protective rights for those who are having their art, their content, used without permission by big corporations who have no interest in compensating creators.

And if you’ve made to the end of this blog post (congratulations!), after all this talk, you might think I’m against AI art.

I am… but I’m also not.

AI is a useful tool. It’s just a tool, used for creating content. Because, let’s face it, it’s everywhere and it’s not going away. People love it, the convenience of it, how it elevates their skills to the next level and helps them create more, faster and better.

But it shouldn’t spell the end of the human-made art (because where else would the training data come from?!). I want to take aside every person who has said making art is now pointless because AI can do it better and give them a good, hard shake (looking at you, Jason Allen). Why would you ever encourage people to stop making art? Who are you to deny someone the satisfaction and gratification of making something?

Anyway.

Despite this freaky, apocalyptic-sounding YouTube video posted by Open AI talking about how it’s imperative that AI is “aligned with human intentions and human values” (or else), despite this humanoid robot named Ai-Da who can discuss and rationalise the art she makes – people aren’t going down without a fight.

Ai-Da the robot posing next to one of her artworks.

After consulting over one thousand artists, computer scientists at the University of Chicago have developed an anti-plagiarism tool called Glaze which “cloaks” the “style features” of an artwork – changing the data of the images just enough to fool AI art generators. When AI uses cloaked images to mimic an artist’s style, the generated images are “much less successful.”

And best of all, Glaze is free to download.

I found this image on this website, captioned: *Original art by Karla Ortiz (left), plagiarized art created by an AI model (center), and the AI-generated image after cloaking the original art (right).* This is how well Glaze works!

Plus, AI isn’t totally omnipotent yet! It’s still bad at stuff, like making hands (it literally can’t figure out how hands work). People’s brains are hard to fool on this one thing – when you see a hand that doesn’t look quite like a hand… you know it’s not a hand.

An AI generated image I made using NightCafe. The prompt was: *a pair of hands with red painted nails holding a pink flower in one hand and a gold coin in the other hand*. The end result is not quite what I had in mind…

I can’t remeber where I found this meme, but you’ve got to admit it’s a good one.

Final thoughts?

At the end of the day (fortunately or unfortunately), AI art is here to stay. It will find its place in the world, but it should not be at the expense of human artists.

So I’ll leave you with this YouTube video I found at the start of my research. It’s only a few minutes long. I watched a lot of videos in the process of writing this blog post, but this one is my favourite.

Keep up with my art journey!