The Japanese knotweed of generative AI is strangling Facebook

The tsunami is drowning the boomers in junk, yet they're loving it (apparently)

Prompt: “complex strands of ivy entangling computer screens which are showing birthday cakes and the Facebook logo, photo-realistic”.

Now sure, nobody likes a smartarse, but I feel the need to say: I told you so. Back in August 2022, I considered the possibilities that were opening up with the arrival of AI image generation tools such as Midjourney, and the fact that people were glomming on to TikTok because its algorithm is designed to keep them entranced. So I pointed out what the key elements existing at the time were:

• Companies whose whole business is built around capturing attention

• AI systems capable of producing limitless amounts of content

• AI systems capable of producing believable-looking pictures of humans (and lots of other things)

• Algorithmic systems which will pick content that humans find compelling

• Humans who like spending time watching content they find compelling

• GAN-generated photos already being used for fake profile pics for marketing or, worse, disinformation and espionage.

Which to me suggested that we would see AI-generated content applied to capture people’s attention, whether they were aware of it or not.

Earlier this week, the excellent new tech website 404 Media¹ published a story with this headline: Facebook’s Algorithm Is Boosting AI Spam That Links to AI-Generated, Ad-Laden Click Farms. This in turn pointed to a preprint by the Stanford Internet Observatory titled How Spammers, Scammers and Creators Leverage AI-Generated Images on Facebook for Audience Growth. It looks at the bizarre phenomenon of Shrimp Jesus (“and, relatedly, Crab Jesus, Watermelon Jesus, Fanta Jesus, and Spaghetti Jesus”)—an AI-generated Jesus whose arms and body are shrimp (or, in the UK, prawns) and points out that Facebook’s algorithm is amplifying stuff like this, just as it did with the fake news written by Macedonian kids during the 2016 US presidential election.

Along the same lines, on Tuesday Chris Alsikkan, who previously came to international notice by posting about the gigantic gap between the Willy Wonka AI advertising and the actual thing, posted an amazing Twitter thread, starting with this:

Don’t worry, we’ll get on to how you can tell those are AI-generated pictures soon

The replies are an absolute carnival. You might think that Alsikkan has shown a talent for spotting AI-generated junk, and you might suspect that this is made up (I don’t think it is), but plenty of other people had examples from Facebook too. This story, and this concern, is not confected.

I’ll deal with how you can tell that the images are AI-generated in a moment, but notice a couple of other things too. The first photo (man with the cake) is posted by an account called “Interesting Planet”, and has the caption “I did it myself, I’m waiting for your feedback 🤗❤️🤗❤️🤗”. The second photo (girl with pizza) is from the same account, and the caption says “This is my first pizza ! 🍕🍕🍕 I will be glad for your marks 👍❤️”

Like colossally bad phishing emails, the captions alone should get your spidey sense tingling. Why would this bloke be saying that rather than something more, well, msaculine? Why is the girl glad for marks, rather than expressing herself like, well, a human would?

The weird thing is that those captions are probably written by a human, or at least programmed by a human—since I suspect that they’re just part of a grab-bag which gets stuck on to the autogenerated pictures.

So, briefly, let’s look at a couple of the AI pictures and see how you can tell they’re not real.

Here’s the pair from above.

• Left:

• T-shirt text isn’t actually text. AIs can’t do spelling (yet?).

• Items behind him are placed in front of cupboards. (They’re strange items, too.)

• If you zoom in, his right eye is looking at you, left eye looking down. Eyes don’t do that.

• Right:

• Her eyes are at different levels.

• The pizza is gigantic yet cut unevenly. Who cuts a pizza like that?

• The chairs behind her have a bench that would hit your knees. Nobody designs kitchens like that.

In both cases, you can’t see their hands—a known weakness, so the generating prompt probably specifies not to include them. Next:

• Left:

• “Grandpal”?

• If he started at age 5 and he’s now 89, does he need to “grow his baking journey”?

• According to the cake, his name is “fulanfer”. (AIs still can’t spell, or make this stuff up.)

• The stripes across the top of his shirt, where it’s open, don’t match—but they do lower down.

• Candles aren’t organised as a human would. None on his right, and one very wonky one?

• Subtle, but: the back of the chair is too far away from him. The chair’s seat would have to be about as deep as the chair is high for the perspective to work.

• Right:

• good grief, where to start? Apart from the Death Star blueberries, that spaceship is made of no material known to humanity. The lack of necessary perspective or humans means that this doesn’t have the same bizarre nature as the others, but it’s still very, very weird.

Now we come to the reactions to these AI-generated pictures, and it has to be said that things aren’t encouraging here either. Here’s yet another example, also from Alsikkan’s thread:

OK, so the photo is obviously AI-generated. (Obvious to you and I; not so much to other Facebook users, judging by the number of shares on it.) The smoothness and regularity of the hair; the peculiar number of fingers on the hand touching the hangar; the discontinuity of the hangar where it goes under the dress. But what about those comments? Those do not, to me, look like humans. The names along are just weird. Sure, I know, people do have weird names sometimes, but I don’t think they have weird names and write EXCELLENT CONGRATULATIONS.

The obvious reason why the bots are there is to boost engagement, so that the posts will be shown to more people (the Algorithm likes engagement!).

So: Facebook is being overrun by bots, both generating the content and responding to the content. The Internet Observatory report (co-authored by Renee DiResta, to whom I spoke extensively for various parts of Social Warming) points out that this isn’t neutral:

We studied 120 Facebook Pages that posted at least 50 AI-generated images each, classifying the Pages into spam, scam, and ‘other creator’ categories. Some were coordinated clusters of Pages run by the same administrators. 

These images collectively received hundreds of millions of engagements. A post including an AI-generated image was one of the 20 most viewed pieces of content on Facebook in Q3 2023 (with 40 million views). 

Spam Pages used clickbait tactics and attempted to direct users to off-platform content farms and low-quality domains. Scam Pages attempted to sell products that do not exist or to get users to divulge personal details; some were posting the AI-generated images on stolen Pages.

As ever, this is all about money—either scamming people directly, or getting them to go to pages stuffed with ads. As the SIO says, it’s exactly the same modus operandi as the Macedonian fake news people, and the concerning point is that you can be pretty sure that once the US election starts to really amp up, the generative AI will be a lot more dramatic. In 2016, it was just fake news with generalised photos. In 2024, it will be fake news with AI-generated fake photos.

And Facebook, it seems, isn’t particularly worried. As Jason Koebler explains in the 404 Media writeup, the spammers tend to put up these images and then instruct readers to click on a link that appears in the first comment, which they’ve written. The reason is straightforward:

Anyone who has used Facebook knows that, for many years, it was possible to post links on Facebook and get a lot of clicks. Facebook has deprioritized link posts in the algorithm, so it is incredibly difficult to get any offsite traffic from Facebook. Image posts still perform well in the algorithm, though. By putting the link to the article in the comments, pinning the comment, and directing people to look in the comments for the link, AI-generated spam pages are able to drive traffic to their spam sites without taking the algorithmic hit that a link post would.

Koebler points out, with graphics, that a Facebook page called “Thoughts” saw an explosion in interactions once it switched to AI-generated content. (The graphics come from Crowdtangle which, hey ho, Facebook is soon going to phase out from public use. Thanks a lot, Facebook.)

The page went from having roughly 250,000 “interactions” on Facebook in November to more than 2 million interactions in December using AI images, and stayed over 2 million interactions in January

The rest of his article shows that Facebook is basically being overrun by this stuff. Shrimp Jesus and his mates are taking it over. And everyone’s ignoring Facebook.

In February, Meta said that it will require labeling of AI-generated images on Facebook and Instagram. Of the thousands and thousands of AI-generated images I have viewed, I have seen exactly one "altered photo" tag, on a sand sculpture of Jesus that has 616,000 likes on the Love God &Love You (Shrimp Jesus) page.

The images are going to get better, and Facebook is being sandblasted by junk. It may be big enough to absorb all this, and the boomer (and older) generation may just noodle there clicking the Like button, and, you hope, not following the links.

However, this isn’t the worst of it. Sure, this stuff is weird and worrying enough.

But just wait until we’re routinely seeing AI-generated video on these posts. If you thought Facebook was an AI-generated wasteland now, well, you ain’t seen nothing yet. Every time you think it can’t get worse… it gets worse.

---

Operational note: I’m going to be off for a couple of weeks while I recover from a minor elbow op.

• You can buy Social Warming in paperback, hardback or ebook via One World Publications, or order it through your friendly local bookstore. Or listen to me read it on Audible.

• You could also sign up for The Overspill, a daily list of links with short extracts and brief commentary on things I find interesting in tech, science, medicine, politics and any other topic that takes my fancy.

1. Subscribe to them. They really deserve it: they’re journalists who were made redundant by Vice and decided to take those lemons and set up their own lemonade stand. And damn good it is.