Do AIs dream of electric sheep embryos?

An Executive Order in the US shows that we shouldn't trust some AI answers. But which ones?

Early human embryos. CC-BY licensed photo by Zeiss Microscopy.

My attitude to AI has, for a long time, been “this sounds great in theory, but let’s see exactly how the practice works out”. In the 1980s, there was tremendous excitement about the prospects for “expert systems”: humans would input their knowledge to systems effectively as a set of IF-THEN rules—for example, to identify an infection in a patient by getting the medic to answer a set of yes/no questions and perhaps entering a little text. They have some success in a number of fields, but according to Wikipedia:

In the 1990s and beyond, the term expert system and the idea of a standalone AI system mostly dropped from the IT lexicon. There are two interpretations of this. One is that "expert systems failed": the IT world moved on because expert systems did not deliver on their over hyped promise

(Though it does say there are alternative explanations: one, that expert systems were rolled into standard tools—which I don’t really think happened—or two, that they cause power struggles inside organisations when the systems escaped the grasp of IT. Personally, I thought they just vanished: they’d been unwieldy, and hard to fill with knowledge (and the knowledgeable humans were a bit wary of filling a computer with their valuable, commercially exploitable wisdom gained over years).

Anyway, the AI tide, having gone out, has now come roaring back, as you may be aware. But now we aren’t filling the AIs with expert knowledge; we’re stuffing it with absolutely everything we can find on the internet, including Reddit, and somehow hoping that this will be enough for accuracy. As has already been seen with the recommendations to use glue as a topping for pizza and eat rocks, this isn’t a good idea.

No matter! People are relying on the new generation of chatbots to answer general knowledge questions, and relying on them to be correct. What’s more, they’re relying on them to answer complicated questions.

Screenshots of Google Gemini responses used as answers to anything should induce an instant eyeroll by now, but people are lazy: the biggest lesson perhaps we should have learnt from the years of social media (so far) is that people won’t even open a new browser tab to check something before saying it. But now that they are being moved to, they’re using the top result—frequently Google Gemini, because Google is the biggest search engine—and relying on that.

There are times where this isn’t helpful. In my many years as a journalist covering technology, science and medicine, it always felt important to actually get things right, and to make sure that what I wrote didn’t somehow get altered to mess up the sense. Covering BSE (aka mad cow disease, which infected 184,000 cattle from 1987 to 2003) and then its human form, variant CJD (which has killed about 180 Britons), required wrapping my head around extremely puzzling science where there were many conflicting suggestions for how the disease was spread (bacterium? virus? It turned out to be neither), and very few experts—nor many internet resources—who could talk about it. This part may sound familiar: despite a detailed inquiry lasting years and testimony from scores of experts, the origin of BSE remains a mystery¹.

BSE and vCJD are prion diseases, caused when a protein in a cell flips from one stable shape to another shape which is harmful, and somehow causes other copies of the same protein to flip in the same way. But does the AI know this?

I’m happy to say that Bing’s AI answer gets this correct. Ditto Google Gemini. Both use reliable scientific sources; for the Google one, it’s from a document updated in May 2024—the same one, in fact, I used in the previous paragraph. (I didn’t know it would use that.)

However.. this week Donald Trump came into office and signed a slew of Executive Orders. One of them relates to sex, though it’s more ponderously titled DEFENDING WOMEN FROM GENDER IDEOLOGY EXTREMISM AND RESTORING BIOLOGICAL TRUTH TO THE FEDERAL GOVERNMENT.

What got people worked up was part of Section 2, “Policy and Definitions”:

(a) “Sex” shall refer to an individual’s immutable biological classification as either male or female. “Sex” is not a synonym for and does not include the concept of “gender identity.”

(b) “Women” or “woman” and “girls” or “girl” shall mean adult and juvenile human females, respectively.

(c) “Men” or “man” and “boys” or “boy” shall mean adult and juvenile human males, respectively.

(d) “Female” means a person belonging, at conception, to the sex that produces the large reproductive cell.

(e) “Male” means a person belonging, at conception, to the sex that produces the small reproductive cell.

We can agree (let’s hope) that “at conception” means the point at which the sperm and ovum fuse to form the zygote. This is very basic biology. At that stage, the zygote has all the DNA it will ever have, and that DNA contains the instructions for every life stage that follows, from blastocyst to embryo to baby to child to adult. That DNA will determine if the adult is male or female².

Logically, therefore, your sex is fixed at conception/fertilisation. And because your sex chromosomes won’t change, neither will your sex. The definitions given there of the two sexes—one that produces large reproductive cells (aka gametes), one that produces small ones—is completely standard in any biology textbook.

And yet… people on social media weren’t having it.

(Amazingly, none of the Community Notes on that tweet has passed muster to become publicly visible—which demonstrates one drawback of the Notes algorithm which insists that “Notes are only approved if people who have disagreed on other topics agree on it”.)

Which brought a riposte from an actual developmental biologist:

But the people claiming the “all embryos begin as female and develop that way for six weeks until if they’re male they change development path” were quick to jump onto Google. And to be fair, for quite a long time—up to about 2000—the prevailing thinking was that human embryos up to six weeks had female characteristics, and it was only at that point with the activation of the SRY gene and related genes (found³ on the Y chromosome) that they became male.

So if you go and ask Bing it will spit back a Copilot result which says this:

Look at the date on that study! 1974—before modern genomics and before the first IVF baby. What’s more, if you scroll down the page past all the garbage to the actual search results, you get this:

Notice how the more recent results confirm what is now understood: embryos are “bipotential” (though they’re already motoring down their specific developmental track towards male or female).

How about Google Gemini?

A little better, but definitely confusing. Also, wrong, even though it’s a reference at the NIH, which is usually a reliable source. “Phenotypically female” implies, well, female. That’s because it’s quoting a 2001 book.

For the correct description, one needs to search the NIH PubMed database more accurately, which turns up a 2020 book called Sexual Differentiation, which begins:

The chromosomal sex of the embryo is established at fertilization. However, 6 weeks elapse in humans before the first signs of sex differentiation are noticed. Sex differentiation involves a series of events whereby the sexually indifferent gonads and genitalia progressively acquire male or female characteristics.

“Sexually indifferent” is a precise enough phrase. No default-female embryos here.

But the scale of the problem is evident. Google and Bing get this wrong. (Shout out to DuckDuckGo whose first result is correct.) They stuff the wrong answer—incredibly outdated in an area of science!—into their AI, and there’s no indication of whether it will ever change. People are misinformed, and they can claim the backing of the two biggest search engines. Although, big although, ChatGPT gets this right:

Not exactly, though it's a common misconception. All human embryos start with the potential to develop into either sex, but they aren’t technically "female" by default. Here's how it works:

In the earliest stages of development, all embryos have a pair of structures called gonadal ridges and two sets of ducts (Müllerian and Wolffian ducts), which can develop into either male or female reproductive organs. The gonads at this stage are "indifferent," meaning they haven’t differentiated into testes or ovaries yet.

This, though, gives us a glimpse of the scale of the problem. Search engines are wrong. ChatGPT is correct. (Grok is correct too. Amazing.) Some of PubMed is wrong. Lots of people are wrong, thought the woman who wrote the EO was quite careful to be correct by not specifying chromosomes⁴, just sex.

I worry that this is going to get worse. It was bad enough when Google was effectively the sole resource and was Googlebombed; now we have multiple sources whose provenance is in effect hidden but trusted too much. And there’s so much investment wrapped up in them that they’re not going to turn them off.

Plus people have a real hatred of having to change their minds about something. If an AI contradicts what they think, I think it’s more likely they’ll go and ask a search engine which will bolster their view, or vice-versa. (I realise you might say that about my attitude here. Except I can point to the scientific book from 2020 which formed my view.) It’s almost like what used to happen with newspapers, when people would gravitate towards one which supported their world view rather than constantly challenging it.

Maybe we just have to wait for the tide to retreat a bit. But the trouble is that most humans remain hungry for information, but not quite hungry enough to search any further than they have to in order to find something that will confirm their priors. We may be hunter-gatherers of information. But we’re quite lazy ones.

---

• You can buy Social Warming in paperback, hardback or ebook via One World Publications, or order it through your friendly local bookstore. Or listen to me read it on Audible.

You could also sign up for The Overspill, a daily list of links with short extracts and brief commentary on things I find interesting in tech, science, medicine, politics and any other topic that takes my fancy.

• Back next week! Or leave a comment here, or in the Substack chat, or Substack Notes, or write it in a letter and put it in a bottle so that The Police write a song about it after it falls through a wormhole and goes back in time.

1. The two principal theories are that it was imported from another country (which only pushes the origin question back one stage) or that it arose spontaneously, possibly through a genetic mutation; genetics can predispose humans to prion diseases such as fatal familial insomnia. Another possibility is that it somehow jumped species from sheep, where it has long been recognised, and called scrapie. However, scientists mostly ruled the latter out. BSE spread because infected (but perhaps not symptomatic) cows were sent to abattoirs, and their remains—especially the spine and brain—were used for bonemeal, which was then fed to more cows. Can you say feedback loop?

2. Please don’t say “but intersex!” It’s an outdated, misleading word: the correct term, adopted by medics and scientists since 2006, is “people with differences of sex development (DSDs)”. Like all of us, they’re all still either male or female, even though they may have an unusual (but not always) have an unusual complement of sex chromosomes.

3. *almost always. But translocation during meiosis (when the normal cell splits to become a gamete) can push the SRY gene and others onto the X chromosome; or the SRY gene can be inactivated. Genetics and embryology aren’t simple.

4. See footnotes 2 and 3