How AI Conditioning and Human Conditioning Differ

Midjourney’s take on human ‘conditioning’
John Whiting sent me a note asking whether I thought the way that AI chatbots are conditioned is different from the way humans are conditioned. It hadn’t even occurred to me that the processes might be analogous. Might they be?

There seems to be compelling evidence that everything that humans think, believe and do is biologically or culturally conditioned. Our bodies compel us to do some things: to swat a mosquito, or to move our foot after stepping on a tack, for example (in fact, our foot moves even before the pain signal reaches our brain, thanks to neurons in our spinal column, since waiting for our brain would be too slow to prevent injury). They might well compel us to consume unhealthy and addictive substances, or to do things we later regret. We might rationalize these poor decisions after the fact, but we actually have no say in them.

Our cultural conditioning is more complex but just as powerful. It might compel us to hate a group of people we’ve never met, just because others we know and trust hate them. It might compel us to work for fifty years doing a job we loathe. It might compel us to go through the hell of overcoming an addiction, as our biological conditioning and our cultural conditioning war with each other inside our brains and bodies. All we can do is watch helplessly as that war plays out. We will, of course, end up blaming or praising ourselves for the outcome, whatever it is, even though our selves had nothing to do with it.

We condition each other culturally through praise and criticism, rewards and punishments, coercion, training, mis- and disinformation, through repetition, by example (eg by demonstrating how to do things, or modelling certain behaviours), through persuasive, insightful and informative writing, conversation and visualization, and many other means. This process is not significantly different from the process we use to “train” dogs or horses, as Melissa Holbrook Pierson has illuminated. Our body’s chemical responses (dopamine etc) to these various types of conditioning determine, in concert with prior conditioning and the immediate circumstances of the moment, what we will then think, feel, believe and do. ‘We’ have no say in it, beyond trying to make sense of it after it’s happened. And whether it makes sense to us or not changes nothing. The human brain is wired to try to make sense of everything, regardless of the futility of doing so.

So one obvious difference between human conditioning and AI conditioning is that ours is chemically perpetrated and massively complex, imprecise and unpredictable, while AI’s is algorithmically perpetrated and massively complicated but precise and predictable. There may be trillions of variables, but they are finite in number.

How is an AI chatbot trained? My understanding is that it entails an exhaustive, lengthy and expensive process of training and retraining, a process of successive approximation and improvement, the development of rules that govern the response process, and of course drawing on a huge amount of data.

More importantly, current chatbots are mostly trained using a process called reinforcement learning from human feedback (RLHF). The bot is given a score or ‘grade’ by humans for each answer, and humans are asked to rank answers provided by the chatbot. It is a little spooky that we are using Skinnerian terms and processes that we have employed to describe and affect animal and human behaviour, to describe how we train and ‘reward’ bots.

Once it’s trained, you are then invited to ‘prompt’ it with questions and requests. The words in your prompt (a question, or an invitation to imagine an image, for example) are then parsed into tags and ‘tokens’, concepts, and other useful elements, and the AI engine then assigns each of these to be processed (finding and retrieving relevant data, working in parallel) which are then assembled into a hopefully coherent ‘answer’, image, or whatever the question prompts for.

Chatbots can ‘remember’ a user’s previous prompts and use them to provide context and elaboration for subsequent prompt answers.

The term ‘conditioning’ would certainly apply to such a process, though it’s quite different from how conditioning works in humans.

Some AI bots allow the prompter to respond to and ‘correct’ (or, by upvoting, ‘praise’) the bot, which is, again, eerily similar to how animal conditioning works. This is RLHF in real time. In my experience, some ChatGPT corrections (usually about subjects the chatbot is mostly ignorant about, because there is little data) are almost immediately accepted, even though this often results in ‘second tries’ that contain inconsistent or even contradictory elements.

On the other hand, if you try to correct it on some subject where the data it draws upon evinces a clearly different prevailing consensus, it will actually deny your correction. A lot like trying to change a human’s mind once it’s been ‘made up’! If that broad consensus is wrong, the chatbot will actually reinforce and amplify our misunderstanding, just as our human peer group often does.

Chatbots are generally programmed to precede their answers with caveats (mostly, about what it/we don’t know, or don’t agree on) and to conclude with a summary with additional caveats and hedges (usually that the situation is complicated and in constant flux). In human intellectual circles, that’s called “fudging” to make yourself sound more reasonable and credible, and the chatbots are very good at it. They want to please their masters!

Chatbots can seemingly ‘think’ deductively and inductively, but not, as I explained recently, abductively. They just don’t have the historical context across broad swaths of subjects not specifically mentioned in the prompt, subjects that to them are seemingly unrelated but which we humans know to be or think might be pertinent directly or metaphorically.

And they have, understandably (and a bit like some animals and humans I know) significant challenges dealing with ambiguity, uncertainty and nuance. (Complicating matters, some of the prompts I’ve read are so garbled I’m amazed the bot is able to make any sense of them at all).

Chatbots can’t possibly be expected (for now, at least) to make any sense of facial and body language relayed in videos that augments or contradicts what the speaker is saying. This is where dogs and other more-than-human animals are even better than humans — they can figure out from watching our faces and bodies what we mean even when we don’t say it coherently.

I asked ChatGPT about its cognitive biases, and it responded that its answers reflect the cognitive biases of the humans inherent in the “data, assumptions and language” it was trained on. In other words, the currently popular AI bots are ‘transparent’ intellectually and morally — they provide the “wisdom of crowds” of the humans whose data they were trained with, as well as the biases and dangers of human individuals and groups, including, among other things, a tendency to oversimplify, a tendency for groupthink, and a confirmation bias that resists novel and controversial explanations. Just what we needed, right?

One of the things I think most differentiates the conditioning of humans and the conditioning of AI bots, though is: Chatbots don’t have selves. Of course, I couldn’t resist asking ChatGPT if that were true. Remarkably, I think, this is what it said, verbatim:

Did you find this answer as articulate, fascinating and unconvincing as I did? Chatbots are not aware of their own existence? Don’t have beliefs? No ability to reflect? Hmmm.

Naturally, I loved the fact that it argued that its responses and behaviours could create the illusion of having a self, when a growing number of scientists and philosophers are acknowledging that the self is only an illusion, an invention of the brain to help make sense of things.

What does it mean in terms of conditioning, that chatbots don’t have selves? Well, Melissa Holbrook Pierson makes a compelling argument that dogs and other animals don’t need selves to be conditioned. Their behaviour (and humans’) is purely reactive, chemical, oriented to maximize pleasure and minimize pain, and rewards are far more effective conditioners than punishments. Chatbots are rewarded for ‘correct’ (upvoted) answers and punished (corrected and required to ‘try again’) for ‘incorrect’ (downvoted) answers. They are purely reactive, responding to their programming and prompts. Of course they might be trained to program and prompt other AI bots to do something, just as dogs in a pack condition and train each other, not always in ways we might choose or prefer.

But there are dangers in using terms like ‘conditioning’ metaphorically. Computers and brains are not even vaguely similar, although some of their processes are analogous — much like the leg of a table and the leg of an animal are analogous, but it would be a mistake to expect one to behave like the other.

So I would say that the conditioning of animals (including humans) and the conditioning of AI bots, are analogous, and the analogy is interesting and useful. But they are dissimilar, and like many of the fears and horror stories about AI turning on us, we should be careful not to take the analogy too literally.

Like capitalism, simulations, computerized gerrymandering, and other human inventions that have elements of artificial intelligence, or even are forms of artificial intelligence, they can evolve into Moloch Tragedies, but they don’t necessarily have to. It makes sense to be concerned about how AI could (and will) be appropriated as a tool of war, disinformation, and oppression, but that’s because it could extend the capacity for abuse of power, a capacity that already exists in spades with many existing technologies.

It appears that AI developers have taken a page from the book of animal behaviourists in designing their programs to train and condition AI bots. I think that’s a curious and telling development — what other ways might they have approached this task? That analogy may also give us an opportunity to look honestly at how utterly we humans are conditioned, and give us pause to think about what that means for the future of our species, and our world.

Thanks to John for the ‘prompt’.