Is intelligence all there is?

I am taking a course online on AI Safety. It is run by Lens Academy and it is pretty good. I’m happy to be doing it, in part because we are waiting for the reviews to come in for the book and in the mean time there is only so much editing you can do.

The course is structured with readings and videos taken from online sources, with accompanying questions. There does not seem to be any tests or quizzes, but perhaps that is yet to come. What it does have, which is new to me, is an AI Tutor who you can pose questions to. The tutor sits in a window beside the main reading (or viewing videos) window. There is also a left pane with navigation. The AI tutor seems to be powered by a fairly high end system, perhaps a custom installation of Claude or ChatGPT. (Later I learned that it is Claude Haiku with some special training - basically all AI Safety literature).

This afternoon I was reading an article by Nate Soares (from way back in 2015), titled Four Background Claims. You can read it yourself, here: https://intelligence.org/2015/07/24/four-background-claims/ Soares is the co-author, with Elizer Yudkowski of the book, If Anyone Build It, Everyone Dies. So, he’s a pretty big deal.

In the conversation about the article with the AI Tutor, I got into one of the bug-bears I have with some of the LessWrong crowd. Actually, not just them but many in the AI space. I am sure they are very intelligent people but because they are so smart they tend (in my opinion) to overemphasize the importance of intelligence in everything. Here’s an example:

In Soares essay, he advances four claims. The first one is this: “Humans have a very general ability to solve problems and achieve goals across diverse domains.” He then goes on to say, “We call this ability “intelligence,” or “general intelligence.”

That isn’t even identified as a controversial claim or assertion. But it should be. Think about it for a bit. The “general ability to solve problems and achieve goals” has other potential explanations or factors. What about persistence? Or creativity? Or ambition? Or any number of factors that a reasonable psychologist might list as factors in human success ‘across diverse domains.’

When you attribute all of the success of humans to intelligence, you really box yourself into a corner. And it isn’t just our mental abilities. What about emotion? Caring can be a driver. Or nurturing. The role of mothers and grandmothers, for example. What about biomechanics? Having opposable thumbs is a big deal for achieving goals across diverse domains. I could go on.

Our challenges in the coming years, as we tackle the impending arrival of “actor AI” or even superintelligence will be difficult enough without muddying the water on the topic of intelligence.

The problem goes deeper.

We are not only leaving aside other important aspects of human capability when we focus on intelligence. We also blur or lose focus on things that we’ve known for centuries about the ability of humans to dominate their environment and others: the dynamics of power, and political economy. The role of emotions. Being smart is not enough. You need power. Being smart can help you get power but so can being ruthless. So can lying. So can being born in the right family or marrying the right person or just simply being in the right place at the right time.

In our CASX framework we have tried to distill the things that make a machine (an “AI”) an actor or, to be precise, the things that move an AI down the gradient from tool to actor. While Capability, Autonomy, Scale, and Access are a good starting point and - importantly - they are operationally crisp in a way that “general intelligence” or “superintelligence” is not, I think we might have to consider a more nuanced look in the future. Should we adapt our four to include venality? And what about ruthlessness? No, probably not, but it doesn’t hurt to think about these things.

A more serious question for us: how to bring the role of intelligence back in to the discussion about regulating AI, after you have completed your CASX evaluation. We have a solution for this, I believe, but for now, these are questions for the future.

For now I am glad I had that chat with the AI Tutor at the Lens Academy.