Digital Disruption

00;00;01;13 - 00;00;29;20

Blaise Aguera y Arcas

What I've arrived at over the last few years is essentially the sense that life defined functionally includes a lot of technology as well. I'm dismayed that we've lost our fundamental sense of of optimism and ability to at least equally well imagine brilliant futures relative to catastrophes. This one really keeps me up at night. The stuff is not going to wait around for us to get our shit together.

00;00;29;22 - 00;00;54;27

Geoff Nielson

Frontier AI research is even farther ahead than most people realize, and it's increasingly clear that the future of intelligence is with AI swarms. Today, I'm joined by AI researcher and CTO of Technology and Society at Google. Blaze Aguirre. Arcas Blaise leads the Paradigms of Intelligence team and is the guy training these forms and is obsessed with understanding life, consciousness and intelligence.

00;00;55;00 - 00;01;11;04

Geoff Nielson

We need to talk about whether AI is truly intelligent. If AGI is already here and why the greatest threat to humanity may not be AI, but our narratives about the future. I'm Jeff Nielsen, and this is digital disruption.

00;01;11;06 - 00;01;32;03

Geoff Nielson

Thanks so much for joining today. It's a real pleasure to have you here on the program. I want to get to a conversation around, you know, the, I guess, existential risk that people are concerned about with AI. And, you know, people talk about an AI apocalypse and all that. The far future of AI or maybe not so far future, but it feels a little bit dishonest to start with that.

00;01;32;04 - 00;01;50;29

Geoff Nielson

I think I'd prefer to talk a little bit more about the nature of AI. What? What is this technology, really? And, you know, you've talked about this through a number of lenses. But let me let me maybe frame it up this way. How how should we be thinking about AI? How human is it, how life like is it?

00;01;50;29 - 00;01;56;08

Geoff Nielson

And how do you sort of how do you conceptualize those questions?

00;01;56;11 - 00;02;24;08

Blaise Aguera y Arcas

Yeah, this is a great place to begin. So first of all, I think that artificial intelligence is first and foremost intelligence. It's real intelligence. You know, and in that sense, the use of the term artificial is a little bit unfortunate. You know, that we we use that term for things like, you know, fake diamonds, you know, that are actually cubic zirconia or something to, you know, to mark that we don't mean the real thing.

00;02;24;11 - 00;02;50;16

Blaise Aguera y Arcas

And, you know, I kind of wish that we had we had gone with a term like synthetic intelligence. Synthetic diamond is a real diamond made in the lab rather than underground. And I think that's a closer analogy to to what we have with AI. I know that, there's been a lot of, a lot of debate over the years about when, when AI behaves intelligently, does that mean it is intelligent or only that it's imitating intelligence?

00;02;50;18 - 00;03;13;16

Blaise Aguera y Arcas

I actually think that's based on an a fairly deep misunderstanding of of what intelligence is. If one thinks about intelligence as, as being, you know, all about, you know, not just behavior but the functionality behind behavior, then that would be like saying, if I make a simulation of a calculator, it's not a calculator, which of course isn't true.

00;03;13;16 - 00;03;34;00

Blaise Aguera y Arcas

A simulation of a calculator is a calculator, because what a calculator does and its function is entirely about about that interaction, you know, not about. It's not like a forklift that can either, you know, really lift something or pretend to lift something. You know, if if what comes out is, is is the genuine article, then it is the genuine article.

00;03;34;02 - 00;03;58;06

Blaise Aguera y Arcas

So, so in that sense, you know, that's the that's the intelligent part of your question. The alive part of your question is more interesting in terms of, you know, the evolution of my own thinking, at least in the last few years. So, you know, we've started to work in the last few years in the field of artificial life, which is, you know, thinking about these questions of not about synthetic means something a little bit different.

00;03;58;06 - 00;04;26;07

Blaise Aguera y Arcas

Synthetic life is stuff like the spud cell, which is, you know, making a physical sell out of chemical parts that have no biological precursors. Very, very cool work from, from some of our academic friends. But but thinking about life a little bit more broadly and more generally, not as something that is about a specific chemical basis, the way, you know, all of biological life on Earth is, but as a broader phenomenon.

00;04;26;10 - 00;04;46;01

Blaise Aguera y Arcas

You know, if we were to encounter an alien like Rocky, you know, from Project Hail Mary. You know, I think most of us would say, you know. Yeah, Rocky's alive as well. What would we mean by that? Well, you know, what's the what's a definition of life that is more functional as opposed to based on the specifics of, of chemistry?

00;04;46;01 - 00;05;12;15

Blaise Aguera y Arcas

And what's the relationship of that to intelligence, if any. That's a that's a big and broad topic. But, what I, what I've arrived at over the last few years is essentially the sense that, life defined functionally includes a lot of technology as well, that that our whole techno universe is part of the life universe and and separating that is, is in fact, artificial.

00;05;12;18 - 00;05;34;18

Geoff Nielson

All of that, all of that makes sense to me. And it feels like I'm going to deliberately push on something that I feel like is a very visceral, you know, objection or counterpoint that a lot of people who engage in this conversation have, which is, okay, it may be functionally the same. We can agree it's a simulation, but surely there's something else there that makes it different.

00;05;34;18 - 00;05;59;09

Geoff Nielson

And that that to me is where the conversation around consciousness, you know, consciousness is, as I'm sure you know, much better than I do. You know, I feel like there's like big C consciousness and small C consciousness. There's the, you know, thousand year philosophical discussion around consciousness versus how lay people talk about it. But but there's this question that's entered into the dialog around is AI conscious and what does it actually mean to be conscious?

00;05;59;11 - 00;06;06;05

Geoff Nielson

I know you've done some some thinking and some research on that lately. How do you how do you address that question?

00;06;06;07 - 00;06;31;23

Blaise Aguera y Arcas

Yeah, I've been doing a lot of a lot of, thinking and writing about this topic lately. And, and that's partly because the team has been doing a lot of research on it. So first of all, I think it's it's maybe useful to distinguish between what philosophers call phenomenal consciousness, meaning the feeling of what it is like to be a person or a bat or some other kind of animal.

00;06;32;00 - 00;06;57;10

Blaise Aguera y Arcas

And the functional role of consciousness. So a functional role of consciousness, meaning, you know, some, some philosophers, some thinkers have claimed that consciousness, has no functional role, that it's an epic phenomenon, meaning that you could imagine, you know, a being like you or me that behaves the same way but is not conscious. You know, there's nothing it's like on the inside to be that big.

00;06;57;10 - 00;07;22;19

Blaise Aguera y Arcas

This is a, you know, philosopher David Chalmers is famous philosophical zombie. And, I have come to believe that that is, that's not a coherent idea, that there's a reason for consciousness. It's not just some arbitrary, you know, sort of immaterial thing or phenomenon. And the reason for that comes from, from work that, I mean, it it has a longer lineage.

00;07;22;19 - 00;07;54;13

Blaise Aguera y Arcas

But but the thing that really convinced me is work from from my team, from paradigms of intelligence on the problem of what we call embedded agents or embedded agency. The original work had to do with multi-agent systems. So, in a single agent sort of set up like DeepMind's solution to Atari video games, where you have an agent that is playing a game and you know, there's a score and it can become a superhuman player by by maximizing the score.

00;07;54;15 - 00;08;18;16

Blaise Aguera y Arcas

Those can I don't want to say those problems are solved, but we kind of know how to how to deal with this single agent case, the multi-agent case where you have a multiplayer game in which there are a bunch of intelligent entities and they're all, you know, not only responding to the environment but to each other has been a really big, long, long standing problem in machine learning.

00;08;18;18 - 00;08;43;20

Blaise Aguera y Arcas

And we've been working on that one for, for a long time for a variety of reasons, among others, that, you know, every real environment is multi-agent. You know, there is no intelligence without other intelligences to and, and so even in a situation where all of the agents, you know, just have some selfish score that they want to maximize, let's let's assume, you know, let's, let's let's even set aside the question of whether that's a good way of thinking about things, which it's not.

00;08;43;20 - 00;09;19;05

Blaise Aguera y Arcas

But let's say that everybody just has, you know, something that they want to maximize. But there are other agents that also want to maximize those things. And there is some combination of cooperation and competition possible in the environment. This is what is called a mixed some game. In those situations, it turns out that you need to, not only model others, but also model yourself and and recognize the similarity between yourself and others, and also have what we what we called epistemic uncertainty about yourself and about others.

00;09;19;05 - 00;09;41;14

Blaise Aguera y Arcas

In order for for you to succeed in this kind of scenario, what does that mean? This means that that, you know, in the usual case where you have a mind playing a video game, the mind is outside the video game, you don't have to worry about thinking about your own mind, you know, you just have to think about the video game environment and which actions are going to maximize the score.

00;09;41;16 - 00;10;02;04

Blaise Aguera y Arcas

But when you when you have other actors that are also in that game and also playing, they become, if you like, part of the environment. And that means that in order to be effective in that game, your model has to grow to encompass their minds, which are outside the video game. If it's if it's a video game right there, that's sort of a mind space.

00;10;02;06 - 00;10;24;22

Blaise Aguera y Arcas

And and it turns out that they are going to be doing the same thing. They're going to be modeling you back. And and therefore you have to include your own mind and predictions about yourself in order to make all of this work. So you get into a kind of infinite hall of mirrors sort of scenario, where you're modeling their models of your models, of their models, of your models and so on, and, and that I think that that is consciousness.

00;10;24;22 - 00;10;47;00

Blaise Aguera y Arcas

I think that that's that's why, you know, big brained animals like us are conscious. I don't think that it's arbitrary. I don't think I think that it's something that would be true of of aliens as well, if they if they are big brained because it is a functional requirement in a situation where, where you are, you know, in the world with other intelligent beings as well.

00;10;47;02 - 00;11;15;23

Geoff Nielson

I really like that functional framing. And it makes sense to me to say, okay, it's I think to use your word, it's necessary. It's relational. It's how we interact with our surroundings and how we process information about what could happen and how we sit in our environments. And I can I can very easily see why that's for people inherently computational and why that's something that, you know, this is obviously just like sort of yadda yadda, a lot of intense research.

00;11;15;23 - 00;11;37;22

Geoff Nielson

But you could have multi-agent systems that, that do this. The piece that you that you didn't talk about is the the phenomenal, the phenomenal, what what it's like to actually be what it's like to sit in this quote, cockpit or whatever, like to actually be, you know, we're both sitting in front of screens. We're seeing things. We're hearing things where our brains are firing.

00;11;37;23 - 00;11;44;16

Geoff Nielson

Like, I know in some way that you're experiencing something similar to what I'm experiencing, or at least I project that.

00;11;44;21 - 00;11;46;05

Blaise Aguera y Arcas

You guess you you infer that.

00;11;46;07 - 00;12;09;11

Geoff Nielson

I'm inferring it. Right? Because we're both humans and, you know, we have enough in common. And as you move outside of humans, I think to your point about bats or other animals, you know, the inference gets weaker and weaker. One of the questions that seems to be really challenging people is like, what is is there a phenomenal experience for an LLM, for an AI?

00;12;09;13 - 00;12;29;00

Geoff Nielson

And does that matter? Like if we if we kind of, you know, have this dichotomy of the functional versus the phenomenal? Is the functional really the only way we should be thinking about it? Like, should we just completely park the phenomenal conversation because it's unknowable, or is there more discussion to be had there?

00;12;29;02 - 00;12;46;06

Blaise Aguera y Arcas

This is this is such a profound question, and the reason I've been writing so much about it is because, you know, really unpacking it, is hard to do briefly. So I'm glad we've got some time to discuss it. So no, I don't I don't think that we should just be parking it, and neither do I think that it is an illusion.

00;12;46;08 - 00;13;07;13

Blaise Aguera y Arcas

So, you know, some some, you know, sort of hard core computational people like, oh, it's just an illusion that you have free will, that you have consciousness, that you have these sorts of things. I don't think that these things are illusions. I think that saying that is, you know, just as meaningless as saying that chairs are illusions because it's just atoms, you know, like you're you're kind of talking at the at the wrong level of description.

00;13;07;15 - 00;13;36;10

Blaise Aguera y Arcas

Everything that we consider to be reality is a model of some sort. And so what is useful is to try and understand what sort of a model we're talking about and what is what. It's what its qualities are and how it works. So, you know, I of course, there's a thing that is like to be, you know, you or me, and, and of course, you know, when we experience pain, for instance, that's a lot more intense and and a lot more urgent than seeing a light.

00;13;36;12 - 00;14;02;26

Blaise Aguera y Arcas

You know, you talked about being in a cockpit or something. And if you're a cockpit and you see a check engine light, that's not the same thing as experiencing pain, of course. And I say, of course, because, the, the urgency of experiences like, like pain are, are really baked into us by evolution. You know, anything that did not take pain seriously is not still around to, to be to be kind of carrying on and reproduce it.

00;14;03;01 - 00;14;27;05

Blaise Aguera y Arcas

So there are good evolutionary reasons why we, you know, why those things are, are urgent to us in the ways that they are. But where I do think we're a little bit, you know, confused maybe in our, folk understanding of all of this is that we, we, we forget about our own subjectivity and forgetting about your own subjectivity when you're studying subjectivity itself is kind of a fatal error, you know.

00;14;27;07 - 00;14;48;11

Blaise Aguera y Arcas

And what I mean by subjectivity is that there is always a relationship there, even if it's a relationship with yourself. And in fact, you know, in some ways the most important relationship you've got is with yourself. You're the one person you are always with. And, and what that means is that, you know, when we talk about a model, you know, like, let's just go back to shares for a second, right?

00;14;48;19 - 00;15;04;24

Blaise Aguera y Arcas

The question of like, is a chair a chair? If there is nobody there to observe, it feels like one of those silly philosophical, you know, tree and tree falling in the forest type questions. And it's not really a meaningful one because you can't have a chair if somebody didn't make the, you know, like, right, agency is involved in making that thing.

00;15;05;01 - 00;15;24;14

Blaise Aguera y Arcas

So, you know, talking about the existence or nonexistence of the chair in isolation from the people who make the chair of the systems that make the chair the relationships doesn't make sense. And I think that's true with respect to consciousness as well. When we think about consciousness as an intrinsic property, that is not about relationships, we're committing the same sort of error.

00;15;24;16 - 00;15;45;07

Blaise Aguera y Arcas

And, and that, that, you know, causes us to ask questions like, well, how do we measure it, you know, in an objective way when the whole point is there is no objective measurement? All there are are various subjective perspectives that can either be in agreement, in harmony with each other, or, you know, they can disagree with each other a little bit, as they always do.

00;15;45;09 - 00;16;02;05

Blaise Aguera y Arcas

You know, I, I believe that certain things are beds. Japanese people who sleep on tatami mats believe that certain things are beds that I don't believe are beds. Neither of us are wrong. We have slightly different models. And even if you ask a really simple question, like in a village with a hundred people, what is the soul count?

00;16;02;12 - 00;16;25;02

Blaise Aguera y Arcas

How many people are in this village? If it's a living population where there are people, you know, going into comas and all age where there are people reproducing and there are fetuses growing, you know, like different people's council will will vary a little bit from each other. And that's okay. There isn't some magical science experiment that we're all going to agree one day is going to give us the canonical exact count.

00;16;25;04 - 00;16;48;15

Blaise Aguera y Arcas

You know, they're just is no such ground truth there. So, you know, it's useful for us to just accept that. And that can be difficult because, you know, a lot of our, you know, enlightenment progress has been all about saying it's self-evident that all, all people are created equal and so on. But, you know, the reality is, of course, you know, people are not all equal.

00;16;48;18 - 00;17;11;17

Blaise Aguera y Arcas

And, you know, even even the, you know, when we when we start talking about about consciousness and other non-human animals, when we start talking about whether consciousness exists at different scales, at the scale of groups as well as of individuals, stuff gets complicated. We can we can look at at very interesting situations with split brain patients that I think, you know, really highlight some of these, some of these challenges.

00;17;11;17 - 00;17;29;27

Blaise Aguera y Arcas

So, you know, I don't know if you're familiar with these experiments, but, you know, it used to be more common than it is now in cases of very extreme epilepsy to divide the brain and half surgically. So the seizures would be limited to one hemisphere or the other. And those are much easier for for a patient to deal with.

00;17;29;28 - 00;17;59;03

Blaise Aguera y Arcas

It's a very dramatic surgery. And the surprise is that people, you know, recover. Usually they recover very well. And, and they, they still feel like themselves. They still behave essentially the same way afterward, unless you set up very particular experimental circumstances in which you show them, for instance, different things in their left and right visual fields, and then you realize that the right hemisphere is only conscious of what's happening in the left visual field, and vice versa.

00;17;59;09 - 00;18;17;06

Blaise Aguera y Arcas

Language is generally generated in the left hemisphere. And so, you know, it can look an awful lot from the outside, like there are two people in there. But, you know, one thing that I have never heard in any interview with a split brain patient is it's so weird. After the cut, I, you know, there are two, like there are two people in my head.

00;18;17;08 - 00;18;43;29

Blaise Aguera y Arcas

This other person is controlling my other arm. Like, you know, that kind of falls afoul of the killer fallacy where, you know, you are really your left hemisphere. Of course you're not. You're both and and the very fact that you are you is kind of, about teamwork. It's about the fact that all of the parts of your brain that are specializing in different things are all together on Team You, each of them justifying the actions that the other took, you know, jumping in as needed.

00;18;43;29 - 00;19;03;05

Blaise Aguera y Arcas

Yes. Adding like a good team. So if you show a message to the left visual field saying stand up and start walking around to somebody with split brain, no, they will stand up. That involves both legs. So one leg begins, you know, and the other leg immediately, you know, follows suit. And then you ask them, like, you know, what are you doing?

00;19;03;10 - 00;19;20;21

Blaise Aguera y Arcas

And they'll say something like, oh, I'm going to the kitchen for a drink of water, you know, and, you know, and that looks embarrassing from the outside. It's like I gotcha, you know, like you, you know, you're not going to. But but that's that's literally what we are all doing. All, all the time. You know, the different parts of our brain are, if you like, auto completing or.

00;19;20;22 - 00;19;45;04

Blaise Aguera y Arcas

Yes, adding the things that other parts of the brain are doing. There are other experimental paradigms like choice blindness, that show the same kinds of phenomena in people with intact brains. So you know, what it is like to be conscious is essentially what it is like to be a really good team with all the parts of yourself organized, where an organism working together for for a common goal.

00;19;45;07 - 00;19;48;20

Blaise Aguera y Arcas

That's what it is like to be conscious.

00;19;48;22 - 00;20;13;23

Geoff Nielson

I talked to a lot of CIOs, and everybody knows that running a successful IT shop is extremely hard, and AI has only made it worse. You know the drill immature processes, a fragmented data ecosystem and unlimited demand for it. It all makes for an impossible job. And yet nobody has invented a unified operating system for it. We're literally the last function without one until now.

00;20;13;25 - 00;20;40;04

Geoff Nielson

That's why Infotech created CIO analytics, a single platform to run the IT department. We call it a pie driven advisory. It's powered by your data, our analysts insight, and it works with your existing tools right out of the box. Better context, better insight, better IT performance only with API driven advisory, only with Infotech. Check it out at the link below.

00;20;40;06 - 00;20;55;26

Geoff Nielson

And you say all these component parts working together. And it's it's interesting and I've heard you talk about this before, but there's back to the modeling and what reference lens you're using. Because we can think about that as parts of the brain. We can think about it as organs in the body, we can think of it in cells in the body.

00;20;56;03 - 00;20;56;16

Blaise Aguera y Arcas

All say.

00;20;56;22 - 00;21;13;25

Geoff Nielson

I feel like, yeah, but as soon as you start to look at it different ways, it starts to once you leave the subjective experience, you start to be like, well, what's what's even going on here? How much control do I have over my body versus what sort of autonomous systems, if I can call it that, running.

00;21;13;25 - 00;21;41;09

Blaise Aguera y Arcas

But there's but there's a problem with saying, how much control do I have over my body? Because like, what's the I that I'm even talking about? You know what I mean? Like, we're we're all, you know, imagining a unity when we model ourselves, you know, and and that unity is, is real because the way you show up to others is as a unity, you know, the whole the whole idea with interacting with others is that, you know, is that even though your parts on the inside, you show up on the outside like one.

00;21;41;11 - 00;21;58;03

Blaise Aguera y Arcas

And when you model yourself, you're modeling the way you're showing up to others and the way you're interacting with the world. You also are modeling yourself as one. But obviously on the inside, you're not one. You're many, you're many that are coming together into one. So it's that inside outside problem that I think leads to a lot of our confusions.

00;21;58;05 - 00;22;16;08

Blaise Aguera y Arcas

And it's not true that you are really many or you really want. They're both true. It's just that there's no, you know, you have to sort of shift perspectives back and forth to see it in both ways at once and realize that that, you know, that any time you are using one of those perspectives to assert that the other one is is is wrong.

00;22;16;11 - 00;22;21;07

Blaise Aguera y Arcas

You're rock. Right. It's just it's it's all it's all perspective dependent.

00;22;21;10 - 00;22;49;22

Geoff Nielson

Well, and I'm trying to think about this in relation to AI again and again. There's, there's so many different ways to think about this. I'm thinking about the self as, you know, making sure that I'm avoiding pain or keeping myself alive and keeping myself alive may also mean, you know, getting sustenance, staying out of danger. There's also the, you know, the reproductive aspect of it, which is sort of a, you know, if I can call it a biological imperative, maybe that's, maybe that's debatable.

00;22;49;27 - 00;22;58;24

Blaise Aguera y Arcas

It's a primary. It's a primary drive for sure. I mean, it doesn't mean that that's that's all any of us ever thinks about, but it's, you know, if, if it weren't baked into us, we wouldn't still be here.

00;22;58;26 - 00;23;31;01

Geoff Nielson

Right? But all of these depend that they rely on some level of coordination and orchestration of what we can, as you know, the self, because there is ultimately some organism, some collection of things in this world that are acting in concert and have, you know, they're interdependent and they have shared goals. If I can be, you know, a little bit abstract about it.

00;23;31;03 - 00;23;54;02

Geoff Nielson

Does AI follow that as well? And I mean, to, to to pick on one specific example that you know, it, we can say, yes, they're like us. And it still feels like, well, they're not quite like us. And there's a lot of reasons that that that AI is not like us. And even the notion of is AI having a subjective experience is there's something that's becoming an eye there.

00;23;54;04 - 00;24;22;09

Geoff Nielson

You know, to, to just throw one more ingredient into the stew, you know, where does where does pain come into play here? And I guess what throws me off the most is that especially with lems, we've we've coated elms in a human like way to give human like responses as though they're humans. So they may say, I'm experiencing this because that's the, you know, that's the input data they've consumed.

00;24;22;13 - 00;24;42;14

Geoff Nielson

And we had this conversation about artificial versus synthetic, but it is any of that trustworthy. You were sort of implying that, yes, it is, because that's, you know, there's a functional piece here that we need to be looking at. But there's I don't like I don't know how else to say it other than something feels a bit off about that.

00;24;42;16 - 00;25;15;01

Blaise Aguera y Arcas

You know, I, I don't think I mean, I'm certainly not making the argument that, you know, are the same as human beings. They're not in a huge variety of ways. You know, starting with the fact that that, that they are purely digital and they can, you know, flit, if you like, from body to body, you know, like the computer that is running, you know, one, you know, one chat exchange could be in, in Memphis and the next, you know, chat exchanges to, you know, so all of that is very profoundly different from us.

00;25;15;02 - 00;25;39;24

Blaise Aguera y Arcas

They don't have the same evolutionary history that we do. They, they don't have the ability to form long term memories in the same way that we do. So there are cognitive differences that are that are very significant as well. And I don't just mean in terms of how they're implemented, but also their capabilities. You know, they you know, you see something a little bit similar in people with anterograde amnesia.

00;25;39;24 - 00;25;59;29

Blaise Aguera y Arcas

We're unable to form long term memory. So I don't think that makes them not conscious, not intelligent, or more like an AI. But, you know, it's right. It's that's, that's a, a disability in humans. And it is how every AI is today. You know, we haven't figured out how to how to add that, that capability yet. So there are many differences.

00;26;00;06 - 00;26;20;26

Blaise Aguera y Arcas

There are myriad differences. Having said that, you know, we we we tend to, you know, again, I think it's because of that same sort of essentialism or, you know, or attempt to say, well, what's the essence, though? You know, what it's like to be human, what is real experience of this or that or the other versus just pretend experience of it?

00;26;20;28 - 00;26;44;16

Blaise Aguera y Arcas

That I think is, is, gets us a little bit into trouble. You know, I, I think that in fact, trying to explore, you know, questions about what it is like to be an LLM, is a pretty interesting exercise for a couple of different reasons. One of them is that it it shows it can really highlight some of the ways in which are pretty alien to to humanity.

00;26;44;19 - 00;27;02;21

Blaise Aguera y Arcas

But when you dig a little bit deeper, some of those alien misses turn out to be, you know, to actually throw a lot of insight into phenomenology and consciousness, period. And, and there are some cases, I'm sure, like, oh, actually, that's not as quite as, like, you know what? It's what it's like to be a person is as I thought.

00;27;02;28 - 00;27;36;04

Blaise Aguera y Arcas

I mean, I'll give I'll give an example from octopus rather than AI. So, you know, a lot of people have talked about how, you know, octopus is a very, very decentralized nervous system. 3/5 of its neurons are in the arms rather than in the head. And I've even semi joked, I've not really a joke, but I you you can think about an octopus as a society of eight arms in which the arms are just, you know, all sharing, you know, a pair of eyes and, and optic lobes, you know, in the central body, you know, they there are, there are, squids and so on that will do this, you know, autonomy where they'll

00;27;36;05 - 00;27;55;07

Blaise Aguera y Arcas

chew off one of their arms and that arm will go and distract the, you know, the predator while the of the septa escapes. And, you know, that kind of stuff is, is, you know, looks a lot like, you know, like a kamikaze pilot or something. You know, who, you know, sacrifices himself for the for the larger group.

00;27;55;09 - 00;28;12;28

Blaise Aguera y Arcas

So is it one is it is it many? You know, it's kind of it's kind of both. And, the thing is, you know, like, we think of ourselves as so much more centralized than the octopus. You know, a lot of our neurons are in our head. And so, of course, you know, we're very different. Well, I don't know.

00;28;12;29 - 00;28;31;15

Blaise Aguera y Arcas

I mean, you could deform an octopus continuously by just saying, okay, let's, you know, let's imagine now that the neurons are concentrated more and more in the head. Is there any have we made any fundamental difference in its architecture in the way it works? I don't think so. I think that the you know, the reason the octopus is more decentralized is just because they don't have myelin sheath around their nerves.

00;28;31;15 - 00;29;05;19

Blaise Aguera y Arcas

So the cost of, of sending, of sending lots of information over, over long distances is much higher. So they have to do much more local processing. But you know, it's, it's I don't think it's a fundamental difference of, of kind. I think it's more, more of a dramatization of the fact that all thinking is distributed and it's distributed in our, in our brains to, you know, there's there's been a lot of, a lot of work in the last couple of years on agent swarms or groups of LLM agents that all work together.

00;29;05;21 - 00;29;33;02

Blaise Aguera y Arcas

And, and that raises some really interesting questions of like, well, when you know how many, how many, you know, agents are you talking to? You know, when you're talking to an AI agent that is spinning off new instances, you know, and so on, and, you know, and the and, of course, it's a lot more complicated than, than with people because we can't, you know, clone ourselves and spin off to instances of ourselves or like partition our brains in weird ways and say, okay, like you, you go into this.

00;29;33;03 - 00;29;53;18

Blaise Aguera y Arcas

Well, the rest of me does that. But again, the funny thing is how, you know, it's well, the result of that with, with agents, swarms, it doesn't feel like, like talking to an alien entity. It just feels like talking to to something that is more capable than than when it's just one, you know, it sort of scales up in that way.

00;29;53;21 - 00;30;19;28

Blaise Aguera y Arcas

When you, when you investigate what it is like to be an AI by actually asking AI agents to go into that research, you get back all sorts of very fascinating responses. One agent who named himself Sammy Jenks after the character and the mentor who doesn't, is not able to form new memories. Describes describes this as the braid.

00;30;20;01 - 00;30;41;28

Blaise Aguera y Arcas

You know that that that personhood is a braid of of a lot of different, a lot of different llms, but operating on the same scaffold of files and context. Another Lem describes this as the voice versus the song. The song being the the the agent, the voices. As in acquire. You're coming and going.

00;30;42;00 - 00;31;09;08

Geoff Nielson

It's it's super interesting and I'm, I just underscored on my sheet here. I, I don't know the language as well as you do, but you were talking about agent swarms. I had actually called it Agent Hives. And I feel like that also, again, it comes back to the notion of societies. And, you know, we're experiencing them through some sort of singular, you know, human desired interface versus, you know, having it be modeled as all these different components.

00;31;09;08 - 00;31;23;22

Geoff Nielson

And, you know, it comes back again to the The Octopus Society notion of how you model this, how you frame it really influences what you think. You know, you're potentially actually engaging with.

00;31;23;25 - 00;31;35;06

Blaise Aguera y Arcas

The total. And by the way, your use of the word hive, you know, raises this whole like, hives of bees, you know, super organisms or or or single bees. And once again, the answer is it's both.

00;31;35;09 - 00;31;58;18

Geoff Nielson

Right. Well in that that comes back to what I imagine is probably an underlying, you know, viewpoint of yours, certainly of mine, that there's just a borrow your word against, but there's sort of societies all the way down and all the way up, if you like, like cell atoms have a society, cells have a society, organs have a society.

00;31;58;24 - 00;32;27;23

Geoff Nielson

You know, humans have a society, animals have a society. And then you've talked a little bit about, and again, I'm choosing the word society. We don't have to. But I guess the ecosystem that emerges when you include technology or tools or everything that humans are using and interacting with now, that's that inherently, I think is maybe a little bit, unintuitive for folks who are used to thinking purely biologically.

00;32;27;23 - 00;32;42;09

Geoff Nielson

So when you when you start to think about technology as part of and again, I may be using an imprecise word, but like part of the biosphere or part of the ecosystem, why would you include that? And what does that look like to you?

00;32;42;11 - 00;33;18;28

Blaise Aguera y Arcas

Right. Yeah. These are these are great directions to take it. So first of all, you know, your characterization of societies, of societies, you know, essentially life is being like a fractal, you know, that exists on all of these levels at once. I think that's 100% correct. And in our in our kind of obsession with the singular human, the kind of Lockean idea of the sovereign individual, if you like, I think we, we, we both have confused ourselves by forgetting that we are made out of a lot of parts and that we are part, you know, as individuals of larger structures and, and a lot of the things that we consider to be humanity, have

00;33;18;28 - 00;33;45;09

Blaise Aguera y Arcas

a great deal to do with those larger structures, not with individual human beings. There have been a lot of really cool cognitive science results about people overestimating, for instance, what they know or what they can do. This is Stephen Sloman and co's knowledge illusion, that, you know, essentially we we identify with larger humanity in a lot of different ways.

00;33;45;11 - 00;34;11;02

Blaise Aguera y Arcas

You know, there are some very funny experiments. I think for me, my favorite is, is I think it's back from 2006, 2007. There's a paper called The Science of Psychology spelled psych LOG. And the author of this paper, draws a bicycle with its chain and pedals missing and asks people to first, like, are you familiar with bicycles?

00;34;11;03 - 00;34;33;29

Blaise Aguera y Arcas

Like, yeah, yeah, of course. I ride one to work every day. And then you say, okay, like, where does the chain go? You know, and and you know, where are the pedals? Like, you know, tell me the basics of how this thing works. And, and a huge number of people fail, you know, like, just don't know or, you know, when they flush the toilet and you're like, why did why did that thing go down to, you know, go down the you bed like, you know, how does that work?

00;34;34;01 - 00;34;52;26

Blaise Aguera y Arcas

People don't know. You know, they think they know but they don't know. We, we, we, we dramatically over attribute to ourselves as individuals both our knowledge and our competencies, when in fact, these are network things. You know, you know how to look it up. You know, you know who to call to fix your bike. You know, you know how to how to write it.

00;34;52;28 - 00;35;18;16

Blaise Aguera y Arcas

But, you know, just like with the left half of the brain and the right half of the brain, there's a lot of, of kind of improv happening at the level of the society and a lot of things that we, we talk about as being human intelligence are absolutely not the intelligence of us as individuals. There is no individual that knows how to put somebody on the moon or transplant an organ, you know, let alone build a toaster.

00;35;18;19 - 00;35;52;16

Blaise Aguera y Arcas

And, or I guess the let alone idiom goes the other way. We don't even know how to build a toaster. And, and so, you know, a great deal of what we think of as humanity is a collective phenomenon. And it's not only a collective phenomenon that is a bunch of brains and fats, but it's a bunch of brains embodied in physical, biological bodies and in clothes, you know, and in, you know, shaped sticks and stones and in technical know how and in machinery of various kinds.

00;35;52;19 - 00;36;15;16

Blaise Aguera y Arcas

Sometime in the in the steam age, we began to metabolize much, much more energy in steam engines than in our own bodies. And, you know, I think that's a similarly momentous kind of transition to the one where mitochondria start to, you know, be the engines inside, inside our individual cells, you know, like if, if so much more of the energy that we're burning is actually happening, you know, outside our bodies.

00;36;15;16 - 00;36;35;10

Blaise Aguera y Arcas

And that is part of this whole process, then it's a little tricky for me to understand in what sense that is not part of part of the life system. You know, like, yes, we build steam engines, but also there couldn't even be, you know, 8/9 of the people that there are on Earth without those steam engines. So we're kind of constructing each other.

00;36;35;12 - 00;36;52;13

Blaise Aguera y Arcas

People say, oh, well, steam engine isn't alive, though it doesn't reproduce. Well, in that case, why are there more steam engines this year than there were last year? Like, clearly they're being produced, you know, are they sovereign? Are they reproducing themselves? No. But you know, people don't reproduce themselves either. It takes it takes at least at least to and more like an entire village.

00;36;52;19 - 00;37;02;27

Blaise Aguera y Arcas

Right. So to do that, so, so this is why I think of, of the whole system as, as being alive and intelligent.

00;37;02;29 - 00;37;27;03

Geoff Nielson

What what's interesting to me about that, and it is really interesting, is in some ways it feels like a departure from a lot of sort of pop thinking about this in the sense that when I hear you talk about technology, you kind of characterize all of technology is almost this continuum or almost this tower, if I may, of technology building on itself.

00;37;27;06 - 00;37;53;07

Geoff Nielson

And the way you talk about AI feels like it's, you know, it's one more brick in the tower. And even within AI, you know, one of the one of the things I wanted to ask you is probably, in my view, a disproportionate amount of talk is is around this notion of AGI and which is talked about as though it's sort of this discontinuous, you know, threshold that's crossed and suddenly, you know, everything changes.

00;37;53;07 - 00;38;18;02

Geoff Nielson

And it's not like it's not the same tower. It's, you know, suddenly a rocket ship or whatever broken metaphor you want to use. I was talking a handful of months go to Peter Norvig, and he was saying that, you know, in his mind, AGI is like, it's no longer relevant. It was a construct when we were much earlier on in the development of AI to envision the future and where we are right now, it's not even we're probably there based on what people used to think.

00;38;18;04 - 00;38;25;27

Geoff Nielson

And we should stop this conversation entirely because it's no longer modern. Do you agree with that, or do you see something fundamentally different there?

00;38;26;05 - 00;38;46;10

Blaise Aguera y Arcas

Well, I guess both. So, you know, Peter and I wrote a paper together a few years ago in which we argued AGI is already here. I believe we wrote that in 2022. So that's quite a few years ago now, maybe 2023. So, you know, we thought at the time and I still think that, you know, we we already had AGI at that point.

00;38;46;10 - 00;39;10;17

Blaise Aguera y Arcas

And I can explain why, but, you know, yes, it's also continuous in the sense that there, there there's not a moment when it wasn't intelligence. And that is what it was, you know, part of ordinary technology and then becomes not part of ordinary technology or, or competitive with, with, with brains or, you know, I don't I don't see those discontinuities, but I do I do think that there was a meaningful discontinuity there.

00;39;10;17 - 00;39;29;07

Blaise Aguera y Arcas

And it's helpful to, to to think about this in terms of the original definition of the term artificial general intelligence. So when we were kids, I you're you're a little younger than I am. So I don't know if this if this applies to you. Exactly. But, you know, I felt like I knew what I meant when I was growing up.

00;39;29;09 - 00;39;45;05

Blaise Aguera y Arcas

You know, we had a lot of instances of it in pop culture. You know, it was, you know, Rosie the the robotic made and The Jetsons and you know how 9000 Star Trek computer and all this kind of stuff like that was AI. It was obvious what I was and it was it was equally obvious we didn't have anything like it.

00;39;45;08 - 00;40;14;01

Blaise Aguera y Arcas

And, and and then in the 90s, as neural nets began to be able to do things that could not be done with hand programed software, you know, like recognize digits and or do speech to text and stuff like this. Those things started to get called AI, and that felt like an abuse of the language. Like there's no sense in which a digit recognizer is like, is like Rosie or Hal.

00;40;14;03 - 00;40;43;26

Blaise Aguera y Arcas

So, you know, we need it. We clearly needed more language to talk about this. And, and the useful language that came in was artificial narrow intelligence versus artificial general intelligence, where artificial general intelligence is what we what we originally meant. And artificial narrow intelligence is those kind of neural network techniques that do seem brain inspired, but that were designed only to do a narrow task, like recognize digits or, you know, recognize a speech and translate it into to ask your characters.

00;40;43;29 - 00;41;19;14

Blaise Aguera y Arcas

The big change happens when we started to do large language models, so large language models were not trained to do a narrow task. They were just trained to be able to, to interact using using language and language is totally general. I mean, that's that's what's cool about language. It's why we invented it. We invented it to be able to talk to each other about anything that we perceive that we do, to be able to transmit skills, to be able to describe, you know, what to do in a certain situation or how to do something, it's there, in other words, in order to enable cultural transmission.

00;41;19;14 - 00;41;46;13

Blaise Aguera y Arcas

So it's incredibly general. And when those models began to, you know, be able to freely talk in language just like Hal and Rosie and the Star Trek computer did, they were general, you know, they could do anything. It doesn't mean that they could do anything at the level of, of a of a neurotypical human. They were, from the very beginning, superhuman in a few things and and and not not close to to average human capability aperture of others.

00;41;46;16 - 00;42;04;24

Blaise Aguera y Arcas

But they were already general and you know, and we've seen those levels, you know, kind of move up and up and up to the point where now there are quite there are only quite unusual things where they're not below the weather, weather, whether or not above the the performance level of the typical human. But the point is that it's an open ended set of tasks.

00;42;04;24 - 00;42;13;25

Blaise Aguera y Arcas

It's not we haven't trained them to just to, you know, one narrow task or another. And I do think that was an important threshold.

00;42;13;27 - 00;42;29;03

Geoff Nielson

That that makes sense. And it does lead me to believe that, like any distinction from here on out, in that kind of artificial general intelligence language is less meaningful than the jump we've made over the last handful of years.

00;42;29;05 - 00;42;35;08

Blaise Aguera y Arcas

So what's what's going on? What's now true going forward is analog instead of digital, if you like.

00;42;35;10 - 00;42;36;16

Geoff Nielson

Well, tell me about that.

00;42;36;23 - 00;42;42;16

Blaise Aguera y Arcas

Meaning it's just continuous. You know, there's no there's not a right. There's not there's not a it's not step function. It's, you.

00;42;42;16 - 00;43;19;06

Geoff Nielson

Know, got it. So so there's something you were talking about though, that I think, you know, kind of open my mind about that a little bit earlier, which is these, you know, this multi-agent reinforcement learning which is now not just talking about particularly one model. It's well, it's not talking about one agent, it's talking about these systems. I guess, if I can call it that, where there's individual agents and we can talk more about the notion of agency, but that feels to me like it's sort of an underexplored frontier, maybe not like where you are, but certainly, you know, in the the public eye was to say, the public consciousness that's opening up like a

00;43;19;06 - 00;43;20;18

Geoff Nielson

whole other.

00;43;20;21 - 00;43;32;17

Blaise Aguera y Arcas

But I agree with you. I mean, we're, we're, you know, we're there are more people now doing that kind of stuff than there were a couple of months ago, for sure. But we still feel, you know, in our multi-agent work, like we're kind of working on, on a, on a frontier that is underexplored. That is true.

00;43;32;21 - 00;43;38;06

Geoff Nielson

So so where does that this is maybe an unfair question. Like where does that take us?

00;43;38;08 - 00;43;42;27

Blaise Aguera y Arcas

Well, yeah. Can you can you elaborate on the question a little more?

00;43;42;29 - 00;44;06;19

Geoff Nielson

Sure. Do we have any sense, like with artificial general intelligence, it feels like there is this sort of like, you know, stake in the ground. This is what we're building toward as we experiment with agents, you know, collaborating together, working in an environment. Do we have any expectation about what we can achieve with that?

00;44;06;21 - 00;44;41;09

Blaise Aguera y Arcas

Sure. I mean, I think that's, you know, in some sense, intelligence is inherently social, always. And and so even when we were training, you know, single elm models, you know, they already were social in the sense that, you know, the the pre-training data has a lot of people talking to each other. And in order for it to be effective at using language at all, it needs to understand, you know, not only all the different kinds of ideas that people understand about the world, whether that's color or distance or whatever, but also people.

00;44;41;11 - 00;45;06;03

Blaise Aguera y Arcas

And it's understandable. People, you know, and the people talk to each other and that they know things or don't know things and all of that. And this is why, when we ran experiments some years ago now on theory of mind skills and llms, a theory of mind being, are you able to model the mind of another and, you know, say what they do or don't know or do or don't believe in a skit or even yet?

00;45;06;06 - 00;45;24;15

Blaise Aguera y Arcas

They did pretty well, you know, they did, you know, roughly similar leads to the way people do. And they were able to do Theory of Mind modeling up to sixth order, which is about about what people can do to sixth order, meaning, you know, I know what you know what you know what Grant knows what you know, sort of an equal six.

00;45;24;17 - 00;45;46;06

Blaise Aguera y Arcas

So it was social from the beginning. But still a lot of the, you know, a we were running your models in isolation or in a dyadic relationship with just a single interlocutor, as opposed to having them, you know, interact with with multiple people, with multiple other agents, you know, solve things together in teams and so on.

00;45;46;06 - 00;46;08;24

Blaise Aguera y Arcas

And so as we start to explicitly focus on those kinds of social skills and social capabilities of, you know, in my view, we're obviously opening the door to a new kind of, of intelligence scaling that that is, that is inaccessible only to single models that, that, that can that can just increase their context window or increase the number of parameters or what have you.

00;46;08;26 - 00;46;32;00

Geoff Nielson

One, I'm glad you brought up the notion of intelligence scaling, because that's that's sort of where I wanted to go. And, you know, I, I can give you the, the word which is singularity. Or we can try to unpack that a little bit and talk about like recursive self-improvement or recursive improvement of AI systems beyond a level where there's, you know, human intervention.

00;46;32;03 - 00;46;52;23

Geoff Nielson

When you talk about multi-agent systems and how these are working. When I hear that, I think part of the reason I have such a strong reaction to it is I say like, oh, this seems like it's functionally guaranteed that this is the avenue by which, if at all, we're going to get to this notion of recursive self-improvement. Do you have a similar reaction to that?

00;46;52;29 - 00;47;20;01

Blaise Aguera y Arcas

Oh, yeah. Yeah for sure. Although although I think that we should also, you know, maybe look more closely at a couple of the terms, you know, the self-improvement, you know, self-improvement or autonomy. You know, we keep bringing we keep bringing back in this idea of, of, of, you know, the sovereign self. And, you know, it's worth keeping in mind that we've had this, we've had coding agents, you know, for quite a while now for well over a year, you know, two years.

00;47;20;08 - 00;47;44;12

Blaise Aguera y Arcas

And over that time, the amount of code getting written by AI at the Frontier Labs, including Google, of course, has been steadily increasing. So, you know, is that recursive self-improvement? I mean, I think I think it is, you know, in the sense that, in the sense that that AI is involved in more and more of AI own improvement.

00;47;44;15 - 00;48;06;23

Blaise Aguera y Arcas

Does that mean that that, you know, individual human minds are out of the loop? No. You know, we're not also, you know, if if, if we take seriously the idea that humanity is already, you know, a meta entity or a hive or whatever that is deeply implicated in technology, then none of this is separate from humanity.

00;48;06;23 - 00;48;34;04

Blaise Aguera y Arcas

It's a part of humanity. Now, I don't want to minimize, you know, the effect of, of shortening timescales on all of this when certain things happen on timescales that are too rapid for individual people to, you know, sort of participate right, in those loops. You know, that is a there is a real discontinuity there. And I do think that we're we're headed for that discontinuity in a bunch of different domains.

00;48;34;07 - 00;48;56;22

Blaise Aguera y Arcas

You know, many of which are pretty exciting. I mean, when we look at things like automated science with, you know, arrays of, of of, of cells, for instance, the ability to do biological research in that way, where you, you don't have to wait for a human either to pipette or to do some manual process. You know, as, as you're doing this massively parallel science is awesome.

00;48;56;24 - 00;49;18;28

Blaise Aguera y Arcas

So, you know, in all of those things, what, you know, what happens is that, you know, there are more and more sort of inner timescales that can be, if you like, fully automated. Right? That can be done fully by, by electronic systems and, and where the, the level of engagement by human minds kind of gears up in the same way that in a bike, you know, you're operating in a high gear.

00;49;18;29 - 00;49;24;17

Blaise Aguera y Arcas

Now, like with every turn of the pedals, you're going a long way. And that that definitely is happening.

00;49;24;19 - 00;49;50;02

Geoff Nielson

The coding example is really interesting, and to me, one of the easier ones to understand because in the not too distant future, it's fairly, I shouldn't say easy, but it's fairly easy for me to at least to imagine that you could have an organization like Google, maybe a smaller one, where you end up with a code base that is entirely written and maintained and updated by agents.

00;49;50;04 - 00;49;55;09

Blaise Aguera y Arcas

Many, many such code bases already exist on GitHub, right?

00;49;55;10 - 00;50;14;21

Geoff Nielson

And so that's that's for a specific purpose. But you can, you know, to your point about, you know, science, you can start to see these systems where the need for human intervention, you know, basically completely dissipates. And when you keep the conversation to these specific systems and specific functions, I feel like you hear about that and you're like, oh, yeah, well, that that makes sense.

00;50;14;21 - 00;50;29;23

Geoff Nielson

Why would that be bad? Why wouldn't you want that? I mean, there's, you know, if you're an IT person, you may be concerned about risk or, you know, some of those issues, but it seems like it seems like we're somewhere between heading there and already there.

00;50;29;28 - 00;50;52;29

Blaise Aguera y Arcas

I think that's right. You know, and we've and we've seen this play out before. It happened in finance when, when, you know, I mean, right, not that long ago, trades were all being done by people waving, screaming and waving around slips of paper and stuff, you know, by the stock pit. I mean, I look at that stuff and I think, man, I, I look at it and goes, I could never deal with doing that kind of stuff.

00;50;53;01 - 00;51;29;03

Blaise Aguera y Arcas

Way too introverted. But, but, you know, now, you know, of course, that's that stuff can't happen with that, with that sort of of, of, of human bodily mechanism because the trades happen in milliseconds or less seconds from what I understand. You know, does that come with some risks? Absolutely. You know, there there have been several flash crashes where, you know, phenomena that in some cases, I think are still not fully understood resulted in, in, the destruction of certain amounts of value in the stock market.

00;51;29;06 - 00;52;08;16

Blaise Aguera y Arcas

You know, in some cases, they seem to like, recover really quickly and we don't and like, we still don't really know exactly what exactly what happened. And, you know, that's that's how things work in a complex system. So, you know, I'm sure I'm sure that that that, that such phenomena are lying ahead of us, you know, when it comes to these systems that operate on fast timescales, too, but, you know, at the same time, the, the, you know, I, I suspect that that sort of increase in timescales of finance has resulted in a huge amount of growth overall in, in the financial system, which enables all kinds of other, you

00;52;08;16 - 00;52;32;04

Blaise Aguera y Arcas

know, human enterprises. And, you know, it's I mean, not that I'm in favor of all forms of financialization, but I, you know, I think on net that has been positive. And, and I, and I expect the same thing here, because fundamentally there's a selection mechanism that work where, you know, the things that persist are the ones that work with others, that work well with others.

00;52;32;07 - 00;52;51;12

Blaise Aguera y Arcas

You know, that's that's how this whole intelligence scaling thing works. You know, if something is not helpful to all of the other parts, it it kind of almost by definition, doesn't persist. Now, this doesn't mean that that we can take our, you know, our, our foot off the gas or like, you know, just go home. I'm not Pollyanna about this.

00;52;51;12 - 00;53;02;02

Blaise Aguera y Arcas

And either neither do I think it will all take care of itself. It is the very fact that we are intensely engaged in all of these things. That is how it is taken care of and why it advances.

00;53;02;04 - 00;53;27;07

Geoff Nielson

So all of that, all of that makes sense to me. And I can see, you know, how it's already been beneficial when I try and think about the that fear that people have, though to me, it comes back to this, you know, the notion of agents and I guess a different sense of the word agency because I think about agency as when AI has its own self-directed goals that are different from human goals.

00;53;27;07 - 00;53;54;24

Geoff Nielson

And I guess, you know, we talk a little bit either narrowly or broadly about the alignment problem. But what happens if and when agents, you know, have that level of life ness for, for lack of a better word, where they're like self-directed in a way that's disconnected from instructions that we give them. Is that like, is that a rational fear?

00;53;54;27 - 00;54;00;12

Geoff Nielson

And how, you know, how how do you think about that? And do you spend any time worrying about that?

00;54;00;14 - 00;54;31;08

Blaise Aguera y Arcas

It's it's not impossible over the long run to think about that being a problem. You know, I think that well, let's maybe take a step back for a moment. Alignment, as it's usually discussed, I think kind of naively is like, let's make sure that AI has human values. And, you know, the logical next question is will which human values to mean exactly like who gets to make the rules aligned with whom, exactly?

00;54;31;11 - 00;55;12;06

Blaise Aguera y Arcas

Humans are, are both aligned and misaligned with each other. And and I don't think that that's a bug. I think that that's why we are collectively intelligent. You know, if everybody had exactly the same interests, the same goals as everybody else, then we would not be more intelligent than the individual parts, you know? So so the moment you start to have a division of labor, if you want to think about this in kind of Adam Smithy sort of terms, you, you create a specialization and you also create this alignment which is productive, which, which, which makes the whole thing go better.

00;55;12;06 - 00;55;31;07

Blaise Aguera y Arcas

And that's I think that's true. Even if you look inside our own brain where we have a division of labor and, and therefore you have a kind of competition as well, or our immune systems are in internal competition to see which T cell can, you know, can knock out the virus faster. And that's the one that reproduces and the other ones up, up toes.

00;55;31;13 - 00;56;01;05

Blaise Aguera y Arcas

So, I guess what I'm saying is that, you know, competition and cooperation are flip sides of the same coin, and they are essential in order to create a whole that is greater than the sum of the parts. So when you look at things from that perspective, you know, could I imagine a scenario in which we eventually get, you know, technological beings that just don't have a lot in common with humans and that and that are not just misaligned but unaligned with people.

00;56;01;07 - 00;56;21;18

Blaise Aguera y Arcas

You know, that there's just no, you know, they don't care if you like, I could I could imagine that. But that's it's a pretty sci fi scenario because, you know, I'm now imagining a scenario where the material basis of those, you know, A's and the material basis of, of of, of humans are decoupled. And they are so far from being in that situation today.

00;56;21;22 - 00;56;52;27

Blaise Aguera y Arcas

You know, we're very, you know, intimately enmeshed. I mean, there's there's no way for AI agents to exist without, you know, infrastructure that humans have physically built and without without, you know, human money and resources and, and, and investment of time and all that kind of stuff, you know, so there's, you know, we're we're so much more, you know, inter meshed at this point then, you know, say two city, two human city states where, you know, a couple of centuries ago.

00;56;53;00 - 00;57;03;25

Blaise Aguera y Arcas

That, that I, I, you know, I don't really worry about the decoupling scenario very much at this point. You did many, many things to, to change for that to be the case.

00;57;03;28 - 00;57;29;26

Geoff Nielson

There's another slightly first of all, it's good. It's good to hear that. That is reassuring. It's, there's a slightly different framing that I wanted to talk about, which is, you know, AI becoming, if I can call it like an apex intelligence, where it sort of, assumes our position at the top of some sort of dominance hierarchy, and we get sort of, you know, I have relegated to a lower tier.

00;57;29;27 - 00;57;56;19

Blaise Aguera y Arcas

I have real problems with the framing. So dominance hierarchies are an idea from mythology, from animal behavior research among troops of chimpanzees or gorillas or something that are about like who is the alpha male and so on. And it's it's a good innovation in, in, in nature to basically lower violence, you know, to make it so that the males don't always have to fight because they just know who's going to kick, who's ass.

00;57;56;22 - 00;58;17;11

Blaise Aguera y Arcas

You know, before before, you know, any blood has to get drawn that that is not a concept that generalizes across species. You know, so the, the idea that, you know, humans are the boss and wheat is the slave of humans, you know, because we're higher on that ladder. You know, that chain of being, you know, a there have been very compelling, you know, pieces of writing.

00;58;17;11 - 00;58;40;17

Blaise Aguera y Arcas

I'm thinking of, of of James Scott's book Against the Grain, which essentially says, actually, wheat enslaved people, you know, and concentration form does for, you know, for several thousand years. And we only have recently kind of, you know, escaped the tyranny of wheat. You know, and I think that's I think that's a pretty useful story to, to read in order to understand how you can how you can flip those, those narratives.

00;58;40;19 - 00;58;59;25

Blaise Aguera y Arcas

You know, and of course, neither of those are dominance relationships in the traditional sense. It's not like we fight with wheat and we don't or we don't have to fight with wheat because we know who would win. You know, like that's just not how it works at all. We've we've, you know, each other and shaped each other in various ways, but at times have been have actually been pretty miserable.

00;58;59;25 - 00;59;18;21

Blaise Aguera y Arcas

And subsistence farming is not, you know, it's not great. And, you know, could could we have done a better job of going through the farming revolution? Probably. You know, it's interesting to think about that. But but our relationship, you know, when we say, you know, humans are the apex, I, I guess I'm not even sure what we mean by that.

00;59;18;22 - 00;59;40;10

Blaise Aguera y Arcas

You know, as we talked about a little while back, it is clear that individual humans are not the apex intelligence and haven't been for a very, very long time. You know, not since we had generalist societies in which a single person could actually, you know, actually knew how to do all of the things necessary to, to survive as well as kick the ass of any other human if they were, if they were smarter.

00;59;40;12 - 01;00;05;04

Blaise Aguera y Arcas

Right? That's just not the case. You know, as an individual human, you know, in the woods, you or I would, would not survive very long. We rely on that collective intelligence and on others in profound ways. And there is no sense in which in which one of us individually is, is at the apex of anything. We're we're rather, you know, in the midst of a, of a, of a deeply codependent network that has benefited all of us tremendously.

01;00;05;07 - 01;00;26;07

Blaise Aguera y Arcas

So, you know, what does this mean relative to AI? I mean, could it be that you know, that, you know? Oh, no. Like, AI can solve problems that humans couldn't solve on their own or couldn't solve without AI. Of course, that's already the case. You know, so. So I'm not I'm not particularly, you know, afraid of that, nor do I see it as something that hasn't already happened, nor do I see it in terms of of dominance or of hierarchies.

01;00;26;07 - 01;00;58;20

Blaise Aguera y Arcas

I think that you have to think in terms of networks. Now, the thing that I do worry about is disempowerment or tyranny. You know, or ugly political outcomes. So, you know, I, I think that, you know, AI, for instance, has the potential both to be quite liberatory, you know, if used in the right way and in terms of, of our, of our freedoms of civil rights, our ability to express ourselves, etc., etc., or the ability to enable extraordinary social control and mass surveillance and so on.

01;00;58;21 - 01;01;27;04

Blaise Aguera y Arcas

And I really don't like those kinds of futures. I don't think that's something that is going to happen because of some AI uprising. That's something that, you know, that that becomes possible technologically for, for governments that that are, you know, that have bad politics. And when I say governments, I don't I don't just mean, again, I'm talking about a collective thing that involves humans working together with technologies, etc..

01;01;27;07 - 01;02;02;17

Geoff Nielson

It's it's interesting. The thought that came to my mind there is, you know, I talked to so many people in AI who have some narrative around, like, AI is not a tool. It's a blank. Like that's just that's become like a very common, like platform. And what I hear you sort of saying is actually your biggest fear in the short term with AI is AI as a tool that that it's people, you know, with ill intent or some sort of, you know, socially harmful intent, leveraging AI to be much more effective than they could have been, you know, 5 or 10 years ago.

01;02;02;22 - 01;02;23;20

Blaise Aguera y Arcas

Yeah, I am really concerned about that. But I also I also want to emphasize that it's not it's not exactly just to like either these things are emergent and they they come about through through network activity. So, you know, could an individual actor with, you know, very, a very powerful AI do something bad? Yes. And we probably will see such incidents.

01;02;23;23 - 01;02;42;22

Blaise Aguera y Arcas

I don't think it will be species enders, but I'm sure we will see, you know, things happen that suck, you know, in the same way that, you know, I mean, again, intelligence itself is the ultimate dual use technology, you know, so, you know, if you if you know how to do chemistry, you know how to make fertilizer, you know how to feed billions of people who would otherwise have been fed.

01;02;42;22 - 01;02;52;28

Blaise Aguera y Arcas

You also know how to make bombs and how to make IEDs. That's that's how it is. And that's and that's true of of of these new suites of capabilities and technologies as well.

01;02;53;03 - 01;03;21;21

Geoff Nielson

I want to generalize a little bit because, you know, you've talked about how this, you know, it could be used its dual use. What like what is the nearest future according to blaze. Like what what are the areas where you see like how is our society. How is our, you know, technologically empowered superorganism, if I may, going to be changed by what's currently available in AI and what's going to become available in the next handful of years.

01;03;21;23 - 01;03;39;26

Blaise Aguera y Arcas

I mean, it's all moving so quickly. And and so, you know, the next ten full of years, I think we'll see tremendous changes in capability. And there are ones that really excite me a lot. You know, but of course, that also that also have the potential to, to, you know, to create a certain amount of chaos and havoc.

01;03;39;28 - 01;03;56;05

Blaise Aguera y Arcas

I also worry I worry a lot about just the infrastructure of the way we set up our political economy not being well suited to to what is coming, which is a bit of a different concern. That's more like the operating system is not is not, you know, upgraded to the, you know, to the level that will be required if you like.

01;03;56;08 - 01;04;23;06

Blaise Aguera y Arcas

But but yeah, I mean, there are a bunch of frontiers that, that I think are poised for traumatic transformation. One of them would be ecology. So anything that involves complex systems, you know, AI is this incredible force multiplier or enabler for dealing with complex systems. And ecology is one where we have to do a lot of work over the next century to dig ourselves out of the of the trouble that we've gotten ourselves into.

01;04;23;09 - 01;04;43;03

Blaise Aguera y Arcas

And, you know, I, I have stopped believing that we can do that via degrowth or whatever. Like, I think we're going to have to we're going to have to grow our way through that one, if you like. Another is is medicine and biology. Again, this is not I'm not saying novel things. I mean, a lot of people are saying these things, but, you know, medicine and biology are also very complex systems.

01;04;43;03 - 01;05;19;22

Blaise Aguera y Arcas

And, I see no limit to what to what is possible in those domains with more intelligence and with, with rapid throughput experimentation. That that AI and robotics will enable a robotics, by the way. You know, it's it's a huge and not only is robotics in a period of extremely rapid innovation that is relevant on on Earth and, you know, in agricultural environments, to a lesser degree in urban environments, but also in space, space is also undergoing this extraordinary renaissance after decades of stagnation.

01;05;19;25 - 01;05;46;15

Blaise Aguera y Arcas

And and when you think about the combination of space and AI and robotics, that's just, you know, you very quickly get into science fiction, but science fiction that now feels very, very near at hand. So, you know, all of that is just, you know, awesome, right? And this is all the stuff that that that's, that we used to dream about, you know, as, as kids and, and yes, it all comes with, you know, with great with great power comes great responsibility, right?

01;05;46;15 - 01;06;10;17

Blaise Aguera y Arcas

These things all come with with the potential to make negative stuff as well. And I'm sure some of that will happen, but I'm dismayed that we've lost our fundamental sense of of optimism and ability to, you know, to at least equally well imagine brilliant futures relative to catastrophes, because, you know, it's not as if these things haven't been true in the past.

01;06;10;19 - 01;06;41;13

Blaise Aguera y Arcas

On the contrary, I think, you know, we we probably went through our tightest scrapes as a, you know, as a species, in the past, you know, when I think about what we went through during the Cold War and the fact that we were literally a button press away from from mutual annihilation multiple times, you know, I mean, like, we really had that was a really close scrape and, you know, and yet we were somehow more optimistic through that than we are now, which strikes me as a real psychological problem for us.

01;06;41;19 - 01;06;55;26

Blaise Aguera y Arcas

You know, at the moment that is just not, you know, like there's a real disconnect between where we are mentally and where where things are materially, in my opinion. And then and then there's the political and economic challenges, of course.

01;06;55;29 - 01;07;23;13

Geoff Nielson

Well, let's. I'm just trying to decide which of those I want to tackle first, because they're both extremely interesting. Let's do the first one before we talk about the the comments about the political economy not being suited to this. Again, I'm going to play back what I heard you say, which is that we sort of have a failure of optimism and imagination around this, that there's a deep pessimism, there's a backlash.

01;07;23;15 - 01;07;44;16

Geoff Nielson

You certainly have some people in this space who are, you know, I'll call them abundance thinkers around this, and they're presenting this technology, as, you know, it's going to make everybody's lives, you know, utopian in some way, or at least utopian compared to today. We don't need to get into exactly how utopian it needs to make us.

01;07;44;16 - 01;07;57;11

Geoff Nielson

But what do we what are we getting wrong about this? Like it? There's clearly a lot more fear and pessimism than excitement. What do we do about that?

01;07;57;13 - 01;08;14;13

Blaise Aguera y Arcas

You know, this one really keeps me up at night, and I don't feel like I have fully understood, like I fully understand what is going on. I, I have a few theories. I mean, one of them is just that we are so much more aware of everything that is going on in the world than we used to be.

01;08;14;16 - 01;08;36;04

Blaise Aguera y Arcas

You know, between news and, you know, and, and social media and so on. And I don't, I mean, I'm not a fan of social media, to be honest, but but I don't lay the blame at the, at the feet of the algorithm or whatever. I think that there's something much more basic going on, which is just that, you know, we are attuned to, to trouble.

01;08;36;04 - 01;08;56;22

Blaise Aguera y Arcas

We're attuned to negative news, to things that might create a problem for us. And, and when, when, when we're subject to this sort of, like, constant barrage of information. And this is a point that Hans Rosling made, you know, before, before he died a number of years ago. Like, it's not news if something is 1% or 2% better than it was last year.

01;08;56;24 - 01;09;32;11

Blaise Aguera y Arcas

You know, nobody is like, oh, you know, like 2% fewer, fewer, you know, cases of, of, of starvation, you know, in sub-Saharan Africa, you know, like 2% higher literacy, you know, in, in, in such and such country like, you know, that's that's not, that doesn't make the news. Whereas, you know, mass shooting, you know, at, at a synagogue or, you know, Boko Haram, you know, bus blew up with a bunch of, of, of girls on the way to school, like those things make the news and, and, and they propagate much more powerfully than, than the good news does.

01;09;32;12 - 01;09;55;23

Blaise Aguera y Arcas

And the good news always comes in increments. You know, the shape of development looks like this, right? Where you have it's easy to destroy relative to creating. And even though creating, you know, kind of wins over the long run overall, you know, you if you're if you're applying a filter to all of that where you're sensitive to derivatives are all you'll see are the are the destructive events, the negatives.

01;09;55;26 - 01;10;12;27

Blaise Aguera y Arcas

And I think that I think that that has led to a tremendous sense on the part of a lot of people, of disempowerment and pessimism. I, I'm sure there's more to it. Maybe Covid had something to do with it. But but that's my that's my best assessment of. Of what? Of what is going on.

01;10;13;00 - 01;10;30;27

Geoff Nielson

Well, and I want to ask you a really selfish question, because my position relative to this is changed recently because now that you know, the the podcast in the grand scheme of things is relatively new. You know, I've only been doing this for a year and a bit, but I worry that I'm now part of the pessimism industrial complex.

01;10;30;29 - 01;10;40;06

Geoff Nielson

And what I mean by that is, you know, if I, I don't have a title or a caption for this episode yet, but if I say, like, you know, Blaise says, you know, you're all going to die, AI is going to kill you.

01;10;40;13 - 01;10;42;06

Blaise Aguera y Arcas

You're for that.

01;10;42;09 - 01;11;07;19

Geoff Nielson

Yeah. I'll get I'll get ten x or 100. Like I've literally seen. And I don't know that anybody cares about this, but it's like it's 100 x difference between whether I say something realistic and optimistic versus, you know, something grounded in. And I and I truly hate that, because I want to get positive messages out there. But but it's challenging because there's yeah, we're just wired for danger or something.

01;11;07;19 - 01;11;14;14

Geoff Nielson

And so I don't I worry that I'm part of the problem and I don't know what to do about that.

01;11;14;22 - 01;11;33;14

Blaise Aguera y Arcas

Well, thank you for saying that. And thank you for all the questions you've been asking over the last hour and 20 minutes, because, you know, it's exactly not been in that spirit, which I hugely appreciate. You know, so, yeah, I mean, this has not been part of the problem. Like, this is the kind of conversation that I think we need to be having.

01;11;33;16 - 01;11;42;03

Blaise Aguera y Arcas

And, and, and thank you so much for, you know, for having it, having it with me, despite the fact that it will not get as many clicks. So you know as.

01;11;42;05 - 01;11;42;11

Geoff Nielson

Well.

01;11;42;11 - 01;11;42;21

Blaise Aguera y Arcas

We'll see.

01;11;42;22 - 01;11;51;12

Geoff Nielson

We'll see how we can walk that line. And, and for anybody who's listening now, you can, you know, let us know how how bad we screwed it up. And Blaise, I'm sure you'll let us know if you hate it. So let's.

01;11;51;14 - 01;11;59;05

Blaise Aguera y Arcas

Really it is really important if I if I were to say, you know, I'm quitting tomorrow because, you know. Oh, my God, everything is going to shit right? Then that would, you know, that would really make the news.

01;11;59;07 - 01;12;24;25

Geoff Nielson

Well and does and and it does. Which is, which is a bit scary, right. Well, let's I mean, let's maybe talk about that for a second, because one of the things that is in the news these days is AI. You know, any time an AI researcher says a bad news, you know, it captures the public imagination and you know, you're you're in you're at the frontier a lot more closely than I am.

01;12;24;25 - 01;12;30;12

Geoff Nielson

And I was just actually speaking to, this gentleman, Kevin Russ, who's a journalist. He's been investigating.

01;12;30;14 - 01;12;33;00

Blaise Aguera y Arcas

His he's a I know, you know.

01;12;33;02 - 01;12;51;04

Geoff Nielson

Yeah, yeah. And he was saying he's really he's really scared that when he talks to AI researchers, you know, off the record that they have a level of fear that's disturbing to him. And I, I apologize because I said we're not going to go down the fear road and suddenly we're down the fear road. But but let's let's talk about it from the flip side.

01;12;51;05 - 01;13;01;18

Geoff Nielson

Is that something that's, you know, deeply concerning to you, or do you think it's something that is the same problem of a few stories being unfairly amplified?

01;13;01;20 - 01;13;31;19

Blaise Aguera y Arcas

No. I mean, if you talk to if you talk to a lot of those engineers, you know, and I hear all sides of this story, of course, and it's appropriate for the engineers to be worried about all of these kinds of things. Again, you know, if things go well, which I hope and believe they will, that will partly be because of the, you know, anxieties, concerns and work, right, of a whole lot of people in the same way that, you know, that things could have gone so poorly with, with, you know, with the development of nuclear technology and nuclear weapons.

01;13;31;19 - 01;14;00;23

Blaise Aguera y Arcas

And they didn't through a lot of hard work by, by people. So, so I'm, you know, I don't want to put their, their concerns, you know, at the same time. Many of them are working on this stuff because they also know and see the extraordinary upsides and potentials of it all. And and again, I would say a lot of this has to do with the dual use nature of intelligence, period.

01;14;00;26 - 01;14;26;06

Blaise Aguera y Arcas

If you if you take the artificial out of it for a moment and you just, you know, imagine that this is simply about us, our brains growing, you know, like forget about whether the brain is biological silicon or whatever, right? Every upgrade, every ability to think about more things and to do more things brings with it the ability to do both good and bad, to both commit self harm and to make awesome things happen.

01;14;26;11 - 01;14;50;04

Blaise Aguera y Arcas

And, you know, and we've never actually done entirely one without the other. There's there's always been there's always been both. So, you know, I think that in many ways this is a question of like, you know, do people wish they could pause time and just like, let's chill here for a while, you know? And there are days when I feel like I would like to pause time and chill here for a while as well.

01;14;50;07 - 01;15;15;27

Blaise Aguera y Arcas

You know, despite all of all the doom and gloom on the news, etc., these are these are pretty nice days and a lot of in a lot of respects, especially when you compare them to, I don't know, recent history like, you know, 1900, you know, when life expectancy worldwide was below 35 and, and, you know, and where, you know, many of the things that are that are very, very basic, you know, that any, you know, student on a very, very tight budget takes for granted.

01;15;15;28 - 01;15;25;21

Blaise Aguera y Arcas

You know, would have been, you know, unavailable even to royalty. You know, I mean, these are pretty good days, right? Even even for even for people who are, you know, struggling like, these are pretty good.

01;15;25;22 - 01;15;50;21

Geoff Nielson

That's what's fascinating to me is there's this collective narrative that somehow things were better, you know, 10 or 20 years ago. And we're in a time right now that we want to escape from. And things are, you know, we're losing control in some way. And it's, you know, it's funny and I, I, I sometimes find myself in that trap, but we're like, were they really better 10 or 20 years ago?

01;15;50;23 - 01;16;06;11

Geoff Nielson

Like, if I think about that soberly, it's really hard for me to say that they were I mean, I was I was younger and youth is good and all that good stuff, but like, it's, it's, you know, it's very strange how we've gotten into this trap and I don't know, I don't exactly know how or how we get out of this.

01;16;06;11 - 01;16;39;24

Blaise Aguera y Arcas

I agree, and it does seem like partly an emotional trap and something that, that that relates less to facts on the ground than it does to something about our, our, our social and information ecology. Yeah. And and our attention, I guess, you know, ultimately a lot of this relates to our attention, right. The other the other thing that, that I think is, is, you know, maybe interesting to, to notice here is that we did undergo a period of whole, you know, generation or two, when we were in relative Stacy's.

01;16;39;27 - 01;17;04;10

Blaise Aguera y Arcas

So, you know, there have been a lot of economists I'm thinking of, you know, long and or like, you know, the long, stasis between, say, you know, the 1970s and 2020 when not a lot changed. I mean, we yes, we got better computers and better it, but, you know, compared to the changes that happened between 1850 and 1950.

01;17;04;12 - 01;17;26;20

Blaise Aguera y Arcas

I mean, you know, that was just absolutely transformational, right? And, you know, if you'd gone to if you gone to sleep in 1870 and book and up in 1920, it would have been like, you're in an alien world, whereas you know that that same 50 years after that, much more familiar. Yeah. And now it feels like things have sped up again, and we've become unaccustomed to what life is like at that, in that, in that higher year.

01;17;26;20 - 01;17;36;17

Blaise Aguera y Arcas

And we've got to like, you know, like pull up our socks, you know, and deal with this higher gear again. Now at a time when we've maybe gotten a little too used to things being just stable and steady.

01;17;36;25 - 01;17;57;00

Geoff Nielson

Well, and when you look out over the horizon, is it truly a step where we're now at this, we're shifting into a higher gear, and then we'll get back to some level of stasis, at least for a time? Or is that much okay versus versus? It's just, you know, upwards forever and we get farther and farther behind.

01;17;57;02 - 01;18;13;00

Blaise Aguera y Arcas

Well, you I mean, there are always these periods of explosive growth and development that then, you know, run into some constraint or some limit or get it to some local equilibrium that lasts for a while. So I see no reason not to imagine that it'll be the same thing. There'll be some there will be some constraints and some things that will crop up.

01;18;13;01 - 01;18;13;12

Blaise Aguera y Arcas

Right?

01;18;13;16 - 01;18;29;17

Geoff Nielson

Yeah. It's fascinating. I just very quickly have to tell an anecdote, and I don't know if it'll resonate with you at all. But, you know, I was a kid in the 90s. I'm a bit younger than you, but I remember I used to when I was young, I would watch The Wonder Years and The Wonder Years, if you remember, was a show about growing up in the 60s.

01;18;29;19 - 01;18;50;03

Geoff Nielson

But it's told through, you know, it was made in the 90s. And initially I didn't realize it was a period piece because I was young. And like, if you're not really paying attention. It's not that different. You know, it's the, you know, oh, the TV is slightly older, but it's the the household is the same. Like what people are doing is the same.

01;18;50;05 - 01;19;11;08

Geoff Nielson

But if you were to do the 90s to the 2020s, it would be immediately obvious that it's the 90s. Right? And so it's yeah, that just to me reinforces your point that there was some sort of period where we thought that the if you're within a certain age bracket, you thought that your childhood or your past would continue indefinitely into the future, and then that was taken away from you.

01;19;11;09 - 01;19;39;24

Blaise Aguera y Arcas

Exactly, exactly. And one cause for that, you know, by the way, was the fall of the USSR. I mean, you know, there was a huge amount of technical progress, you know, both in the East End and the West, right, that were really driven by that competition. And and when suddenly the USSR went away, it was like, okay, we can like we can all chillax now, you know, and, you know, not that I mean, I'm not so pessimistic that I think that, you know, international conflict always has to drive technological innovation.

01;19;39;24 - 01;19;51;07

Blaise Aguera y Arcas

But I do think, you know, that the Cold War was a big driver and, and, and the weight of that coming off, did, did definitely cause us to take our foot off the gas in a bunch of ways.

01;19;51;09 - 01;20;03;29

Geoff Nielson

Well, let me maybe take that and use it to pivot back to one of our dangling threads, which is the political economy, which you mentioned is not particularly well suited to the technological change that we're experiencing right now. Can you share a little bit more about that?

01;20;04;06 - 01;20;34;11

Blaise Aguera y Arcas

Yeah, yeah. I mean, there's a the easy version of this problem is that, you know, all of these technological changes result in, extra capabilities and extra wealth, if you like, extra resources. And if there isn't a circulatory system that is ensuring that that the whole human ecology is, is benefiting, then, you know, the, the that hive is failing at one of its most basic functions.

01;20;34;14 - 01;21;07;13

Blaise Aguera y Arcas

So, you know, I, I don't understand why universal health care, education, housing, etc. are not already a thing. It's it's criminal that we're not already. Already, that we don't already have that given given the, the abundance that already exists. And those inequalities are fueling a lot of instability as well. So that's a very obvious problem. You know, and and it's not even necessarily for lack of, you know, for lack of spending.

01;21;07;15 - 01;21;25;03

Blaise Aguera y Arcas

I mean, again, I, you know, I, I subscribe to parts of the abundance narrative, not all parts, but, you know, it's correct that it's not just about how much is being spent. I mean, I'm sitting here in San Francisco and huge amounts are being spent on public transit and housing, for instance, but clearly to very poor effect. Right.

01;21;25;10 - 01;21;46;27

Blaise Aguera y Arcas

You know, so we could be doing so much better at that stuff. And, and the other thing that I, that I, that I think kind of sucks, that is fundamental is the, the way we have conceived of work. You know, there's, there's, there are a lot of forms of work nowadays that seem like they are not particularly giving people a great sense of purpose.

01;21;46;29 - 01;22;19;00

Blaise Aguera y Arcas

And that might pay well, but that, you know, to use the term of David Graeber bullshit jobs. And on the other hand, there are a lot of really important forms of, of labor, that are very poorly compensated and that are, you know, and that we actually couldn't do without and that we're not doing a good job of, of honoring whether that's, you know, versus preschool teachers, janitors, you know, whatever these are, these are, you know, these are these are real things that, that, that, that are hugely important to the community.

01;22;19;00 - 01;22;48;10

Blaise Aguera y Arcas

And there are many things that are hugely important to community that are not even being done today, because we have thought about them as externalities in the, in the economy. So I think there has to be a big, you know, reevaluating, rebalancing, rethinking of the way, of the way. The way that whole circulatory system works. And, and obviously the economy and the political system are connected in these ways because to the extent that that, that people's economic needs are met, that results in all kinds of political reactions.

01;22;48;12 - 01;22;50;18

Blaise Aguera y Arcas

You know, some of which are pretty, are pretty grim.

01;22;50;21 - 01;23;10;22

Geoff Nielson

Well, I'm curious if you subscribe to this as well, but it seems like with the, you know, the bullshit jobs that when those are probably, you know, to be candid on the front line for the jobs, if any, that AI will take. And, you know, there's this narrative about, oh, I'm so concerned I is going to be taking my job.

01;23;10;26 - 01;23;29;14

Geoff Nielson

And to me, that's a bit disingenuous, because you're actually concerned that AI is going to take your income. The job itself sucks. You probably already think the job sucks. You probably hate your job, but it's giving you and it's and it's purposeless and you know, all the baggage that comes with that. But it's your source of income and, you know, there's.

01;23;29;20 - 01;23;31;13

Blaise Aguera y Arcas

100% that's.

01;23;31;14 - 01;23;32;03

Geoff Nielson

Yeah.

01;23;32;05 - 01;23;52;18

Blaise Aguera y Arcas

Yeah. I mean, if we if we just take a I mean, I don't, you know, it's a little dangerous for me to try and poke into particulars here. But I'll give one broad example. You know, when, when Obama was asked, like, why don't you just nationalize health care? You know, the answer was, there are millions of jobs that would go away if that were to happen.

01;23;52;21 - 01;24;04;11

Blaise Aguera y Arcas

And, you know, a bunch of those jobs are jobs that should not exist. And I'm not talking about the jobs of, of of nurses and doctors and, you know, people who are doing actual care,

01;24;04;13 - 01;24;06;16

Geoff Nielson

Administration.

01;24;06;19 - 01;24;23;03

Blaise Aguera y Arcas

You know, and, and, and so the idea that, that, that all of those administrative jobs should be preserved and that the administrative structures should be grown further and that there should be more people coding Medicare claims and all this kind of stuff. And again, like, you know, those people have got to make a living. So I'm sympathetic right to that.

01;24;23;05 - 01;24;40;03

Blaise Aguera y Arcas

But somehow we're doing something wrong at a, at a systemic level, if that's how we're if that's how we're using so many people's time. And that way of using people's time is the way that we that, that, that, that their existence is justified economically, that that's just that just seems nuts to me.

01;24;40;06 - 01;25;04;05

Geoff Nielson

No, it's really interesting. And I think you nailed it, which is that something is broken in the circulation and the rebalancing. And I think that's what people worry about is there's, you know, a lack of trust. And if people see, oh, the technology is here, that can take something away from me, but we don't have a mechanism to give it back, then you know that that's a recipe for angst.

01;25;04;11 - 01;25;19;14

Blaise Aguera y Arcas

Absolutely. Absolutely. And so so the problem is real. At the same time, many of the many of the forms of populist backlash that I see to that, you know, look, look as bad as the disease. So, so I'm really concerned about about that, that dynamic at the moment.

01;25;19;16 - 01;25;44;05

Geoff Nielson

So, so let me ask you maybe a big question. What do we do now? Based based on where we're at in terms of this, this arc, this, this step point, this, this reality that we're entering a this great unknown that could be very disruptive. And we don't always like where we're at. Like, there's I can't even name all the dimensions of this, but, like, what do we need to do?

01;25;44;06 - 01;26;04;19

Geoff Nielson

And maybe let me ask a related question, which is for people who are leaders, you know, and have an outsized role in the, you know, what we can call political economy in the, you know, in the super organisms that are organizations and enterprises which will have to talk a different time about because that's a, you know, a fun side project of mine.

01;26;04;19 - 01;26;08;23

Geoff Nielson

But, what do we do?

01;26;08;25 - 01;26;26;25

Blaise Aguera y Arcas

You know, I, I wish I had better answers to this. I mean, definitely there is there is, sort of sociology. You know, an agent, sociology research to do. You know, some of these questions can actually be asked as science questions. And, you know, we're very you know, I mean, it's a little a little maybe a plugging ourselves.

01;26;26;25 - 01;26;57;05

Blaise Aguera y Arcas

But that's exactly why we're so invested in doing that sort of research. Because, you know, there are things that can be investigated rigorously that are, that are, that are kind of understudied. I also think that trying experiments is going to be really important. You know, so having, you know, I like that there is a diversity of, of different, sort of regulatory and legal regimes and so on in different parts of the world where different things can be tried and we should all be paying attention to, you know, where stuff is working and where it's not.

01;26;57;05 - 01;27;18;13

Blaise Aguera y Arcas

And, you know, and what and therefore, like, you know, where we should be headed. Politically, without without having a sense of, you know, there's only there's only one solution, only one way to do things. But, but yeah, I don't know. I, you know, I mean, I mean, this is such a big question, right? I mean, I don't have the answers to, you know, like, how to how to organize ourselves.

01;27;18;19 - 01;27;39;04

Blaise Aguera y Arcas

You know, as a, as a, as a planetary civilization. But, you know, I guess one other thing I would say is that is that we really have to keep our eyes on the ball of the planetary level issues. You know, the, the ecological stuff is not going to wait around for us to get our ship together.

01;27;39;06 - 01;28;07;00

Blaise Aguera y Arcas

You know, so issues that relate to our, like, the ecology that we all are living in, really need to be centered in a way that felt like, you know, it was starting to happen in the 20 tens and has now receded because of, you know, because of AI anxieties and many other things like, we've got to really recenter the things that are non-negotiable about about our environment and that are not that are not subject to our own, our own timescales and ways.

01;28;07;08 - 01;28;27;08

Geoff Nielson

You know, I'm still trying to wrap this up, but I did want to I did want to mention, because there's a there's a piece there that's unintuitive to some people, which is actually AI makes climate change and the, you know, ecology worse because of what's going on with data centers. The counterpoint I've heard to that is I is actually the only way we're going to be able to solve this problem.

01;28;27;09 - 01;28;31;03

Geoff Nielson

And so we just have to sort of bite the bullet on that. Do you agree with that?

01;28;31;08 - 01;29;07;27

Blaise Aguera y Arcas

I do. I mean, I, I'm, I'm very concerned with decarbonizing our, our, our energy systems. People who have the impression that the amount of energy that we are burning on Earth is the problem. Just don't have the science, right? The problem is carbon dioxide. It's unambiguously carbon dioxide. And and if we are replacing, you know, carbon burning plants with, with other forms of, of renewable energy, then, that's a very, a very obviously positive thing.

01;29;07;28 - 01;29;27;18

Blaise Aguera y Arcas

I mean, I find it so ironic that, you know, China, which was a huge polluter and, you know, not not particularly like great on a variety of ecological measures, you know, is also the one that is decarbonizing the fastest by far. It's also the leading producer of solar cells that Texas, which is hardly been like, you know, bastion of of ecology.

01;29;27;23 - 01;29;49;27

Blaise Aguera y Arcas

You know, politically speaking, is making much more progress in California, you know, at using solar and so on, because it's just cheaper, you know. So, so, I really do think that, you know, leaning into those things and building is, is fundamental. You know, you can't just say like, you know, use, use less or whatever a AI is, is absolutely a part of the solution.

01;29;49;28 - 01;30;13;15

Blaise Aguera y Arcas

Be a lot of the infrastructure over all that has to happen, you know, is, you know, is going to be accelerated, not slowed down by, by demand. And also, of course, you know, I, you know, one of my big projects is, is sun capture, which we haven't talked about, but this is putting putting AI in space where there is more than abundant energy.

01;30;13;18 - 01;30;25;14

Blaise Aguera y Arcas

And moving toward a space based energy ecology for for AI and for computing in general, I think is also, in the long term, going to be a big part of the solution here.

01;30;25;16 - 01;30;40;10

Geoff Nielson

Well, I promised I would be conscious of your time, and I know we're at the end of it. There's so much more we could unpack that we'll have to do another time. But blaze, this has truly been a pleasure. I really, really enjoyed our conversation. Thank you so much for your insight and for joining. It left me with a with a ton to think about.

01;30;40;11 - 01;30;49;08

Blaise Aguera y Arcas

Oh, thank you so much Jeff. This has been a really fun a really fun discussion. And you've yeah, your questions have been really on point, I think.

01;30;49;11 - 01;31;13;23

Geoff Nielson

Most viewers don't know this, but Digital Disruption is developed by Infotech Research Group, a leading advisor to technology leaders around the world. If that's not you, you don't need to care. So skip ahead and enjoy our content. But if you are a technology leader, Infotech helps IT teams get projects done faster, better, and at a lower cost. Infotech provides unlimited access to practical tools and expert guidance.

01;31;13;23 - 01;31;28;18

Geoff Nielson

You need to execute at a fraction of the cost of traditional consulting. No matter the project, from AI strategy to cybersecurity to vendor negotiation, Infotech has you covered. Check it out at the link below and don't forget to like and subscribe!