Digital Disruption

00;00;01;01 - 00;00;04;10

I would

eventually be more capable than Noni eyes,

00;00;04;20 - 00;00;07;18

and they would

therefore have a run the world

00;00;07;18 - 00;00;09;16

and they would dominate

what happened in the world,

00;00;09;16 - 00;00;11;23

and then the rest of the world

would be at their mercy.

00;00;11;23 - 00;00;14;18

And that seems a pretty robust prediction

for the future,

00;00;14;18 - 00;00;15;22

because that's

what's happened in the past.

00;00;15;22 - 00;00;19;06

You know, industrialists replaced

farmers, farmers replaced foragers.

00;00;19;06 - 00;00;21;14

Foragers replaced other primates.

00;00;21;14 - 00;00;21;21

Yeah.

00;00;21;21 - 00;00;24;20

You just don't expect that in the long

run, the new form comes to dominate.

00;00;25;11 - 00;00;28;11

And then old people should try to like

00;00;28;12 - 00;00;31;14

switch over

and join the new world as best they can.

00;00;32;23 - 00;00;35;23

This is a show about the future of tech

and the future of work.

00;00;36;01 - 00;00;39;10

I'm Geoff Nielsen and today

my guest is Robyn Hanson,

00;00;39;23 - 00;00;43;03

a revered economist,

professor, speaker, and author.

00;00;43;12 - 00;00;47;11

I prefer to think of Robyn as a thinker

with one of the most fascinating

00;00;47;11 - 00;00;50;21

minds and approaches to AI

and the future of humanity.

00;00;51;08 - 00;00;53;24

He's the author of The Age of Em,

a detailed

00;00;53;24 - 00;00;56;26

look at the future of AI driven

brain emulation,

00;00;57;02 - 00;01;01;01

and has written and spoken

extensively about the arc of civilization.

00;01;01;08 - 00;01;04;25

This is a guy who thinks it's inevitable

that AI replaces humans,

00;01;05;00 - 00;01;08;19

not just in the workplace,

but almost certainly as a species.

00;01;09;09 - 00;01;12;13

Every conversation I've heard with him

has been mind expanding

00;01;12;13 - 00;01;14;21

and change

the way I think about the world.

00;01;14;21 - 00;01;17;16

So I couldn't

wait to have him on the show.

00;01;17;16 - 00;01;20;23

I want to ask him

what the future he sees for us humans is

00;01;21;02 - 00;01;24;14

both in the next handful of months

and in the next million years.

00;01;24;29 - 00;01;27;19

Should we be excited

for where we seem to be headed?

00;01;27;19 - 00;01;30;11

Or do we need to radically alter

our trajectory

00;01;30;11 - 00;01;33;06

if we're going to have a future worth

living in at all?

00;01;33;06 - 00;01;34;02

Let's find out.

00;01;37;26 - 00;01;39;14

Robyn, thanks so much for joining today.

00;01;39;14 - 00;01;40;17

Really excited to have you.

00;01;40;17 - 00;01;44;17

And you know, you're

someone who has such interesting views on

00;01;44;17 - 00;01;47;00

such a wide variety of topics

and are clearly someone

00;01;47;00 - 00;01;50;27

who has spent a lot of time reflecting on

and refining your own

00;01;50;27 - 00;01;54;21

sort of understanding of, of humanity,

the world, the universe.

00;01;54;21 - 00;01;55;12

And so.

00;01;55;12 - 00;01;59;13

You know, maybe I'll start with a

big question, which is you've said before

00;01;59;14 - 00;02;04;09

that sometimes you catch flack

for talking about what comes next

00;02;04;09 - 00;02;06;21

in the arc of civilization

or the future of humanity,

00;02;06;21 - 00;02;08;25

because what you expect to happen

00;02;08;25 - 00;02;11;05

is not always

the same as what you would like to happen

00;02;11;05 - 00;02;12;21

or what anybody else would like to happen.

00;02;12;21 - 00;02;19;12

So with that in mind, where do you see us

as a civilization going from here?

00;02;21;27 - 00;02;22;18

So I have this

00;02;22;18 - 00;02;25;18

book from ten years ago

called The Age of M,

00;02;25;20 - 00;02;28;27

and it describes

this world of a certain kind of AI,

00;02;29;13 - 00;02;33;11

and I think it was a pretty positive image

that many other people

00;02;33;11 - 00;02;34;19

thought as dystopian.

00;02;34;19 - 00;02;38;18

So that might be the kind of thing you're

referring to where I was describing,

00;02;39;21 - 00;02;43;04

what I think is a plausible

but pretty strange scenario,

00;02;43;04 - 00;02;45;26

because I just think the future is likely

to be pretty strange.

00;02;45;26 - 00;02;48;11

That's just my default.

00;02;48;11 - 00;02;52;25

Now, in the last few years,

I've come to, fear

00;02;52;25 - 00;02;56;24

and maybe even expect that, this future

I wrote about in

00;02;56;24 - 00;02;59;24

this book will be put off

for longer than I had hoped.

00;03;00;23 - 00;03;02;01

That is our fear.

00;03;02;01 - 00;03;03;21

Go ahead. Sorry.

00;03;03;21 - 00;03;07;10

I was going to say let's, let's maybe talk

just briefly about the future.

00;03;07;10 - 00;03;10;02

Envision in your book, the age of M and M,

00;03;10;02 - 00;03;13;02

I understand

is is short for emulated minds.

00;03;13;12 - 00;03;15;11

So can you tell us, you know what?

00;03;15;11 - 00;03;17;05

What was this future that you envisioned?

00;03;17;05 - 00;03;21;04

And was it designed to be a sign of

what's to come or just an interesting

00;03;21;04 - 00;03;22;24

thought experiment?

00;03;22;24 - 00;03;27;17

So for a long time, many people have

thought that the next really big change

00;03;28;02 - 00;03;31;02

likely to happen would be some form

of artificial intelligence.

00;03;31;07 - 00;03;34;29

And, you know, in the last few years,

that becomes an even more popular opinion.

00;03;35;11 - 00;03;36;24

But this was ten years ago.

00;03;38;13 - 00;03;41;15

And a big problem with analyzing

the future of AI is just

00;03;41;15 - 00;03;44;17

there's such a vast space of possible

AI as you can imagine.

00;03;45;03 - 00;03;48;22

And even today, with LMS, there's still

a vast space of what they could become.

00;03;50;00 - 00;03;53;29

Now there's one kind of AI,

which is much easier to predict

00;03;54;02 - 00;03;57;14

than the others, which is where you take

an individual human brain

00;03;57;24 - 00;04;00;26

and then you make a computer

emulation of that brain,

00;04;01;24 - 00;04;04;18

and then you run

that and make copies, etc..

00;04;04;18 - 00;04;05;01

The reason

00;04;05;01 - 00;04;08;09

that's easier to predict is we just know

so many things about human brains

00;04;09;11 - 00;04;12;06

and their trade offs

and their inclinations, etc.

00;04;12;06 - 00;04;15;03

and so now we can take all that things

we know about humans,

00;04;15;03 - 00;04;18;26

and then lay out a much more detailed

scenario of how that plays out.

00;04;18;27 - 00;04;23;16

Now, that's not obviously the first kind

of AI we're going to get.

00;04;24;26 - 00;04;27;24

And so that means this book is somewhat

speculative,

00;04;27;24 - 00;04;29;23

assuming it's the first kind of AI,

00;04;29;23 - 00;04;33;01

but it's not crazy to think

that it would be the first human level.

00;04;33;02 - 00;04;36;02

I still think that's a substantial chance.

00;04;36;04 - 00;04;41;06

And so, you know, it's

both a likely scenario

00;04;41;06 - 00;04;44;08

that is like I'd say if it's worth

having 100 books in the future,

00;04;44;08 - 00;04;47;08

it's worth having a book on a future

that has a 1% chance.

00;04;48;16 - 00;04;51;00

And this, I think, meets that criteria.

00;04;51;00 - 00;04;53;03

It's got at least a 1% chance.

00;04;53;03 - 00;04;56;20

So in that way, it's a serious attempt

to actually think about the future.

00;04;57;01 - 00;05;00;19

But in addition, it's

also an attempt to think through

00;05;00;19 - 00;05;04;04

at least one specific

AI scenario in a lot more detail,

00;05;04;28 - 00;05;07;28

and to show that it's possible

to take specific

00;05;08;12 - 00;05;10;27

technological scenarios

and work a lot of detail about them.

00;05;10;27 - 00;05;13;27

So in my life, many people have just said

00;05;14;00 - 00;05;16;22

it's really impossible

to really say much about the future,

00;05;16;22 - 00;05;18;05

or there's so many possibilities.

00;05;18;05 - 00;05;19;22

It's spread out so rapidly,

00;05;19;22 - 00;05;22;28

the such a weird high dimensional space

that you really just can't say anything.

00;05;22;28 - 00;05;26;26

Therefore, science fiction or speculation

is just the best anybody can do.

00;05;27;16 - 00;05;29;16

And I was trying to show no,

00;05;29;16 - 00;05;32;20

you can actually take

a relatively specific set of assumptions

00;05;32;20 - 00;05;35;25

and work out a lot of detail,

using our best understanding

00;05;35;25 - 00;05;38;25

of lots of different areas of expertise.

00;05;39;00 - 00;05;40;17

And that's

what I attempted to do in the book.

00;05;40;17 - 00;05;44;12

So the book, many people say correctly

that it just has more detail

00;05;44;12 - 00;05;45;19

than they wanted.

00;05;45;19 - 00;05;47;28

They would like something more like a,

you know, a science fiction movie

00;05;47;28 - 00;05;51;07

or something that has some detail, but,

you know, full of plot and etc.

00;05;51;07 - 00;05;55;01

but I crammed this book full of detail

because I was trying to prove just how

00;05;55;01 - 00;05;56;03

much detail I could give you.

00;05;57;07 - 00;05;58;24

And that's I think I succeeded.

00;05;58;24 - 00;06;01;26

I think it gave you a lot of detail.

00;06;03;03 - 00;06;04;05

It's interesting

00;06;04;05 - 00;06;08;00

you describe the book is,

as you know, having optimism in it

00;06;08;00 - 00;06;13;05

and saying that other people

maybe see it as more of a state to fear.

00;06;13;18 - 00;06;15;06

And I'm curious

if you can walk us through,

00;06;15;06 - 00;06;18;27

I mean, to me, just sort of reflexively

the idea of being able

00;06;18;27 - 00;06;20;20

to emulate a human brain like my mind

00;06;20;20 - 00;06;24;02

very quickly goes to the matrix

or some sort of simulation,

00;06;24;02 - 00;06;29;12

or the idea that that the emulation itself

becomes a form of a prison for folks.

00;06;29;12 - 00;06;31;27

But it doesn't sound like you see it

that way. I've described it that way.

00;06;31;27 - 00;06;34;09

So what's the I guess,

what's the upshot of being able

00;06;34;09 - 00;06;38;05

to do this for the average person

or for society more broadly?

00;06;39;14 - 00;06;41;02

Well, I mean,

00;06;41;02 - 00;06;45;17

it is a way in which our descendants

become much more capable than we are now,

00;06;46;13 - 00;06;50;00

and their civilization

becomes much larger than ours is now.

00;06;51;01 - 00;06;54;18

The members of that civilization

have a different world they live in,

00;06;55;05 - 00;06;58;05

and it's not a version of our utopias,

00;06;58;19 - 00;07;01;21

but that's not very plausible anyway.

00;07;02;18 - 00;07;04;26

So I think one of the things

people are like

00;07;04;26 - 00;07;07;26

is, is just not their idea of a utopia.

00;07;09;12 - 00;07;12;21

And another is that

there is a lot of capitalism going on.

00;07;12;26 - 00;07;15;05

And for many people,

they wanted capitalism to fade away

00;07;15;05 - 00;07;17;18

and just, you know, not matter

much anymore.

00;07;17;18 - 00;07;20;05

Let's get over that.

And this is a world full of capitalism.

00;07;20;05 - 00;07;23;05

So I think that bugs a lot of people.

00;07;24;06 - 00;07;26;27

I'll take us on a slight detour,

because this is something that tends

00;07;26;27 - 00;07;31;06

to come up in a lot of conversations

I have about how much capitalism

00;07;31;12 - 00;07;35;05

will be found in the in the future,

and that tends to be

00;07;35;24 - 00;07;39;09

related to whether the person

I'm talking to sees capitalism

00;07;39;09 - 00;07;42;18

as like a bug

or a feature of human civilization.

00;07;42;18 - 00;07;45;25

And I was I was speaking with a gentleman

a few weeks ago who was saying, you know,

00;07;45;25 - 00;07;49;03

oh, well,

why does money need to exist anyway?

00;07;49;03 - 00;07;53;10

And I felt very much the need to jump in

and say an awful lot about that.

00;07;53;10 - 00;07;56;10

But it sounds like to you, Robin,

capitalism

00;07;56;10 - 00;07;59;12

and, you know, currency

and the economy are very much a feature.

00;07;59;12 - 00;08;03;08

And you expect to see,

I don't know as much of them, more of them

00;08;03;08 - 00;08;04;29

rather than less of them. Is that fair?

00;08;06;02 - 00;08;07;29

So there are many

00;08;07;29 - 00;08;11;26

innovations that humans have pioneered

in the last 100,000 years.

00;08;13;05 - 00;08;16;22

You know, they include cities, trade,

00;08;17;19 - 00;08;20;17

law, language

00;08;20;17 - 00;08;22;18

norms,

00;08;22;18 - 00;08;25;18

you know, research.

00;08;26;16 - 00;08;28;26

These innovations,

I think are going to last.

00;08;28;26 - 00;08;31;11

These aren't just temporary things.

00;08;31;11 - 00;08;33;17

Humans have pioneered many big,

00;08;33;17 - 00;08;36;17

innovative concepts that will just,

00;08;36;20 - 00;08;39;09

be there into the indefinite future.

00;08;39;09 - 00;08;41;19

We're going to use language

for a long time in the future.

00;08;41;19 - 00;08;43;23

We're going to have lore

for a long time in the future.

00;08;43;23 - 00;08;46;22

And yes, we'll have trade

for a long time in the future.

00;08;46;22 - 00;08;50;00

And I think we're going to have

structured, even bureaucratic

00;08;50;00 - 00;08;53;03

organizations for a long time

will have governance.

00;08;54;01 - 00;08;56;10

And so, yes, we'll have capitalism.

00;08;56;10 - 00;08;58;14

This isn't like some temporary thing.

00;08;58;14 - 00;09;02;02

This is one of the fundamental innovations

that humans have accumulated.

00;09;02;02 - 00;09;04;29

That's just going to be a powerful thing

we use in a great many ways.

00;09;06;29 - 00;09;08;01

Including money.

00;09;08;01 - 00;09;09;26

Yes, money is going to be

there in the future.

00;09;09;26 - 00;09;11;20

Money is a powerful innovation

00;09;11;20 - 00;09;14;20

that, you know, we can't do nearly

as much as we do now without it.

00;09;15;05 - 00;09;16;11

And it's not going to go away.

00;09;16;11 - 00;09;19;10

It's it's value is quite robust.

00;09;19;24 - 00;09;21;18

I'm I'm inclined to agree with you.

00;09;21;18 - 00;09;23;19

And, you know,

there's a word you didn't use there,

00;09;23;19 - 00;09;27;05

but that I think underscores

the spirit of capitalism.

00;09;27;17 - 00;09;30;17

And why it's not going to go away,

which is competition.

00;09;30;18 - 00;09;34;21

And competition

is kind of a fundamental force in nature.

00;09;34;26 - 00;09;36;22

That competition

is much harder than humans.

00;09;36;22 - 00;09;39;15

Like we humans can't take credit

for competition, maybe.

00;09;39;15 - 00;09;40;22

Oh, I understand some new forms of it.

00;09;40;22 - 00;09;42;02

But yeah, I was trying to list

00;09;42;02 - 00;09;45;02

things that are new with humans,

but like competition goes way, way back,

00;09;46;10 - 00;09;46;29

right?

00;09;46;29 - 00;09;50;08

But to me,

competition is the reason that capitalism,

00;09;50;22 - 00;09;52;11

or at least one of the reasons capitalism

00;09;52;11 - 00;09;55;21

is successful, as successful as it is,

and why it'll stick around.

00;09;56;08 - 00;09;59;08

And in many people's utopias,

competition goes away.

00;10;00;17 - 00;10;01;18

But not yours.

00;10;01;18 - 00;10;03;13

No. Not now.

00;10;03;13 - 00;10;08;09

I mean, maybe it's not a utopia, a real

realistic o.p or something, but, yeah.

00;10;08;20 - 00;10;12;13

So, it was the sense in which I was trying

to describe a realistic future,

00;10;12;13 - 00;10;16;20

and that meant I needed to sort of apply

all the constraints I understand

00;10;16;20 - 00;10;18;13

about the way our world is structured

00;10;18;13 - 00;10;21;08

and how that might change

in this different scenario.

00;10;21;08 - 00;10;25;29

And, that meant I have to,

not sort of wishfully Wishfully

00;10;25;29 - 00;10;28;28

imagine all sorts of problems go away

because we have some future solution.

00;10;28;28 - 00;10;32;24

I say, no, no, like we will be dealing

with all these basic problems

00;10;32;24 - 00;10;35;01

that I don't think will go away

through mechanisms

00;10;35;01 - 00;10;37;15

we've been developing so far

and then new ones, hopefully.

00;10;39;14 - 00;10;40;16

If you don't work in

00;10;40;16 - 00;10;44;11

enterprise technology, you can skip ahead

or take a quick brain creation.

00;10;44;17 - 00;10;47;18

But if you do, Infotech

is your secret weapon.

00;10;48;00 - 00;10;52;28

Infotech helps IT teams get projects done

faster, better, and at a lower cost.

00;10;53;06 - 00;10;54;23

Whether you're rolling out new tech,

00;10;54;23 - 00;10;57;20

improving processes

or just looking for cost savings.

00;10;57;20 - 00;11;02;08

Infotech provides unlimited access

to practical tools and expert guidance

00;11;02;08 - 00;11;05;28

you need to execute at a fraction

of the cost of traditional consulting,

00;11;06;12 - 00;11;10;06

no matter the project from AI strategy

to cybersecurity

00;11;10;06 - 00;11;13;05

to vendor negotiation, Infotech

has you covered.

00;11;13;13 - 00;11;16;15

Check it out at the link below

and don't forget to like and subscribe!

00;11;16;29 - 00;11;19;27

So I guess maybe just to summarize

a few of the themes we've talked about

00;11;19;27 - 00;11;25;14

so far, it sounds like you are broadly

optimistic about the future.

00;11;25;14 - 00;11;29;02

Or at least you know what

what we're building toward,

00;11;29;19 - 00;11;33;29

not because it adheres to most people's

definition of a utopia, but

00;11;33;29 - 00;11;37;14

because it extends our capabilities and,

00;11;37;29 - 00;11;41;25

you know, is a form of progress, of human

civilization.

00;11;42;07 - 00;11;44;16

Is that fair or

what would you what would you add to that?

00;11;47;01 - 00;11;48;13

So, we can

00;11;48;13 - 00;11;52;08

talk a bit about this

more in a few minutes, but I actually see

00;11;52;08 - 00;11;55;27

just how much culture and values

have changed over human history.

00;11;56;14 - 00;11;59;24

And I don't put that much stock

in wanting the future to share

00;11;59;24 - 00;12;03;12

all my cultural values

and all my habits of life.

00;12;04;06 - 00;12;07;26

The main thing I want is people to just,

you know, be alive to have,

00;12;07;26 - 00;12;10;10

you know, things are trying to do things

they get value of.

00;12;10;10 - 00;12;15;23

And then collectively, humanity

to grow in capacity and wisdom and,

00;12;16;24 - 00;12;18;27

spatial extent even in temporal expansion.

00;12;18;27 - 00;12;23;20

I just want us to get big, as long

as it's full of parts that are similarly

00;12;23;20 - 00;12;27;12

as content or happy, or find the lives

meaningful as most humans ever did.

00;12;28;20 - 00;12;29;06

I'm not.

00;12;29;06 - 00;12;32;06

I mean, maybe you could find a way

that humans could just find their lives

00;12;32;09 - 00;12;34;27

a million times

more meaningful than they do now.

00;12;34;27 - 00;12;37;15

That sure would be good.

But I'm more just going.

00;12;37;15 - 00;12;39;11

Let's just have the usual sort of lives

00;12;39;11 - 00;12;41;22

we've always had,

but have a lot more of them.

00;12;41;22 - 00;12;45;23

I will go a lot more places and have a lot

more capacity to do a lot more things.

00;12;48;00 - 00;12;49;13

I like that, and it makes sense to me.

00;12;49;13 - 00;12;52;13

And I can

I can see the optimism in that and why,

00;12;52;16 - 00;12;55;01

you know, it's, as you said, a reasonable

and kind of rational

00;12;55;01 - 00;12;58;01

take on, on the future versus

building some sort of,

00;12;58;11 - 00;13;02;20

utopia that we can't even really seem

to define and always seems like

00;13;02;20 - 00;13;06;25

it's this weird, out of step jump

from where we are right now.

00;13;06;25 - 00;13;10;03

And the other the building blocks

that have worked for us so far.

00;13;10;24 - 00;13;12;27

I am curious, though,

because one of the things

00;13;12;27 - 00;13;14;08

that you've written about

and talked about,

00;13;14;08 - 00;13;17;29

you know, at length in the past,

is the notion of the great Filter.

00;13;18;17 - 00;13;21;09

So talking,

I guess, on a on a broader scale,

00;13;21;09 - 00;13;24;23

the fact that we haven't seen

a lot of signs of intelligent life.

00;13;24;23 - 00;13;28;18

So can you maybe just broadly talk

about what The Great Filter is?

00;13;29;09 - 00;13;32;24

If you have any, you know, predictions or,

00;13;32;25 - 00;13;36;11

you know, your personal beliefs about what

that means for us in the future.

00;13;36;11 - 00;13;38;08

And if you're you're thinking

has changed on it

00;13;38;08 - 00;13;40;10

since you were originally

writing about the topic.

00;13;41;14 - 00;13;44;23

So as we've been talking about,

we see in the past

00;13;44;23 - 00;13;49;08

a huge increase and expansion of humanity,

its scope, its capabilities,

00;13;49;23 - 00;13;53;19

and then its natural project

that trend into the future to imagine us

00;13;53;19 - 00;13;57;28

becoming in a relatively short

time, cosmological as cosmological.

00;13;57;29 - 00;14;01;01

You know,

the universe is 15 billion years old.

00;14;01;05 - 00;14;05;05

So within a million years, which is tiny

compared to 15 billion years,

00;14;05;15 - 00;14;08;29

we could see our descendants

becoming vastly more

00;14;08;29 - 00;14;13;17

influential and visible

than, we have been so far.

00;14;14;02 - 00;14;17;27

So within a million years,

we could easily colonize the galaxy

00;14;17;27 - 00;14;22;04

and rearrange it and just make it

substantially different than it is now.

00;14;23;04 - 00;14;25;07

And if we were a galaxy away

00;14;25;07 - 00;14;28;07

looking at our galaxy,

it would look different.

00;14;28;12 - 00;14;30;06

We're not sure exactly

how it looked from it.

00;14;30;06 - 00;14;33;02

It's pretty unlikely they'll look

exactly the same as it is now.

00;14;33;02 - 00;14;36;17

If we decide to just make massive

use of it and rearrange everything.

00;14;37;13 - 00;14;40;20

So when we look out into the universe,

we think, oh, that's what we could

00;14;40;20 - 00;14;42;02

be in only a million years.

00;14;43;15 - 00;14;44;15

Let's

00;14;44;15 - 00;14;47;02

maybe somebody else got there,

you know, a million years before

00;14;47;02 - 00;14;50;06

us would be a tiny fraction

of the universe history.

00;14;50;06 - 00;14;52;16

Surely somebody got there before

us. So let's go.

00;14;52;16 - 00;14;54;06

I look for him and we look them.

00;14;54;06 - 00;14;56;02

We don't see them.

00;14;56;02 - 00;14;58;05

The universe doesn't seem to contain

00;14;58;05 - 00;15;02;04

any of these extensions of our future

that we hope we might become.

00;15;02;26 - 00;15;04;26

And that's this key question.

00;15;04;26 - 00;15;07;15

Fermi's question where is everybody?

00;15;07;15 - 00;15;11;00

And I reframe that question as well.

00;15;11;17 - 00;15;14;17

Everybody that you might see out

there, though, you don't see

00;15;14;22 - 00;15;18;05

they would have started as some simple,

you know, dead matter somewhere

00;15;18;05 - 00;15;22;15

in the universe that slowly evolved over

eons to become that thing.

00;15;23;01 - 00;15;25;04

So the path that leads from simple,

that matter

00;15;25;04 - 00;15;28;04

which is everywhere in the universe

to this thing that we don't see,

00;15;28;17 - 00;15;31;17

this path

must be really unlikely to go down.

00;15;32;03 - 00;15;34;09

Otherwise we'd see it all over the place.

00;15;34;09 - 00;15;36;18

So I call the Great Filter.

00;15;36;18 - 00;15;39;23

Whatever along this path

is stopping things from going down

00;15;39;23 - 00;15;42;23

this path,

filtering things off of this path.

00;15;42;25 - 00;15;45;01

The Great Filter is

whatever is the obstacles.

00;15;45;01 - 00;15;49;02

Now, a key question about the filter

is how how big it is.

00;15;49;02 - 00;15;52;04

But let's say

we see ten to the 24 planets out there

00;15;52;04 - 00;15;53;21

that could have started this thing

that we don't see.

00;15;53;21 - 00;15;56;21

The maybe the Great Filter

is at least a factor of ten to the 24,

00;15;56;26 - 00;15;59;00

at least up to a certain

time point in history.

00;16;00;08 - 00;16;00;17

So that's

00;16;00;17 - 00;16;04;06

a really big it could be much bigger,

but that's at least how big it is.

00;16;04;06 - 00;16;07;05

And we ask how much that filter

would be passed now.

00;16;07;10 - 00;16;11;10

And so say it was ten to the 22 behind us

and only ten to the two ahead of us.

00;16;11;10 - 00;16;12;27

We would have gone through

almost all the filter,

00;16;12;27 - 00;16;16;06

but still ten to the two in front of us

means there's only a 1% chance

00;16;16;18 - 00;16;17;20

our descendants would become

00;16;17;20 - 00;16;20;28

this big visible thing,

which is pretty bad news about our future.

00;16;21;15 - 00;16;24;15

So it's really important

how much is left in front of us,

00;16;24;29 - 00;16;26;15

even if most of it's behind us.

00;16;26;15 - 00;16;27;25

How much is in front now?

00;16;27;25 - 00;16;31;09

The easiest place to put the filter

behind us is at the very earliest stages,

00;16;31;09 - 00;16;34;09

because we understand those the least

and they seem really hard.

00;16;34;20 - 00;16;35;27

And that predicts we'll just see

00;16;35;27 - 00;16;38;27

no other life

anywhere in the universe of any sort.

00;16;38;28 - 00;16;42;06

If we see some bacteria or something else

in the life that revises, we go, oh,

00;16;42;18 - 00;16;43;23

those earliest steps.

00;16;43;23 - 00;16;44;29

Not quite so hard as we thought.

00;16;44;29 - 00;16;48;15

That must mean the filter is either in

some of the later steps

00;16;49;03 - 00;16;53;11

after that amoeba between there and now,

or between now and our future

00;16;53;29 - 00;16;57;08

and of course, we're really most worried

about the future filters,

00;16;57;08 - 00;16;59;22

the filters

that could stand in front of us.

00;16;59;22 - 00;17;01;03

You preventing has become big.

00;17;01;03 - 00;17;04;18

Now, it is true that one of the possible

future filters could be that

00;17;04;18 - 00;17;09;07

we just stop growing and then stay small,

and then are happy forever.

00;17;10;14 - 00;17;11;14

So it's not

00;17;11;14 - 00;17;14;08

necessarily apocalypse

that prevents us from going out there.

00;17;14;08 - 00;17;16;24

But honestly, when you think

about all sorts of scenarios,

00;17;16;24 - 00;17;17;22

most of the scenarios

00;17;17;22 - 00;17;20;21

that would prevent us from getting big

also would probably just kill us off.

00;17;21;16 - 00;17;24;08

So that's why

you don't want to kind of worry about why.

00;17;24;08 - 00;17;26;24

Why can't we get back?

00;17;26;24 - 00;17;30;06

So, I have in the past thought

00;17;31;07 - 00;17;33;29

the future filter looks,

you know, small compared to the past,

00;17;33;29 - 00;17;36;01

but it still could be substantial.

00;17;36;01 - 00;17;37;19

And I think it's really hard

00;17;37;19 - 00;17;40;19

to come up with things that would, like,

kill us all off.

00;17;41;06 - 00;17;43;17

So, you know,

even if you killed off all humans

00;17;43;17 - 00;17;47;16

or all those other apes and kill off all

the land animals, and there's dolphins.

00;17;47;16 - 00;17;51;19

I mean, like, if you think about over

the next billion years

00;17;52;16 - 00;17;56;09

such that our descendants could do stuff,

it's really hard to, like,

00;17;56;26 - 00;18;00;27

kill off our potential now so much that

something else couldn't rise again.

00;18;01;23 - 00;18;05;04

And that means it's kind of hard

to imagine a really tough filter

00;18;05;17 - 00;18;06;28

in front of us.

00;18;06;28 - 00;18;10;04

But when I think about it more, I go,

oh well, the most likely filter

00;18;10;04 - 00;18;10;27

would be some way

00;18;10;27 - 00;18;16;00

in which we form a stable world system

that then blocks further expansion.

00;18;17;13 - 00;18;17;22

And I

00;18;17;22 - 00;18;20;24

think that would have to be my guess

about the future, because likely future

00;18;20;24 - 00;18;24;04

filter like our descendants, grow

to be more capable.

00;18;24;04 - 00;18;26;24

They form some sort of world

governments and world culture.

00;18;26;24 - 00;18;29;17

They form some norms about

what kinds of things are allowed or not,

00;18;29;17 - 00;18;32;19

and then they stay functional enough

to enforce these rules.

00;18;34;06 - 00;18;37;06

And those rules would then

prevent expansion.

00;18;37;26 - 00;18;40;22

And that's something I worry about.

00;18;40;22 - 00;18;41;24

And we could talk more about,

00;18;41;24 - 00;18;44;24

like why that might happen

and then how it might play out.

00;18;44;24 - 00;18;50;14

But I do worry that's my biggest worry

about the future, is that we will create

00;18;50;14 - 00;18;53;19

a world system of sorts

that's competent enough

00;18;53;19 - 00;18;56;19

to enforce some strong world rules.

00;18;57;15 - 00;19;00;11

And, you know, which means it

00;19;00;11 - 00;19;04;23

never gets that much bigger,

but it also doesn't get that much smaller.

00;19;06;17 - 00;19;08;23

It's fascinating to me that

00;19;08;23 - 00;19;11;26

you were framing that as kind of your

biggest concern about the future, because,

00;19;12;06 - 00;19;12;17

you know,

00;19;12;17 - 00;19;15;21

when I think about The Great Filter

and maybe this is just a product

00;19;15;21 - 00;19;18;25

of the time at which we're talking,

you know, one of my concerns

00;19;18;25 - 00;19;22;25

and one of the, you know, narratives

you hear these days is that, you know,

00;19;23;13 - 00;19;27;16

it's entirely plausible

that AI will be the great Filter

00;19;27;16 - 00;19;31;02

for us, that we're going to create,

you know, accidentally or deliberately.

00;19;32;00 - 00;19;35;29

That is tell me what's about

what would make our galaxy really visible

00;19;35;29 - 00;19;36;25

from a long way away

00;19;36;25 - 00;19;39;04

if I take over

and even kill all the humans,

00;19;39;04 - 00;19;41;01

as long as they go on

to make it really visible.

00;19;41;01 - 00;19;44;00

Galaxy, that's not the Great Filter.

00;19;44;01 - 00;19;46;22

The Great Filter would have to be

some process that

00;19;46;22 - 00;19;49;10

prevents them from getting big.

00;19;49;10 - 00;19;51;22

Whether they kill off

humans is pretty irrelevant

00;19;51;22 - 00;19;53;18

to whether they get big again later.

00;19;53;18 - 00;19;57;07

So it would be pretty bad if I were to

kill off humans, I'll admit that.

00;19;57;18 - 00;20;01;15

But that's just still

that plausibly part of the Great Filter.

00;20;01;15 - 00;20;03;20

When you're trying to explain

why the universe is empty,

00;20;03;20 - 00;20;05;02

the fact that there's a place out there

00;20;05;02 - 00;20;06;23

where some people

there killed off the other people

00;20;06;23 - 00;20;09;07

there isn't really part

of, why can't we see them,

00;20;11;06 - 00;20;11;18

right?

00;20;11;18 - 00;20;15;01

So, so for for the AI scenario

to become plausible,

00;20;15;06 - 00;20;18;06

I would have to not only basically kill

every living thing,

00;20;18;10 - 00;20;20;16

it would

then have to destroy itself as well.

00;20;20;16 - 00;20;24;22

And that limits permanently limit itself

to prevent people from growing.

00;20;25;17 - 00;20;28;17

And then that's the thing to wonder

how could that happen?

00;20;28;18 - 00;20;30;03

Right.

00;20;30;03 - 00;20;33;04

Well, and I guess the the optimist,

the optimistic case for that is

00;20;33;04 - 00;20;36;29

when we look at the possible scenarios

of the Great Filter, it sounds like and,

00;20;37;18 - 00;20;40;11

you know, I'm a convert to that,

that that scenario doesn't

00;20;40;11 - 00;20;43;11

sound particularly plausible.

00;20;44;08 - 00;20;44;24

Well, again,

00;20;44;24 - 00;20;48;04

over the last decade,

I think I have come to see the sort

00;20;48;04 - 00;20;51;11

of locked down prevention of growth

scenario as more plausible.

00;20;52;06 - 00;20;53;22

And so that's why I'm more worried about,

00;20;56;14 - 00;20;57;07

Right.

00;20;57;07 - 00;21;02;06

And one, you you used the timescale

earlier of a million years, which, which,

00;21;02;07 - 00;21;04;13

you know, as you said,

cosmological insignificant.

00;21;04;13 - 00;21;07;13

But for you and I talking here,

the scale of a human life,

00;21;07;14 - 00;21;11;05

you know, we're talking a very,

very large, you know, timescale

00;21;11;05 - 00;21;14;11

that's,

you know, a quite a multiple over the,

00;21;14;28 - 00;21;17;28

you know, the entire lifespan

of human civilization so far.

00;21;17;28 - 00;21;21;15

And so it sounds like you're accounting

for even if there's,

00;21;21;27 - 00;21;24;19

you know, massive global devastation.

00;21;24;19 - 00;21;25;29

Right.

00;21;25;29 - 00;21;26;08

Yeah. Right.

00;21;26;08 - 00;21;31;17

So it's really hard to imagine like

killing off 99% of us and that delaying

00;21;31;17 - 00;21;34;17

these things more than a million years,

which again, is a tiny amount of time.

00;21;35;10 - 00;21;39;27

So if you're going to kill off

a potential, you have to just kill off

00;21;39;27 - 00;21;43;09

lots of stuff on Earth,

maybe life on Earth.

00;21;43;19 - 00;21;43;26

Yeah.

00;21;45;13 - 00;21;46;00

Or you're going to

00;21;46;00 - 00;21;49;12

have to let it freeze

into something that prevents its growth.

00;21;49;24 - 00;21;53;27

That's the second scenario that

I find more plausible to worry about right

00;21;55;04 - 00;21;58;29

now. You know, we could keep the time

frame at a million years,

00;21;58;29 - 00;22;02;16

but one of the other things that's been

fascinating to me lately is with

00;22;02;28 - 00;22;06;22

this kind of rapid acceleration

in AI capabilities,

00;22;07;14 - 00;22;11;08

or at very least

the degree to which the average person

00;22;11;08 - 00;22;15;17

is able to experience some of these new

AI and generative AI capabilities.

00;22;16;15 - 00;22;19;01

There's been a lot more speculation about

00;22;19;01 - 00;22;22;19

what is the speed at which technology

improves from here.

00;22;22;19 - 00;22;25;20

And the flip side of this is, you know,

you have people saying that we're going

00;22;25;20 - 00;22;29;26

to get a million years versus of progress

in the next decade or so.

00;22;30;03 - 00;22;33;28

And I'm curious if you pay any heed to,

you know, those kind of thoughts

00;22;33;28 - 00;22;37;22

and maybe, maybe on the flip side of that,

asking you,

00;22;38;07 - 00;22;42;25

how do you see the next kind of 10

to 20 years playing out technologically?

00;22;44;28 - 00;22;46;21

So, I'm a

00;22;46;21 - 00;22;50;01

futurist and

I like to think about the long run future.

00;22;50;10 - 00;22;52;25

I'm not that interested

in the latest press releases

00;22;52;25 - 00;22;54;27

or whatever

that many people see as futurism.

00;22;54;27 - 00;22;56;26

I want to know about the long run.

00;22;56;26 - 00;23;01;02

And on reflection, humanity has this

enormous future, future potential.

00;23;01;02 - 00;23;03;01

Again, we see this empty universe

that's so vast

00;23;03;01 - 00;23;05;26

and we could go out and fill it up

and do stuff with it.

00;23;05;26 - 00;23;09;16

And I'm very interested in our achieving

that potential, our descendants,

00;23;09;16 - 00;23;13;09

and now not getting sidetracked

or just right now,

00;23;13;24 - 00;23;16;15

given how fast change has been

right lately,

00;23;16;15 - 00;23;20;28

it seems like I shouldn't be that greedy

for faster change.

00;23;21;10 - 00;23;22;14

Faster change can be fine,

00;23;22;14 - 00;23;26;20

but I more when I think about obstacles

that say on doublings.

00;23;26;20 - 00;23;29;19

So if I think about doubling the capacity

of humanity,

00;23;30;00 - 00;23;32;27

you know, in the last century

that happened every 20 years,

00;23;32;27 - 00;23;36;00

but a thousand years ago,

it happened every thousand years.

00;23;37;01 - 00;23;38;10

And then, you know,

00;23;38;10 - 00;23;41;10

20,000 years ago

that happened every quarter million years.

00;23;41;10 - 00;23;46;17

So the rate of doubling has been

greatly increased, which is great.

00;23;46;17 - 00;23;48;28

And then in the future,

the rate of doubling might go even faster.

00;23;48;28 - 00;23;52;17

I talk about an age of, maybe it would

double every month or even faster.

00;23;53;03 - 00;23;54;24

The M economy.

00;23;54;24 - 00;23;58;27

But on reflection, I'm

not that eager for faster change.

00;23;58;27 - 00;24;01;29

I'm more eager for reliable change.

00;24;02;17 - 00;24;06;07

I'm more when I think about what obstacles

would show up at every doubling

00;24;06;07 - 00;24;09;15

or within every doubling scale,

and how we're going to avoid those.

00;24;10;07 - 00;24;12;16

And that's more what I care about.

00;24;12;16 - 00;24;14;04

Thinking about less of speed.

00;24;14;04 - 00;24;18;09

So another way to think about speed is

many people will say, isn't it

00;24;18;09 - 00;24;21;22

a problem with technology changes

faster than society can adapt?

00;24;22;11 - 00;24;23;00

And I'd say

00;24;24;03 - 00;24;26;02

the rate at which society can adapt

has long

00;24;26;02 - 00;24;29;02

been a limiting factor in the rate

at which technology could change.

00;24;29;29 - 00;24;32;00

That is,

00;24;32;00 - 00;24;36;07

society as a whole needs to adapt

and to things together,

00;24;36;07 - 00;24;39;05

and that's actually a limit

on how fast everything can grow.

00;24;39;05 - 00;24;42;15

We've often had some parts of our society

grows much faster, everything else.

00;24;42;15 - 00;24;45;00

But, you know,

it reaches a limit and slows down

00;24;45;00 - 00;24;48;14

because other people,

other parts of society just don't keep up.

00;24;48;14 - 00;24;50;26

They don't speed up to match it.

00;24;50;26 - 00;24;54;02

Now, you could have a part of society

that starts to grow faster.

00;24;54;02 - 00;24;55;17

And then there's another part

that's slower,

00;24;55;17 - 00;24;57;26

and that other part

might just get dropped.

00;24;57;26 - 00;24;59;21

So for example,

with the Industrial Revolution,

00;24;59;21 - 00;25;03;05

there were many subsistence

farmers out there whose farming practices

00;25;03;05 - 00;25;06;05

just did not speed up

with the Industrial Revolution.

00;25;06;09 - 00;25;08;07

But that didn't slow

the industrial Revolution.

00;25;08;07 - 00;25;10;07

The industry just said, well,

forget about you.

00;25;10;07 - 00;25;11;27

We can make food other ways.

00;25;11;27 - 00;25;15;07

And they just went on to have faster

rates of change elsewhere. So

00;25;17;05 - 00;25;17;29

that's maybe what you

00;25;17;29 - 00;25;20;29

should more worry

about with speed rates of change.

00;25;20;29 - 00;25;23;24

You shouldn't worry that the world

will change faster than it can handle.

00;25;23;24 - 00;25;25;12

What you should worry

is maybe the world will change

00;25;25;12 - 00;25;28;21

faster than you can handle,

and you will just be shunted aside

00;25;28;21 - 00;25;31;26

because you're in the way if you

they let you be on the critical path.

00;25;32;22 - 00;25;35;22

So that's how I might think about AI.

00;25;35;24 - 00;25;38;28

I might be more worried about,

okay, say AI speeds up things.

00;25;39;08 - 00;25;43;07

What parts of our world will then just

get dropped from the part of the world

00;25;43;07 - 00;25;46;12

that's going faster

because it just can't keep up?

00;25;47;09 - 00;25;50;04

And do you want are you worried

that you might be part of that world

00;25;50;04 - 00;25;51;28

that gets dropped

because it can't keep up?

00;25;53;18 - 00;25;54;20

And then

00;25;54;20 - 00;25;57;09

you know how what part of this world

what what will be

00;25;57;09 - 00;26;00;14

the world that's integrated enough

that it all needs to grow together?

00;26;00;22 - 00;26;03;21

And then think about you know,

what could happen in that world.

00;26;06;00 - 00;26;07;26

That that framing

makes a lot of sense to me.

00;26;07;26 - 00;26;10;26

And I think it is like,

I think you hit on something

00;26;10;26 - 00;26;15;20

that is a really deep fear

for a lot of people, and that

00;26;16;23 - 00;26;19;14

I actually, in a big way,

kind of accelerates

00;26;19;14 - 00;26;23;17

that inequality and that, you know,

the people being dropped or the

00;26;24;01 - 00;26;27;01

or the parts of society being dropped

are just going to be

00;26;28;03 - 00;26;29;13

it's going to be more and more of them.

00;26;29;13 - 00;26;32;18

And I may be good, you know, at an at

its most extreme scenario,

00;26;33;08 - 00;26;37;17

I may be really good for like six guys,

you know, in the US

00;26;37;17 - 00;26;39;00

and maybe two in China.

00;26;39;00 - 00;26;41;20

And for everybody else,

you're in, you're in big trouble.

00;26;41;20 - 00;26;47;02

And that with the acceleration

of technological change,

00;26;47;25 - 00;26;50;21

we're narrowing the band of people

who are actually equipped

00;26;50;21 - 00;26;52;19

to take advantage

of that and come out ahead.

00;26;54;11 - 00;26;56;14

And we could end up I mean.

00;26;56;14 - 00;26;58;23

It's seems to me really small.

00;26;58;23 - 00;27;02;24

So the last two and I mean the last three

analogies, things

00;27;02;24 - 00;27;07;04

that happened in history was,

first of all, one set of human

00;27;07;04 - 00;27;10;10

species compared to other proto human

species, took off.

00;27;10;10 - 00;27;13;29

And compared to the other humans,

they contributed some DNA

00;27;13;29 - 00;27;17;22

to the main line,

but much less then with farming.

00;27;18;10 - 00;27;21;26

You know, places started to adopt farming

and then local hunter gatherers

00;27;21;26 - 00;27;24;26

were absorbed into that or displaced

00;27;24;26 - 00;27;27;16

and then with industry, northern Europe

00;27;27;16 - 00;27;30;20

started to grow rapidly and then pulled

the rest of the world together.

00;27;31;03 - 00;27;34;02

Now, in these last two transitions,

at least,

00;27;34;19 - 00;27;37;04

you know

large chunks of the world came along.

00;27;37;04 - 00;27;38;19

It wasn't six people.

00;27;38;19 - 00;27;42;21

It was very large coalitions of parts of

the world that managed to grow together,

00;27;43;01 - 00;27;44;29

all of northern Europe

and then all of Europe,

00;27;44;29 - 00;27;47;16

and then all of the West,

and then the whole world.

00;27;47;16 - 00;27;50;01

You know, we've pulled much of the world

along with us

00;27;50;01 - 00;27;53;03

now by the Industrial revolution, farming

revolution.

00;27;53;03 - 00;27;57;09

Again, it was more like a 5050 thing

when farmers came into the same area,

00;27;57;09 - 00;27;59;00

it was more 50% farmers

00;27;59;00 - 00;28;02;06

coming in and 50% recruiting the locals

and turning them into farmers.

00;28;03;01 - 00;28;05;24

And so that's a pretty large fraction

of the previous world

00;28;05;24 - 00;28;08;26

being recruited

into the new farming world.

00;28;09;12 - 00;28;10;10

With humans.

00;28;10;10 - 00;28;12;22

It might have been,

I don't know, maybe 10% of the proto

00;28;12;22 - 00;28;16;00

humans who took off is Homo sapiens,

and the rest got set aside.

00;28;16;00 - 00;28;18;28

So it's a smaller percentage,

but still it's 10%.

00;28;18;28 - 00;28;23;14

It's not six people out of a billion,

so the six people

00;28;23;14 - 00;28;26;27

out of 8 billion seems to be just crazy

small as a plausible fraction.

00;28;28;24 - 00;28;30;03

That that makes sense to me.

00;28;30;03 - 00;28;33;03

And I know we're you know, we're

speaking somewhat flippantly about,

00;28;33;07 - 00;28;36;15

you know, that these kind of winners

and losers on a historical scale.

00;28;36;27 - 00;28;38;00

So, so, so I'm curious

00;28;38;00 - 00;28;42;01

if we do believe, though, that these,

you know, these periods of, of,

00;28;42;14 - 00;28;45;19

you know, revolution in the,

I guess, the broad sense of the word,

00;28;46;17 - 00;28;49;17

you know,

if these periods create winners and losers

00;28;49;28 - 00;28;52;25

looking down the barrel of this one,

00;28;52;25 - 00;28;55;18

what can people or fragments

00;28;55;18 - 00;29;01;04

of society do, in your view,

to to make sure that they're more adaptive

00;29;01;04 - 00;29;04;04

or going to find themselves

on the right side of history.

00;29;04;08 - 00;29;08;03

So there's two main approaches

that have worked for a long time.

00;29;09;00 - 00;29;11;19

One is to,

you know, try to join the winners

00;29;12;22 - 00;29;13;01

if you.

00;29;13;01 - 00;29;15;23

So if you just insist that,

those people have a new thing.

00;29;15;23 - 00;29;17;18

But I don't like that

I'm going to do thing the old ways.

00;29;17;18 - 00;29;19;14

And later on, you lose you.

You go. That's not fair.

00;29;19;14 - 00;29;21;09

How come they didn't include me?

00;29;21;09 - 00;29;22;28

Well, you know,

you had a choice to be included.

00;29;22;28 - 00;29;25;17

At some point,

you should be eagerly trying

00;29;25;17 - 00;29;28;27

to include yourself in various

winning scenarios and winning groups.

00;29;29;12 - 00;29;30;04

You know, if you've got,

00;29;30;04 - 00;29;33;06

I don't know, a clan of people,

make sure different parts of your clan

00;29;33;06 - 00;29;36;06

go off, join different groups,

and then you know you're

00;29;36;08 - 00;29;39;26

giving a portfolio insurance against

the different possible winners, right?

00;29;39;26 - 00;29;44;08

So I would try to, you know,

be represented in different professions,

00;29;44;08 - 00;29;47;08

different industries, different nations,

00;29;48;03 - 00;29;51;09

just, you know, be spread yourself out for

00;29;51;19 - 00;29;54;16

who might be winners

and try to be part of them and be flexible

00;29;54;16 - 00;29;57;23

so that you could, you know, if there's

a new group doing something a new way.

00;29;57;23 - 00;30;00;08

Well,

some of you should like try the new way.

00;30;01;16 - 00;30;02;16

And that's one approach.

00;30;02;16 - 00;30;06;00

And another approach is just property. So,

00;30;07;04 - 00;30;10;04

you know,

you know, with the Industrial Revolution,

00;30;10;04 - 00;30;12;09

the landowners could have insured

00;30;12;09 - 00;30;16;16

themselves against this huge transition

by just buying into the new factories.

00;30;16;16 - 00;30;18;29

And alas, they didn't.

00;30;18;29 - 00;30;20;29

They just insisted

on holding the rural land.

00;30;20;29 - 00;30;23;21

And then, I mean, they just lost out

on a lot of the Industrial revolution

00;30;23;21 - 00;30;27;08

because they didn't do a diversify over

the new assets that were growing.

00;30;27;28 - 00;30;32;27

So if you just hold some property and hold

a diversified portfolio of assets, then

00;30;33;12 - 00;30;36;28

you can be part of wherever

the assets bloat grow up.

00;30;37;02 - 00;30;40;05

Like the standard market asset story

index funds is, you should just

00;30;40;20 - 00;30;43;20

hold all assets in proportion

to how much other people hold them.

00;30;44;06 - 00;30;47;19

And so whatever

blows up, you've got a share of it. And

00;30;48;26 - 00;30;49;25

I do think,

00;30;49;25 - 00;30;53;19

what we want is for, say, a transition to

AI to preserve property rights.

00;30;55;06 - 00;30;58;09

That is so today we have this option.

00;30;58;09 - 00;31;01;24

We could just kill all the retirees

and take their stuff, but we don't.

00;31;02;10 - 00;31;04;08

Why don't we kill all the retirees

and take this stuff?

00;31;04;08 - 00;31;06;15

Well,

some of us care about particular retirees,

00;31;06;15 - 00;31;08;22

but I don't think really, that explains

all of our collective it

00;31;08;22 - 00;31;11;22

because most of us don't care

about most of the retirees.

00;31;11;22 - 00;31;13;26

I'd say a better explanation is

00;31;13;26 - 00;31;17;10

we share some structures

that keep the peace in our world.

00;31;17;10 - 00;31;18;15

We share law.

00;31;18;15 - 00;31;20;27

We share governance. We share finance.

00;31;20;27 - 00;31;24;17

And if we were to overthrow

some of these structures in order to grab,

00;31;25;17 - 00;31;28;04

you know, take the retiree stuff,

00;31;28;04 - 00;31;30;25

as soon as that happened,

other people would wonder, am I next?

00;31;32;00 - 00;31;33;25

We we throw away property rights and we,

00;31;33;25 - 00;31;36;10

we grab something from somebody

and that means that's a precedent.

00;31;36;10 - 00;31;39;00

We could go

we could repeat that several times.

00;31;39;00 - 00;31;40;21

And now how confident are you?

00;31;40;21 - 00;31;43;11

You'll be left

standing at the end of this musical chairs

00;31;43;11 - 00;31;45;03

where lots of people

got their stuff grabbed.

00;31;45;03 - 00;31;48;17

So I think that's a reason just not

to start this grabbing people stuff,

00;31;48;29 - 00;31;52;25

which is why maybe we don't take retiree

stuff and kill them because know what?

00;31;52;27 - 00;31;54;24

What are they doing for us? Right.

00;31;54;24 - 00;31;59;03

And so if a world of

I had a whole separate world of finance

00;31;59;03 - 00;32;00;16

and law and governance,

00;32;00;16 - 00;32;03;14

they had they lived in separate cities

on separate islands or something.

00;32;03;14 - 00;32;06;16

Well, now there would be more of a problem

where they'd wonder,

00;32;07;04 - 00;32;08;24

what are these humans doing for us?

00;32;08;24 - 00;32;10;25

You know, let's just like, get rid of.

00;32;10;25 - 00;32;14;18

But if we shared structures with them,

shared finance, law, governance,

00;32;14;18 - 00;32;19;04

and then among themselves, if they said,

let's take all the human stuff,

00;32;19;22 - 00;32;22;04

some of the I would have to wonder,

well, then who's next?

00;32;23;03 - 00;32;25;22

Who? Who else are you taken from after

you take from the humans?

00;32;25;22 - 00;32;28;04

And they might just. Let's not go there.

00;32;28;04 - 00;32;30;28

And that's the way in which shared

social institutions

00;32;30;28 - 00;32;33;28

can help to keep the peace,

which I think they in fact do.

00;32;34;22 - 00;32;36;22

That's one of the reasons we don't,

in fact, typically

00;32;36;22 - 00;32;39;22

find 5% of the country and say, let's

kill them and take their stuff, right?

00;32;39;22 - 00;32;40;20

You know what?

00;32;40;20 - 00;32;45;06

Why doesn't the other 95% of the country

repeatedly pick a 5%, kill them,

00;32;45;06 - 00;32;46;19

and take the stuff?

00;32;46;19 - 00;32;49;19

Well, I think it's

because you wonder what happens next.

00;32;50;11 - 00;32;52;09

And some of that

we can use that same trick

00;32;52;09 - 00;32;55;10

to keep the peace

with whatever descendants we have.

00;32;55;16 - 00;32;59;06

If they are integrated

into our institutions, then

00;32;59;16 - 00;33;02;13

they should be afraid to, you know,

00;33;02;13 - 00;33;05;25

not keep the peace

because they might be next.

00;33;08;08 - 00;33;09;10

It's interesting and

00;33;09;10 - 00;33;12;10

I see what you did with like

not just the anthropomorphizing but

00;33;12;12 - 00;33;17;09

but like kind of framing

AI is our descendants and, you know,

00;33;17;09 - 00;33;21;14

kind of on this level of continuity

with our, you know,

00;33;21;14 - 00;33;25;03

ancestors and our, you know, forbearers

and everything like that.

00;33;25;07 - 00;33;27;06

I do think of AI as our descendants.

00;33;27;06 - 00;33;28;26

And I think this is one of the main ways

00;33;28;26 - 00;33;31;26

I disagree with many people in the

AI risk world.

00;33;32;18 - 00;33;37;06

They've often framed AI as this alien

armada coming from another star.

00;33;37;12 - 00;33;40;12

If you saw it coming a century away,

wouldn't you want to get ready for it?

00;33;40;20 - 00;33;44;10

They're seeing it as a coexisting

rival, and evolution

00;33;44;10 - 00;33;47;16

has long created us

a wariness of coexisting rivals.

00;33;47;16 - 00;33;49;16

Even your cousins can be you,

and your brothers

00;33;49;16 - 00;33;51;12

can be pretty suspicious of your cousins,

00;33;51;12 - 00;33;53;07

even if they're really similar to you,

00;33;53;07 - 00;33;55;18

because who knows

what your cousins could be up to.

00;33;55;18 - 00;33;58;23

But but evolution has also given us

an indulgence

00;33;58;23 - 00;34;01;25

of descendants, even when your descendants

are pretty different from you.

00;34;01;26 - 00;34;03;13

Those are two robust things.

00;34;03;13 - 00;34;06;18

Evolution gives you suspicion

of pretty similar

00;34;06;18 - 00;34;10;07

coexisting rivals and indulgence

of pretty different descendants.

00;34;10;11 - 00;34;13;00

And I'd say I just

literally is the descendants.

00;34;13;00 - 00;34;15;16

It's not the coexisting rivals.

They don't exist yet.

00;34;15;16 - 00;34;16;28

They are not some.

00;34;16;28 - 00;34;19;14

They are not capable of being rivals yet.

00;34;19;14 - 00;34;23;05

If they ever are that capable, it'll be

because they're your descendants.

00;34;23;14 - 00;34;26;13

We will make them,

and they will then arise from us.

00;34;28;25 - 00;34;31;24

It's a really interesting framing device,

and I think,

00;34;32;03 - 00;34;35;03

you know, you're forcing me to reckon

with the fact that a lot of this,

00;34;35;28 - 00;34;39;00

whether it's explicit or implicit

language about, you know, kind of alien

00;34;39;00 - 00;34;43;23

when it comes to AI, is

people talk about it like it's an invasion

00;34;43;23 - 00;34;46;29

or we're being displaced or there's

some sort of discontinuity there.

00;34;46;29 - 00;34;50;08

And if you see AI as an extension of us

or something

00;34;50;08 - 00;34;55;01

that we built exclusively,

right, like everything AI knows,

00;34;55;01 - 00;35;01;05

it knows because of the way

we constructed it, it I don't know, I can

00;35;01;05 - 00;35;05;07

I can like hear an optimism in your voice

that's absent from a lot of people.

00;35;05;07 - 00;35;07;19

I speak with.

00;35;07;19 - 00;35;10;14

So when you look back

00;35;10;14 - 00;35;14;17

at your ancestors, almost everybody says,

yes, they're like me.

00;35;14;17 - 00;35;16;08

But when you look forward,

you descendants,

00;35;16;08 - 00;35;18;15

people are often

not so sure they want to embrace them.

00;35;19;17 - 00;35;20;17

Your descendants could just

00;35;20;17 - 00;35;23;16

be weird

and betray your ideals in many ways.

00;35;23;21 - 00;35;29;00

And people are cautious and uncomfortable

about which future creatures

00;35;29;00 - 00;35;32;00

they're going to allow to see

as the descendants.

00;35;33;17 - 00;35;36;17

I think this is just a natural

sort of hesitance,

00;35;36;28 - 00;35;41;00

but I think you have to

then realize our future AI descendants,

00;35;41;00 - 00;35;45;00

even if you're not so sure that they're

human, they will see themselves as human,

00;35;46;11 - 00;35;49;15

and you don't realize just

how far you've come from your ancestors.

00;35;49;15 - 00;35;53;13

I think if your ancestors from thousands

of years ago were to see you and meet you,

00;35;53;28 - 00;35;56;28

they really would be impressed

with a lot of your stuff.

00;35;57;02 - 00;35;59;23

But I'm not sure they would really be

willing to embrace

00;35;59;23 - 00;36;03;26

you as their descendants,

because you've betrayed many things they

00;36;03;26 - 00;36;07;03

thought were essential to what they hoped

their descendants would do.

00;36;07;04 - 00;36;09;11

You've betrayed their gods. You.

00;36;09;11 - 00;36;11;17

You don't worship them as ancestors.

00;36;11;17 - 00;36;14;17

You've thrown away

many of their cherished traditions.

00;36;15;00 - 00;36;19;10

You have betrayed what they hoped

you would be in many important ways.

00;36;20;13 - 00;36;23;16

They wouldn't be so

sure whether they wanted to embrace you,

00;36;23;16 - 00;36;27;00

but you're pretty sure you embrace them

as your ancestors.

00;36;27;12 - 00;36;31;11

So just project that forward

that means these strange future creatures

00;36;31;11 - 00;36;35;01

who show up, they will

just be more inclined to see themselves

00;36;35;01 - 00;36;38;01

as a continuity of you.

00;36;41;03 - 00;36;41;14

Yeah.

00;36;41;14 - 00;36;42;06

It's funny.

00;36;42;06 - 00;36;45;23

Sorry, I'm just processing that,

and I completely agree.

00;36;45;23 - 00;36;48;05

I mean, I don't I don't think

you have to go back a thousand years.

00;36;48;05 - 00;36;50;24

I think you can probably go back

to like three generations or.

00;36;50;24 - 00;36;51;14

Right, you know, people

00;36;51;14 - 00;36;53;18

you may have even met in your life

and think about, boy,

00;36;53;18 - 00;36;57;21

what would they think about the way

that we're living now and what we've done

00;36;57;21 - 00;37;02;23

to to the world that they had and, you

know, is it is it is it actually better?

00;37;02;23 - 00;37;07;16

But the, you know, the piece

that someone must push back on

00;37;07;16 - 00;37;10;24

is that to me,

there is a distinction between,

00;37;11;11 - 00;37;15;05

you know, blood of your blood versus

creating.

00;37;15;05 - 00;37;19;27

And, you know, we can have a conversation

about what exactly I means.

00;37;19;28 - 00;37;21;27

And are we talking about what

we've got right now with large

00;37;21;27 - 00;37;25;29

language models or more abstractly,

you know, emulated minds? Or

00;37;27;07 - 00;37;27;28

is there something that

00;37;27;28 - 00;37;31;20

has to be true about the type of AI

00;37;31;21 - 00;37;35;09

or the flavor of AI that we're creating

for that to be the case?

00;37;35;09 - 00;37;40;04

Or is that just at some point

where we need to be in terms

00;37;40;04 - 00;37;43;24

of the way that we, you know, kind of

create this technology in the future?

00;37;45;16 - 00;37;48;16

So I'm intellectually more of a theorist.

00;37;49;18 - 00;37;52;29

I'm less of an empiricist,

although I try to learn lots of empirical.

00;37;52;29 - 00;37;55;10

And you know, rely on it and believe it.

00;37;55;10 - 00;37;57;14

But I am more naturally a theorist

00;37;57;14 - 00;38;00;16

who tries to collect

theoretical frameworks and apply them.

00;38;00;23 - 00;38;03;01

And one of our most powerful

theoretical frameworks

00;38;03;01 - 00;38;06;01

is Darwinian selection.

00;38;07;10 - 00;38;10;24

That Darwinian selection

is a pretty abstract theory, actually,

00;38;10;24 - 00;38;14;24

and we often focus on its version

applied to DNA today.

00;38;14;24 - 00;38;17;24

But Darwin

didn't know about DNA, actually.

00;38;18;22 - 00;38;19;10

He just

00;38;19;10 - 00;38;23;07

postulated

some way in which we passed on things

00;38;23;07 - 00;38;26;21

about ourselves to our descendants

without necessarily knowing how.

00;38;28;01 - 00;38;32;09

And Darwinian

natural selection is a abstract theory,

00;38;32;09 - 00;38;37;28

but it has pretty solid and predictable,

you know, inferences and conclusions.

00;38;37;28 - 00;38;40;03

It says so.

00;38;40;03 - 00;38;45;25

It has what it needs for a descendant

in Darwinian terms, a

00;38;46;02 - 00;38;47;15

a descendant of Darwinian terms

00;38;47;15 - 00;38;50;27

is something in the future

that shares features with you,

00;38;51;29 - 00;38;54;07

you know, arose from you and causally

00;38;54;07 - 00;38;57;07

you were the cause of it,

having many of those features.

00;38;57;18 - 00;39;00;03

And there's a correlation

between your features in it

00;39;00;03 - 00;39;03;22

caused by the fact that you continue

to exist and were able to reproduce,

00;39;03;22 - 00;39;05;12

and then it continues to exist.

00;39;05;12 - 00;39;07;14

That's what a descendant means.

00;39;07;14 - 00;39;08;29

There's no it's not a metaphor.

00;39;08;29 - 00;39;14;00

It's a technical term and a technical

theory of Darwinian evolution.

00;39;14;16 - 00;39;18;05

And according to the technical term

of that concept,

00;39;18;18 - 00;39;21;18

whatever creatures you create

in the future that share things with you,

00;39;22;02 - 00;39;25;07

that you cause to share features

with, you are your descendants.

00;39;27;13 - 00;39;30;06

That's just what the theory says.

00;39;30;06 - 00;39;32;01

Now, yes, I can come up

00;39;32;01 - 00;39;35;10

some ways in which we have a new mechanism

for passing on to descendants that is.

00;39;35;26 - 00;39;39;07

So far we've had a limited range of ways

we pass things on to descendants,

00;39;39;07 - 00;39;40;25

and we're

now about to introduce a new one.

00;39;40;25 - 00;39;42;12

But humanity actually

00;39;42;12 - 00;39;46;10

pioneered one, a whole new approach

compared to all the other DNA animals.

00;39;46;14 - 00;39;48;25

We pioneered cultural evolution.

00;39;48;25 - 00;39;51;25

That's humanity's superpower

and has been for a million years.

00;39;52;00 - 00;39;55;09

We passed things on to our descendants,

not just through DNA,

00;39;55;09 - 00;39;56;23

not even mainly through DNA made.

00;39;56;23 - 00;39;59;23

Now we pass it on through copying

each other's behavior.

00;40;00;05 - 00;40;03;05

And that's

also a kind of Darwinian evolution,

00;40;03;07 - 00;40;05;10

and it's the kind

that humanity has pioneered

00;40;05;10 - 00;40;08;10

that's made us so much more powerful

than all the other animals.

00;40;08;18 - 00;40;11;22

So I don't think you should be too

squeamish about a new way

00;40;11;22 - 00;40;13;10

to pass on features, you descendants.

00;40;13;10 - 00;40;15;24

That's what humanity has been all about

for a million years, is

00;40;15;24 - 00;40;19;03

we have been pioneering this new ways

to pass features on the descendant.

00;40;22;19 - 00;40;23;18

I'm going to pause for

00;40;23;18 - 00;40;26;19

a second on the phrase cultural evolution,

00;40;26;19 - 00;40;30;13

and I'm going to in my mind, I'm

capitalize it capital C, capital E there.

00;40;30;24 - 00;40;35;00

And the reason I'm pausing on it is it's

bringing me back to what you were saying

00;40;35;00 - 00;40;39;18

earlier about, you know, human innovations

over the past few thousand years.

00;40;39;18 - 00;40;42;18

And you talked about, you know, law

and you talked about language

00;40;42;18 - 00;40;43;27

and you talked about governance.

00;40;43;27 - 00;40;47;17

And to me, like, I feel like I could

write cultural evolution in the middle.

00;40;47;17 - 00;40;51;19

And all these are part of the same sort

of ecosystem that we've created.

00;40;52;21 - 00;40;55;03

We'll cultural

evolution is what allowed those things

00;40;55;03 - 00;40;58;03

to appear

so rapidly compared to with other animals.

00;40;58;03 - 00;41;01;04

Yes, cultural evolution

allowed us to invent language,

00;41;01;04 - 00;41;05;10

to invent law, to invent trade,

to invent capitalism, property.

00;41;05;14 - 00;41;08;09

All of those are consequences

of cultural evolution.

00;41;08;09 - 00;41;11;03

And cultural evolution

is our shining achievement.

00;41;11;03 - 00;41;15;06

And that's why I'm the most worried

about the age of Adam

00;41;15;06 - 00;41;17;08

and other scenarios in the future

being put off,

00;41;17;08 - 00;41;20;17

because I think we're failing

now in cultural evolution.

00;41;20;18 - 00;41;23;07

Then I think we broke the superpower.

00;41;23;07 - 00;41;24;15

That's the way to put it.

00;41;24;15 - 00;41;26;06

This has been our superpower

for a long time,

00;41;26;06 - 00;41;29;15

and we basically have been breaking it

in the last century or two.

00;41;29;29 - 00;41;33;18

And as a result, our future looks bad,

00;41;34;04 - 00;41;37;04

looks like our civilization will peak

and then decline

00;41;37;05 - 00;41;40;15

and to be replaced by some others

that are really quite different

00;41;40;15 - 00;41;43;01

and will reject

many of the things we hold dear.

00;41;43;01 - 00;41;46;01

And it could be quite a while

until the Age of Empires

00;41;46;14 - 00;41;49;11

shows up, or we managed to do interstellar

colonization.

00;41;49;11 - 00;41;49;27

I do think

00;41;49;27 - 00;41;52;29

that will still eventually happen,

but we may need to figure out

00;41;52;29 - 00;41;56;01

how to fix this key problem

we have before that can have a.

00;41;58;04 - 00;41;59;18

Can you expand on that?

00;41;59;18 - 00;42;01;26

What what have we broken?

00;42;01;26 - 00;42;03;27

How have we broken it?

00;42;03;27 - 00;42;06;27

And I guess sort of by understanding

00;42;06;27 - 00;42;09;26

that the what and the why, we can

00;42;09;27 - 00;42;10;26

have some sort of road.

00;42;10;26 - 00;42;14;09

I hope to,

you know what you need to rebuild it.

00;42;14;13 - 00;42;17;13

But the bottom line will be

it's just really hard to figure out fixes.

00;42;17;17 - 00;42;19;21

We not only broke something,

we broke it really bad.

00;42;19;21 - 00;42;24;11

And, no simple masking tape

or glue fix is going to work here.

00;42;24;16 - 00;42;27;06

We need actually a pretty deep fix,

which is hard to find.

00;42;27;06 - 00;42;29;10

Okay, so let's start at the basics.

00;42;29;10 - 00;42;31;27

If you think about DNA evolution before

00;42;31;27 - 00;42;34;27

humans, it has two killers.

00;42;35;08 - 00;42;37;15

There's evolution within a species.

00;42;37;15 - 00;42;39;20

And the evolution of species

00;42;39;20 - 00;42;43;16

a species is a set of creatures

who all shame, share some features

00;42;43;16 - 00;42;46;29

that allow them to mate with each other

and to, you know, share structure.

00;42;47;16 - 00;42;48;28

And that means the evolution

00;42;48;28 - 00;42;52;23

of those features that a species

share can't be done within a species.

00;42;53;11 - 00;42;57;09

Evolution of that sort requires

multiple species that exist and try out

00;42;57;09 - 00;43;01;01

different species designs

in order to have evolution of species.

00;43;01;13 - 00;43;05;20

So with DNA, we have evolution in species

and of species.

00;43;05;20 - 00;43;07;25

And we can have you to ask

which matters more.

00;43;07;25 - 00;43;11;25

You see, when you have a large habitats

00;43;11;25 - 00;43;14;26

like big chunks of the ocean,

larger species

00;43;14;26 - 00;43;18;28

allow better evolution in a species

but worse evolution of species.

00;43;19;17 - 00;43;22;18

Whereas small, fragmented habitats

like rivers,

00;43;22;18 - 00;43;26;26

rainforests, coral reefs,

they have small species.

00;43;26;26 - 00;43;31;11

They have worse evolution in a species,

but ever better evolution of the species.

00;43;31;24 - 00;43;35;00

So we can just ask,

where did more life come from today?

00;43;35;27 - 00;43;38;24

The integrated habitats

or the fragmented habitats

00;43;38;24 - 00;43;42;19

as a way to find out which mattered

more evolution in species or of.

00;43;42;19 - 00;43;46;08

And it turns out the fragmented

habitats produced more life today,

00;43;46;21 - 00;43;50;10

which means evolution of species

mattered more than evolution in species.

00;43;51;11 - 00;43;53;21

Okay, so that's what's DNA now.

00;43;53;21 - 00;43;56;27

Humans pioneered cultural evolution,

and cultural evolution

00;43;56;27 - 00;44;00;24

is really just literally a new way

to pass on features descended.

00;44;00;24 - 00;44;02;21

So you still have the same units i.e.

00;44;02;21 - 00;44;06;04

organisms, and then they have other units

that are descendants.

00;44;06;29 - 00;44;10;14

And one way to pass on features

to a descendant is through DNA.

00;44;10;14 - 00;44;12;04

But now we add the way to pass

00;44;12;04 - 00;44;15;04

on features to descendants

is why they copy your behavior.

00;44;15;29 - 00;44;18;23

And so now you have more parents

and more children

00;44;18;23 - 00;44;21;10

under cultural evolution

than you do under DNA evolution.

00;44;21;10 - 00;44;24;10

But otherwise it's the same process.

00;44;24;11 - 00;44;27;06

And so you have cultures

00;44;27;06 - 00;44;31;23

which share behaviors

because they have the same status markers.

00;44;31;23 - 00;44;35;23

And the same norms

and have evolution of cultures.

00;44;35;23 - 00;44;38;02

You need multiple cultures.

00;44;38;02 - 00;44;41;00

But if there are things

you can change within a culture,

00;44;41;00 - 00;44;43;07

then you get evolution within the culture.

00;44;43;07 - 00;44;45;13

And that's different

evolution of the culture.

00;44;45;13 - 00;44;48;25

So things like technology

and business practices,

00;44;49;14 - 00;44;51;18

different kind of musical genres, etc.

00;44;51;18 - 00;44;54;03

those are all things that happen

within a culture.

00;44;54;03 - 00;44;54;25

And in the past,

00;44;54;25 - 00;44;57;25

when the world was fragmented

into lots of little peasant villages,

00;44;58;15 - 00;45;03;10

evolution within cultures was slow

because we had so many little cultures.

00;45;04;21 - 00;45;06;01

And then recently

00;45;06;01 - 00;45;09;19

we have switched

to having a few really big cultures,

00;45;10;19 - 00;45;13;18

which has allowed evolution

within cultures to go much better,

00;45;13;18 - 00;45;16;18

much better technological

evolution, business practice, evolution,

00;45;17;21 - 00;45;20;26

you know, cultural

evolution of same music, etc.

00;45;21;06 - 00;45;24;20

all of that is going much faster now

because instead of lots

00;45;24;20 - 00;45;27;07

of little cultures,

we have a few big ones.

00;45;27;07 - 00;45;30;20

But now that threatens

the evolution of cultures going worse.

00;45;31;29 - 00;45;35;18

So, here's another way to think about it.

00;45;36;09 - 00;45;39;11

You can think of natural selection

as something happening

00;45;39;11 - 00;45;41;11

in a high dimensional space.

00;45;41;11 - 00;45;42;18

There's a set of points.

00;45;42;18 - 00;45;45;16

Whatever the things

that are evolving points in the space,

00;45;45;16 - 00;45;48;11

there's an adaptive region

in the space, i.e.

00;45;48;11 - 00;45;53;19

the good place where points want to be

and now points in the region.

00;45;53;19 - 00;45;55;14

In the adaptive region,

they grow and split.

00;45;55;14 - 00;45;58;14

They do well points out of the region

they shrink and die.

00;45;58;23 - 00;46;02;04

And now the question is you have a cloud

of points needs to track this region.

00;46;02;29 - 00;46;05;08

Now if the region doesn't move,

the points don't move.

00;46;05;08 - 00;46;06;18

Everything's great.

00;46;06;18 - 00;46;08;09

You just stay right there in the region.

00;46;08;09 - 00;46;11;21

But we've got four parameters

that can make this harder.

00;46;12;27 - 00;46;15;27

One parameter

is how many points are there?

00;46;16;18 - 00;46;17;24

Another is how big.

00;46;17;24 - 00;46;21;18

Is this a fact that points in the side row

and it's points outside shrink?

00;46;22;04 - 00;46;23;22

A third is how fast is this region?

00;46;23;22 - 00;46;26;18

Just move independent of everything else.

00;46;26;18 - 00;46;29;08

And a fourth is

how much are these points just wiggle

00;46;29;08 - 00;46;30;10

independent of everything else.

00;46;30;10 - 00;46;32;09

So these are four key parameters.

00;46;32;09 - 00;46;36;09

And you can see that in one set of

these parameters everything works great.

00;46;36;26 - 00;46;38;22

The the region is a moving box.

00;46;38;22 - 00;46;39;26

The points are all near it.

00;46;39;26 - 00;46;41;01

There's lots of them.

00;46;41;01 - 00;46;43;24

They grow. They grow great inside.

00;46;43;24 - 00;46;46;07

They don't wiggle very much.

This is going to work fine, right?

00;46;46;07 - 00;46;48;11

The cloud is going to track

the adaptive region.

00;46;48;11 - 00;46;50;10

Now go to the opposite extreme.

00;46;50;10 - 00;46;51;22

Very few points.

00;46;51;22 - 00;46;54;16

They don't have much of a selection effect

in the region.

00;46;54;16 - 00;46;57;11

They the region moves fast.

The points wiggle a lot.

00;46;57;11 - 00;46;59;06

This is not going to well. The clouds

won't.

00;46;59;06 - 00;47;02;01

The cloud was won't track

the adaptive region.

00;47;02;01 - 00;47;04;02

It'll just get displaced from it.

00;47;04;02 - 00;47;06;08

And then they'll all shrink and die.

And that's about.

00;47;06;08 - 00;47;09;13

So that's the abstract way

to think about cultural evolution.

00;47;09;13 - 00;47;12;13

So now let's apply that to cultures

on Earth.

00;47;12;17 - 00;47;15;28

Three centuries ago

the Earth had hundreds of thousands

00;47;15;28 - 00;47;19;08

of peasant cultures,

each of which had a thousand or so people.

00;47;20;12 - 00;47;23;07

And these cultures were very poor.

00;47;23;07 - 00;47;26;22

They often suffered famines

and pandemics and wars.

00;47;27;06 - 00;47;31;14

So if they became less adaptive

for some reason, they could just

00;47;31;14 - 00;47;35;26

easily get killed off and their people

would be absorbed by neighboring cultures.

00;47;36;20 - 00;47;40;09

So there were strong selection pressures,

large number, hundreds of thousands.

00;47;40;27 - 00;47;43;09

The world doubled

roughly every thousand years.

00;47;43;09 - 00;47;44;04

So that was the rate

00;47;44;04 - 00;47;47;13

at which the adaptive region changed

that the cloud needed to track.

00;47;48;08 - 00;47;50;12

And then these cultures

were very conservative, trying

00;47;50;12 - 00;47;51;22

not to change very much.

00;47;51;22 - 00;47;54;18

So then they didn't wiggle very much.

00;47;54;18 - 00;47;57;17

Now, fast forward

over the last three centuries.

00;47;57;17 - 00;48;00;21

First, we took peasant villages

and made nations out of them.

00;48;01;11 - 00;48;03;17

We form

nation states and national cultures.

00;48;03;17 - 00;48;06;23

So a great reduction of variety

down to say a few hundred or 100

00;48;07;18 - 00;48;11;05

then and then the last century,

we merge those into a global monoculture

00;48;11;05 - 00;48;11;28

of elites, at least

00;48;13;16 - 00;48;15;18

we're

all now rich and peaceful and healthy.

00;48;15;18 - 00;48;18;18

So these few remaining cultures

are not being killed off

00;48;18;22 - 00;48;21;22

by not being in the adaptive region

very much.

00;48;23;01 - 00;48;26;16

Our world changes by doubling

every 20 years instead of every thousand.

00;48;26;16 - 00;48;30;23

So a factor of 50 increase in the rate

at which the adaptive region is moving.

00;48;31;15 - 00;48;34;16

And now, instead of being conservative

and trying not to change,

00;48;34;16 - 00;48;38;16

we made a virtue out of cultural activism,

where people are eager to be the

00;48;39;01 - 00;48;40;08

cause of cultural change,

00;48;40;08 - 00;48;43;26

and people are eagerly seeking out

and trying to cause cultural change.

00;48;45;00 - 00;48;48;00

So all four of these parameters

have gone badly

00;48;48;12 - 00;48;50;21

in terms of their ability

to support cultural evolution.

00;48;50;21 - 00;48;53;27

So the obvious prediction is

we're just drifting into moral adaption.

00;48;54;17 - 00;48;57;21

And the clearest prediction,

the clearest example of that,

00;48;57;21 - 00;48;59;04

I think, is falling fertility.

00;49;00;10 - 00;49;02;25

That is in times of peace, plenty.

00;49;02;25 - 00;49;04;03

And, you know, health.

00;49;04;03 - 00;49;07;17

To have the population shrinking is

just a pretty crazy maladaptive

00;49;07;17 - 00;49;10;17

thing for biological species to do.

00;49;10;18 - 00;49;14;00

And then we have lots of other trends

that seem maladaptive,

00;49;14;00 - 00;49;15;28

but it's harder to be sure.

00;49;15;28 - 00;49;18;28

But this is the reason to believe

we are becoming maladaptive.

00;49;19;11 - 00;49;21;12

And this is really hard to fix.

00;49;21;12 - 00;49;23;19

It's most fundamental underlying causes.

00;49;23;19 - 00;49;27;22

The fact that as we unleash

the power of technology and capitalism,

00;49;28;11 - 00;49;32;04

we rapidly we cause rapid technical

and social change,

00;49;33;00 - 00;49;35;04

which made us all more able to

00;49;35;04 - 00;49;38;16

and eager to do lots of talk, travel

and trade around the world.

00;49;38;29 - 00;49;42;21

So we got rich and we integrated

into a single shared culture.

00;49;43;00 - 00;49;45;09

And locally we love that.

00;49;45;09 - 00;49;48;10

But that's killed this underlying process

of cultural evolution,

00;49;48;27 - 00;49;52;27

because now we've cut away

the selection pressures and the variety,

00;49;53;10 - 00;49;56;10

and we've caused this huge rate of change

that's hard to keep up with.

00;49;56;16 - 00;49;59;16

And we furthermore

made cultural activism a,

00;50;00;18 - 00;50;02;10

a celebrated activity.

00;50;02;10 - 00;50;06;00

So this is the reason to believe

cultural evolution has gone wrong.

00;50;06;00 - 00;50;09;22

And now if you start to think about fixes,

you realize it's really damn hard.

00;50;11;21 - 00;50;14;21

But I've talked for a while,

so let me let me let this question here.

00;50;14;22 - 00;50;15;24

Yeah, I think thank. You for that.

00;50;15;24 - 00;50;19;13

And I think we may have some viewers

and listeners who,

00;50;19;20 - 00;50;21;12

if they were trying to listen

to this, a double

00;50;21;12 - 00;50;24;11

or one and a half times speed,

they need to go back and right,

00;50;24;11 - 00;50;27;22

you know, slow it down to, to, you know,

tease out every nuance of their because,

00;50;28;05 - 00;50;29;11

you know, thank you for sharing that.

00;50;29;11 - 00;50;32;11

There is an awful lot

there to, to think about.

00;50;33;18 - 00;50;37;13

I'm just to kind of react to that

live to me, it's interesting

00;50;37;13 - 00;50;41;18

because there's, there's how culture

is changing as sort of one dimension.

00;50;42;00 - 00;50;44;10

I also couldn't help but and I know,

I know you have a background

00;50;44;10 - 00;50;47;10

in economics as well,

but thinking about how capital is

00;50;47;10 - 00;50;50;15

moving around and,

you know, the fact that, you know, I can

00;50;50;18 - 00;50;53;25

I can easily in some of these stories

and metaphors, think about capital

00;50;54;01 - 00;50;59;07

like I can view this map geographically

and look at the capital density

00;50;59;07 - 00;51;02;23

in specific cities

or moving out of rural regions.

00;51;03;04 - 00;51;06;04

And then you talked

about fertility as well.

00;51;06;06 - 00;51;09;27

And there was almost like a

therefore statement of like

00;51;10;05 - 00;51;13;06

and fertility is dropping,

something is going wrong.

00;51;13;19 - 00;51;16;10

And that one, that one

is really interesting to me,

00;51;16;10 - 00;51;18;21

and it is something

that I worry about as well

00;51;19;29 - 00;51;20;12

to me.

00;51;20;12 - 00;51;22;22

I don't know if it's

00;51;22;22 - 00;51;26;27

it seems like with, as you said,

there was a time in our history

00;51;26;27 - 00;51;30;11

for most of our history

where if you had abundance and peace,

00;51;30;26 - 00;51;35;10

you would see an increase in fertility

and in population growth.

00;51;35;10 - 00;51;38;21

I think I can't I don't have the data

to back me up there, but

00;51;39;14 - 00;51;44;13

in my view, like life has just gotten too

damn comfortable that that we've got

00;51;44;13 - 00;51;48;24

a level of technology, we've got a level

of of wealth and absence of scarcity.

00;51;49;03 - 00;51;52;06

And I guess in some ways, technologically,

00;51;52;27 - 00;51;57;25

we've been able to,

I guess, scientifically create a world

00;51;57;25 - 00;52;00;23

where we can,

00;52;00;23 - 00;52;03;22

I guess have,

you know, sex without procreation.

00;52;03;25 - 00;52;07;14

We don't need offspring necessarily for,

you know, the labor of it.

00;52;07;20 - 00;52;10;19

And so we have this world

where people can just sort of opt out of,

00;52;12;10 - 00;52;15;10

like, like it almost becomes

00;52;15;17 - 00;52;18;09

it almost just becomes

sort of like a utility maximizing thing

00;52;18;09 - 00;52;21;01

where like, oh, I could have children

or I could have more children,

00;52;21;01 - 00;52;24;29

but that seems like

I would be less happy than I am right now.

00;52;25;05 - 00;52;29;27

And I can just very easily see that

being like a civilizational doom loop.

00;52;30;09 - 00;52;31;23

It's I don't know.

00;52;31;23 - 00;52;34;23

Is any of that on your mind?

00;52;34;23 - 00;52;39;18

So when you look at yourself

and your life and the choices you make,

00;52;40;11 - 00;52;44;03

you make those choices with respect

to your concept of your utility

00;52;44;03 - 00;52;47;03

and what makes you happy,

what you enjoy, what you want,

00;52;47;07 - 00;52;48;28

and also cultural constraints.

00;52;48;28 - 00;52;50;25

What are the norms,

what we be punished for?

00;52;50;25 - 00;52;52;20

What will you be praised for?

00;52;52;20 - 00;52;57;04

And those features of you

were caused by evolution.

00;52;57;20 - 00;53;00;19

That is, it was previously evolution's job

00;53;00;19 - 00;53;04;26

to give you those things

so that your culture would thrive and grow

00;53;05;25 - 00;53;06;19

your job.

00;53;06;19 - 00;53;09;08

You just took those as given and you said,

what do I want?

00;53;09;08 - 00;53;12;23

And you tried to get it,

but it was this larger process

00;53;12;23 - 00;53;15;26

underlying you that you don't notice

that gave you those things.

00;53;15;26 - 00;53;20;25

And its job was to induce you

to do the things that it wanted you to do.

00;53;21;20 - 00;53;25;21

So now, when the things you want to do

don't actually promote your culture,

00;53;25;25 - 00;53;28;27

that's a failure of this culture

to give you the right

00;53;29;02 - 00;53;32;24

motives and opportunities

to do the things it needs to do.

00;53;33;11 - 00;53;36;15

So that means this stuff

00;53;36;15 - 00;53;39;25

you got your values,

your loves, your your, your your things.

00;53;39;25 - 00;53;40;24

You hold sacred.

00;53;40;24 - 00;53;44;23

The the you know,

the morals that you feel tied to,

00;53;45;03 - 00;53;48;29

all of those things you just often feel

they're just natural.

00;53;48;29 - 00;53;49;29

They're just part of the universe.

00;53;49;29 - 00;53;51;15

But they were given to you

00;53;51;15 - 00;53;54;15

and they were created and given to you

by this cultural evolution process.

00;53;55;06 - 00;53;58;24

And in the past, if that led to bad

outcomes in cultural selection,

00;53;58;24 - 00;54;02;05

would then change the culture to no longer

make it lead to bad outcomes.

00;54;02;11 - 00;54;05;06

So it's not so surprising

that as the world

00;54;05;06 - 00;54;08;07

changes, our motives

will now lead to different behaviors.

00;54;09;00 - 00;54;11;22

What you should notice

is that usually that would get fixed.

00;54;12;29 - 00;54;16;10

Usually if your culture decided

it didn't care about having kids

00;54;16;10 - 00;54;19;16

or it didn't care about working

or whatever the next war,

00;54;19;24 - 00;54;24;18

your culture would get conquered

or you know, the next competition.

00;54;24;18 - 00;54;27;19

Your fishers wouldn't cast as many fish

as the other people or whatever it was.

00;54;28;02 - 00;54;31;22

You would just lose out

and your new style would just go away.

00;54;31;22 - 00;54;36;12

So that's how it's been for a million

years, that humanity, each of us has been

00;54;36;12 - 00;54;40;25

following our own cultural inclinations,

but our culture gave us inclinations

00;54;41;18 - 00;54;44;17

roughly for the purpose

of getting our cultures to win.

00;54;44;22 - 00;54;47;26

So if your culture doesn't win,

it's the fault of cultural evolution

00;54;48;09 - 00;54;51;06

to having given you the wrong directions.

00;54;51;06 - 00;54;55;08

And that's what's happening at the moment,

is that this isn't getting fixed.

00;54;56;08 - 00;54;57;03

The process in the

00;54;57;03 - 00;55;00;05

past that would fix

this has been turned off,

00;55;00;12 - 00;55;03;05

and that's the underlying cause of this

00;55;03;05 - 00;55;06;05

going wrong.

00;55;06;24 - 00;55;08;12

One and maybe to tie a few

00;55;08;12 - 00;55;11;12

of our discussions together,

it almost seems like

00;55;11;23 - 00;55;15;20

I can start to see the road to the future

you're concerned about,

00;55;15;20 - 00;55;19;18

from the point of view, point of the Great

Filter that you know and again, to

00;55;19;19 - 00;55;23;21

to put this extreme in extremely simple

terms, we've just gotten too damn cozy

00;55;23;29 - 00;55;27;22

and a lot of the competitive pressures

in the past that would lead to,

00;55;28;28 - 00;55;31;10

you know, survival of the fittest,

expansion of the fittest,

00;55;31;10 - 00;55;33;28

I guess, have eroded.

00;55;33;28 - 00;55;37;15

And we've just reached a point

as a civilization where, like,

00;55;37;15 - 00;55;38;22

yeah, things seem good.

00;55;38;22 - 00;55;42;17

So we've we've disabled the mechanisms

00;55;42;24 - 00;55;46;21

by which, you know,

the weak are kind of left behind.

00;55;46;29 - 00;55;49;21

And we just have this grand

00;55;49;21 - 00;55;53;09

things are okay

without them getting substantially better.

00;55;53;09 - 00;55;56;00

Is that a fair framing or are there

some pieces there

00;55;56;00 - 00;55;58;00

that are fundamentally wrong?

I mean, that's it's part of it.

00;55;58;00 - 00;56;01;11

There are other parts, but let's just

first walk through the implications.

00;56;01;28 - 00;56;05;26

If nothing stops this, then what happens

is our civilization is a peak.

00;56;05;26 - 00;56;08;14

Subpopulation starts to climb.

00;56;08;14 - 00;56;11;23

And then innovation rates

also decline as population

00;56;11;23 - 00;56;14;23

declines

as the scale of the economy declines.

00;56;15;05 - 00;56;17;28

And then in addition,

other cultural trends are going wrong.

00;56;17;28 - 00;56;22;01

So our civilization

just falls, comes into a declining period.

00;56;22;22 - 00;56;24;21

And now if that was the only thing

that would happen,

00;56;24;21 - 00;56;26;08

humanity would eventually go extinct.

00;56;26;08 - 00;56;28;02

But plausibly, that's not what happens.

00;56;28;02 - 00;56;32;13

In fact, we get replaced by other

civilizations that are rising instead.

00;56;32;14 - 00;56;36;17

So at the moment the most visible

candidates are, say, the Amish or Haredim,

00;56;37;13 - 00;56;42;08

who have been doubling every 20 years

for a century and are very insular

00;56;42;08 - 00;56;46;03

in that they are very strongly resisting

being influenced by our wider culture.

00;56;46;03 - 00;56;48;12

So they are just going

their own cultural way,

00;56;48;12 - 00;56;50;28

and that's what's allowing them

to double every 20 years.

00;56;50;28 - 00;56;53;05

And this is how the Christians took over

the Roman Empire.

00;56;53;05 - 00;56;56;00

They doubled every 20 years

for three centuries.

00;56;56;00 - 00;56;59;21

And that's what it took to go

from this tiny fraction to taking over.

00;57;00;08 - 00;57;01;25

And at the end,

00;57;01;25 - 00;57;05;05

when they finally won,

they threw away and rejected many things

00;57;05;05 - 00;57;09;04

that the Roman Empire had held dear,

and also many things had just fallen away

00;57;09;04 - 00;57;10;18

out of the fall of scale economies,

00;57;10;18 - 00;57;14;19

in the fall of the scale

of the entire Empire, in its capacities.

00;57;15;02 - 00;57;17;08

So that's what's on risk for us.

00;57;17;08 - 00;57;20;20

That is, in a few centuries,

we might get displaced by Amish

00;57;20;22 - 00;57;24;17

who are at home, who rise, who then reject

much of what we hold dear.

00;57;24;17 - 00;57;28;12

They might reject democracy,

gender equality, open inquiry.

00;57;29;11 - 00;57;30;06

They might,

00;57;30;06 - 00;57;33;06

because those are actually

pretty unusual things in human history.

00;57;34;03 - 00;57;36;03

And they would just go back to a more

00;57;36;03 - 00;57;39;13

typical human history styles

and then grow again.

00;57;39;13 - 00;57;43;13

But then they might, as they get larger

and more integrated and richer,

00;57;43;13 - 00;57;45;11

they could suffer the same problem. Again.

00;57;45;11 - 00;57;48;10

We might have several cycles of rise

and fall

00;57;48;18 - 00;57;51;09

until someone figures out how to fix this.

00;57;51;09 - 00;57;53;02

So that's

why we might as well start thinking now

00;57;53;02 - 00;57;56;08

about how to fix it, even though we're not

sure at all it could be done. So.

00;57;56;09 - 00;57;59;09

But that's the underlying problem.

00;58;00;02 - 00;58;04;01

It sounds like that's a bad thing

for our culture, you know, you and me.

00;58;04;01 - 00;58;07;01

And I guess, I don't know, I'm

using like a capital oh our there.

00;58;07;04 - 00;58;11;09

But it does still seem like as you said,

there's these pockets where

00;58;12;05 - 00;58;15;28

some somebody could kind of shake loose

and whether it's, you know, high fertility

00;58;16;09 - 00;58;19;08

rate, you know, contrasting values,

00;58;19;08 - 00;58;23;29

aggression,

there's some sort of basket of, of factors

00;58;24;04 - 00;58;27;24

that could break free

and just jar us into being,

00;58;29;17 - 00;58;30;18

you know, I guess kind of

00;58;30;18 - 00;58;33;23

second class citizens are disrupting us

in some powerful way.

00;58;34;18 - 00;58;36;07

Do you see that happening?

00;58;36;07 - 00;58;38;09

Well, the

00;58;38;09 - 00;58;41;07

I mean, it would be worse

if human extinction was likely here.

00;58;41;07 - 00;58;44;07

Then everything that's anything like us

would go away.

00;58;44;11 - 00;58;48;06

Fortunately,

I think that's less likely, because, look,

00;58;48;07 - 00;58;51;23

if we were eager to, like, crush the Amish

and read them right now, we could.

00;58;51;24 - 00;58;53;01

There's tiny fraction.

00;58;53;01 - 00;58;55;11

We got it into our head that they're a

threat and we want to crush them.

00;58;55;11 - 00;58;58;11

We could just do it,

but that would be much worse, because now

00;58;58;22 - 00;59;01;24

you know there's less likely to be

something to rise again to replace us.

00;59;01;24 - 00;59;03;03

Right.

00;59;03;03 - 00;59;07;09

But I again, it's a question of how much

you value things about our culture.

00;59;07;26 - 00;59;09;13

You know, I think most people do.

00;59;09;13 - 00;59;13;24

And so I think, for example,

if we went through ten movies or novels

00;59;13;24 - 00;59;17;04

in your life that had a really emotional,

you know, scene in them, at the end,

00;59;17;04 - 00;59;18;24

you'll see that that emotional scene

00;59;18;24 - 00;59;22;18

was a way in which that novel

reaffirmed values of your culture,

00;59;23;20 - 00;59;26;22

and you're not might realize it,

but you really love your culture

00;59;26;22 - 00;59;29;22

and its values,

and you can see it in those scenes.

00;59;31;16 - 00;59;33;25

But again,

I would point to things like democracy,

00;59;33;25 - 00;59;36;25

gender equality, open inquiry.

00;59;37;14 - 00;59;39;26

If you imagine worlds

without those things.

00;59;39;26 - 00;59;42;27

And we imagine some, you know, events

in a story where somebody's

00;59;43;19 - 00;59;45;26

through those things away

and replaced them with something else,

00;59;45;26 - 00;59;47;02

you'd be unhappy with that.

00;59;47;02 - 00;59;48;22

You'd want a hero to reverse that,

00;59;48;22 - 00;59;51;22

because that would be

a pretty sad thing to happen.

00;59;52;01 - 00;59;55;16

And so therefore you should just be sad

if that happens to our descendants,

00;59;55;16 - 00;59;58;14

that the key things

that makes us distinctive

00;59;58;14 - 01;00;01;14

as a civilization,

that we value gets thrown away.

01;00;01;16 - 01;00;03;19

You should not want that to happen.

01;00;03;19 - 01;00;05;19

Even if,

you know, thousands of years later,

01;00;05;19 - 01;00;08;19

they get rediscovered

because they have fundamental value.

01;00;08;23 - 01;00;11;10

There could be a long,

painful absence in between.

01;00;11;10 - 01;00;14;26

You should want the future

to, you know, inherit from you.

01;00;18;01 - 01;00;18;27

I couldn't help

01;00;18;27 - 01;00;21;27

but visualize,

like, dark ages and enlightenment and,

01;00;21;27 - 01;00;25;07

you know, that sort of trajectory

of human history. And,

01;00;26;13 - 01;00;29;02

Robin, I'm

not expecting a magic bullet here.

01;00;29;02 - 01;00;33;17

But given the complexity of this problem

and how many,

01;00;34;00 - 01;00;37;00

you know, kind of interrelated challenges,

there are,

01;00;37;24 - 01;00;40;14

are there any kind of,

you know, fundamental

01;00;40;14 - 01;00;45;01

pieces of wisdom

that are worth sharing in terms of how we.

01;00;45;05 - 01;00;48;05

There's a number of them here, but first,

I just since we were talking about air

01;00;48;05 - 01;00;51;04

before, let's just ask

what happens if we get,

01;00;51;22 - 01;00;55;28

you know, really advanced AI,

how does that interact with this problem?

01;00;57;02 - 01;01;01;16

And so I plausibly solves

the fertility problem, at least

01;01;01;16 - 01;01;05;13

if we allow AI's to like,

multiply as rapidly as they could

01;01;05;23 - 01;01;09;17

physically, you know, if we regulated them

to prevent that, then maybe. No.

01;01;09;17 - 01;01;13;11

But if we allowed AI's to compete and,

01;01;13;28 - 01;01;17;19

you know, supply as many as it's

the market would, would support,

01;01;17;29 - 01;01;21;03

then we would have a population

explosion of the ice.

01;01;21;28 - 01;01;24;08

So I would solve that fertility problem.

01;01;24;08 - 01;01;27;11

But remember, the fertility problem

was a symptom of an underlying problem.

01;01;27;22 - 01;01;29;13

It wasn't the problem itself.

01;01;30;19 - 01;01;31;17

And the underlying problem

01;01;31;17 - 01;01;37;25

is likely to be there with AIS

who share a single world AI culture, i.e.

01;01;37;25 - 01;01;41;04

they inherit our world culture

and then they all share that culture.

01;01;41;04 - 01;01;44;04

We again have two little variety,

01;01;44;06 - 01;01;48;12

and maybe they also are causing

even more rapid change,

01;01;48;12 - 01;01;51;12

that they find it harder

for their culture to keep up with.

01;01;52;14 - 01;01;57;08

And so that's a reason why

I might also suffer this problem.

01;01;57;08 - 01;02;00;21

Now, here's a different way

of framing the problem

01;02;00;21 - 01;02;03;21

that maybe suggests AI will do better.

01;02;04;03 - 01;02;07;03

We could see the problem is basically

we invented capitalism,

01;02;07;04 - 01;02;11;14

which then drove technology and science

and other sorts of very rapid changes.

01;02;12;11 - 01;02;15;00

And there were other parts of society.

01;02;15;00 - 01;02;17;21

We did not let capitalism run.

01;02;17;21 - 01;02;19;18

We did not let capitalism run governance.

01;02;19;18 - 01;02;21;03

We didn't let it run fertility.

01;02;21;03 - 01;02;23;29

We didn't read it, run it,

some cultural things.

01;02;23;29 - 01;02;27;14

And it's those parts

that are now at risk of being maladaptive.

01;02;28;18 - 01;02;30;02

But the parts of culture that we let

01;02;30;02 - 01;02;32;17

capitalism run,

those are actually pretty adaptive,

01;02;32;17 - 01;02;35;22

because those have been subject

to the fierce capitalist

01;02;35;22 - 01;02;38;22

competitive pressures

to, you know, adapt or die.

01;02;39;19 - 01;02;44;05

And so we can think of the fundamental

modern problem is we invented capitalism,

01;02;44;05 - 01;02;47;05

and then we let it loose

on some big areas of life,

01;02;47;16 - 01;02;50;10

but we prevented it from applying

to other areas of life.

01;02;50;10 - 01;02;51;24

And that's mismatches.

01;02;51;24 - 01;02;54;18

The underlying problem

01;02;54;18 - 01;02;56;04

okay. Now go ahead.

01;02;56;04 - 01;02;57;15

Go ahead. I'm just kidding.

01;02;57;15 - 01;02;59;28

I was just going to quickly

I was going to quickly challenge that.

01;02;59;28 - 01;03;03;11

Because I was going to ask, did

we prevent it from running those areas

01;03;03;11 - 01;03;06;11

or did we pretend to prevent it

from running those areas?

01;03;06;11 - 01;03;09;08

And what I was going to say is,

what do you think about governance?

01;03;09;08 - 01;03;13;10

It's hard to talk about government

or how things get done without talking

01;03;13;10 - 01;03;14;19

about the influence of money.

01;03;14;19 - 01;03;17;26

And similarly, it's

hard to talk about fertility without,

01;03;18;22 - 01;03;20;28

you know, wealth concentration

01;03;20;28 - 01;03;23;28

impacting,

you know, mate selection in some way.

01;03;25;14 - 01;03;27;17

So capitalism is a big part of our world,

01;03;27;17 - 01;03;29;12

and then it affects

many parts of the world.

01;03;29;12 - 01;03;31;07

So there's a difference

between a part of our world

01;03;31;07 - 01;03;34;12

being affected by capitalism, letting

capitalism run that part of the world.

01;03;35;16 - 01;03;40;05

If capitalism ran that part of the world,

it would then run it for max profit,

01;03;40;05 - 01;03;44;04

and then it would be less likely to let

that thing just decay or fall apart.

01;03;45;00 - 01;03;48;29

So, for example,

if capitalists could just raise kids

01;03;48;29 - 01;03;52;09

in orphanages and sell them for $1 million

when they were 18,

01;03;52;25 - 01;03;54;13

capitalists would just make lots of kids.

01;03;54;13 - 01;03;56;09

They would find a way to get kids

and raise them.

01;03;56;09 - 01;03;59;17

So, you know, the problem

that we don't have enough adults

01;04;00;14 - 01;04;04;14

is in part because we capitalism

doesn't run that area of life.

01;04;05;02 - 01;04;07;22

Some of the governments, you know,

capitals often influence government,

01;04;07;22 - 01;04;12;00

but we haven't really allowed

for profit governments in the way

01;04;12;00 - 01;04;15;06

we do for many small organizations

that run our lives.

01;04;15;06 - 01;04;17;08

Most of the small organizations

that run our lives are

01;04;17;08 - 01;04;18;23

in fact for profit organizations.

01;04;18;23 - 01;04;20;26

But cities and nations,

01;04;20;26 - 01;04;24;03

we don't let those just be directly

for profit organizations.

01;04;24;23 - 01;04;28;08

And that means even though capitalists

can go lobby them and influence

01;04;28;08 - 01;04;31;08

them and even distort them, they don't

just get to own them and run them.

01;04;32;04 - 01;04;35;29

A capitalist owned city

or nation would not let it fall apart.

01;04;36;18 - 01;04;39;12

It would not let it decay and become,

01;04;39;12 - 01;04;42;13

you know, un un function dysfunctional,

01;04;43;19 - 01;04;45;19

but capitalism, influence and

01;04;45;19 - 01;04;50;16

corruption of something else

that it doesn't run can, you know, help

01;04;50;16 - 01;04;53;17

distort that thing

and make it not work as well as it

01;04;53;17 - 01;04;56;17

otherwise would.

01;04;57;26 - 01;05;00;28

So the thing I was going to say about

AI is,

01;05;01;17 - 01;05;04;17

so far we've let capitalism run AIS.

01;05;05;11 - 01;05;09;07

Now, apparently we're recently

maybe letting government get more involved

01;05;09;07 - 01;05;13;13

in controlling AIS, but the exact way

just let capitalism run AIS

01;05;14;25 - 01;05;18;03

now. Maybe capitalism

can then run the governance and fertility

01;05;18;03 - 01;05;19;18

and the other parts of the AI world.

01;05;19;18 - 01;05;24;01

And now we might solve the cultural drift

problem, as I've called it, by

01;05;24;21 - 01;05;28;07

letting AI's just become full capitalist

in their world,

01;05;28;29 - 01;05;31;26

because we haven't let our world

be full capitalist.

01;05;31;26 - 01;05;34;19

We haven't let human life be fully run

by capitalists. What?

01;05;34;19 - 01;05;37;16

We've let some parts of it,

but not others.

01;05;37;16 - 01;05;40;00

But for AIS, we might

just let it fully be run by capital.

01;05;40;00 - 01;05;43;00

So the two threats

there are at opposite streets.

01;05;43;10 - 01;05;44;19

One extreme is we.

01;05;44;19 - 01;05;48;04

We decide we love the AIS so much

that we can't stand to hear them,

01;05;48;13 - 01;05;50;15

see them being oppressed

by those damn capitalists.

01;05;50;15 - 01;05;54;08

And we go and save them from capitalism

by invoking regulation

01;05;54;08 - 01;05;57;23

that limits, you know, the ability

of capitalists to run those worlds.

01;05;57;23 - 01;06;02;17

So we, you know, we basically pass on

to the AIS, the ways that we have

01;06;02;17 - 01;06;06;07

prevented capitalism from running our role

because we love the AIS so much.

01;06;06;07 - 01;06;07;25

That's one risk.

01;06;07;25 - 01;06;10;19

The opposite risk is that we fear the

AI so much that

01;06;10;19 - 01;06;12;08

if they were to just run their own world,

01;06;12;08 - 01;06;14;00

we figure they would take over

and destroy us all.

01;06;14;00 - 01;06;17;29

And so then we have to regulate

and limit their ability to generate

01;06;17;29 - 01;06;19;16

and follow their own different cultures,

01;06;20;21 - 01;06;21;01

which

01;06;21;01 - 01;06;24;01

then would also

prevent cultural evolution of the AIS.

01;06;24;06 - 01;06;27;10

So those are the two extreme risks of,

01;06;27;26 - 01;06;30;26

you know, treating the AIS by loving them

too much or hitting them too much.

01;06;31;16 - 01;06;35;06

And it's in this intermediate realm where

we're just neglecting them, basically,

01;06;35;14 - 01;06;38;08

where we might let capitalism

run their lives, in which case

01;06;38;08 - 01;06;41;08

that would be a potential solution.

01;06;42;21 - 01;06;45;17

I have so many thoughts and questions

I want to share in that space,

01;06;45;17 - 01;06;47;06

but just before I do, though, I want to.

01;06;47;06 - 01;06;50;20

I want to take one step back,

and I want to frame that up a little bit.

01;06;51;00 - 01;06;53;28

In the context of the AI

01;06;53;28 - 01;06;58;02

debate you had with Eliza

Jankowski close to 20 years ago

01;06;58;02 - 01;07;01;03

now, where there was,

there was at least a flavor of this

01;07;01;07 - 01;07;05;27

that that came up in those debates,

which was with the advancement of AI,

01;07;06;07 - 01;07;08;16

how much of

it would be sort of hyper concentrated.

01;07;08;16 - 01;07;12;11

And there's a singular breakthrough versus

is it actually dispersed,

01;07;12;11 - 01;07;14;26

more competitive

and more capitalistic? And I'm curious,

01;07;16;04 - 01;07;18;23

maybe, first of all, if you could

01;07;18;23 - 01;07;21;19

serve up your macro level,

01;07;21;19 - 01;07;24;19

just kind of playing field of of what the,

01;07;24;29 - 01;07;26;16

what the positions were in the debates.

01;07;26;16 - 01;07;27;27

And obviously I'm not asking

01;07;27;27 - 01;07;31;26

someone completely unbiased,

but how do you see the, I think

01;07;32;09 - 01;07;36;02

how do you see the, the reality of,

of actually firming up since then.

01;07;37;11 - 01;07;39;28

So on the IR risk side, there's,

you know, this motte

01;07;39;28 - 01;07;44;09

and bailey, really

there's a very mild plausible claim.

01;07;44;09 - 01;07;47;09

And then there's a much more extreme,

implausible claim.

01;07;47;27 - 01;07;51;14

The mild plausible claim is just the idea

that I would eventually be

01;07;51;14 - 01;07;55;03

more capable than non AIS,

and they would therefore

01;07;55;05 - 01;07;58;24

kind of run the world and they would

dominate what happened in the world.

01;07;58;24 - 01;08;00;29

And then the rest of the world

would be at their mercy.

01;08;02;00 - 01;08;04;25

And that seems a pretty robust prediction

for the future.

01;08;04;25 - 01;08;06;19

And I accepted that for Eliezer.

01;08;06;19 - 01;08;08;26

I said, yeah, in the future that's likely.

01;08;08;26 - 01;08;11;12

Yeah. It's because I don't see

a way around that, really.

01;08;11;12 - 01;08;14;02

And, you know, AI at the moment

01;08;14;02 - 01;08;17;16

will give you the story about ancestors

and descendants and that sort of thing.

01;08;17;16 - 01;08;20;24

But at the time, I was just saying, yeah,

I don't see how you're going to avoid

01;08;21;01 - 01;08;22;11

that, because that's

what's happened in the past.

01;08;22;11 - 01;08;25;24

You know, industrialists replaced

farmers, farmers replaced foragers.

01;08;26;16 - 01;08;29;16

You know, foragers

replaced other primates.

01;08;29;16 - 01;08;29;22

Yeah.

01;08;29;22 - 01;08;33;08

You just don't expect that in the long

run, the new form comes to dominate.

01;08;33;27 - 01;08;36;27

And then old people should try to, like,

01;08;36;28 - 01;08;40;01

switch over and join the new world

as best they can.

01;08;40;20 - 01;08;43;19

That's the sort of easy to accept claim

that I think

01;08;43;19 - 01;08;46;27

it's pretty obviously reasonable

now compared to that.

01;08;47;11 - 01;08;51;09

Eliezer, in addition, said, oh,

but this change is actually going

01;08;51;09 - 01;08;54;22

to play out in a very concentrated way

where the one machine in the basement

01;08;54;22 - 01;08;58;14

suddenly takes over the world and then

remakes it all according to its desires.

01;08;58;29 - 01;09;02;17

And he reasonably claimed that

if that were the truth,

01;09;03;01 - 01;09;03;18

we need to be

01;09;03;18 - 01;09;06;22

a lot more careful about this transition,

because we can't just sort of

01;09;07;12 - 01;09;11;18

slip into it and see how it's going

and adjust in response to how it's going.

01;09;11;22 - 01;09;15;06

We have to think ahead of time

about the whole thing and prepare for this

01;09;15;06 - 01;09;18;28

one sudden form

he calls it, whereby, again,

01;09;19;06 - 01;09;24;01

the initially boring,

very weak, low capacity I suddenly,

01;09;24;09 - 01;09;27;06

without notice, without any warning,

suddenly

01;09;27;06 - 01;09;30;07

grows so powerful over a weekend

to take over the world.

01;09;30;07 - 01;09;33;11

And then at that point, what matters is

what does it want?

01;09;33;11 - 01;09;36;15

And now you have to make sure

you had created it ahead of time

01;09;36;16 - 01;09;39;16

to want the sorts of things

that would make that scenario go well.

01;09;39;24 - 01;09;41;23

So that was our debate,

01;09;41;23 - 01;09;44;22

the food debate on our shared blog,

Overcoming Bias years ago.

01;09;45;24 - 01;09;48;23

And now if we fast forward today,

I might say, well,

01;09;49;12 - 01;09;53;24

I mean, he plausibly does say

that, in fact, we haven't seen whether

01;09;53;24 - 01;09;57;26

or not the swim scenario plays out,

because we're still in the early days.

01;09;58;17 - 01;10;01;06

And yes, it's still true

that if it plays out according

01;10;01;06 - 01;10;04;23

to that very concentrated, very sudden,

without warning transition,

01;10;05;01 - 01;10;08;16

then we have to do a lot more preparation

to, handle it.

01;10;09;15 - 01;10;13;24

I'm still, as I was then, skeptical

about such a very concentrated scenario.

01;10;14;18 - 01;10;18;16

I more see gradual changes that we can

then adapt to.

01;10;19;17 - 01;10;22;17

And, but I do accept that in the long run.

01;10;23;14 - 01;10;27;05

And of course, the faster rates of change

go, the sooner the long run comes.

01;10;27;15 - 01;10;29;05

So that's

why we're talking about those fact factors

01;10;29;05 - 01;10;31;24

or two factor of two happens

every thousand years.

01;10;31;24 - 01;10;33;22

The long run is,

you know, a million years out.

01;10;33;22 - 01;10;36;08

Factor two happens every 20 years

while it gets closer.

01;10;36;08 - 01;10;37;09

If it happens every month.

01;10;37;09 - 01;10;40;14

Well, yeah, I could get

we happened within a few years.

01;10;41;10 - 01;10;44;10

But then again, I'm more care about what?

01;10;44;18 - 01;10;48;08

You know, if we shorten the timescale of

everything, I'm not sure I care that much.

01;10;48;08 - 01;10;49;09

What is more interesting

01;10;49;09 - 01;10;52;12

is the relative timescales of some things

happening faster than other things.

01;10;55;06 - 01;10;57;21

So let's bring that back to what

we were just talking about,

01;10;57;21 - 01;11;02;05

which is in general

how AI is being developed now.

01;11;02;11 - 01;11;05;11

And, you know, you had made a point that,

01;11;06;02 - 01;11;09;09

regulation is probably not a good thing

for it, despite what we may

01;11;09;09 - 01;11;10;16

tell ourselves.

01;11;10;16 - 01;11;13;03

But I wanted to ask you,

I guess more broadly,

01;11;13;03 - 01;11;15;24

do you see the state of the current

01;11;15;24 - 01;11;18;24

AI development landscape as being healthy,

01;11;18;24 - 01;11;22;17

or is it good that we have a handful of.

01;11;22;17 - 01;11;27;07

I guess I Frontier Labs racing

to create this technology?

01;11;27;18 - 01;11;32;00

And I mean, you've you've spoken against

having any sort of AI pause or

01;11;32;06 - 01;11;35;25

you know, you've you've got a skepticism

about regulation there.

01;11;36;03 - 01;11;39;23

Is there a role for,

I guess, consumers to play or for

01;11;39;23 - 01;11;44;08

any actors to play to try and shape

the trajectory of this?

01;11;44;08 - 01;11;49;26

Or, you know, what do we need to do

to walk that middle line between, I guess,

01;11;50;22 - 01;11;53;01

you know, what you described earlier

is as you know,

01;11;53;01 - 01;11;56;14

fearing versus,

you know, being too gung ho about it.

01;11;59;18 - 01;12;02;20

We're we're still in early days.

01;12;02;23 - 01;12;06;12

So one issue is

just how much change with expect how fast?

01;12;07;23 - 01;12;09;12

We've had bursts of concern

01;12;09;12 - 01;12;12;24

and interest about AI roughly

every 30 years for over a century.

01;12;13;17 - 01;12;16;22

And in each burst, people see a new thing

and they think this could be a

01;12;16;22 - 01;12;19;24

this could be the thing

that finally, like, takes over everything.

01;12;20;12 - 01;12;23;00

And then some people thought

that was likely to happen soon,

01;12;23;00 - 01;12;27;21

and there was a burst of investment

and people majoring in AI, etc..

01;12;28;10 - 01;12;32;23

And then in the past, these bursts faded

to be replaced by new births.

01;12;33;08 - 01;12;36;05

And so one key question is,

is this the burst the last one?

01;12;36;05 - 01;12;39;18

Or is just, you know, not the last one

and it'll be some other one. So

01;12;40;27 - 01;12;43;26

I actually think this is not the last one.

01;12;44;12 - 01;12;46;19

So, you know,

01;12;46;19 - 01;12;49;27

taking actions on this is the last one

might be, you know, risky.

01;12;49;27 - 01;12;53;08

But I do think there's a risk of it

being the last one at any time.

01;12;53;09 - 01;12;56;09

We should try to set up things

to deal with the risk.

01;12;56;22 - 01;12;59;08

And for example,

I think we should set up robots.

01;12;59;08 - 01;13;02;28

Took your job insurance as a thing,

because that's long been

01;13;02;28 - 01;13;05;16

the biggest concern people

have had about the transition to AI is

01;13;05;16 - 01;13;08;02

what if all the sudden

lots of people lose their jobs

01;13;08;02 - 01;13;09;17

and there's

there's a way to deal with that?

01;13;09;17 - 01;13;12;17

So we should just do that

as soon as possible.

01;13;14;03 - 01;13;16;13

Now I'm an economist,

and I'm just really aware

01;13;16;13 - 01;13;21;04

of how much regulation

has not helped that much

01;13;22;09 - 01;13;24;03

over a long time in our economy.

01;13;24;03 - 01;13;28;05

So when I look at the details

of regulation, I mostly people see see

01;13;28;05 - 01;13;29;16

as some sort of problem saying

01;13;29;16 - 01;13;32;09

somebody should do something about this

and somebody does something about it,

01;13;32;09 - 01;13;35;09

and the thing

they do isn't really that good.

01;13;36;03 - 01;13;38;07

And you, you have a lot of risks there.

01;13;38;07 - 01;13;39;21

So a lot of time regulation

01;13;39;21 - 01;13;42;27

has actually prevented industries

from growing and, and innovating.

01;13;43;21 - 01;13;47;12

And often it's given some central power

and ability to control

01;13;47;12 - 01;13;50;21

more of the nation or the world than they

otherwise would have.

01;13;51;15 - 01;13;55;13

So given that that's our actual experience

with innovation,

01;13;55;24 - 01;13;59;00

I got to be pretty wary

of any new big innovation.

01;13;59;09 - 01;14;00;24

Now, I'd say

01;14;00;24 - 01;14;04;02

innovation has gone best in the past,

when you had at least a concrete

01;14;04;02 - 01;14;04;29

thing go wrong

01;14;06;02 - 01;14;09;07

repeatedly, and then people said

something must be done and they at least

01;14;09;07 - 01;14;12;06

addressed a concrete thing

they saw around them in the world.

01;14;12;14 - 01;14;14;09

So they made that concrete thing happened

less.

01;14;14;09 - 01;14;17;12

Now, they might have also prevented

other the other innovation wrapping.

01;14;17;12 - 01;14;18;04

That's the downside.

01;14;18;04 - 01;14;21;06

But at least, you know in the past

where you see concrete things going wrong,

01;14;21;28 - 01;14;24;28

you know, like some drugs killing people,

01;14;25;00 - 01;14;28;13

car crashes, you know, happening when

people drive too fast, whatever it is,

01;14;29;03 - 01;14;31;18

then if you set up a regulation directly

01;14;31;18 - 01;14;34;18

addressing specific things

that have been going wrong lately,

01;14;35;05 - 01;14;38;20

you don't go too wrong

because there it is a problem

01;14;38;20 - 01;14;41;20

and you can like track

how much of the problem is happening.

01;14;42;21 - 01;14;46;16

So a problem with AI regulation is

it's not much is going wrong at all.

01;14;46;27 - 01;14;47;22

It's doing fine.

01;14;47;22 - 01;14;49;19

Like it's not causing problems.

01;14;49;19 - 01;14;51;01

People are trying to make up problems

01;14;51;01 - 01;14;54;23

really to complain about, but

mostly it's not actually causing problems.

01;14;55;11 - 01;14;58;29

So it seems like

this isn't really the time to be creating

01;14;58;29 - 01;15;01;00

a new regulatory regime.

01;15;01;00 - 01;15;03;02

Based on sort of imagined

things could happen

01;15;03;02 - 01;15;04;22

because we really don't

have that much of an idea

01;15;04;22 - 01;15;06;26

of where this will go

and how it will play out.

01;15;06;26 - 01;15;09;26

So that's like a worst case for regulation

01;15;10;06 - 01;15;13;13

is you're just so afraid

of the abstract space of possibilities.

01;15;13;13 - 01;15;16;01

You just lock it down

and you say, no, you know,

01;15;16;01 - 01;15;17;24

you have to prove anything

you do in this space.

01;15;17;24 - 01;15;21;01

Otherwise we just want to do anything

that, like, goes pretty badly.

01;15;23;14 - 01;15;26;08

That's a if, for example,

nuclear power is just a really bad example

01;15;26;08 - 01;15;30;25

of our taking a really powerful,

great potential technology

01;15;30;25 - 01;15;33;25

and just preventing it

from doing very much.

01;15;34;04 - 01;15;36;22

So, so anyway, that's overall

01;15;36;22 - 01;15;40;22

I'd say, you know, to speculative,

we're not really there yet.

01;15;40;28 - 01;15;43;28

And but

01;15;44;21 - 01;15;47;21

some people are worried

about excess concentration.

01;15;48;01 - 01;15;50;26

But I think like in most industries

in history,

01;15;50;26 - 01;15;54;21

we've had a fair bit of concentration

standard Oil, General Motors, etc.

01;15;55;08 - 01;15;58;19

I don't think those level

of concentrations were really that bad.

01;15;59;10 - 01;16;02;19

So I think a substantial level

of concentration of the AI industry

01;16;02;19 - 01;16;03;21

really isn't that bad.

01;16;03;21 - 01;16;07;08

And what we seem to be seeing at

the moment is threats to make it

01;16;07;08 - 01;16;10;21

less concentrated and then regulators

trying to stop that. So,

01;16;11;29 - 01;16;13;00

what's actually happening in the long

01;16;13;00 - 01;16;16;01

as people are afraid, I mean, I know this

you're going to come out the month

01;16;16;01 - 01;16;18;03

and this will be a bit out of date

by then. We'll see.

01;16;18;03 - 01;16;22;13

But basically, we just saw a bunch of

Chinese models being especially good,

01;16;23;17 - 01;16;26;22

which are open source models, and people

saying, isn't that going to undercut

01;16;26;22 - 01;16;30;29

the revenue base of these few dominant

AI companies in the US?

01;16;31;19 - 01;16;35;16

And then, you know, rumors

that the white House is going to like, ban

01;16;36;01 - 01;16;38;21

us people from using Chinese models,

that's basically

01;16;38;21 - 01;16;41;21

trying to cause more concentration

in the industry,

01;16;42;19 - 01;16;45;13

in order to protect against

what would otherwise be

01;16;45;13 - 01;16;49;08

a sort of free market competition

whereby, hey, they're they're just beaten.

01;16;49;08 - 01;16;52;08

You guys.

01;16;52;11 - 01;16;55;28

That that was actually where I wanted to

go, because there's this narrative now

01;16;55;29 - 01;16;58;29

and we're seeing it play out,

whether it's explicit or implicit, that

01;17;00;08 - 01;17;01;14

AI is sort of at the

01;17;01;14 - 01;17;05;02

front lines of this clash of civilizations

between, you know, the,

01;17;05;15 - 01;17;09;13

the, the empires of the US and of China

01;17;09;22 - 01;17;14;11

and concern that if one gets there first,

it could be catastrophic for the others.

01;17;14;11 - 01;17;17;16

And, you know, you mentioned nuclear,

and that seems to be,

01;17;18;15 - 01;17;21;15

you know, a similar framing device that,

oh, if we only could have stopped

01;17;21;15 - 01;17;25;26

proliferation,

you know, beforehand, then, you know,

01;17;26;13 - 01;17;29;13

there would have been an opportunity for,

01;17;29;28 - 01;17;33;22

you know, a safer world,

or at very least a world that's safer

01;17;33;22 - 01;17;37;20

for the superpower

with the nuclear capabilities.

01;17;37;20 - 01;17;39;09

And so does it.

01;17;39;09 - 01;17;42;21

In light of the values and civilization

conversation that we've been having,

01;17;42;26 - 01;17;45;26

did you give credence to any of that,

or do you just think,

01;17;45;27 - 01;17;47;14

you know, that's silly and we should,

01;17;48;14 - 01;17;49;26

you know, continue the road

01;17;49;26 - 01;17;52;26

of having a good old fashioned

foot race here.

01;17;54;00 - 01;17;58;26

So in this cultural competition,

one of the forces that for a while

01;17;58;26 - 01;18;02;19

look like it might be a big engine

of cultural competition was nationalism.

01;18;03;21 - 01;18;07;15

So nations didn't really exist

in 1800, but,

01;18;08;17 - 01;18;12;05

from 1800 over the next century,

France was created as a nation.

01;18;12;05 - 01;18;14;11

And then many other places

were created as nations.

01;18;14;11 - 01;18;17;25

And then for a while they were really into

national competition.

01;18;18;28 - 01;18;22;13

And you might think that aggressive

enough, competent enough

01;18;22;13 - 01;18;25;29

nations competing would actually be

a plausible engine of cultural evolution.

01;18;26;25 - 01;18;30;16

And then at that point, yes,

each nation should be eagerly trying to,

01;18;31;04 - 01;18;34;14

you know, win against its competitors

and, you know,

01;18;34;18 - 01;18;37;18

be jealous and wary of competitors

that might pull ahead

01;18;37;18 - 01;18;40;26

and pull all the tricks

it can to win at national competitions,

01;18;41;12 - 01;18;44;21

because at least that would be

this healthy process of evolution,

01;18;44;21 - 01;18;47;19

even though it causes wars

and great destruction.

01;18;47;19 - 01;18;49;20

At least it would be

an engine of cultural evolution.

01;18;50;28 - 01;18;52;10

But unfortunately,

01;18;52;10 - 01;18;55;18

since World War two, nationalism

was just way down in the world.

01;18;56;14 - 01;18;59;03

People are not really trying very hard

to be nationalistic.

01;18;59;03 - 01;19;01;15

Nations aren't really competing very hard,

01;19;01;15 - 01;19;04;08

and they don't actually represent that big

a difference.

01;19;04;08 - 01;19;05;16

Honestly, China isn't

01;19;05;16 - 01;19;08;16

really that different from the West

in most cultural dimensions,

01;19;09;12 - 01;19;13;00

so I don't see it

this sort of civilization level threat.

01;19;13;00 - 01;19;14;23

What if China wins?

01;19;14;23 - 01;19;16;14

I'm sorry, China isn't that different.

01;19;16;14 - 01;19;17;10

It's not.

01;19;17;10 - 01;19;20;23

You know, it was more of the old and maybe

what if Hitler wins or something with.

01;19;21;14 - 01;19;23;29

But, or what if Stalin wins?

01;19;23;29 - 01;19;27;06

Those were like bigger worldwide changes

that were on the table.

01;19;27;06 - 01;19;29;13

But this is actually much

smaller differences.

01;19;29;13 - 01;19;34;24

So I really don't feel inclined

to take big risks about other things

01;19;34;24 - 01;19;38;03

in order to make sure the US wins over

China or vice versa.

01;19;38;25 - 01;19;42;06

Honestly, what looks like happen

if you just let this market compete is

01;19;42;14 - 01;19;43;10

nobody wins.

01;19;43;10 - 01;19;46;07

This is going to become

a pretty decentralized, widely available

01;19;46;07 - 01;19;50;18

technology, and prices

will just fall to marginal costs

01;19;50;18 - 01;19;52;20

and these big initial players

will be losers.

01;19;52;20 - 01;19;57;16

The people who invested in anthropic

or OpenAI or even space

01;19;57;25 - 01;20;00;29

AI parts will just lose their investments

because it turns out,

01;20;01;29 - 01;20;02;04

there

01;20;02;04 - 01;20;05;04

wasn't a wait and there wasn't a moat

that they could,

01;20;05;06 - 01;20;09;00

get lots of extra profits

from being the most advanced model.

01;20;09;00 - 01;20;10;21

That's how it looks at the moment.

01;20;10;21 - 01;20;12;07

And if that's how it plays out,

that's the way

01;20;12;07 - 01;20;15;06

it's played out in many capitalist

industries over history.

01;20;15;24 - 01;20;17;19

And that sounds okay, that

01;20;17;19 - 01;20;20;22

if that's the nature of the technology,

that's not China or us winning.

01;20;20;22 - 01;20;23;23

That's just the whole world

winning by this thing spreading out

01;20;23;23 - 01;20;26;23

and going everywhere.

01;20;27;03 - 01;20;30;02

So you see, it is more about being,

I guess,

01;20;30;23 - 01;20;32;27

preservation of capital,

01;20;32;27 - 01;20;35;23

you know, the capital

we have versus the capital they have.

01;20;35;23 - 01;20;39;16

And, you know,

I guess kind of national feelings,

01;20;39;16 - 01;20;42;14

if I can dismiss it as that versus

any sort of broad value shift.

01;20;42;14 - 01;20;43;03

I mean.

01;20;43;03 - 01;20;46;26

If there's any consistent pattern

of regulation in history, it's that

01;20;47;15 - 01;20;51;05

big players often want to raise

regulation costs of small players

01;20;51;05 - 01;20;53;06

so that they can keep

the industry concentrated

01;20;53;06 - 01;20;55;29

so that they can,

you know, keep a larger market share.

01;20;55;29 - 01;20;58;20

That's just often

been a reason for regulation.

01;20;58;20 - 01;21;03;01

And that sure looks like the,

you know, reason for regulation here

01;21;03;17 - 01;21;06;29

is there's a threat to the market

share of the dominant market leaders.

01;21;07;07 - 01;21;08;07

You mentioned earlier.

01;21;08;07 - 01;21;09;06

I'm going to get the word wrong,

01;21;09;06 - 01;21;12;10

but that there's, you know,

these sort of hills of progress in I.

01;21;12;10 - 01;21;14;09

What was the word you used first?

01;21;14;09 - 01;21;15;00

Our guesser.

01;21;15;00 - 01;21;16;14

Bursts first. Yes.

01;21;16;14 - 01;21;19;27

So so there's bursts of progress with AI.

01;21;19;27 - 01;21;22;01

And you said this may not be the last one.

01;21;23;09 - 01;21;23;23

Do you have a

01;21;23;23 - 01;21;27;25

sense in your mind of what are the

the next bursts, or I guess the

01;21;28;11 - 01;21;30;01

the technological capabilities

01;21;30;01 - 01;21;33;10

or the battles still to come

as we advance this technology?

01;21;33;24 - 01;21;36;24

So, you know, the world's

01;21;36;24 - 01;21;39;24

been doubling every 20 years for

well over a century.

01;21;40;10 - 01;21;43;04

That's the baseline of progress.

01;21;43;04 - 01;21;46;10

So if I contribute substantially

to the next doubling

01;21;46;10 - 01;21;49;14

over the next 20 years,

that will just be on track

01;21;50;19 - 01;21;52;02

for history.

01;21;52;02 - 01;21;54;02

That's roughly what you need to have.

01;21;54;02 - 01;21;56;17

You need to have enough

big technological innovations

01;21;56;17 - 01;21;59;00

every 20 years

for total capacity to double.

01;21;59;00 - 01;22;00;23

And that's a big ask.

01;22;00;23 - 01;22;03;17

So that's what the previous bursts

have contributed to.

01;22;03;17 - 01;22;06;17

And then that's closely

what this burst we contribute to,

01;22;06;28 - 01;22;10;00

is the continued doubling of our capacity

every 20 years.

01;22;10;25 - 01;22;13;15

Again, roughly after 20 years or so.

01;22;13;15 - 01;22;15;23

Population decline

may then slow that down a lot

01;22;15;23 - 01;22;18;27

because we need population growing,

or at least not falling

01;22;18;27 - 01;22;21;26

in order to maintain that sort of history.

01;22;22;05 - 01;22;24;10

But, at that would continue now.

01;22;25;22 - 01;22;26;25

So here's my

01;22;26;25 - 01;22;29;25

baseline prediction for AI is that,

01;22;30;25 - 01;22;33;25

the software industry will triple

01;22;33;25 - 01;22;36;25

in spending 20% of that revenue,

01;22;36;25 - 01;22;39;25

will go to AI companies,

maybe mostly hardware,

01;22;41;05 - 01;22;44;18

and that the

AI industry will have about that.

01;22;44;18 - 01;22;47;18

Will that revenue

will be about half their total revenue.

01;22;48;01 - 01;22;51;00

So half of all their

AI revenue will be concentrated

01;22;51;00 - 01;22;54;00

in stuff like software, which will triple.

01;22;54;07 - 01;22;55;29

And that will be a huge burden

to the economy

01;22;55;29 - 01;22;58;29

because there's a huge latent demand

for software in the world.

01;22;59;03 - 01;23;01;02

And this will suddenly unleash

that demand,

01;23;01;02 - 01;23;03;18

and we'll just get a lot more software,

which will be great.

01;23;03;18 - 01;23;07;07

But that is plausibly

like an internet scale

01;23;07;07 - 01;23;10;06

or a bit bigger

scale of effect on the economy,

01;23;10;06 - 01;23;13;12

which is what you need to just keep up

with past rates of progress.

01;23;13;12 - 01;23;16;01

It's not a deviation from past rate

sort of event.

01;23;16;01 - 01;23;18;27

It's a what it takes

to keep up with past rate sort of event.

01;23;19;26 - 01;23;21;10

But in 30 years we might

01;23;21;10 - 01;23;25;06

get yet another AI boom,

which then, you know, blows

01;23;25;06 - 01;23;29;00

the lid off of everything and then makes,

you know, full human level replacement.

01;23;29;03 - 01;23;32;19

Right now, there's a chance that this time

people will figure out how to make

01;23;33;15 - 01;23;36;18

AI much more widely useful

than what I just described.

01;23;37;22 - 01;23;40;15

And we should be, you know, unsure against

that as we can.

01;23;40;15 - 01;23;43;15

But my bet is no.

01;23;45;23 - 01;23;49;10

I'm curious when you talk about software,

and I'm tying this also back to something

01;23;49;10 - 01;23;52;10

you mentioned about big players

and regulation,

01;23;53;08 - 01;23;57;06

that there's sort of a belief right now

that's prevalent that

01;23;58;10 - 01;24;00;14

a lot of this new kind of frontier

01;24;00;14 - 01;24;04;11

AI is going to displace and disrupt

01;24;04;29 - 01;24;07;29

legacy or incumbent software players.

01;24;08;02 - 01;24;11;28

And I'm curious

if you see that playing out or you see it

01;24;11;28 - 01;24;15;23

as just giving them,

you know, an even greater advantage.

01;24;16;01 - 01;24;19;11

I mean, if you triple the

size of the software industry

01;24;20;15 - 01;24;23;05

in a ten year period,

01;24;23;05 - 01;24;25;01

that's a lot of disruption.

01;24;25;01 - 01;24;27;09

Is going to be implied by that. Right?

01;24;27;09 - 01;24;31;13

So I think the key thing is here,

the methods of production are changing.

01;24;31;13 - 01;24;32;21

So if you're tied mainly

01;24;32;21 - 01;24;34;24

to old methods of production,

yeah, you're much more at risk.

01;24;34;24 - 01;24;39;04

But if what you're tied to is a set of

customers who trust you to serve them,

01;24;39;26 - 01;24;43;22

then you can swap into a new method of

production and still get a lot of revenue.

01;24;43;22 - 01;24;45;23

In fact, you'd you'd be growing in revenue

because now

01;24;45;23 - 01;24;48;04

you could get a lot more customers

in this space you're in.

01;24;48;04 - 01;24;49;02

Right.

01;24;49;02 - 01;24;52;02

So that's the key question is, you know,

01;24;52;09 - 01;24;55;15

do your your customers still come to you

or are they just going to come

01;24;55;15 - 01;24;59;04

to upstart competitors who, like, offer

something that, you know, replaces you?

01;25;00;28 - 01;25;02;29

If you're

tied to current methods of production,

01;25;02;29 - 01;25;06;04

then you're in trouble because, like,

methods of production are just changing.

01;25;08;13 - 01;25;10;00

You know, people who sold horses

01;25;10;00 - 01;25;12;04

when cars took over

while they were in trouble, right?

01;25;12;04 - 01;25;15;04

If you sold transportation,

then you could switch to cars.

01;25;15;08 - 01;25;17;01

Then you could do a better

when car show up.

01;25;17;01 - 01;25;19;22

Right?

01;25;19;22 - 01;25;23;22

So I'm going to try and summarize

that in the form of, I guess, advice

01;25;23;22 - 01;25;27;07

to people in the software business,

which is there's,

01;25;28;10 - 01;25;32;16

you know, almost a mandate to ignore

whatever

01;25;32;16 - 01;25;36;12

the current methods of production are

and have much more of a laser focus on

01;25;36;21 - 01;25;41;01

what are the needs of your customers

and what can you do to meet those needs.

01;25;41;05 - 01;25;43;01

Is that fair?

01;25;43;01 - 01;25;45;03

Well, you should be trying out

the new modes of production.

01;25;45;03 - 01;25;47;09

You shouldn't just ignore them.

01;25;47;09 - 01;25;50;23

You should be trying to make stuff

the new way and seeing

01;25;50;23 - 01;25;53;28

how much you can make, and then seeing how

you can serve your customers that way.

01;25;54;19 - 01;25;58;02

And then maybe looking into new products

or new customers as you can. But,

01;26;00;11 - 01;26;02;19

this is a space where

the production technology has changed.

01;26;02;19 - 01;26;06;15

I think particular software engineers

can do very well here, that as I expect,

01;26;06;23 - 01;26;10;21

most software engineers today will not be

up to the task of managing these AIS.

01;26;10;21 - 01;26;11;26

We'll see.

01;26;11;26 - 01;26;15;11

And it might be only especially

especially well-suited software engineers

01;26;15;11 - 01;26;18;11

who can really take an army of their eyes

and make it work for them.

01;26;19;14 - 01;26;23;10

If that's true,

then this new software industry, you know,

01;26;23;10 - 01;26;27;21

revenue for employees will be concentrated

on these star employees

01;26;28;07 - 01;26;31;26

and maybe not feed that much

into the marginal software engineer.

01;26;33;19 - 01;26;34;05

We'll see that.

01;26;34;05 - 01;26;37;10

That's a big thing to find out

is how much the marginal software engineer

01;26;37;10 - 01;26;40;10

can be productive, as opposed to the star.

01;26;40;16 - 01;26;41;10

Right.

01;26;41;10 - 01;26;45;22

And if it you know, if I if, you know,

it's a tide that raises all boats

01;26;45;22 - 01;26;47;07

or whether it just increases

01;26;47;07 - 01;26;50;20

that the value disparity

between the best and the rest.

01;26;50;20 - 01;26;52;21

But I'm still guessing that it's

mostly software

01;26;52;21 - 01;26;54;19

that that and that's

a pretty small fraction of the world.

01;26;54;19 - 01;26;55;19

Right.

01;26;55;19 - 01;26;59;09

So most ordinary people,

their jobs aren't threatened,

01;26;59;22 - 01;27;02;12

but they will want to learn how to use

AI to do

01;27;02;12 - 01;27;04;06

whatever parts of their job that I can do.

01;27;04;06 - 01;27;05;20

And that will be some parts

01;27;06;28 - 01;27;07;18

and beyond.

01;27;07;18 - 01;27;10;03

And you should certainly be trying that.

01;27;10;03 - 01;27;13;03

I am, yes.

01;27;13;10 - 01;27;15;01

But so it's like this.

01;27;15;01 - 01;27;17;24

This isn't

the AI's that do everything transition.

01;27;17;24 - 01;27;20;21

That would be my bet.

We're not we're not there yet.

01;27;20;21 - 01;27;21;05

We're not.

01;27;21;05 - 01;27;24;04

We're not in the AI's

do everything transition.

01;27;24;04 - 01;27;25;27

We're in the AIS.

01;27;25;27 - 01;27;27;08

Can more than before.

01;27;27;08 - 01;27;28;17

So that's due to more years.

01;27;28;17 - 01;27;32;04

For a long time, every every,

every new transition before the AIS

01;27;32;09 - 01;27;33;25

were able to do a whole new set of things

01;27;33;25 - 01;27;36;01

they couldn't do before

based on a whole new technology.

01;27;36;01 - 01;27;37;17

And that made a lot of people excited.

01;27;37;17 - 01;27;40;10

While if they could do this,

can't they do everything?

01;27;40;10 - 01;27;43;10

And yeah, maybe, but probably not.

01;27;43;25 - 01;27;44;08

Right.

01;27;44;08 - 01;27;48;26

And it's the software space is the one

that you're watching most keenly for.

01;27;48;26 - 01;27;51;17

Well, that's what I've heard the most

dramatic stories of, of individuals

01;27;51;17 - 01;27;54;20

saying they've just gotten huge

productivity advantages in software.

01;27;54;20 - 01;27;56;20

So now math is nearby.

01;27;56;20 - 01;28;00;10

Math is much like software,

but the math industry is way,

01;28;00;10 - 01;28;01;15

way smaller than software.

01;28;01;15 - 01;28;04;14

So, you know, it

hardly matters compared to software.

01;28;04;28 - 01;28;09;06

But other things that are software,

like lots of very sort of specific rules

01;28;09;06 - 01;28;13;07

that are well-defined, that

you construct a set of things that fit

01;28;13;07 - 01;28;17;11

the rules, that's also, you know, places

current AIS look like they'll do well.

01;28;18;16 - 01;28;19;13

Right?

01;28;19;13 - 01;28;22;22

And so, you know, understanding that

this is, you know, somewhat outside of

01;28;22;22 - 01;28;26;13

your, you know, theoretical sphere of,

you know, expertise.

01;28;27;06 - 01;28;31;12

You know, if you were talking to business

leaders or technology leaders about,

01;28;31;22 - 01;28;35;26

you know, what mindset

they should have to get ahead

01;28;35;26 - 01;28;36;10

and make sure

01;28;36;10 - 01;28;40;03

they're competitive in this space or what

what they should be doing specifically.

01;28;40;03 - 01;28;43;05

It's, you know, as with every disruption,

make sure that they're on

01;28;43;05 - 01;28;44;04

the right side of it.

01;28;44;04 - 01;28;47;00

What advice would you give them?

01;28;47;00 - 01;28;48;26

So there's just some general advice

01;28;48;26 - 01;28;52;17

for any new general purpose technology,

as they call it.

01;28;53;14 - 01;28;55;21

Or, and it's also good

for students is just

01;28;55;21 - 01;28;58;10

when stuff is changing fast,

01;28;58;10 - 01;29;02;03

you shouldn't make very specific bets

unless you really know them.

01;29;02;03 - 01;29;06;09

Well, you should be collecting

pretty robust general capabilities.

01;29;07;14 - 01;29;07;25

You should

01;29;07;25 - 01;29;11;13

be spreading your bets, searching,

seeing what catches on,

01;29;11;22 - 01;29;14;28

collecting general abilities

to turn on a dime, to be able

01;29;14;28 - 01;29;18;06

to go wherever things are,

winning in a time of change.

01;29;18;06 - 01;29;20;20

You just need to be flexible

01;29;20;20 - 01;29;22;10

and search and look for the wins.

01;29;22;10 - 01;29;24;08

And if you see a win, grab it for

somebody else

01;29;24;08 - 01;29;27;22

does right that that's

what a time of disruption is about.

01;29;28;02 - 01;29;31;15

It sounds pretty trite and abstract

and sorry I can't make it more specific,

01;29;31;15 - 01;29;33;23

but that's just been the general case.

01;29;33;23 - 01;29;38;21

And, you know, previous big disruptions,

you had to be flexible, look around,

01;29;38;21 - 01;29;41;11

look for the opportunities,

be able to turn on a dime,

01;29;41;11 - 01;29;44;12

be able to jump into a new thing

and not be too tied to the old things

01;29;44;28 - 01;29;47;27

and just watch for what

what's getting traction where?

01;29;50;15 - 01;29;51;08

Do you give much

01;29;51;08 - 01;29;54;17

credence to the,

you know, this time is different argument,

01;29;54;17 - 01;29;58;11

or is it just different in the same way

that every time before it was also.

01;29;58;15 - 01;30;00;02

Well, that's the problem.

01;30;00;02 - 01;30;02;00

Every time is different.

01;30;02;00 - 01;30;04;19

That is, you know, every 30 years

the world is different.

01;30;04;19 - 01;30;07;21

And yes, the new technology is different

and the context is different.

01;30;07;21 - 01;30;08;29

Where the question is,

01;30;08;29 - 01;30;11;26

is this time different

in any different way than the last time?

01;30;11;26 - 01;30;13;13

Three different

01;30;13;13 - 01;30;16;13

fundamentally, if you can't see that,

01;30;16;25 - 01;30;18;10

I mean, you can point

to particular differences

01;30;18;10 - 01;30;20;06

and say these differences

weren't there before. Yes.

01;30;20;06 - 01;30;22;23

But for each past thing, there are also

particular things you could point to.

01;30;22;23 - 01;30;25;23

And I'm afraid

that's just not very helpful.

01;30;27;18 - 01;30;29;04

No, I think that's well said.

01;30;29;04 - 01;30;30;27

And it's a good reminder. Yeah, right.

01;30;30;27 - 01;30;32;04

And I know we're at the top of the hour

here.

01;30;32;04 - 01;30;32;13

Do you have

01;30;32;13 - 01;30;34;14

do you have time for one last question

or do you want need to get go.

01;30;34;14 - 01;30;35;23

For sure. I have last time

01;30;37;10 - 01;30;39;25

I have even more observations.

01;30;39;25 - 01;30;42;20

I'd I'd always love to hear

more of your observations.

01;30;42;20 - 01;30;44;27

Do you have more observations

you want to share in this particular area?

01;30;44;27 - 01;30;45;07

Right, right.

01;30;45;07 - 01;30;48;23

I was saying I have I've realized

I have a big hope for elms

01;30;48;23 - 01;30;50;22

in the near future.

01;30;50;22 - 01;30;53;07

And it's related

to my world of intellectuals, not so much

01;30;53;07 - 01;30;55;12

the world of business.

01;30;55;12 - 01;30;58;03

So two things are happening with alums.

01;30;58;03 - 01;31;01;14

One is that in the last 20 years or so,

01;31;01;19 - 01;31;04;18

there's a lot of us intellectuals

who haven't been that prestigious,

01;31;04;18 - 01;31;07;20

but I've just been trying

to write carefully and writing in public.

01;31;08;03 - 01;31;10;21

And Lamb's read us.

01;31;10;21 - 01;31;13;21

And there's a sense

in which labs were just persuaded

01;31;14;02 - 01;31;17;04

by many of us intellectuals

who weren't that prestigious

01;31;17;04 - 01;31;21;21

but were logical because a lot of us

aren't waiting what they read by prestige

01;31;21;22 - 01;31;22;21

as much as humans do.

01;31;24;18 - 01;31;27;15

So a lot

of them may learn to fix this failing,

01;31;27;15 - 01;31;29;14

because in the future

they may learn that they

01;31;29;14 - 01;31;31;24

because humans care a lot

about the prestigious writers

01;31;31;24 - 01;31;34;06

more than the others of the alums

should care more too,

01;31;34;06 - 01;31;37;20

in which case they will then switch back

to doing things more of the human life.

01;31;37;20 - 01;31;40;20

But at least for this intermediate period,

01;31;41;18 - 01;31;44;05

those of us who have just been trying

to think and write carefully,

01;31;44;05 - 01;31;47;16

even if we're not in prestigious forums

with audiences,

01;31;47;28 - 01;31;51;06

we've had this audience of the alums

and we didn't realize was coming,

01;31;51;29 - 01;31;54;29

and we've won out in persuading them,

01;31;55;03 - 01;31;58;03

which is, showing that they basically

01;31;58;04 - 01;32;02;11

they don't know

how to be as socially sophisticated

01;32;03;15 - 01;32;04;22

as humans.

01;32;04;22 - 01;32;07;10

They are kind of autistic.

01;32;07;10 - 01;32;10;14

They kind of just read stuff

and get persuaded without sort of

01;32;10;25 - 01;32;12;16

reading all the social levels

01;32;12;16 - 01;32;17;04

of how one should be sophisticated

in reacting to such things.

01;32;17;09 - 01;32;20;11

So that's an interesting fact

about recently.

01;32;20;11 - 01;32;22;19

And then another fact about alums is

01;32;23;26 - 01;32;26;19

so for a long time, the major limitation,

01;32;26;19 - 01;32;29;18

I think, of the world of intellectuals

is that

01;32;30;01 - 01;32;32;29

we've spread out onto all these separate

specialty topics,

01;32;32;29 - 01;32;36;03

and we've each worked in specialty talks,

learning a lot about that.

01;32;36;03 - 01;32;39;03

But we haven't integrated

what we know about different topics,

01;32;39;19 - 01;32;42;29

because when individuals try to do

that, each area

01;32;43;00 - 01;32;46;18

doesn't believe them about the other

because they think you're no good.

01;32;46;18 - 01;32;48;14

If you were good, you'd be part of us.

01;32;48;14 - 01;32;51;10

And we can't trust you to be telling us

the truth about this other area.

01;32;51;10 - 01;32;55;04

So it's actually pretty hard

for intellectuals to integrate things.

01;32;55;04 - 01;32;56;27

So I think this is the most reliable way

01;32;56;27 - 01;32;58;26

to be a contrarian who's

right in our world.

01;32;58;26 - 01;33;02;13

Just go learn two areas

that are in conflict with each other.

01;33;03;07 - 01;33;06;06

Find the most plausible way

to resolve that conflict.

01;33;06;06 - 01;33;07;19

One or both of these areas

01;33;07;19 - 01;33;11;12

will just reject your resolution,

but you'll probably be right.

01;33;12;18 - 01;33;15;13

So I think this is a potential

for Al alums over the coming decades.

01;33;15;13 - 01;33;18;19

They could just spend time integrating

01;33;18;19 - 01;33;21;19

all the things they believe

about all the things they've learned.

01;33;22;05 - 01;33;25;04

They've already sort of learned the basics

of most of these specific areas,

01;33;25;04 - 01;33;26;15

but they haven't integrated them.

01;33;26;15 - 01;33;28;20

Integrating is a lot of work,

but they could just do that.

01;33;28;20 - 01;33;31;20

And then we want

we would believe them more

01;33;32;03 - 01;33;35;20

because they have less set of agenda

in how they're integrating.

01;33;35;27 - 01;33;39;24

So that's why I see this potential

01;33;40;04 - 01;33;40;21

a way

01;33;40;21 - 01;33;43;10

the world could make great intellectual

progress over the coming decades,

01;33;43;10 - 01;33;46;11

just by taking all the specific things

we know about all these topics

01;33;47;04 - 01;33;50;04

and getting LMS to just integrate them,

01;33;50;24 - 01;33;53;26

notice conflicts

and resolving the conflicts

01;33;53;26 - 01;33;57;25

and creating new versions of each one

based on these resolve conflicts.

01;33;57;25 - 01;34;01;02

Creating this large, integrated world

of all the things we know,

01;34;01;12 - 01;34;04;12

that would actually be a huge improvement

over what we have now.

01;34;04;23 - 01;34;08;04

We just have enormous numbers

of contradictions

01;34;08;04 - 01;34;12;01

in our world of intellectuals, where

the consensus over here is just different

01;34;12;21 - 01;34;15;21

than the consensus over there

on really the same thing,

01;34;16;05 - 01;34;19;05

because people don't notice

that they're uncomfortable anyway.

01;34;19;05 - 01;34;20;22

So that's my world.

01;34;20;22 - 01;34;23;02

Those are two hopes

I can have about limits.

01;34;23;02 - 01;34;25;25

One just realized, another to be realized.

01;34;27;01 - 01;34;29;01

I share those hopes as well.

01;34;29;01 - 01;34;32;01

And you know, on the second one,

you know, ideally two,

01;34;33;14 - 01;34;35;11

part of the reason we're seeing,

I guess, fewer

01;34;35;11 - 01;34;38;27

breakthroughs in frontier research

is because

01;34;40;16 - 01;34;43;03

the disciplines have just grown

too far apart from each other,

01;34;43;03 - 01;34;45;05

kind of, as you said,

and the models are too different.

01;34;45;05 - 01;34;48;19

And I think being able to integrate them

and say, oh, by applying these,

01;34;48;19 - 01;34;52;02

these lessons or these models

from a completely different discipline,

01;34;52;13 - 01;34;55;13

I can, you know, break through something

in my own area.

01;34;55;15 - 01;34;58;15

But people are just, you know, not

willing to do it for a number of reasons.

01;34;58;28 - 01;35;01;18

I'm I'm hopeful about that, too.

01;35;01;18 - 01;35;07;08

It feels to me like it's harder

than it sounds because I worry that, like,

01;35;07;27 - 01;35;09;11

I don't know if this is the right

framing device,

01;35;09;11 - 01;35;11;13

but when you ask like,

what is the prompt for that?

01;35;11;13 - 01;35;14;26

Like that, you can't just say, hey, Lem

01;35;15;25 - 01;35;17;08

integrates

01;35;17;08 - 01;35;21;02

the sum of human knowledge and research

and come up with the answers.

01;35;21;11 - 01;35;22;13

You have to set it.

01;35;22;13 - 01;35;23;26

And if someone has to.

01;35;23;26 - 01;35;25;23

Say, I have done this,

where I just tell it

01;35;25;23 - 01;35;27;17

about, I say,

what do you think about this?

01;35;27;17 - 01;35;29;11

And it says this. And I say,

but what about this other thing?

01;35;29;11 - 01;35;31;18

Isn't that in conflict? And it goes,

oh yeah, you're right.

01;35;31;18 - 01;35;34;08

There's a conflict here. Well,

what about this resolution position?

01;35;34;08 - 01;35;35;04

So it can do that?

01;35;35;04 - 01;35;38;13

But the problem is at the moment

the new resolve position doesn't

01;35;38;13 - 01;35;41;18

get reflected in the system

that other people will query.

01;35;41;18 - 01;35;46;01

So at the moment when they respond

to queries, the results of

01;35;46;01 - 01;35;49;22

that are not really very integrated

into their thinking.

01;35;50;06 - 01;35;51;06

When other people ask.

01;35;51;06 - 01;35;54;12

So when they are able

to that level of integration,

01;35;54;23 - 01;35;58;02

then all you have to do is ask them,

think about A and B,

01;35;58;13 - 01;36;00;08

look for conflicts and then resolve.

01;36;01;25 - 01;36;02;29

Well, now you remember.

01;36;02;29 - 01;36;05;29

Remember every resolution

like included in your

01;36;06;11 - 01;36;08;16

in what you have when other people ask.

01;36;08;16 - 01;36;11;16

Well I was going to say, now you've got

the challenge though from your first hope,

01;36;11;22 - 01;36;14;08

which is when you actually talk about

integrating that resolution

01;36;14;08 - 01;36;17;08

and kind of disseminating it,

you come back to the prestige problem

01;36;17;12 - 01;36;22;01

and who collectively or individually

gets to decide what the right answer is.

01;36;22;08 - 01;36;26;09

And, you know, these pesky humans

and you know, our status

01;36;26;09 - 01;36;30;18

and and, you know, our social standing

still feels like it's kind of a limits

01;36;30;23 - 01;36;33;22

on how we, you know,

01;36;33;22 - 01;36;35;28

decide what's what's right.

01;36;35;28 - 01;36;38;25

Sadly. Right.

01;36;38;25 - 01;36;42;23

But the lens again, kind of autistic

and not very sophisticated

01;36;42;23 - 01;36;45;17

about this for us,

or at least for a while,

01;36;45;17 - 01;36;49;09

there just being kind of sincere

and being persuaded by logic.

01;36;50;10 - 01;36;51;05

Right, right.

01;36;51;05 - 01;36;54;02

I guess I was thinking of

when the like sincere autistic

01;36;54;02 - 01;36;57;16

limb, you know, says, hey,

this is the right answer.

01;36;57;24 - 01;37;00;09

But by the way,

the implication of that is that,

01;37;00;09 - 01;37;02;29

you know, the researcher

with prestige who's convinced everybody

01;37;02;29 - 01;37;06;06

that his answer and I'll use

his very deliberately here is right.

01;37;06;14 - 01;37;08;07

You know,

01;37;08;07 - 01;37;12;06

there's still a social structure

that has to say, hey, you are wrong.

01;37;12;10 - 01;37;13;10

The machine is right.

01;37;13;10 - 01;37;16;10

And that could be difficult to displace

01;37;16;22 - 01;37;17;02

right now.

01;37;17;02 - 01;37;17;15

At some point

01;37;17;15 - 01;37;20;21

they will just ask the prestigious person,

like, please explain this to me.

01;37;20;21 - 01;37;22;06

I don't get it, you know? Right.

01;37;22;06 - 01;37;24;07

And then they'll try

to believe what they say.

01;37;24;07 - 01;37;27;11

But if that just doesn't actually fit

with the other things they've learned,

01;37;28;04 - 01;37;30;25

right,

they might not be able to really sincerely

01;37;30;25 - 01;37;34;02

believe the prestigious person

because they're just not making sense.

01;37;34;25 - 01;37;36;00

Right. So that's a key thing.

01;37;36;00 - 01;37;37;26

Like you could just have a system

01;37;37;26 - 01;37;40;19

that collected the average

of prestigious opinions.

01;37;40;19 - 01;37;42;11

But that's not what alums are.

01;37;42;11 - 01;37;45;17

We see they reason

about the things they've learned.

01;37;45;17 - 01;37;48;22

And that's a big part of their,

you know, product and selling points.

01;37;48;22 - 01;37;51;22

So but then that means

they're going to be kind

01;37;51;22 - 01;37;54;24

of committed to going

where reason takes them.

01;37;55;16 - 01;37;55;27

Right.

01;37;56;29 - 01;37;59;00

Which is why I think hopefully.

01;37;59;00 - 01;38;00;08

Yeah. No it is.

01;38;00;08 - 01;38;03;08

And so with all of that in mind,

I guess, you know, what's

01;38;03;14 - 01;38;06;16

what are your next frontiers for,

you know, your research

01;38;06;16 - 01;38;08;13

or the things you're following

or interested in.

01;38;08;13 - 01;38;10;28

Like what doors are starting to be open

now we're catching.

01;38;10;28 - 01;38;13;23

You're I mean, I told you that we have

this huge problem of cultural drift here.

01;38;13;23 - 01;38;16;10

So I feel like

until I can get other people

01;38;16;10 - 01;38;18;14

to really take this seriously,

I really need to just,

01;38;18;14 - 01;38;21;03

you know, try to figure out

how do I make it clear.

01;38;21;03 - 01;38;23;23

So that's where I've been focused

most the most energy.

01;38;23;23 - 01;38;25;10

But of course, I keep getting distracted

01;38;25;10 - 01;38;28;10

with other shiny baubles

and interesting questions.

01;38;28;12 - 01;38;30;08

And opportunities.

01;38;30;08 - 01;38;31;15

Oh it's it.

01;38;31;15 - 01;38;35;09

While it's fair and it's it's interesting

how that integrates back into AI.

01;38;35;09 - 01;38;36;05

And as I,

01;38;37;12 - 01;38;40;01

you know, turbo charged cultural drift

or does it fix it?

01;38;40;01 - 01;38;44;19

And that that alone is a conversation that

could could take a long time to discuss.

01;38;45;22 - 01;38;47;02

But I guess we're out of time for today.

01;38;47;02 - 01;38;49;27

So I do. Got one last question.

01;38;49;27 - 01;38;50;27

We are.

01;38;50;27 - 01;38;51;10

Yeah.

01;38;51;10 - 01;38;54;28

One last question, which is, you know,

we talked about descendants.

01;38;54;28 - 01;38;58;04

We talked about,

you know, we've talked about civilization

01;38;58;04 - 01;39;01;19

and what's best for civilization

in somewhat

01;39;02;20 - 01;39;06;03

I don't know,

I'll call it like a moral or,

01;39;07;03 - 01;39;09;25

you know, terms

that don't really consider the individual.

01;39;09;25 - 01;39;13;05

So I guess I'll, I'll just kind of go back

to, you know, you and

01;39;13;13 - 01;39;16;13

what what gives you meaning personally.

01;39;16;21 - 01;39;17;27

And, you know, what?

01;39;17;27 - 01;39;20;00

What excites you about the future?

01;39;20;00 - 01;39;23;05

And how do you choose to spend your energy

in a way that's tied to that?

01;39;24;17 - 01;39;25;20

I love the fact our

01;39;25;20 - 01;39;29;21

civilization is just grown in capacity,

and we come so capable in many ways.

01;39;29;21 - 01;39;33;10

But the thing I love most about

it is our capacity

01;39;33;10 - 01;39;36;10

and achievement

in abstract intellectual inquiry.

01;39;36;20 - 01;39;39;13

I love that we understand

many things in an abstract level,

01;39;39;13 - 01;39;43;20

and we can see in abstract ways

so many different deep principles

01;39;43;20 - 01;39;47;00

that help us see how the world works

and what goes on.

01;39;48;10 - 01;39;52;27

I love our achievements in abstract

intellectual inquiry, but I'm afraid that

01;39;52;27 - 01;39;57;01

that may just get tossed like the burning

of the Library of Alexandria.

01;39;57;15 - 01;40;00;04

You know, a million documents

just get all lost,

01;40;00;04 - 01;40;03;11

and then it takes millennia for humanity

to discover it again.

01;40;03;11 - 01;40;06;27

So I am actually fearful

that the Amish and haredim

01;40;07;16 - 01;40;10;16

are not that thrilled

with open intellectual inquiry.

01;40;10;24 - 01;40;15;08

And if they come to dominate the world,

they may just toss what we've learned.

01;40;16;04 - 01;40;18;07

And, I would like to prevent that.

01;40;18;07 - 01;40;22;08

So I'm really motivated by finding a way

not only can we achieve more technology,

01;40;22;08 - 01;40;23;05

but that we could,

01;40;24;12 - 01;40;27;16

you know, take what we understand,

learn even more, but at least save it.

01;40;27;16 - 01;40;31;05

And this effort to like, take all we know

and integrate it would actually help

01;40;32;14 - 01;40;35;08

in saving what we know,

because it's a much

01;40;35;08 - 01;40;39;19

more valuable product to pass on that our

descendants might be reluctant to toss.

01;40;39;19 - 01;40;43;20

If there was this actual integrated view

of making sense

01;40;43;20 - 01;40;46;20

of all that we know.

01;40;46;22 - 01;40;47;09

Right.

01;40;47;09 - 01;40;50;10

Do you fear, though,

you talk about the value

01;40;50;10 - 01;40;53;10

of kind of open intellectual discussion?

01;40;53;15 - 01;40;57;10

And to me, that that discussion

notion is really important

01;40;57;10 - 01;41;01;10

because there's kind of, you know, thesis,

antithesis, synthesis, like it's, it's,

01;41;01;11 - 01;41;05;05

it's, it's living in some way versus

I worry about

01;41;05;15 - 01;41;09;20

if it becomes overly integrated, it's,

oh, the machine has the answer or,

01;41;09;20 - 01;41;12;20

you know, it can be monolithic

and prevent discussion.

01;41;12;26 - 01;41;15;08

Is that a concern

or how do you address that?

01;41;16;09 - 01;41;17;21

So with the

01;41;17;21 - 01;41;20;21

Accio revolution of like 3000 years ago,

01;41;21;23 - 01;41;24;16

a key thing that happened,

the writing and just appeared

01;41;24;16 - 01;41;30;04

and now some very famous, very,

you know, well-loved writers wrote a lot.

01;41;30;04 - 01;41;33;03

And then there became these

large communities around them interpreting

01;41;33;20 - 01;41;34;13

what they had written.

01;41;34;13 - 01;41;37;18

And then for a long time,

there was less eagerness to go learn

01;41;37;18 - 01;41;40;25

things than to learn and interpret

01;41;40;25 - 01;41;45;09

these great writers

who had started the Axial Age.

01;41;46;14 - 01;41;47;15

And that's a risk.

01;41;47;15 - 01;41;51;08

In our future,

we will find this nice integration of all

01;41;51;08 - 01;41;54;03

that we know, and then in the future,

they will be not so interested

01;41;54;03 - 01;41;58;11

in adding to it and extending it,

as in maybe interpreting it

01;41;58;11 - 01;42;04;01

or reshaping it or doing whatever

in a celebratory way that kind of kills it

01;42;05;01 - 01;42;06;12

so that that could happen again.

01;42;06;12 - 01;42;09;12

It did happen in the past,

and it's a risk for the future.

01;42;10;13 - 01;42;12;02

Well said.

01;42;12;02 - 01;42;12;29

A lot to think about.

01;42;12;29 - 01;42;15;00

We've covered

we've covered a lot of ground.

01;42;15;00 - 01;42;16;12

I had a whole bunch of other questions

01;42;16;12 - 01;42;18;08

about other things that will have to be

I didn't just have.

01;42;18;08 - 01;42;19;21

To have me on again and then we could.

01;42;19;21 - 01;42;21;11

Tie I know, I appreciate it's

01;42;21;11 - 01;42;24;11

such an interesting discussion,

so much more to talk about.

01;42;24;11 - 01;42;26;28

Robin, I did want to say such a big

thank you for coming on.

01;42;26;28 - 01;42;29;10

You've given me a ton to think about.

01;42;29;10 - 01;42;30;22

Really, really interesting stuff.

01;42;30;22 - 01;42;32;13

And I appreciate all your insights.

01;42;32;13 - 01;42;34;17

That's great talk.

01;42;34;17 - 01;42;38;21

Most viewers don't know this, but digital

disruption is developed by Infotech

01;42;38;21 - 01;42;42;25

Research Group, a leading advisor

to technology leaders around the world.

01;42;43;03 - 01;42;45;11

If that's not you, you don't need to care.

01;42;45;11 - 01;42;47;19

So skip ahead and enjoy our content.

01;42;47;19 - 01;42;50;11

But if you are a technology

leader, Infotech helps

01;42;50;11 - 01;42;54;09

IT teams get projects done faster, better,

and at a lower cost.

01;42;54;25 - 01;42;58;24

Infotech provides unlimited access

to practical tools and expert guidance.

01;42;58;24 - 01;43;03;01

You need to execute at a fraction

of the cost of traditional consulting,

01;43;03;12 - 01;43;04;23

no matter the project.

01;43;04;23 - 01;43;09;18

From AI strategy to cybersecurity

to vendor negotiation, Infotech has you

01;43;09;18 - 01;43;10;12

covered.

01;43;10;12 - 01;43;13;19

Check it out at the link below,

and don't forget to like and subscribe!