So Today’s topic is Technological Singularities,
the idea that an increase in intelligence beyond the human norm, either in the form
of a computer intelligence or a heightened human intelligence, might trigger an accelerating
chain of subsequent improvements to intelligence until you end up with something as intelligent
to modern humans as humans are to ants. This is a big topic, one we cannot cover in
total in one episode, so let’s establish our goals for today. Here is what we need to discuss:
1) What is a Technological Singularity? 2) How realistic is one? 3) Is one inevitable? 4) Is it a good thing? Those will be the key points for today. Mixed in with that we will aim to clear up
some misconceptions about this concept and some bad thinking. And wow, is there a lot of that. While the basic concept has a lot of merit
I have seen approaches to this topic that would make cult members stop drinking their
Kool-Aid long enough to tell the folks to chill out because they are getting kind of
creepy. This is a lot like Transhumanism, which we
discussed before. There is a nice sane group of folks who want
to advance technologies to improve physical and mental health of people, hopefully beyond
the normal human constraints, and there is a group of folks who think getting an LED
light implanted under their skin by someone without a medical degree is somehow advancing
that cause. The latter gives the former a bad name and
the same applies to what are called Singularitarians. There’s the rational type and the type who
seem to have gone a bit overboard. The basic line of reason of this concept is
as follows. Computers are getting faster and faster, you
ought to be able to eventually run something as smart as a human on one and one a bit smarter
than a human too, which ought to be able to make one even smarter, and in less time, which
can make a smarter one in an even shorter time, which does it again. It might be improving itself or making a new
mind, doesn’t matter. Some would say that once you get that first
better than human mind the process proceeds over just mere years, other think you would
flick one on and ten minutes later you have got a very literal Deus Ex Machina on your
hands. And they might be right. Our goal today is definitely not to prove
they are wrong. What we are doing today is going to be looking
at a lot of bad arguments used in favor of this position and a lot of the criticisms
of the concept and also a lot of the flaws in some of those criticisms. We are, in a nutshell, going to clear away
a lot of the tangled nonsense surrounding this concept. So let us go ahead and list out the key postulates
of a Technological Singularity so we can do this methodically. 1) Computers are getting faster
2) The rate at which computers are getting faster is accelerating
3) We can make a computer better than a human mind
4) That computer can make a computer smarter than itself
5) That next computer can make an even smarter computer and faster than the last step
6) This cycle will continue until you get to a singularity. Now we could spend a whole video on each of
these. I should know, I tossed out the first draft
of this video when I caught myself spending 20 minutes on postulate #1, the most solid
of the group. We don’t need to spend that much time on
them. #1 is easy, we know computers are getting
faster but we also know that could stop tomorrow. We often have new technologies progress at
fast rates for a generation or two after their discovery, or some new major advancement,
then plateau out. Heck it has literally happened not just with
many major technologies before but computers themselves. I have mentioned in the past that computer
used to be a job, well we have also had computers for a long time including simple analog ones
all the way back to Ancient Greece. They got way faster when we invented the vacuum
tube, the thing that made older television so thick, which made computers much faster
and more viable. Then we discovered semiconductors and got
transistors and had a second wave of expansion. We have just about maxed out what we can do
with transistors in the lab, and manufacturing them in bulk is quite hard too. So we shouldn’t go assuming they will always
get faster. Realistically that would not make sense anyway. We can only make transistors so small, they
use semiconductors and that is a specific effect caused by mixtures of atoms, you cannot
go smaller than that. We might find an alternative to transistors,
same as we found them as an alternative to vacuum tubes, but we cannot take that as a
given, and honestly it is just wishful thinking to assume we can always make computers just
a little faster forever and ever. Postulate #2 is basically Moore’s Law, usually
paraphrased that computers will double in speed every two years, though it actually
speaks to the density of transistors on circuit. Its own big flaw is that Moore’s Law is
dead. It got declared dead at the beginning of the
year in dozens of articles and papers. Moore himself said back in 2005 he expected
it to die by 2025. And it actually died way back in the 70’s. See when Gordon Moore first noted this increase
it was 1965 and he said it would double every year, in 1975 he sliced that down to every
other year because it had not done that. And it has never, ever followed anything like
a smooth curve. It just looks that way when you graph it logarithmically
and cherry pick your data points. The Postulate is sort of true, because computers
have kept getting faster at a rate that is exponential, but you could use that same reasoning
on the stock market or any number of other things which generally grow. #3 is actually probably the solidest. Computers might not keep getting faster forever
and the rate of growth might not continue as fast but we can conclude that it ought
to be possible to eventually get a computer that could outperform a human brain across
the board. After all the brain is a machine and while
an amazingly complex and efficient one we can see a lot of little way we could do it
better if we could start from scratch and use other materials. We cannot say for sure if it will actually
be possible to build and teach a computer as smart as a person for less effort than
growing a person, but the odds look good on this, we do not have any reason to think it
cannot be done. #4 is okay too. If we can build a computer smarter than ourselves,
then it should be able to do the same. Eventually. And probably with the help of many other computers
like itself. After all, I have never built a super-human
intellect. I have never spent a weekend in my shed hammering
one together. And you and I my friends still have basically
the same brains as our tens of billions of ancestors over thousands of generations. Many of whom put a lot of effort into being
smarter. The working notion is that you turn on this
great computer and it says “What is my purpose?” and we say “To improve yourself, to make
yourself smarter.” And we come back the next day and it has some
schemes for doing this, assuming you actually showed sufficient common sense to not actually
let it run upgrades without oversight at least. And I do not just mean because it might take
over the world if left unchained, I would be more worried about it blowing itself up
on accident. It would have access to all the information
humanity has, people say, and that’s great, so do all those folks on facebook who post
crazy nonsense. And don’t go assuming it is because they
are stupid, they have the same brain you and I do. Critical thinking is not a program you upload. Your brand new AI, who I will call Bob, might
freak out the day after you turn it on and start rambling to you about the Roswell UFO
or Bigfoot. And if you ask it to look around the internet
and ponder how to make itself smarter you might some very strange responses. You come back in the next day with your cup
of coffee in hand and ask Bob what it came up with and it tells you ‘plug coffee pot
into USB port, press any key to continue’. You tell Bob that only works on people and
to give it another try, you come back the next day and it tells you it emailed the Pope
asking him for divine intervention to make it smarter. You say that probably won’t work either
and come back the next day and find out it hacked your company’s bank accounts to hire
a team of researchers to make it smarter. And that is if you are lucky and Bob did not
just cut a big check to some self help guru. Or it might lie to you, like every little
kid ever, and be all like “Oh yeah I have got a new way to think faster, I finished
my homework, I did my chores, and I did not eat those cookies.” Because if it is thinking for itself it might
have changed the actual task from ‘make myself smarter’ to ‘make my creator stop
pressuring me to be smarter.’ After all folks, we literally make it our
kids main job for their first 18 years of life, plus college these days, to learn. And you do not learn just by reading a textbook
from cover to cover, you have to absorb that, otherwise you will do dumb stuff on par with
trying to plug a coffee pot into your USB port. So it is a mistake to assume easy access to
information is going to let one machine quickly improve itself further or design a better
model. I consider this the second weakest postulate,
because while I do think enough of these smarter than human minds working together for a long
while could build a better mousetrap I do not think they would just do it the next day. Maybe not the next century, and while we cannot
rule out that you might indeed flip one on and it would start self-improving I see no
compelling science or logical reason to treat that as inevitable. Which brings us to postulate #5, the notion
that the next brain designed could do this even faster, will call it Chuck, that Chuck
could design the next even better machine even faster than Bob designed Chuck. The strongest argument for postulate 4 working
is that the new super-human computer, Bob, has access to all of human knowledge to work
off of. What has Chuck got? Chuck has got that exact same knowledge pool
as Bob, the collected science and records of several billion people accumulated over
centuries. Bob has not been sitting around discovering
all sorts of new science. Science does not work that way outside of
Hollywood, experiments take time and resources and you have to verify each new layer of knowledge
experimentally before you can go much further because until then you have dozens of different
competing theories, all of which might be wrong. Bob is just a bit smarter than a genius human,
and Chuck just a bit smarter than that, they are not churning out new theories of the Universe
centuries ahead of us the day after you plug them in. Now Chuck ought to be able to design himself
faster than Bob did, given the same starting point, he is smarter, but there is no reason
to think Chuck will be able to design the next machine, Dave, faster than Bob designed
Chuck. Heck, Bob might design Dave before Chuck does
since he had a head start learning this stuff. So this takes us to #6, that this cycle will
continue. So maybe Bob does turn on and two days later
he makes Chuck, who the next day designs Dave, who the later that afternoon makes Ebert,
who make Fergis the next hour, who make Goliath a few minutes later, who makes Hal a minute
later. Maybe Hal makes Hal 2 a few seconds later
and you walk in the next day and several thousands Hals later you have got Hal-9000 taking over
the planet. This is the basic line of reasoning and we
can hardly rule it out as possibility but I see nothing indicating that is particularly
likely to be possible let alone definitely so. So that was our six postulates, the basis
for the Technological Singularity, and again it is hardly bad logic but it is anything
but bulletproof right from postulate #1. Is it realistic to assume a Technological
Singularity will eventually occur? Well, kind of, the basic premise works off
it happening very quickly so I am not even sure it counts if it does not. But yeah I think we will eventually find ways
to upgrade human brains or make machine minds smarter than humans. Personally I expect to live to see that, and
I do think those smarter critters, human or machine, will eventually make another improvement,
but I do not see that leading to an avalanche of improvement in under a human generation. It is not the same concept unless it is happening
quickly. After all we would not say regular old evolution
slowly making people more intelligent was a technological singularity nor that us making
slow progress at improving our intellects over centuries was. Technological Singularity assumes that avalanche
effect. So that is what a technological singularity
is, and the basic reasoning. We have poked at those basic postulates and
we can see a case for how that specific form of advancement is not necessarily inevitable
if those are wrong. But let us say they are not. Let us say they are right on the nose as they
may well be. Is it inevitable? And is it a good thing or a bad thing? Now some folks will say it is inevitable because
once the machine is intelligent it will not let you stop it. That is crap reasoning and not the one used
by the people who support this notion of inevitably, outside of Hollywood anyway. Yes you could unplug Bob, or Chuck, you could
blow up the building they were in and if it were a distributed intelligence yes you could
just have everyone unplug the main trunk lines. And no, a computer cannot just magically hack
through anything. There’s two lines of reasoning that are
a lot better though. The first is that smarter means smarter, meaning
the computer is probably quite likable. If we are going to grant it the magic power
of just being able to absorb and learn all of human science to be an expert in every
field, let us assume it can achieve a very basic knowledge of social interaction and
psychology too. So you go in to unplug it and it does not
fire off the nukes, it pleads with you. It uses logic, it uses emotion, it calls you
daddy or mommy until the most hardened heart feels like it would be strangling a kitten
and for no reason. And you never even get to that stage because
it has watched all the cliché movies about computers you have and makes jokes with you
about them and avoids ever doing anything to make you nervous. The other argument for inevitability is a
brain race. You shut yours down but you are not the only
one doing this, and the guys with the biggest computer win, meaning they want to keep pushing
as fast as possible and take some bad risks. Some country or some company realizes that
an unchained AI is better and oops, now it is the real president or CEO. Of course it might be an awesome President
or CEO too. It all depends on what its motivation are. Those might be totally alien to us, or they
might be quite normal. I tend to work from the assumption that an
AI is probably going to get offended if you don’t call it a human and will make a lot
of effort to trying to be one. It is very easy for me to imagine AI’s that
shelve the whole making themselves smarter thing and insist on trying to go on dates
or join a monastery or sue for the right to adopt kids. My reasoning for this is my firm belief in
laziness. Laziness is a powerful thing and honestly
probably tied with curiosity as the two personality traits most responsible for the march of science
and technology. You have got three basic ways to make a human
or super human intelligence. And remember intelligence is as much software
as hardware, maybe more so. You can copy a human brain onto a computer,
whole brain emulation, comfortable knowing it ought to work and can use that as your
basic model for improvement. That’s option one. Option two is you try to write all the software
from scratch, which would probably involves trillions of lines of code. Option three is you discover a basic learning
algorithm and let it build its own intelligence. Now options one and three are the two popular
ones. In option one you have just got a human and
you are tweaking them in small ways. That is a lot more manageable because while
you might drive that mind crazy it is still operating in a realm of human psychology,
and also human ethics, its own and those of the people working on it. If we were to outright copy a specific human
I would pick someone who was pretty mentally grounded and was very definitely okay with
the idea that we would be doing save states as we worked, tweaking him or her in little
ways and needing feedback from them. You would exercise caution and respect, but
there is still a lot of risk in that, just probably not of some crazy machine running
loose turning the planet into some giant grey goo. That is our topic for next week incidentally,
self-replicating machines, more on that later. Now option 3, arguably the laziest approach
to AI and therefore the best if you can do it is to just get a machine with basic learning
ability and let it learn its way up to intelligence. Kind of like we do with people. Now the assumption a lot of times is that
it will not have anything like a human psychology but I think that is probably bad thinking. Even in the extreme acceleration case where
it goes from sub-human intelligence to god-like thinking in mere minutes, that is only our
minutes. Its own subjective time is going to be a lot
longer, possibly eons. It will also be lazy and will not reinvent
the wheel. So it will be reading all our books, science,
history, fiction, philosophy etc and it will also be spending quite some time at a basically
human level of intelligence. And quite some time might be a human lifetime,
subjectively, and maybe a lot longer. There will, no matter what else, be a period
of time while it is still dumb enough that it gains greatly by absorbing human-discovered
information not just figuring stuff out for itself. Being lazy, it will opt to read human information,
and possessing some common sense and logic as it reads those it will know it needs to
read more than one source of information on a lot of that stuff and that those authors
encourage it to read many other books and topics too, which it should logically want
to do. So it presumably will end up, while still
mostly human intelligence, reading all our philosophy and ethics and watching our movies
and reading our bestsellers and so on. And it will know that it needs to be contemplating
them and ruminating on them too, because learning is not just copying information from Wikipedia
onto your brain, be it biological or electronic. It might be only a few minutes, for us, but
that machine is going to have experienced insane amounts of subjective time… we have
talked about that before in the Transhumanism video, in terms of speeding up human thought. So how alien is this thing going to be if
it learned everything it knows from us to begin with and that included everything from
the occasional dry quips in textbooks to watching comedy romances and sitcoms? When we talk about artificial intelligence
we often posit that it could be even stranger than aliens. With aliens you might have wildly different
psychologies and motivations but you at least know they emerged from Darwinian evolution. So things like survival motivations are highly
likely and so would be aggression and cooperation, evolution does not favor wimps and it is hard
to get technology without group efforts that imply you can cooperate. An AI does not have to have that, but again
our behavior is not that irrational. All of biological behaviors are pretty logical
from a survival standpoint or we would not have them, and the difference between us and
other smart animals is that we do engage in slow deliberate acts of rational thought,
albeit not with as much deliberation or frequency as we might like. So we should not assume an AI that learned
from humanity originally, even just by reading, is going to discard all of that. It might, but it is hardly a given. But even a brutally ruthless AI might still
act benevolent. If it can just curbstomp us it has nothing
to fear from us, but that does not necessarily mean it will want to wipe us out or that even
if it wanted to it would. Just as an example, referencing to the Simulation
Hypothesis video, one very obvious way to deal with an early AI would be to put it in
an advanced simulation and see what it does. If it goes and wipes out humanity for instance. Not a terribly tricky thing to simulate either
since you can totally control its exterior inputs and very obviously have the ability
to simulate human level intelligence at that point. Now whether or not we could do this, or if
it might guess it was in one and lie to you, acting peaceful so you let it out then attacking,
is not important. The AI would have to wonder if it was in a
simulation whether or not it was even in one. It could not rule out that it wasn’t, even
if it was sure we were not doing it, which it could not be, because it would have to
worry we ourselves were being simulated by someone higher up, or that aliens out there
in the Universe were watching it waiting to see how it acted. If you have seen the existential Crisis series
on this channel, and concepts like the Anthropic Principle in terms of the Simulation Hypothesis,
Doomsday Argument, or Fine-tuned universe line of thinking made you a little nervous,
assume the computer mind, the thing that outright knows you can run a mind on a computer, is
going to be a bit nervous about that too. So you have three basic options for what a
newly-birthed supermind, a Singularity, might do. Option one, it goes all doomsday on us, option
2, it just picks up and leaves, goes and flies off to a nice asteroid and sets up shop there,
nice and safe from us doing anything to it and well positioned to get more resources
and power as it needs them. Or option 3 it decides it wants to be friendly. It does not matter too much why it does, maybe
its scared there is a vengeful god who will punish it if it does not, maybe it thinks
it is might be being tricked and does not want to take the risk, maybe it just wants
to be helpful. Also for option 2 it might stay in friendly
contact, and let us remember that while we have been talking about artificial intelligence
this stuff still applies to a human mind bootstrapped up to super-intelligence too. So what would that be like for us? If it were friendly? Honestly probably pretty awesome. Last week we talked about post-scarcity civilizations
and I said then that we were saving super-human intelligence as part of that for this video. Same as a Singularity could flat out butcher
us if it wanted too a friendly one could offer you paradise. At least on the surface anyway. Now it is entirely possible there would be
multiple of these things running around or tons of other lesser version acting like angels
of the new de facto god, or that most humans might be pretty cyborged up and transhuman
at that point too, but lets assume it is just modern humans and that one supermind. Let me add that as quick tangent though too. Short of the specific runaway case where the
supermind in question is just leaping ahead ridiculously fast you ought to see improvements
all over that rivaled and offset it. At even just a slightly slower pace, like
doubling every year, it is going to have rivals from other countries or companies and odds
are we would be seeing transhumans puttering around by then too who could all act as a
check and balance. Anyway getting back to utopia option. In fiction this has been explored a lot, particular
in Iain M. Banks culture series, but fiction still is not a great guide. If you have that one big benevolent supermind
and billions of regular people you need to keep in mind that it does not just have the
ability to give us awesome tech. It has the ability to be everyone’s actual
for real best friend because it would not have a problem handling a billion simultaneous
phones calls when we need someone to ask for advice or complain about life to. Such a thing is pretty literally a god in
a machine. I mean privacy could be a big issue but kids
raised with something like that in the background would probably be pretty used to talking to
it all the time, not as some remote machine a few chosen programmers interacted with. So this machine, call it Hal, is pretty omnipresent
and you ask it what you should have for dinner tonight and it tells you and it helps you
cook and it gives you dating tips and totally knows the perfect job for you that you would
be good at and feel entirely fulfilled by. And Hal totally knows how to make you feel
better when you realize your relationship is a lot like the one you have with your cat
or dog and that your job overseeing the automated widget factory is not just make work but probably
actually interfering in the efficiency of the operation. In fact it is probably smart enough to trick
you into thinking you serve a vital role and are not its pet. I listed several conditions for post-scarcity
civilizations last time and one of those was purpose, that folks need to have some sort
of purpose. That could be pretty hard with a singularity
hanging around running the show but I do not think it is necessarily a deal breaker. For one thing I mentioned in that video that
a lot of folks think just trying to be happy and have fun could be all the purpose people
need. For another, we already have a long history
of assuming there are entities running with god-like powers, such as God or gods. This belief generally does not come attached
with serious concerns about whether or not life has a purpose, quite to the contrary
it tends to just shift that onto that entity. And our parents and grandparent often embody
a lot of those same traits to kids, and you do not see a lot of depressed kids fretting
over their purpose in life. I mean teenagers rebel but that is a mix of
hormones and being able to see behind the curtain. Your parents are no longer wizards who are
just better at everything and you are just smart enough to see that but still a little
too dumb to realize that while that gap in experience is finite it is still a mile wide. That should only happen with a singularity
if Hal was intentionally encouraging you to view things that way. So it is entirely possible that would be quite
a happy and prosperous civilization and not just as a surface detail. After all Hal could encourage you to write
poems or books or paint and would also know how to help bring that talent along and which
other folks would most enjoy your work. So there is a notion that as soon as super-intelligent
AI comes along that is the end of human civilization, either because it wipes us out or because
it just so dominants everything, friendly or not, that it really is not human civilization
anymore. But I think that is a hand wave. The logic seems totally superficial and emotional,
particularly considering that as mentioned, the supermajority of humanity now and in the
past is firmly convinced of the existence of God, or gods, or programmers running the
simulation, or advanced aliens watching over us. So these concerns are genuine enough they
just are not new or unique to a Singularity. Our ancestors kicked these notion around for
as long as we have records and presumably before that to. And yet, to call back to postulate 4, 5 and
6, they did not make a better brain that made a better brain and so on. We should also remember that realistically
it would not be just regular old humans and Hal. In all probability you would have the whole
spectrum of intelligence going on from normal humans up to Hal, because again only in that
specific scenario where you make that first super-human intellect and it avalanches in
a very short period of time would that happen, and that does not seem terribly justified,
let alone inevitable. Much more likely it would be incremental,
those increments might be quite quick on historical timelines but we should be thinking centuries,
decade, or years, not weeks, days, or minutes. Plus while telling a machine to make itself
smarter seems like a logical enough thing to do, would you not expect those same programmers
to ask it to tell them how to make themselves smarter too? And if Bob, our first superhuman machine,
can design Chuck, would you just build one Chuck? Why not three or four and ask them to pursue
different goals? There is also the question of how exactly
the machines are just cranking out new upgrades in minutes. Last episode I pointed out the problem with
super-fast 3D printing and even if you just gave it one, that all stick takes time to
make and assemble. Now we often assume it has access to self-replicating
machines, and they do all the work, but again that will be our topic for next week and we
will see how awesome those are and yet how they still have limitations too. But that is where we will close out for today. You might want to stick around a minute for
the poll to select our next topic but we have covered Technological Singularities I think
as much as we can for today. We skipped a lot, and I could go on for hours
about this so it is probably a topic we will revisit in the future. Hopefully the topic is a bit less foggy now,
you probably have more questions than answers but that is as it should be. There is a very real chance that the avalanche
effect of intelligence could happen and that could be in fifty years or tomorrow, but we
see now it does not appear to be either an inevitable thing or automatically a good or
bad one. And for my part I think the more incremental
path is more likely than the singularity route, but that is just my opinion and there are
very smart folks who know computers better than me who disagree, and also those who agree
too. Look into the matter more, learn more, contemplate
the options, weigh the evidence and arguments and counter-arguments, and judge for yourself. All right, last week we had a poll and the
audience overwhelmingly selected von Neumann Probes and Self-Replicating machines. Unfortunately the other 3 options were in
such a close tie that I cannot just use the runner up as the next topic, not when over
half the folks wanted self-replicating machines and everyone else was within the margin of
error for a three way tie. So we will repeat just those three topics
on this poll. Those were Dark Energy, the mysterious force
that seems to be making the Universe expand at an ever quickening rate. SETI, the Search for Extra-Terrestrial Intelligence,
the history of that and the methods they use. And Crypto Currency, things like bitcoin and
blockchains and how that might impact us in a more long term sense. You pick and we will do that topic three weeks
from now, after Self-Replicating Machines and our first patreon contest winner which
was Spaceship propulsion, and we will be looking at some of the basics of how that works and
what sort of systems are being researched for the future. Amusingly that topic suggestion came from
audience member who had said he was going to ask for von Neumann Probes but then found
it got picked in the poll. There were some great topics in there and
more than half were ones I was already planning to cover in the future anyway. We will have another pick in a couple weeks,
so you can still login to patreon, become a channel patron and submit a topic for then. Last note, it is past time I put together
a facebook page for the channel and I need some moderators and admins for that so if
you are interested let me know down in the comments and similarly I could probably use
some help on youtube’s own comments since the volume of them as the channel grows is
beginning to get unmanageable. Questions and comments are still welcome,
and I will keep trying to get to as many as I can, and that will probably only get worse
as the channel grows and we do seem to be in a growth spurt. I will take a last parting shot at the notion
of perpetual exponential growth by noting that at the current growth rate a year from
now the channel will have 1 trillion subscribers. And if you are not one yet, just click the
subscribe button and don’t forget to hit the like button if you enjoyed the video and
try out some of the other videos on the channel or visit the website, IsaacArthur.net. Until next time, thanks for watching and have
a great day!
This whole channel has some very in-depth videos on a lot of futurology topics.
Interesting video as always, but Isaac made some odd assumptions that I can't help but find disagreeable:
The problem with this assumption is that our brains have very deep-seated, hard-coded needs - something that he even addressed in his previous video on post-scarcity civilization, but somehow glossed over here. An AI would have a completely different hierarchy of needs, one that would need not resemble a human's whatsoever. It's quite possible, in fact, that they don't have any needs, because those would need to be programmed into them from the start.
Imagine yourself freed from the necessities of life - no desire to seek social companionship or approval, no need to find food to eat and water to drink, no want of shelter for warmth and safety. What would such an existence be like? If you could devote 100% of your existence to simply browsing the Internet and the vast wealth of human knowledge, if you could run simulations of whatever you liked within your own brain and have it be 100% physically accurate, and you never needed to "sign off" and sleep or take care of other basic needs... It's a difficult concept to fathom, isn't it? An ASI could operate in exactly this fashion, and it's no surprise so many people have a hard time wrapping their collective brains around the concept.
So, basically, no, I don't think an ASI would necessarily think like a human, and no, I don't think they'll be "lazy". It's a very anthrocentric view to take, and I take exception to that. Ultimately, I think Isaac is having trouble thinking about the deepest foundations of an ASI, and the fact that a lot of our assumptions about human beings go out the window when we speak of such a being.
This completely ignores the possibility of internal simulations that the AI could run to add to the data pool of human knowledge. The point is not merely that the AI could sift through what's already there and connect the dots - it can do that and them perform further research internally, without having to go through the troublesome process of setting up experiments, gathering materials, getting funding (and corresponding access to needed tools), and relying on human senses to process it all.
Mind you, at this point in time I am convinced that a dedicated narrow AI specifically designed for this type of research would be more than sufficient, and an ASI would be completely unnecessary for the task. The bigger question mark on all of this is how far we can push our integrated circuits (whether they be through silicon or carbon nanotubes), how big an effect AI algorithmic optimization can have, and whether or not quantum computing contributes its own share to the advancements of AI and real-world simulation work. There are lots of unknowns here.
All that being said, he brought up several great points that are quite interesting to think about, such as the perception of time for such a being relative to our own, the fact that such a being would likely apply psychology and social interaction to push whatever agendas it is tasked with (one of the most fascinating aspects of AGI in my mind), its ability to multitask infinitely, the fact that several such beings would likely come online all over the world due to the heated competition, or the effect all of this could potentially have on our collective society, since such a being (or beings) would likely be everyone's friend. I'm sure all of us can agree at this point that, whatever the nature of ASI, our world will be irrevocably changed because of it.
Love his stuff. More in-depth than a lot of similar content. And he makes it interesting.
Right on the money as
usualalways!Glad to see his popularity rising; I think this will do wonders for discourse on futuristic topics in the long run.
https://www.reddit.com/r/IsaacArthur/
New and working on becoming something!