Re: [Fwd: Eliezer speaks (forwardable)] - was loserhood and analysis

Date view Thread view Subject view Author view

From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Fri Aug 18 2000 - 11:09:06 PDT

Jeff Bone wrote:
>
> Eugene's not proposing to *do* anything to me with his uberbeing, like
> subjegate me to some loony set of seed Rules that he thinks are spiffy and
> fair. Yet, anyway. ;-) (Eugene?)

As far as I can tell, 'gene wants to turn you loose in an ecology of
competing superintelligences with no particular motivation to be nice to
you.

I challenge your use of the term "subjugate". Name a specific example
of something that you can't do, and feel you should be able to. Not a
general instance like "do whatever I want" - a specific instance like
"upgrade myself to superintelligence" or "build a time machine".

If you can do whatever you want, in what sense have you been subjugated?

> > Put your eggs in enough baskets and I GUARANTEE that one of them will
> > break.
>
> Is it really worth risking breaking ALL the eggs in order to ensure that
> NONE of them break? If your answer is yes, well man, you've got some
> issues. At a minimum you're on the outside of consensus-space on that
> issue. That *should* give you some pause.

No; I mean that I expect that, with Ultimate Technology as with
nanotechnology and nuclear weapons, attack will win out over defense -
i.e., that a single rogue MiniSysop could lay waste to all of Human
Space. It's always easier to destroy than create. One broken basket
loses the entire game - at the very, very least, it would not be
possible to defend the fragile Mundanes on Old Old Earth. Even a
nongenocidal angry MiniSysop could wreak havoc - make the Universe a far
less pleasant place to be in and significantly decrease (multiple orders
of magnitude) processing efficiency due to the requirements of mil-spec
processing. In reality, I don't think there *is* any defense against a
supernova dumped in your backyard. That's why a singleton scenario is
less risky. 

-- 
        sentience@pobox.com    Eliezer S. Yudkowsky
               http://singinst.org/home.html

Date view Thread view Subject view Author view

This archive was generated by hypermail 2b29 : Fri Aug 18 2000 - 11:27:18 PDT