From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Thu Aug 17 2000 - 16:05:52 PDT
Jeff Bone wrote:
>
> Actually, on second thought, I'm not worried. There's a reasonable window of
> opportunity to handle the threat; it takes at least (what, 16 hours? Bueller?
> Bueller?) for gray goo to consume the Earth, most of which happens in the last
> minutes.
See "Some Limits to Global Ecophagy" for a technically correct but
conceptually misguided view of the matter.
http://www.foresight.org/NanoRev/Ecophagy.html
> Plenty of time to take out one out-of-control 21 year old lunatic, his
> pet superintelligence, and his bucket of assemblers. ;-)
The 21-year-old, yes. The bucket, yes. The pet superintelligence... HA
HA HA HA HO HO HO *cough* *wheeze* *sputter*.
Pet?
> All it takes is a single
> well-placed tacnuke. And for sure, the folks building this stuff aren't
> politically or socially savvy enough to smooth over things with the Status Quo so
> as to not be perceived as a threat.
Most hostiles won't take us seriously enough to care.
> Having said all that, I'm incredibly surprised to find myself arguing, in essense,
> in parity with Joy. Let me refine that a bit: I actually *completely* believe in
> the desirability of the end-goal Brian and Eliezer are gunning for... but I think
> that in order to "do the right thing" there has to be some sort of logical,
> quantifiable framework for allowing all those who will be impacted to have input
> into what gets done (to them) with / by that technology.
It's called "email". If you've got anything to say, now would be a good
time.
I'm not interested in implementing Eliezer's Rules or Brian's Rules or
Jeff Bone's Rules; I'm interested in implementing The Rules. I don't
think there's a whole lot of slack in the Sysop Instructions; not with
respect to the basics, anyway. Does the Solar System get divided up
evenly among the six billion inhabitants? Yes. Can you do whatever the
heck you want with your piece, as long as it (a) doesn't harm anyone
else without that being's permission and (b) doesn't let you construct a
Sysop-threatening weapon? Yes. Can you harm, or even affect, another
entity without vis permission? No. Do the Amish get to keep farming?
Yes.
I'd have to rate all of that a big honkin' Duh. So what exactly is left
to debate?
Hand it over to the UN, though, and I *guarantee* they'll screw it up.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://singinst.org/home.html
This archive was generated by hypermail 2b29 : Thu Aug 17 2000 - 16:16:43 PDT