Transhuman Goodness

A response to Massimo Pigliucci: “the problems with transhumanism”

2009-07-12T05:20:00.004+01:00

Massimo Pigliucci responded (and was reprinted) to an article on IEET by Kyle Munkittrick called “Transhumanism F.A.Q. : Is Aging A Moral Good?”.

Munkittrick makes some good arguments, and some bad arguments. Pigliucci points out a bad one here:

Munkittrick begins his own response to critics of transhumanism by stating that if anyone has a problem with technology addressing the issues of disease, aging and death then “by this logic no medical intervention or care should be allowed after the age of 30.” This, of course, is a classic logical fallacy known as a false dichotomy. Munkittrick would like his readers to take one of two stands: either no technological improvement of our lives at all, or accept whatever technology can do for you. But this is rather silly, as there are plenty of other, more reasonable, intermediate positions

but Munkittrick is not presenting the transhumanist argument in its strongest form; we can restate the argument in favour of life extension like this: Do you think that the current 50-60-year healthspan followed a 15-30 year senescence and death – a happenstance of historical accident and a limited degree of technological innovation is optimal? If not, then presumably one is faced by the following genuine dilemma: either you think that the ideal healthspan is longer than 50 years, or shorter. If you think the ideal is shorter, then presumably you are planning to commit suicide at some point? Most people, Massimo included, would probably take the other option and argue that a healthspan of longer than 50 years is best for them, technology permitting, in which case they are pro-life extension. [perhaps not pro-immortality for themselves, though]

This form of argument has been generalized by Nick Bostrom as the reversal test.

Moving on, Pigliucci complains about the issue of life extension and population growth. This has been addressed at length elsewhere, but the strongest argument I can think of in favour of life extension is that people who live for a very long time may not keep having children every 30 years; they might decide to have children once and then go and do other things. In this case, population growth is only linear rather than exponential. This kind of population growth is something that we can cope with – by colonizing space and other planets, or by advancing technology to increase the population density on earth – as has happened many, many, many times before.

In the long run, the theoretical limit on human population is given by the available matter and energy in the universe, which we can colonize at a rate which is asymptotically cubic in time – a constantly expanding sphere around earth. If most people have one set of children in their lives, and people who do have more than one set of children do so with longer and longer gaps in between, it would be easy to stay within that cubic bound even out into the far future – say, 1 million or even 1 billion years from now, even if most people lived forever.

Arguments like this show that the population increase/resource limitation problem requires a rate-balancing solution; a careful plan for ensuring that everyone’s right to life is respected whilst at the same time maximizing peoples’ right to reproduce subject to the constraint of resources. It does not warrant a barbaric, premature and absolute slaughter of everyone over a certain age.

True immortality – actually living forever – is probably not worth talking about at the moment, because we are too young and inexperienced as a species to work out what the best choices are. It is better to aim to keep everyone alive for the next thousand years and reconsider our options then.

Transhumanism isn’t just about life extension. H+ is about improving all aspects of our existence, including mitigating the risks of advanced technology and improving the quality of people’s lives. Any transhumanist who claims to ignore risks and blindly support all technology irrespective of its projected effect on our overall quality of life is not in fact a transhumanist – they are a techno-optimist, which is a distinct notion from h+. The work of David Pearce and Nick Bostrom includes much about quality of life, elimination of suffering and living the best possible existence. The singularitarian community is thinking about the creation of a human-benevolent superintelligence which would probably be better at optimizing our experience than we can even imagine. Yudkowsky’s articles on fun theory come to mind here.

And lastly and by far most importantly, the transhumanist movement is closely associated with the more recent rationalist movement which centres around the work of Eliezer Yudkowsky and Robin Hanson. At LessWrong.com, the rationalist movement considers in extreme detail and rigor the ways that humans fail to achieve their goals and the ways that they fail to have accurate beliefs. Someone like Pigliucci, whose blog is called “Rationally Speaking” would seem to be pursuing the same goals as the rationalist community at LessWrong, and I hope that in the future the transhumanist community will follow the principles of rationality (such as recognizing the importance of the human cognitive biases literature) better than it does at the moment.

Will becoming a yokel improve your life and save the planet?

2009-07-04T19:20:00.038+01:00

A critique of Open Source Ecology (OSE)

I saw this post at Sentient Developments by Edward Miller, the underlying idea is open source ecology. If we are to reliably produce good ideas about changing the world here in the blogsphere, then we must prune out the bad ideas; open source ecology is a bad idea if ever I saw one. First, let me quote the main open source ecology wiki page:

Open Source Ecology is a movement dedicated to the collaborative development of tools for replicable, open source, modern off-grid "resilient communities." By using permaculture and digital fabrication together to provide for basic needs and open source methodology to allow low cost replication of the entire operation, we hope to empower anyone who desires to move beyond the struggle for survival and "evolve to freedom."

Here is a quote from Edward's post:

It is this sort of thinking which is required for a peaceful transition to a new era for our civilization. It will allow us to become resilient to the converging threats which face us from ecological destruction to market failure to terrorism. Global supply chains have shown themselves to be exceedingly vulnerable to these shocks. I hope we can overcome these by localizing production by utilizing global knowledge sharing so we can all enjoy the type of future some of the previous guest bloggers have been talking about. … I would like to see the technologies for food production to be as decentralized as possible. Whether that means vertical farms, community gardens, or single family gardens. I think what would make the most sense is to have cheap, mostly automated greenhouses with drip irrigation built into homes as standard practice.

But does it make any sense on a cold, rational level?

1. Will open source ecology allow people to feed themselves using only locally produced food and materials, without requiring everyone that give up their day job and without vastly reducing population densities?

Just how much land do you need to support one person by farming? This reference states that "Depending on climate, soil conditions, agricultural practices and the crop grown, it generally requires between 1,000 and 40,000 m² (0.25 and 10 acres) per person". The UK can thus probably support a population of about 10 million if we all do nothing but farm in our own little farmsteads. In support of this point, according the UK Census of 1801, the UK population was about 8.5 million, and according to another source it was about 4 million in 1600. So, before we developed modern agriculture, we were, in fact, limited to that figure of 50 people per square kilometre – note that 50 people per square kilometre times 250,000 square kilometres (the total area of the UK) is 12.5 million people.

Now, it is possible that in the very best of circumstances, using the best technology available today – hydroponics, fertilizer and artificial lighting, we might achieve the upper bound of 1000 people per square kilometre. But even this is not high enough. To feed all the people in a large city – for example London – using only the available land area of London, equates to feeding 5000 people using just 1 square kilometer. The usable area is of course even less. These people would have to be able to feed themselves by being farmers in their spare time, all using only locally produced tools and energy, without giving up their day jobs with an area the size of a large living room per person for growing crops, purifying water and recycling sewage. Excuse me if I deride this idea as total and utter nonsense, on a par with the beliefs of creationists, rather than being merely implausible.

2. Will open source ecology allow people to feed themselves using only locally produced food and materials, by making everyone into full-time subsistence farmers and dismantling cities so that population density is low?

In order for everyone to survive like this, the cities would have to be dismantled and the population spread evenly over the country – this is because the population in cities is much greater than 1000 people per square kilometer. This would, of course, completely disrupt the country for years and be massively expensive and unpopular. It is, however, plausible that many jobs could be done without cities – for example using telecommunications technology, but there are some jobs that really do require at least 100 people to be in the same place – for example an engineering company or a chip fabrication plant, an army barracks or military base, or a car factory. It is, however, not plausible that people could feed themselves through subsistence farming using only locally made materials, and only 1000 square meters of land whilst holding down a normal job. People have perhaps 1 hour of free time that they could spend doing manual labour per day – and if we look at the videos on the OSE site, we hear reports of “full days of backbreaking manual work” – and on the OSE factor-e-farm they use nonlocally produced fuel and a nonlocally produced engine for their tractor which does the hardest manual work for them. This shows that OSE high-tech subsistence agriculture is not an option that can be kept "on standby" in case of a problem - it is an all-or-nothing switch to a different, more local phase of society. And, of course this means that we have to consider it in a different light: not as a backup for modern society, but as an alternative. As such it has its own increased risks which I will mention later.

3. Will open-source ecology help save the human race from extinction? Will it make our society more resilient?

Given the above analysis - that we simply do not have enough land to live the "local" way, it is clear that open source ecology would not be able to save everyone. But it could certainly save some people - for example the survivors of a particularly bad nuclear war or bioterrorist pandemic. In fact, after such a war, people would probably spontaneously invent something like OSE due to the severe lack of food and the disruption of trade and transportation. So, on this count, OSE is a clear winner - a great fallback option for the "lucky" survivors. However, waiting for most people in the world to die is not exactly the ideal existential risk mitigation strategy; anyone who lives in a city would almost certainly end up dead in any scenario where OSE because useful..

4. Will open source ecology "free us from the necessity of wage labour", "make us freer"?

Now, in a sense, this claim is a little bit too vague to analyze. The implication seems to be that we will have more free time, or that we will spend less time working. Perhaps one could interpret it to mean that we will live in a moneyless society, and we won't have to work to make money any more. Unfortunately, both of these claims are false. Firstly, how much free time you have for a given standard of living is a function of how rich your society is, i.e the GDP per capita. In the UK, this value is high, so life is relatively good. What would happen if we implemented open source ecology? Would we get richer? No! Because of economies of scale, making production more local makes everything more expensive. Fundamentally, our civilization works because of economies of scale. Adam smith discovered this in his seminal work, An Inquiry into the Nature and Causes of the Wealth of Nations; Smith realized that by specializing to produce just one thing, you can produce that thing more cheaply. A lot more cheaply, in fact. Now, since our present high wealth is powered by super-massively large economies of scale, reducing the scale to the farmstead level will decrease our wealth. We will not be able to afford such luxuries as healthcare, education, going to university, meeting more than a small group of people, using computers, travelling around the world, going to art galleries, watching movies, making art, having specialists who make scientific discoveries, etc. We will instead have to spend our time tending the crops or repairing the home-made tractor.

5. Will open source ecology will be good for the environment?

Well, getting rid of 90% of the world's population would undoubtedly be good for the environment. Fewer people means less environmental footprint, but it strikes the author that this is not a politically viable option in the near-term. But suppose, for the sake of argument, that a small number of people take up the open source ecology movement, utilizing a small amount of land that is not really being used at the moment anyway. Will this be good for the environment? Perhaps, though the effect will be insignificant, since the number of people involved would be small.

6. Will OSE protect us from terrorists?

As I have already argued, local living would mean disbanding or vastly reducing the army and the government (as well as pretty much everything else except farmers!). Any larger than average group that decided to arm itself would be able to "gobble up" adjacent groups through military conflict, kill the men and rape the women (humans have done this kind of thing throughout history, and it seems to have been the dominant mechanism which caused large societies to replace small ones). It is only by having the most efficient possible farming techniques that leave a large segment of the population free from the need to farm for food that countries such as the UK and the USA are able to achieve the kind of military dominance we have, by having specialists in fighting, weapons development and espionage. So, although OSE would protect us against terrorists attacking central infrastructure such as power plants or tall buildings, it would leave us wide open to something much, much worse: open military conquest and war crimes by anyone who decided that conquest was more important than local living.

7. Will highly advanced technology make Open Source Ecology work?

With the development of advanced nanotechnology, and robotics (possibly nanorobotics) we are likely to see products and materials with superlative performance that would allow a person to very easily produce enough food to feed themselves in a very small area of land.. But don't hold your breath: such technology will probably arrive around 2050-2100, and even then if your nanoreplicator or farming robot broke down, you would not be able to fix it using locally made tools. So ultra high-tech local subsistence farming is not actually resilient – using high-technology solutions makes you more reliant upon centralized facilities to supply and repair such devices.

8. So what is OSE good for at the moment?

Well, it seems like a natural fallback if the worst does happen. In brighter times - such as those of today - it is a great hobby, rather like having an allotment, a metalworking hobby and a home-science hobby all rolled into one. It also seems to foster a sense of community and social capital, though at the cost of the people involved being somewhat deluded about the significance of what they're doing. Open source ecology is ultimately a hobby movement, and does not stand a serious chance of changing the world for the better in a big way. Though, on the other hand, it is one of the best hobbies I have ever seen, because it contributes to people having a better understanding of how practical things work, to working together with their community as a team, and to people getting both physically and mentally fit.

9. Why have otherwise very intelligent people been sucked in by the OSE’s false claims?

It looks to me like OSE is an example of the worst possible human cognitive bias – motivated cognition. This is what happens when you become emotionally attached to an idea – in this case “decentralization as the solution to all our problems” - and then you start actively looking for arguments in support of it and actively looking for ways to dismiss any arguments against it. As you become more enamoured with the idea, you start to associate all good things with it and disassociate all bad things from it. The proponents of OSE didn’t even say one bad thing about OSE on their website, but they associated every positive thing they could think of with it – it will make us freer, richer, help the environment, free us from the drudgery of work, protect us from terrorists, they even claimed that it could lead to an end to all wars and human conflicts. Motivated cognition is a systematic failure mode of the human mind which everyone is highly susceptible to unless they take active steps to prevent it, known as rationality training. You can find out more about this by looking at LessWrong.com, or the Wikipedia article on cognitive biases.

Comment on this post on Facebook [You have to create a Facebook account and add me as a friend for this to work, but by doing so you'll be able to network with other people interested in the future of humanity, and you'll help to inform other people on Facebook about what is going on]

An insight from the new Transformers film

2009-06-29T22:21:00.005+01:00

Last night I saw Transformers: Revenge of the fallen; overall, it was (of course) a fairly terrible film. Some great action sequences went some way towards making up for the childish plot, cliched characters and relationships and some really, really bad attempts at slapstick/"American style" vulgar comedy.

But there was one scene which really meant something to me. The lead character, Sam, has been "chosen" by the Transformers (the good robots) and the US secret service to act as some kind of liaison between the Transformers and the US government, he also has some kind of family history of involvement with the Transformers; but he "just wants a normal life" - he is a reluctant hero. He starts university and finds that his roommate, Leo, runs a conspiracy theory website that collects reports about the presence of alien robots on Earth. Sam obviously has to pretend that he doesn't know anything about the Transformers, and Sam and Leo get into an argument about whether the conspiracy is true or not, with Sam lying to Leo, and Leo insisting that he'll get to the bottom of the mystery.

Later, Leo and Sam get pulled into a major fight between the Transformers and the Decepticons, and the Decepticons force the government to make Sam a wanted man. Leo gets scared and starts to panic: having been shot at and chased by alien robots and now by the police too, he tells Sam that he's going to turn himself - and all the rest of the fugitives - in to the authorities.

Sam then retorts: "Well this is what you wanted! You've got your conspiracy theory, now stop complaining!"

The very outspoken transhumanist/singularitarian community may, in time, find itself in Leo's position.

Save the human race with a click!

2009-06-13T17:23:00.005+01:00

After having seen that Facebook causes is getting somewhat serious in terms of publicity and donations - but that the top causes were for things that aren't exactly the most important priorities at the moment - I decided to start a cause for saving the human race. (I couldn't find one, but it turned out there were a few, under different names).

For those people who don't already know, the survival of the human race over the next 100 years is a very touch-and-go thing. Martin Rees, estimates that (due to various risks) we stand a 50% chance of terminating ourselves over the next 100 years, and Nick Bostrom puts the figure at >20%.

Most people think of the destruction or decimation of the human race as something that only happens in fiction, not in real life. So it is very hard to convince people to donate time, money or effort to preventing it. Other far less important causes such as pet welfare* receive far, far more money and support. This is simple scope insensitivity combined with Hanson's near/far distinction - people put the end of the human race in the "far" category, so they don't want to do anything about it.

* if you doubt that protecting pets is as important than saving the entire human race, consider that the destruction of humanity implies the subsequent devastation of the pet population as pets run out of food, and the eventual reversion of the survivors to new species of wild animals; a process that would surely cause immeasurable animal suffering, and select for the most violent characteristics in the surviving animals

Comment on this post on Facebook [You have to create a Facebook account and add me as a friend for this to work, but by doing so you'll be able to network with other people interested in the future of humanity, and you'll help to inform other people on Facebook about what is going on]

Athena debunks some Buddhist Quantum goodness

2009-06-13T13:41:00.004+01:00

Athena Andreadis has written a somewhat acerbic and, in my opinion excellent criticism of some Buddhist video about how Buddhism is "compatible" with quantum mechanics. The video is only mild nonsense compared to what we see from Christianity, so I wrote an initially positive comment about it on Sentient Developments. But then I read Athena's Post:

Physicists and mathematicians are aware of these limitations when they use such constructs [such as Schrodinger's equation]. In contrast, when people who are not conversant with a scientific concept use it to lend credibility to shaky or shady conclusions, they become demagogues and/or charlatans. And before anyone trots out the elitism hobby-horse, all I can say is, just have the next person you meet on the street repair your car or give you a haircut. The same logic applies, and no amount of skimming Wikipedia entries will make up for in-depth knowledge and critical thinking.

Buddhism has become fashionable among people who wish to be considered spiritual but not religious, many of them self-proclaimed progressives – hence it’s de rigeur not to criticize it. Some of its prestige comes from politics (primarily the Tibet/China situation, but only because it’s pertinent to US financial concerns), some from the intelligence and charisma of the current Dalai Lama, some from the simple fact that it appears exotic to Westerners when compared to the home-grown Abrahamic monotheisms.

I like the aesthetics of Zen Buddhism very much. However, there is nothing to attract me in the religion’s misogyny (women have no souls and must be reborn as men to attain Nirvana), its primitive cosmology of universe-toting turtles, its punitive stance that suffering is the result of bad past karma, its oppressive policies whenever it gained its share of temporal power (including pre-Chinese Tibet, which was a far cry from Shangri-La) or the dog-like master/disciple formula that I dissected in my critique of that pinnacle of ersatz mythology, Star Wars.

Worse yet, what is the outcome of suppressing desire, Buddhism’s ultimate goal? It’s the fate of the Miranda settlers in Serenity, the fate of any conscious being that gazes obsessively at its navel with the belief that reality is but an illusion. If this is true, why explore or invent? The Western religions have an awful lot to answer for. But at least in their figures of defiance, from Prometheus to Lucifer, they incorporate a key element: striving for something larger than one’s puny self without letting go of one’s individuality.

Yes, I agree completely. Good work, Athena!

Though, in defence of Buddhism I will say that many people need some kind of comforting nonsense, and Buddhist nonsense seems to be relatively harmless compared to Islamic nonsense, for example. It's like when I go to the dentist, he could tell me not to eat any sugary foods, ever. But he knows I'll eat some sugary foods, so he recommends that I eat bananas rather than deep fried Mars bars. I see Buddhist nonsense vs. Islamic or Christian nonsense a bit like this.

Bioconservative and biomoderate singularitarian positions

2009-06-07T12:26:00.004+01:00

Let us define a singularitarian as a person who considers it likely that some form of smarter than human intelligence will be developed in a characteristic timeframe of a century, and that the manner in which this event occurs is important enough to expend effort altering. Given this definition, it is perfectly possible to be a bioconservative singularitarian - that is someone who:

opposes genetic modification of food crops, the cloning and genetic engineering of livestock and pets, and, most prominently, rejects the genetic, prosthetic, and cognitive modification of human beings to overcome what are broadly perceived as current human biological and cultural limitations.

- one can accept the (at present only suggestive) factual arguments of Hanson, Yudkowsky, Bostrom etc that smarter than human intelligence is the only long-term alternative to human extinction (this is what one might call an "attractor" argument - that our current state simply isn't stable), whilst taking the axiological and ethical position that our pristine, unenhanced human form is to be held as if it were sacred, and that any modification and/or enhancement of the human form is to be resisted, even if the particular human in question wants to be enhanced. A slighly more individual-freedoms-oriented bioconservative position would be try very hard to persuade people (subject to certain constraints) to decide not to enhance themselves, or to allow people to enhance themselves only if they are prepared to face derision and criticism from society. A superintelligent singleton could easily implement such a society.

This position seems internally consistent to me, and given the seemingly unstoppable march of technological advancement and its rapid integration into our society (smartphones, facebook, online dating, youtube, etc) via corporate and economic pressure, bioconservative singularitarianism may become the only realistic bioconservative position.

One can even paint a fairly idyllic bioconservative world where human enhancement is impossible and people don't interact with advanced technology any more, they live in some kind of rural or hunter-gatherer world where the majority of suffering and disease (apart from death, perhaps) is eliminated by a superintelligent singleton, and the singleton takes care to ensure that this world is not "disturbed" by too much technology being invented by anyone. Perhaps people live in a way that is rather like one would have found on a Tahiti before Europeans got there. There are plenty of people who think that they already live in such a world - they are called theists, and they are mistaken (more about this in another post).

For those with a taste for a little more freedom and a light touch of enhancement, we can define biomoderate singularitarianism, which differs from the above in that it sits somewhere more towards the "risque" end of the human enhancement spectrum, but it isn't quite transhumanism. As before, we consider a superintelligent singleton running the practical aspects of a society and most of the people in that society being somehow encouraged or persuaded not to enhance themselves too much, so that the society remains a clearly human one. I would consider Banks' Culture to be the prototypical early result of a biomoderate singularity, followed by such incremental changes as one might expect due to what Yudkowsky calls "heaven of the tired peasant" syndrome - many people would get bored of "low-grade" fun after a while. Note that in the Culture, Banks describes people with significant emotional enhancements and the ability to change gender - so this certainly isn't bioconservative, but the fundaments of human existence are not being pulled apart by such radical developments as mind merging, uploading, wireheading or super-fast radical cognitive enhancement.

Bioconservative and biomoderate singularities are compatible with modern environmentalism, in that the power of a superintelligent AI could be used to eliminate damage to the natural world, and humans could live in almost perfect harmony with nature. Harmony with nature would involve a superintelligence carefully managing biological ecosystems and even controlling the actions of individual animals, plants and microorganisms, as well as informing and guiding the actions of human societie(s) so that no human was ever seriously harmed by any creature (no-one gets infected by parasites, bacteria or viruses (unless they want to be), no-one is killed by wild animals), and no natural ecosystem is seriously harmed by human activity. A variant on this would have all wild animals becoming tame, so that you could stroll through the forest and pet a wildcat.

A biomoderate singularity is an interesting concept to consider, and I think it has some interesting applications to a Freindly AI strategy. It is also, I feel, something that I think will be somewhat easier to sell to most other humans around than a full-on, shock level 4, radical transhumanist singularity. In fact we can frame the concept of a "biomoderate technological singularity" in fairly normal language: it is simply a very carefully designed self-improving computer system that is used to eliminate the need for humans to do work that they don't (all things considered) want to do.

Due to various historical co-incidences, the same small group of people who popularized technologically enabled bio-radical stances such as gender swapping, uploading, cryopreservation, etc also happen to be the people who popularized ideas about smarter than human intelligence. When one small, outspoken group proposes two ideas which sound kind of similar, the rest of the world is highly likely to conflate them.

The situation on the ground is that one of these ideas has a viable politico-cultural future, and the other one doesn't: "bioradical" human modification activates so many "yuck" factors that getting it to fly with educated, secular people is nigh-on impossible, never mind the religious lot. The notion that smarter-than-human intelligence will likely be developed, and that we should try to avoid getting recycled as computronium is a stretch, but at least it involves only nonobvious factual claims and obvious ethical claims.

It is thus an important task to separate out these two ideas and make it clear to people that singularitarianism doesn't imply bioradicalism.

Changing accepted public opinion and Skynet

2009-05-22T19:07:00.001+01:00

Crossposted from Less Wrong

Michael Annisimov has put up a website called Terminator Salvation: Preventing Skynet, which will host a series of essays on the topic of human-friendly artificial intelligence. Three rather good essays are already up there, including an old classic by Eliezer. The association with a piece of fiction is probably unhelpful, but the publicity surrounding the new terminator film is probably worth it.

What rational strategies can we employ to maximize the impact of such a site, or of publicity for serious issues in general? Most people who read this site will probably not do anything about it, or will find some reason to not take the content of these essays seriously. I say this because I have personally spoken to a lot of clever people about the creation of human-friendly artificial intelligence, and almost everyone finds some reason to not do anything about the problem, even if that reason is "oh, ok, that's interesting. Anyway, about my new car... ".

What is the reason underlying people's indifference to these issues? My personal suspicion is that most people make decisions in their lives by following what everyone else does, rather than by performing a genuine rational analysis.

Consider the rise in social acceptability of making small personal sacrifices and political decisions based on eco-friendliness and your carbon footprint. Many people I know have become very enthusiastic for recycling used food containers and for unplugging appliances that use trivial amounts of power (for example unused phone chargers and electrical equipment on standby). The real reason that people do these things is that they have become socially accepted factoids. Most people in this world, even in this country, lack the mental faculties and knowledge to understand and act upon an argument involving notions of per capita CO2 emissions; instead they respond, at least in my understanding, to the general climate of acceptable opinion, and to opinion formers such as the BBC news website, which has a whole section for "science and environment". Now, I don't want to single out environmentalism as the only issue where people form their opinions based upon what is socially acceptable to believe, or to claim that reducing our greenhouse gas emissions is not a worthy cause.

Another great example of socially acceptable factoids (though probably a less serious one) is the detox industry - see, for example, this Times article. I quote:

“Whether or not people believe the biblical story of the Virgin birth, there are plenty of other popular myths that are swallowed with religious fervour over Christmas,” said Martin Wiseman, Visiting Professor of Human Nutrition at the University of Southampton. “Among these is the idea that in some way the body accumulates noxious chemicals during everyday life, and that they need to be expunged by some mysterious process of detoxification, often once a year after Christmas excess. The detox fad — or fads, as there are many methods — is an example of the capacity of people to believe in (and pay for) magic despite the lack of any sound evidence.”

Anyone who takes a serious interest in changing the world would do well to understand the process whereby public opinion as a whole changes on some subject, and attempt to influence that process in an optimal way. How strongly is public opinion correlated with scientific opinion, for example? Particular attention should be paid to the history of the environmentalist movement. See, for example, McKay's Sustainable energy without the hot air for a great example of a rigorous quantitative analysis in support of various ways of balancing our energy supply and demand, and for a great take on the power of socially accepted factoids, see Phone chargers - the Truth.

So I submit to the wisdom of the Less Wrong groupmind - what can we do to efficiently change the opinion of millions of people on important issues such as freindly AI? Is a site such as the one linked above going to have the intended effect, or is it going to fall upon rationally-deaf ears? What practical advice could we give to Michael and his contributors that would maximize the impact of the site? What other intervantions might be a better use of his time?

Comment on this article at Less Wrong

Terminator Salvation: Preventing Skynet website now up

2009-05-22T11:17:00.004+01:00

Michael Annisimov has put up a website called Terminator Salvation: Preventing Skynet, which will host a series of essays on the more serious aspects of the creation of human-friendly artificial intelligence.

A picture worth a thousand words

2009-05-12T22:00:00.003+01:00

Someone on facebook expressed skepticism about the prospect of radical human life extension, and I spent a while trying to craft a concise and effective reply. But in the end, I just gave her this picture:
Ageing: "Fun vs Not Fun"

H/T Aubrey De Grey.

Richard Jones' Comments ...

2009-04-30T22:16:00.004+01:00

Roko, it was a pleasure to meet you too, and I'm sorry the meeting was rather brief - no doubt we would have found more to disagree with had we talked longer.

Let me make a few quick comments on the points you mention. Firstly, and this, I think, gets to the heart of what Dale is saying, it's not at all clear that enough conceptual heavy lifting has been done to support the simple notions of intelligence that your arguments depend on. Secondly, while I agree that in principle you can discuss advanced AI entirely independently of Drexlerian nanotechnology, in practise Kurzweilian projections of the timescales on which brains can be scanned in sufficient detail rely on some kind of radical nanotechnology to emerge that can make practical that which with today's technology is very far out of reach. Finally, you're welcome to think in terms of singularitarianism as political activism that seeks to steer the priorities of science towards its goals, but don't be surprised if I don't agree with your assessment of what those priorities should be.

Since Richard is a clever guy and probably the most intelligent, coherent critic of the singularitarian movement I've heard, I've promoted his comment to a post. Feel free to leave comments here or on facebook. Richard Blogs at Soft Machines.

Meeting with Professor Richard Jones - scientists display a remarkable ability to converge on the truth...

2009-04-28T16:00:00.008+01:00

On Friday, I had the pleasure of meeting with Professor Richard Jones who (blogs at Soft Machines) at his Sheffield office. This was motivated by a rather tedious blog-comment argument over on accelerating future, mostly between me and Dale Carrico. In that discussion, Richard briefly commented in support of some of the things that Dale said, so I was curious to meet Richard - a prominent scientist and critic of certain (somewhat careless) claims made by proponents of advanced nanotechnology research.

In my meeting with Richard, I introduced him to some ideas from Eliezer's school of thought. In particular, the following two ideas were pretty important, I felt:

1. That the way we evolved from less intelligent primates indicates that humans are roughly as stupid as it is possible to be without being too stupid to have a functioning society. - Richard was initially skeptical about this position, but on further reflection we agreed that the rate at which technological society had developed from the onset of agriculture to today was so fast on an evolutionary timescale that we could not reasonably expect a modern human newborn to have significantly more cognitive ability and potential than a hunter-gatherer newborn from 13,000BC. Another way of putting this is that humans are at or close to the very beginning of the hierarcy of intelligent life, not the end point.

This position has some very important implications:

It is very likely that, if there is an ultimate limit on how intelligent a system could be in our universe (for example, due to the Bekenstein bound), then it is likely to be a very long way above the human level of intelligence. This is purely a probabilistic argument: it is unlikely that such a bound would be above the minimum required for organized society, but only just above that minimum.
Many technical problems that seem very hard to us would probably be fairly trivial to intelligences that are genuinely smarter than us, because these intelligences would be able to improve themselves deliberately and would therefore not be bound by the slow evolutionary timescale of human intelligence improvement. We should thus expect smarter than human AI to be very powerful indeed. For example, indefinite life extension of human beings would probably fall into the "trivial" category for smarter than human AI. (Why? Well, mere humans have extended their lifespans by a factor of 3, with a consistent increase of 1 year per decade at the time of writing. Given that humans are, on a cosmic scale of intelligence, idiotically stupid, it seems highly likely that a self-improving smarter than human AI bound only by the laws of physics would be able to solve the problem). Similar comments apply to claims like space colonization, revival of cryonics patients, etc.

2. It is possible to accept the claims made by the Singularity Institute about smarter than human AI, whilst rejecting claims made by Drexlerian nanotechnology proponents. These claims are independent and, scientifically speaking, unrelated.

3. There is an important role for transhumanist and singularitarian advocacy which is not science, but political activism. The role of such activism is not to distort scientific findings to make "sceince fiction - like" claims seem more plausible, but to influence the order in which science is done, i.e. to influence priority.

For example, suppose $100 million available today can either be spent on studying the many species of parrots that live in the Amazon Basin, or that same $100 million in 2009 can be spent developing safe smarter than human AI. Science itself cannot decide which piece of research to do now - there is no scientific answer to that question. It is necessarily a political question.

And the singularitarian position is that it is better - by the standards of any sane human being - to spend that money on safe smarter than human AI research.

Overall, it was a pleasure to speak with Richard. We agreed on pretty much everything that we discussed - which was a breath of fresh air compared to the rivalrous feel of the Accelerating Future comments thread from which the visit was inspired.

What is transhumanism?

2009-04-22T11:21:00.006+01:00

Transhumanism is a movement of people who think that we can radically enhance the human condition using technology. “Trans” stands for transitional, so a trans-humanist is someone who thinks of being human as a transitional state - upon which we can greatly improve. Transhumanists are interested in accelerating technological change which may allow some profound changes to occur within our own lifetimes. Transhumanists are also concerned that future technological development is carried out in an ethical and democratic fashion.

That all sounded very abstract, so let me give some more concrete examples of what improving upon the human condition might involve. Transhumanists take a great interest in defeating aging and death. Our current human bodies are frail - we grow old and we die. It doesn’t have to be like this - there is no reason why technologically enhanced people can’t live for many hundreds or even thousands of years, but someone needs to go and develop the technology, and they need to be funded to do so.

Most of us are forced – by material necessity – to work at jobs that we hate instead of doing exciting, creative things that we enjoy and which would make us better people. Again, it doesn’t have to be this way - advances in automation technology, robotics, and artificial intelligence could automate those repetitive tasks that we call work, and leave us to enjoy those activities that we call fun.

Our minds, whilst brilliant in some ways, are severely limited in others. Our psychology is often competitive and jealous; when someone else is successful it can make us feel a little worse, and we frequently attain happiness at the cost of others. We suffer from biases like racism and sexism, irrationalities like fundamentalist religion, addiction and lack of willpower. But your mind is not immutable - with suitable interfacing technologies, you could re-program your mind so that your biological drives match up with your higher intentions.

Technology does not materialize of its own accord, and it will not necessarily be used or developed in the most effective way, or even in an ethical way. The same technologies which will allow us to overcome age-old problems have the potential to create new and deadlier risks. For example, molecular nanotechnology will allow huge advances in medical technology, but it will also facilitate new and deadlier weapons. The transhumanist movement seeks to encourage and embrace the huge positive potential of technology, and to guard against the associated risks.

This introduction is based on Michael Anissimov’s post, Transhumanism as Questioning Our Nature, which I throughly recommend. He put the situation well:

"it’s perfectly possible to be enthusiastic and cheerful about life, while simultaneously acknowledging that our bodies are weak and are minds are twisted by our genes’ shallow goals."

You may also want to see the Transhumanist declaration from the World Transhumanist Association.

Follow Transhuman Goodness on NetworkedBlogs

2009-04-19T19:52:00.005+01:00

... By clicking on This Link

I'm about 2 followers short of making the front page for "philosophy" on NetworkedBlogs, so even a few more people is great. In return, I promise to do some scholarly philosophy posts...

If everyone who read this blog followed me on NetworkedBlogs... wow, I'd easily double my audience through the additional PageRank and links I'd get. So if you think that saving the human race from technological Armageddon, solving third world poverty, curing aging, and colonizing the universe are kind of worth doing, there's a simple thing you can do to help: follow us on NetworkedBlogs!

Michael Vassar hits the nail on the head

2009-04-16T21:53:00.005+01:00

Michael Vassar was recently interviewed by accelerating future. I was delighted to hear that someone else had had basically the same insight as me.

When asked: "Can you tell us the story of how you first found out about SIAI’s mission", Michael responded [emphasis mine]:

"The major influence was my progressive discovery of the inadequacy of the deliberative and decision-making organs of modern society. I saw fundamentalism. I saw the War on Drugs. I saw failure to adequately secure nuclear materials in Eastern Europe and failure to build adequate levies in the world’s richest nation. Eventually I integrated all of these facts into my world-view rather than leaving them as dangling exceptions to an unchallenged assumption that the collective behavior of the world around me was basically sane. It became clear that if a technological singularity this century was pretty likely I should still expect that by default no one with any serious power would react rationally to the possibility until much too late. Warren Buffet, Sam Nunn, and Ted Turner get kudos for being an exception with their Nuclear Threat Initiative, but they are an exception that proves the rule. Yes, the powers that be could really be collectively stupid enough to hear about the singularity, acknowledge it in the occasional speech, but generally ignore it and allow it to happen in whatever manner is the easy default, even if that default is human extinction. They collectively mess up much easier issues all the time."

Daily Galaxy rubbishes Bostrom's simulation argument.

2009-04-12T22:18:00.024+01:00

From the Daily Galaxy:

Philosophy is a vital study for the human race - from the ancient Greeks to the modern day, some of the finest thinkers have examined the human condition and produced valuable insights and conclusions on what it means "to be." Unfortunately much of the other work in the field is dubious, including a recent paper which argues that we're all living in a vast computer simulation. Yes, it WAS written after the first Matrix film but before the sequels. Funny that.
Unfortunately this isn't a fanfic: it's a refereed paper published in the Philosophical Quarterly, which must have been hurting for content. It was written by, Nick Bostrom, the Director of the "Future of Humanity Institute" at Oxford University, the sort of person we'd generally assume to be above such things. But we suppose that even those pondering the fate of the species need publicity and funding too - probably more than most people, in fact.

Reading the paper is a fun game of "Spot the logical flaws" for all the family, with bonus points for every "Warning sign of BS paper" picked out.

The most egregious flaw is the use of a form of the fantastically annoying Bayesian argument: the idea that if we suppose there far more B-type of people than A, then we're more likely to be born as a B than A. It's been which has been used to argue everything from the imminent end of the species to this simulation silliness despite:

a) assuming that we're all somehow stacked up waiting to exist like capsule toys in a spiritual vending machine.
b) Statistics Error No 1: confusing probability with actual fact, and arguing that nothing but the most common option should exist. For example, by the Bayesian argument you and everyone you know is Asian.
c) It's been an equally (in)valid at every stage in human history since we first dropped out of the trees, and was wrong then too.

One wonders whether the author, Luke McKinney, stopped to ponder how reasonable it is to assume that a professor at Oxford University, who set a national record for academic performance in his home country of Sweden, completed three degrees in the time it takes most people to do one, got a PhD from LSE, and is professor and director of a multi-million pound research institute at 34 makes basic statistics errors.

When we disagree with people who are obviously much cleverer than we are, we should seriously consider the hypothesis that they are right and we are wrong, and the reason this is not obvious to us is that we are not clever enough to see our mistake.

For reference, "confusing probability with actual fact, and arguing that nothing but the most common option should exist" is not a common error in statistics (you'd have to be moronically stupid to go from "the probability of X is close to 1" to "X is a certainty"), and Bostrom doesn't commit it. From this comment alone, it is fairly clear that Luke McKinney probably doesn't understand Bayesian Statistics.

I worried out loud on facebook to a vastly intelligent Cambridge mathematician that, since he is a Christian, and much cleverer than me, perhaps my atheism is mistaken?

I'm fascinated by the fact that so many clever cambridge mathematicians are christian. You, M, N.B. ... I am beginning to wonder if maybe I'm wrong about it! If I disagreed with you, M and N on a question of mathematics, I would simply admit that I was wrong and look for my error...

Testing out a new commenting system

2009-04-03T17:03:00.019+01:00

Spreading the word about Friendly AI, humanistic transhumanism and the singularity is one of the major themes of this blog. In fact, it is nothing short of an urgent task.

When I write an interesting post, people are kind enough to comment on it, and we have had some good discussions here. But Only people who have already found the Transhuman Goodness blog see those comments. Most people who know about the existence of the blog already know about this set of ideas, so most of the debate that goes on probably counts as preaching to the choir.

A great way to promote the idea of humanistic transhumanism, or h+, outside the small(ish) community of people who read this blog is to make use of people's social networks on Facebook. So I'm going to try an experiment: instead of allowing comments on my blog page, I will disable comments on the blog and place a link to the item in my Facebook feed for this post. If you want to comment on the post, click this link, and make the comment on Facebook. If you're not on Facebook, then get a Facebook account!

Suppose that 5 people comment on a post this way, each of who have 150 Facebook friends. The fact that they commented on the Facebook feed item for the post means that some significant fraction of 750 people will see the comment thread in their Facebook feeds. Neat!

This post is meant to be an experiment to see if it works. Please, do comment on the link I've placed below.

UPDATE: The link was broken to start with, but I've got it working now so that all you need to comment is a facebook account. Please, if you're reading this, leave a comment!

UPDATE: Thanks to those who commented. Also, if you don't have anything to add to a post, but you agree with the main point of the post, then "like" it on facebook.

Comment on this post on Facebook [You have to create a Facebook account and add me as a friend for this to work, but by doing so you'll be able to network with other people interested in the future of humanity, and you'll help to inform other people on Facebook about what is going on]

Like this post on Facebook [Press the "Like" button on the Facebook page] [A prize will go to the first person who does this... ]

Mijic, Legg and Bayes: is the writing on the wall for humanity?

2009-03-30T08:57:00.008+01:00

Roko Mijic, Shane Legg, Thomas Bayes: three eminent mathematicians (listed here in order of mathematical ability, of course) meet to discuss the future of the human race in Bunhill Fields Cemetry, London. Shane assures me that the large tombstone in the background contains the same atoms as once constituted the brain and body of the inventor of Bayesian statistics.

It was great to discuss some of the problems that AGI/FAI research poses with someone else who is in the know about the subject. Shane has been at the cutting edge of both theoretical and practical approaches to AGI, including working with Marcus Hutter on AIXI and formal definitions of intelligence, and working for both Ben Goertzel and Peter Voss on some not-quite-yet successful practical attempts at AGI. He now works at the Gatsby computational neuroscience centre, a world-leading centre for the study of the human brain.

It was something of a disappointment that when we spoke about the dangers of AGI to humanity, I was the more optimistic one. Shane's way of putting it was something like:

"We're completely f**ked, we've got about 15 years before unfriendly AI kills us"

this is, considering his expertise, very worrying. His research in computational neuroscience gives him access to evidence that I don't have, he told me that fMRI imaging techniques and TMS are being used to identify the machine learning algorithms that the human brain uses for various purposes, and that the rate of progress in neuroscience is fast. As we invent better imaging techniques, we can learn more about the way the brain actually works. He estimates that in about 15 years, neuroscience will have progressed to a stage sufficient for a serious attempt at neuro-inspired AGI. His timescale estimate should really be replaced with a probability distribution, but even so I think he is overoptimistic about how good neuroscience will be for building an AGI - I think that neuroscience will do very well at uncovering brain functions that computer scientists already know how to implement in code, because it is easy to find something if you know what to look for. For functions which we are conceptually confused about (such as "general intelligence"), neuro will not do so well in my opinion.

Currently, as I sit and write this post, I estimate that there are between zero and ten persons working on friendly AI in the whole world. Considering the magnitude of the risk and its high probability (could one honestly put it at less than 5%?) this is not optimal, to say the least.

We discussed a few other things, including the possibility that post positive-singularity humans would get bored with living in a paradise where their individual actions did not matter globally, the way our actions matter today. If I act one way rather than another today, it could make the difference between a negative singularity and a positive one. After a positive singularity, it is likely that this will never be the case again. This brings up the intriguing possibility that more instances of Roko Mijic are simulations from the future inhabited by people who wanted to experience the immense challenge of being just an ordinary fairly clever guy faced with the challenge of saving the entire world using nothing other than his wits. Reasoning anthropically, it seems extremely likely that I am an ancestor simulation - much more so than if I were just an average person with nothing special about them other than having heard the simulation argument. Shane had had the same thought, which was somehow (irrationally, of course) reassuring.

He then brought up the question "do you ever worry that you've joined a weird religious cult, and just not realized it?" - which doesn't worry me at all - I've spent long enough researching this set of ideas on my own, and trust my own intelligence enough to be pretty sure that the singularitiarian position is a reasonable one.

Before I induce any of my readers to get too depressed, I'll end on two positive notes. Firstly, the human mind is not good at taking uncertainty into account, and I think that Shane's opinion should be taken as "I feel rather worried" rather than "my subjective probability estimate of a negative singularity is close to 100%".

Secondly, I came up with a few good ideas for making a good outcome more likely. I claimed that the task of building an FAI probably doesn't need us to understand recursive self-improvement or stability of goal systems under self-modification very much. For, if we develop a smarter than human intelligence and give it the goal of serving the volition of humanity, and tell it that we think instability of its goal under self-modification might be a problem, then it will solve the safe recursive-self-improvement problem better than we ever could. Also, I claimed that there are a lot of people on the planet who would really care about the FAI problem if only someone would tell them about it in a non-crackpot way.

These include philosophers, prominent environmentalists, most of the Guardian reading left-wing middle class, most scientists and academics. Quite a lot of people, really. And the main impediment to progress in FAI is lack of money, credibility and human resource.

Errors of the heart, not of the mind.

2009-03-09T22:03:00.008Z

It's been an awesome journey since I started this blog nearly two years ago. My original goal was to understand exactly what we mean when we say that we want to "make the world a better place using technology". The answer is actually fairly complex - and I feel that very few people within the transhumanist community actually understand enough to have grasped this answer. This is because the answer is, in a way, disappointingly un-transhumanist. The non-existence of external, objective moral standards means that we are far less morally obliged to push a high-technology transhumanist agenda than we otherwise would be, and that even if the future does lead us to the high rates of technological change that transhumanists have so worshiped, the optimal way for us to use that technology will probably look quite mundane compared to the grand vsions of some transhumanists. As Michael Annisimov blogs:

"When we gain the ability to manipulate reality easily, most people will probably not choose to live within the sanitized white hallways of science fiction or the boring monoliths of The Jetsons. We will create more forests, rolling grasslands, huge gardens, splendid castles, and other things we can’t yet imagine. We’re all human, and most humans foster a romantic yearning to recreate some idealistic past. The true past was a place of disease and suffering, but we love the pleasant outlines transmitted to us through stories and our imaginations."

Mostly, the answer is contained in the work of Joshua Greene and Eliezer Yudkowsky's Values sequence on Overcoming Bias. It's funny that a mere year ago I was a staunch moral realist, and didn't appreciate the latter material. For example, buried halfway down the article on "the meaning of right" we find:

There isn't enough mystery left to justify reasonable doubt as to whether the causal origin of morality is something outside humanity. We have evolutionary psychology. We know where morality came from. We pretty much know how it works, in broad outline at least. We know there are no little XML value tags on electrons (and indeed, even if you found them, why should you pay attention to what is written there?)
If you hoped that morality would be universalizable - sorry, that one I really can't give back.

There are many pitfalls along the way to the truth about meta ethics and the "truth" about ethics. Not all of them are factual errors, either. It is entirely possible to completely dis-identify with human values, and identify entirely with technology. Since there are such things as convergent instrumental values, this is not obviously an incoherent position. Richard Hollerith maintains this position, and for a few very low months in my life I explored the implications of replacing naive morality with what one might call techno-worship. In the end, I realized that systems like GSZ trample over almost everything we find valuable, and I realized that this is hard to see because we take everything that is ubiquitous in our lives for granted. The existence of distinct persons. Love. Compassion. Humor. Sacred places and events (marriages, birthdays, celebrations...). Relationships.

For me, one of the most important is the tying of abstract scientific discoveries to specific places and people. Watson and Crick walking into the Eagle and announcing the discovery of DNA has so much more meaning than an anonymous computer of some kind making the same discovery.

To cut a long story short, the correct response to technology we see developing around us is to make technology serve our values, rather than making human beings serve technology. I might call this humanistic transhumanism. Unfortunately, this is a very hard message to sell... because humans like to "believe in" something that separates them from the crowd of other humans. Many humans who style themselves as "transhumanists" and "singularitarians" claim to hold ethical beliefs that are more there for show than for implementation. "I want to become a Jupiter Brain superintelligence", etc. [Write me an essay on the philosophical debates surrounding continuity of personal identity first... then see if you still want to]

Still, it is slightly easier to sell what I would call the rational utopia version of humanistic transhumanism; that world that is bounded below in goodness by Iain M Banks' Culture, that world that is approximated by Nick Bostrom's Letter From Utopia. That is something to dream about; a world that is as close to satisfying your muddled preferences as the rules of logic permit. I have wondered about how good such a world would be, but "pretty darn good, and optimal by construction" is a fair shot. The opportunities for intimacy, knowledge, personal development and growth, etc, etc would be pretty cool.

Slightly embarrassingly, it has taken me about 18 months to come around to the view of ethics that I was so skeptical about when Eliezer Yudkowsky first published the CEV concept on the SIAI Blog. Why is this? Am I stupid? I don't think so. I think that most really good hard scientists suffer from a fairly large degree of misanthropy; the classic image of the lonely physicist working on his own and never making contact with people is, in fact, true in many cases. In order to be the best possible mathematician/physicst, one spends most of one's time doing maths/physics. One loses contact with most normal human values, and I think that many such people become almost religious worshipers of science, or rather they dis-identify with many parts of their psyche that are not concerned with finding out scientific truths.

Goal System Zero or universal instrumental values is the logical axiology one would expect to be invented by a person who has dis-identified with every part of their mind that does anything other than do science.

So, I would diagnose myself has having made an error* of the heart - of being bullied too much at school, of finding social and interpersonal skills so late in life, and of being put under such pressure by the way I was brought up and by a childhood environment based upon conditional (rather than unconditional) love. Luckily for me, I also have a sex drive, and once you have a sex drive you have to go out and relate to people. And once you go relate to people, you realize that there really is more to life than math and science! It is interesting to note that my rejection of the techno-worship, GSZ/UIV stance roughly coincided with the first time in my life that I realized that someone had loved me unconditionally. She knows who she is. There are special moments between you and another human being, using Haidt's terminology there are moments of secular sanctity, moments where the mere atoms that make up the world fall into a configuration that we attach special significance to. And yes, most, if not all of these moments are a just instantiations of games that evolution programmed us to play. But, to quote Stathis from the SL4 list:

Yes, you could say that it is all just a game evolution has programmed us to play, but it's an important game for those so programmed.

This is such a subtle and beautiful message, and many people could learn a lot from it.

* (Strictly speaking this is not an error. As hume put it: Tis not contrary to reason to prefer the destruction of the whole world to the scratching of my finger. )

A one sentence summary of anti-realist meta-ethics

2009-03-02T18:56:00.002Z

"Yes, you could say that it is all just a game evolution has programmed us to play, but it's an important game for those so programmed."

Stathis, SL4 List

Heading for a singularity winter?

2009-03-01T11:29:00.004Z

The AI winter is a (collection of) historical periods of skepticism about progress in AI development, caused primarily by unfounded, overoptimistic predictions about the abilities of AI. Looking at the "History of AI" article on wikipedia, we find the following:

The optimism

The first generation of AI researchers made these predictions about their work:

1958, H. A. Simon and Allen Newell: "within ten years a digital computer will be the world's chess champion" and "within ten years a digital computer will discover and prove an important new mathematical theorem."
1965, H. A. Simon: "machines will be capable, within twenty years, of doing any work a man can do."
1967, Marvin Minsky: "Within a generation ... the problem of creating 'artificial intelligence' will substantially be solved."
1970, Marvin Minsky (in Life Magazine): "In from three to eight years we will have a machine with the general intelligence of an average human being."

Many people are getting similarly excited about the now fairly nebulous concept of "the singularity". But, as Michael Annisimov blogs, most people really don't get it.

I. J. Good, a very clever mathematician, worked out in 1965 that really intelligent machines could potentially make other machines that were even smarter, since the design of smart computers is a mental task. Aparrently, it takes a lot of smartness to realize this one thing: intelligence is somewhat special.

Even today in 2009, with the writings of Yudkowsky, Bostrom, Good, Vinge, etc to guide them, most people think that the word "singularity" means "cool futuristic toys", and there is an emerging tribal debate between so called "pro singularity" advocates who think that the future is going to be full of cool futuristic toys and resemble a science fiction novel, and the "singularity skeptics" who think that the future will be a bit like the great depression, only more environmentally friendly.

I was at a book club in Edinburgh frequented by the likes of Stross and Banks, and an author remarked that "we like to write about the singularity, but we don't take it seriously", as if reality were customizable, as if thinking that smarter than human intelligence is silly actually stops it from happening. "Yes, I'd like human level AI to be plausible but practically impossible in my version of reality, because that way I can keep writing cool novels, but I don't have to go through the painful process of adjusting to the ramifications of it"

Hype is quickly overtaking rational thinking. Consider, for example, the courses supposedly on offer at singularity university. 9/10 of them are not about intelligence:

Future Studies & Forecasting
Networks & Computing Systems
Biotechnology & Bioinformatics
Nanotechnology
Medicine, Neuroscience & Human Enhancement
AI, Robotics, & Cognitive Computing
Energy & Ecological Systems
Space & Physical Sciences
Policy, Law & Ethics
Finance & Entrepreneurship

What on earth is "space science" doing on the curriculum? Ecological Systems? Medicine? Entrepreneurship?!

Meanwhile, I have been doing some research into the actual referent of the term singularity, namely research into the creation of smarter than human intelligence. How many researchers are working towards smarter than human AI? Less than 10, certainly, probably with less than £500K of funding earmarked in total (plenty of people are talking about it, though, for example on the AGI email list, where someone has been suggesting that Islamic holy law is a fantastic start for an AGI goal system. Yes, the AGI list is hothouse of rigorous intellectual effort)

It suddenly dawned on me that with hype maxed out and actual research into human level AI at almost zero, we are headed straight for the singularity winter, in which the "singularity skeptic" tribe will point out to that the singularity did not, in fact, appear out of thin air, much to the apparent surprise of many.

My personal reaction to this is a twinge of disappointment, because it means that it is going to be very close to impossible for me to do good work on smarter than human AI. I almost feel that the best policy for anyone smart to the situation is to live for the present, have a good life, and accept that the future will take its own sweet course, guided not by prudent planning but rather by random chance. Must ... resist ...

;-0

Surprisingly Good Solutions?

2009-02-09T23:42:00.002Z

Tom McCabe writes:

A superintelligently-designed Utopia should contain a large number of qualia which are surprisingly good, in the sense that we’ve never even imagined that point in utilityspace. Such qualia are relatively rare in the present world, but they do exist; imagine a starving African child eating their first chocolate cake. This, in some sense, is a more worthy goal for the future than, say, Iain Banks’s Culture series. The Culture is limited by human imagination, while superintelligently-designed utopias shouldn’t be.

Do we agree with him?

I wasn't convinced on Overcoming Bias a few weeks ago:

It's also interesting to wonder whether the goals of "radical" transhumanists might be a little self-contradictory. With a limited human brain, you can (as a matter of physical fact) only entertain thoughts that constrain the future to a limited degree. Even with all technological obstacles out of the way, our imaginations might place a hard limit on how good a future we can try to build for ourselves. Anyone who tries to exceed this limit will end up (somehow) absorbing noise from their environment and incorporating it into their preferences. Not that I have anything against this - it is how we got our preferences in the first place - though it is not a strong motivator for me to fantasize about spending eternity fulfilling preferences that I don't have yet and which I will generate at random at some point in the future when I realize that my extant preferences have "run out of juice".

This, I fear, is a serious torpedo in the side of the transhumanist ideal. I eagerly await somebody proving me wrong here...

But now I'm not so sure. I invite readers to pitch in.

Responding to Anne Corwin on "dreams of immortality"

2009-02-04T19:21:00.005Z

Over at Existence is Wonderful, Anne Corwin has a post up called:

Thanks, But I'll Take Test Tubes of Enzymatic Slime over 'Fountains of Youth' Any Day

Now, this title and the ensuing post annoyed me, because I think it is exactly the wrong attitude to take towards life extension research. I think that Anne's position is an intelligent, even semi-coherent position, but I think it is a bad approach to take, and I hope that by critiquing it I can add a valuable sense of direction to the h+ and life-extension movements. First things first: let me paraphrase Anne's post. In fact, I'll just quote her, leaving out things that she says which I don't disagree with or don't want to comment on here. Readers are encouraged to read the original post and correct me if I have significantly altered Anne's intended meaning:

... I am very sensitive to what my rhetoritician friend Dale Carrico calls superlativity.

... I didn't initially see what was wrong with going on about "technological immortality". But now I do.

... I do not claim that any of the developments being discussed are somehow representative of an inevitable convergence of technology toward humans being able to run around forever in indestructible robot bodies ... That sort of thing is fine if you're writing science fiction stories, or even just daydreaming for the fun of it, but it has nothing to do with what can be accomplished within a reasonable timeframe given what humans know now.

... Superlative predictions tend to tempt people toward glossing over the "...and then a miracle happens!" step between now, when we still have people dying at 90 of heart failure and a potential future in which five-hundred-year-olds run weekly marathons (or accomplish some other feat associated culturally with vigor and youthfulness).

... I certainly don't think such a future is impossible, but neither do I think one needs to believe in such an outcome in order to make healthcare improvements (for the elderly and everyone else) a priority.

... There is no magical property to handwaving that somehow makes handwaved potential outcomes more likely to happen.

... [Longevity research at ASU being described as researchers "seeking a fountain of youth"] - might be attention-grabbing, but at what cost? Do we want to invoke legends and myths in describing interesting and potentially promising science experiments, or the reality of present-day healthcare needs (and potential real-life improvements thereof)? I would definitely lean toward the latter. It may be less glamorous, but when it comes to science and ethics, I'll take the hard (but ultimately more promising) practicalities of the present reality over daydream-invoking glamour any day.

The debate started in the comments. My second comment contained this:

if people don't have hopes and dreams to motivate them to do/fund the research, they just won't bother.

To which Anne responded, including:

You seem to be insisting (and please correct me if this is a misinterpretation) that people who see it as problematic to talk about longevity in terms of "immortality" and "fountains of youth" are somehow the ones harming the prospects of research.

That doesn't make sense.

Daydreaming and the science fiction can certainly enrich and inspire people's imaginations -- heck, the reason I'm an engineer now has a lot to do with my enjoyment of science fiction growing up.

But to suggest (by analogy) that in order to get to the moon it is somehow necessary to believe in (or sit there hoping really, really hard for) transporters and warp speed is silly.

Being inspired by something and having motivation is not the same as acting like the images that inspire you must be "defended" as potential realities.

That last sentence is telling: Being inspired by something and having motivation is not the same as acting like the images that inspire you must be "defended" as potential realities. Come again?

Ok, let's get down to basics. A poll for my readers: would you spend time, money and effort on anti-aging research if you did NOT think that an extended human health span was a potential reality at some point in the future? It's a no-brainer. Either you think that extended human healthspan is a realistic possibility or not. If you do not think that it is a realistic possibility, then why are you spending time and effort trying to achieve it? And yes, I think that you must defend that outcome as a potential reality; if you can't defend it as a potential outcome, why are you working on it yourself?

Now we come on to the most important disagreement, and one where I hope to change Anne's mind. Both Anne and I agree that funding longevity research today could result in human physical immortality. The mechanism goes something like this:

If we can extend your life enough that you're alive in the year 2150, say, then the technology available then may be rather a lot better than it is now, allowing your lifespan to be extended further. By this point, we are very uncertain about what the world will be like. Old certainties like death might not be around. There are many mechanisms by which death could be abolished in, say, the 22nd century, including mind uploading, progress in AGI, progress in nanotechnology, etc. We can see some scenarios where people would not die. We can see others where they would die. The specific dates mentioned here are not that important. We can think of scenarios where mind uploading happens this century. We can see scenarios where life extension research extends people's lives by more than 100 years, etc, etc.

This argument suggests that immortality is possible, and that it is a non-negligible possibility, and that for some group of people (a rather large group, remember 100,000 people die of old age PER DAY), avoidance of death may depend upon whether or not anti-aging treatments are researched today.

So, should we tell people this? Or, should we de-emphasize this particular aspect of the truth and instead say something like:

"Breaking news: Scientists are working to explore the use of microbial machinery to biodegrade 7-ketocholesterol"

People respond to hopes and dreams, not to oxidative pathways and tiresome details of microbial machinery. If you tell them that a certain line of research may allow them to live forever, and they believe you, they will respond by giving you lots and lots of money to make sure that the research gets done. If you tell them that the exact same research is "exploring biodegredation of 7-ketocholesterol", they won't even hear you.

Personally, I think that it is unethical not to tell people about the potential for immortality and the research that will get us there, in the same way that it would be unethical to "forget" to tell someone about the existence of a chemical that might be a cure to a poison they had ingested. These situations are ethically isomorphic.

Yes, Anne, the possibility of superlative outcomes "distorts" people's thinking, in the same way that if you had been bitten by a snake and I told you that there might be an anti-venom in the locked medical cabinet, your thinking would be "distorted" into doing everything you possibly could to get that anti-venom. This is a good thing.

But I feel that this argument will not sway Anne, who will insist that these are totally different situations (perhaps she will claim that, for some unknown reason, the best way to get the anti-venom to the snake bite is to pretend it doesn't exist? After all, it might not work, might not be in there, etc).

So, as a last ditch effort to convince Anne that superlative dreams are a good thing, I point her to a common enemy:

God's purpose for you

If people don't see a rational way of bettering their existence, they will clutch at other "solutions":

The purpose of humanity is to become the spiritual children of God! The purpose for your life is more than making money, gaining social status and wearing the right clothes. The coupling of the human spirit with the Spirit of God makes possible the development of a new nature and eventually a resurrection to a new life as immortal children of God, joint heirs with Jesus of all things.

Viable psychological niches that are compatible with doing something about death?

2009-02-03T23:37:00.002Z

Bostrom, De Grey and Savulescu declare war on death

My angle on it:

One question to consider is this: why is anti-aging such a hard idea to sell? Many people who I have broached this topic with either:

(a) are totally apathetic and/or quickly change the subject

or

(b) find some obscure argument that they claim is totally compelling as to why we shouldn't do this. I have heard the usual ones: death gives life meaning, we must kill ourselves to make way for future generations, we mustn't do anything about the problem until every last African is well fed, death is a great way to get rid of dictators (and these people will claim that dictators are bad because they kill a few thousand people a year), etc, etc.

or

(c) will argue at great length that curing human aging and death is *obviously* impossible and that I need to come to terms with my mortality, even in the face of fairly strong arguments to the contrary and even when I know much more about the relevant science than they do. People sometimes make up arguments on the spot as to why SENS and/or cryonics won't work.

There is probably an underlying psychological explanation for this behavior. (a), (b) and (c) probably correlate with three distinct ways of coping with our mortality:

(a)-type people , who are usually young, can just pretend it doesn't exist. There's no way in hell they'll get involved with the anti-aging movement, because then they would no longer be able to pretend that death doesn't exist.

(b)-type people, typically religious or environmentalist or very left-wing, have decided that death is a good thing after all, and they are frightened that their protective worldview will collapse if the anti-aging movement succeeds. These people may actually sabotage the movement, bomb rejuvenation clinics, etc. Imagine all of these youthful, wise, rich, popular, happy, successful 100 year olds swanning around in the year 2050 as living proof that death is actually not a good thing. That would be anathema to the (b) worldview.

(c)-type people, usually materialist rationalists, can cope with mortality, but only if it is TOTALLY unavoidable and thus not to be worried about. The idea that aging might be cured at some point in the future is poison to their particular psychological defense: it is only by living with the absolute certainty that death is unavoidable that they can relax about it.

I think that the most potent weapon in the war on death right now is to examine the psychology of those who implicitly support it, and find ways of giving people viable psychological niches that are compatible with doing something about death. This will involve talking to some psychologists and perhaps doing some research.

Minsky, Sloman on commonsense intelligence architectures

2009-02-03T14:31:00.007Z

Marvin Minsky and Aaron Sloman are experts on AI - they've spent their lives doing it. Their best guess for an architecture for an AI with commonsense or "human level" abilities is the following:

Mental spaghetti, anyone?

"Minsky describes “ways-to-think”
as configurations of agents within the mind
that dispose it towards using certain styles of
representation, collections of commonsense
knowledge, strategies for reasoning, types of
goals and preferences, memories of past expe-
riences, manners of reﬂections, and all the oth-
er aspects that go into a particular “cognitive
style.” One source of knowledge relating prob-
lem-types to ways-to-think is the causal diver-
sity matrix discussed at the start of the meet-
ing—for example, if the system were presented
with a social problem, it might use the causal
diversity matrix to then select a case-based
style of reasoning, and a particular database of
social reasoning episodes to use with it.
However, any particular such approach is
likely to fail in various ways. Then if certain
“critic” agents notice specific ways in which
that approach has failed, they either suggest
strategies to adapt that approach, or suggest
alternative ways-to-think, as suggested shown
in ﬁgure 6. This is not done by employing any
simple strategy for reflection and repair, but
rather by using large arrays of higher level
knowledge about where each way-to-think has
advantages and disadvantages, and how to
adapt them to new contexts."

The researchers present clearly didn't have much of a clue about what the use of a human level AI would be, though:

After a good deal of argument, several partic-
ipants converged upon a vision from The Dia-
mond Age, a novel by Neil Stephenson. That
novel envisioned an “intelligent book”—The
Young Ladies Illustrated Primer—that, when giv-
en to a young girl, would immediately bond
with her and come to understand her so well as
to become a powerful personal tutor and men-
tor.
This suggested that we could try to build a
personalized teaching machine that would adapt
itself to someone’s particular circumstances,
difﬁculties, and needs.

Passages like this remind me that the field of the future of humanity is a weird place to be. Minsky, Lenat, Sloman, and whole host of AI experts gather at a conference to discuss how to build the most powerful machine in the universe. In my very humble opinion, their ideas are fairly good. Then they decide that they need a use for it - and decide that personal tutors are the most compelling application of a human level AGI!

Read the original article here.

XKCD on Artificial Intelligence Risks

2009-02-01T17:54:00.002Z

See the original here