The Multiverse According to Ben

100 neural net cycles to produce consciousness?

2009-12-02T07:17:00.003-05:00

This interesting article presents data indicating that it takes around half a second for an unconscious visual percept to become conscious (in the human brain)...

This matches well with Libet's result that there is a half-second lag between unconsciously initiating an action and consciously knowing you're initiating an action...

(Of course, what is meant by "consciousness" here is "consciousness of the reflective, language-friendly portion of the human mind" -- but I don't want to digress onto the philosophy of consciousness just now; that's not the point of this post ... I've done that in N prior blog posts ;-)

My Chinese collaborator ChenShuo pointed out that, combined with information about the timing of neural firing, this lets us estimate how much neural processing is needed to produce conscious perception.

As I recall, the firing of a single neuron's action potential takes around 5 milliseconds ... It takes maybe another 10-20 milliseconds after that for the neuron to be able to fire again (that's the "refractory period") .... Those numbers are not exact but I'm pretty sure they're the right order of magnitude...

So, the very rough estimate is 100 cycles in the neural net before consciousness, it would seem ;)

This fits with the view of consciousness in terms of strange attractors ... 100 cycles is often enough time for a recurrent net to converge to into an attractor basin ...

But of course the dynamics during those ~100 cycles is the more interesting story, and it's still obscure....

Is it really an attractor we have here, or "just" a nicely patterned transient? A terminal attractor a la Mikhail Zak's work, perhaps? Etc.

Enquiring minds want to know! (TM)

Dream of the Multiversal Cylinder

2009-11-16T20:52:00.003-05:00

(I usually reserve this blog for speculations on intellectual topics, but last night I had a dream that seemed sufficiently interesting to post here. So, here goes ;-) ....

In this dream, I moved to a strange foreign nation, and met a beautiful girl there whose ex-boyfriend was making her life very difficult, yet who she was still somehow attached to....

His martial arts expertise alarmed me, and so together with the mother of a friend who lived in this same strange place -- a very short, hunchbacked old lady who walked with a cane and wore a funny straw hat -- I went to a weird old-fashioned section of the city, where we did two things.

First, we paid some old white-bearded "witch doctor" to cast a magical spell on the ex-boyfriend, which caused him to forget having ever known the girl, haha.

Then, we went to a strange store full of ancient relics, and bought this cylindrical wooden container, which I was supposed to keep in my bedroom for good luck, but not to open.

The girl and I walked along the beach and the ex-boyfriend walked right past and showed no sign of recognizing her. This freaked her out a bit, and she asked me to have the spell undone on Dec. 21 2012.

Then I went back to my house, which I suddenly shared with the girl, and of course I had to open the wooden cylinder. She kept telling me not to, but I had to anyway. I opened one end of it, prying it open with a screwdriver, and inside the small cylinder was an infinite space -- a whole multiverse of possibilities.

She just kept staring inside it, looking intent but not saying anything. I asked if she wanted me to close it; but she shook her head no. There were millions of these little intelligent creatures in there, which could see our (and everything's) past and future.... Clearly she was absorbing a lot of knowledge from them ... and so was I ... but it was also clear that we were absorbing somewhat different things.

Then, we looked at each other and, without words, asked each other if we should dive into one of those universes or stay in this one. It was clear that in those universes we could still exist as individuals (and could still be with each other); but would exist in radically different form (some form not constrained by time, though there were other constraints not comprehensible in human terms).

Gradually, we collectively realized that we did not feel like entering that other multiverse at that particular time.

Then, she gave me a look that meant something like: "I will never be afraid of anything relating to human society anymore, nor be afraid of my own emotions, because I can see that this whole world of you and me and humanity and Earth is just a sort of artistic construction, which exists for aesthetic purposes. We have chosen to remain in this universe so as to remain part of this artwork ... "

... and then her unspoken thought faded out before it was done, because someone was in the house walking around and we got distracted by wondering who it was...

... and then I woke up because of the noise of my dad walking around downstairs in my house (he was visiting last night)

... and I tried to fall back asleep so as to re-enter the dream, but failed ...

Causation as Storytelling

2009-10-15T20:41:00.008-04:00

I've pointed out before (and it's not my original observation) that no branch of modern science contains a notion of "cause" more than vaguely similar to the folk psychology notion -- causation, as we commonsensically understand it, is something that we humans introduce to help us understand the world; and most directly, to help us figure out what to do....

[David Orban, in a comment on an earlier version of this post, noted that some formulations of relativity theory contain the term and concept of "causation." But causation as used in that context is really just "influence" -- the restrictions on light cones and so forth tell you which events can influence which other ones, but don't tell you how to distinguish which events are causal of which other ones in the stronger, commonsense usage of the "causation" concept.]

Cause, in our everyday intuitive world-view, is tied to will: "A causes B" means "I analogically intuit that if I were able to choose to make A happen, then B would happen as a result of my choice."

And, I think cause is also tied to storytelling. Causal ascription is basically a form of storytelling.

Think about the archetypal story structure:

If we envision a typical causal ascription as fitting into this structure, we have:

The Setup is the situation in which the causation occurs, the set of "enabling conditions." For instance, we rarely would say that oxygen is the cause of us being alive -- oxygen is considered an "enabling condition" rather than a cause of our life ... it's part of the set-up....

The Confrontation is the introduction of something unusual into the Setup. This must be something that is not always there in the Setup, otherwise one wouldn't be able to isolate it as the cause of some particular events. It's not necessarily a violent confrontation, but it's a violation of the norm. Could be someone shooting a gun, could be a couple having sex, could be a finger pushing down on a computer keyboard. The less expectable and frequent it is, the better -- i.e. the more convincing it will be as a potential cause of some event.

The End is the event being caused.

My suggestion is that, if one digs into the matter deeply, one will find that many of the same patterns distinguishing compelling stories from bad ones, also distinguish convincing causal ascriptions from unconvincing ones.

What Would Aristotle Say in a Situation Like This?

The Aristotelian distinction between efficient and final cause is also relevant here.

"Efficient cause" is what we usually think about as causation these days: roughly, A causes B if

P(B|A,Setup) > P(B|Setup)

and there is some "plausible causal mechanism" (i.e. some convincing story) connecting A and B in the context of the Setup.

"Final cause" is telos, teleology -- A causes B if B, as a goal, somehow reaches back in time and makes A happen as part of the inevitable movement toward B's occurrence.

Modern physics theories have no place for final causes in the Aristotelian sense. But, human psychology does! Very often, when a human seeks a cause for something, what they're doing starting with some event they've observed and trying to find a "good reason" for it.

Why did I fall in love with her? It must have been because she was beautiful ... or smart ... or rich ... or whatever.

Why did my business succeed? It must have been because I was smart ... or because it was the right time ... etc.

Storytelling generally mixes up efficient and final causation in complex ways. Many stories give a feeling of inevitability -- final causation -- by the end. And when postmodernist stories avoid giving this feeling, it's generally done intentionally, with a view toward violating the known psychological norm and doing something disconcerting or shocking.

Convincing causal ascriptions, like compelling stories, tend to mix up efficient and final causation.

Cause and Will

Nietzsche wrote that (paraphrasing) "free will is like the army commander who takes responsibility, after the fact, for the actions of his troops."

Experiments by Gazzaniga, Libet and other neuroscientists have validated that in many cases the reflective, willing portion of the human brain-mind "decides" to do something only well after some other part of the brain has actually already started to do it.

This fits in fine with the notion of causal ascription as storytelling. Willing is a matter of making up a story about how one came to do something. It had better be a compelling story or the illusion of free will will fall apart, which is bad for the maintenance of the self-model!

Causation and Storytelling in Neuroscience, AGI and Early Cognitive Development

One concrete hypothesis that comes out of this train of thought is that, when the neural foundations of causal ascription and storytelling are unravelled, it will turn out that the two share a large number of structural and dynamical mechanisms.

Another hypothesis is that, if we want our AGI systems to be able to ascribe causes in humanlike ways, we should teach our AGI systems to tell and understand stories in a humanlike way.

I strongly suspect that one of the major roles that storytelling plays in human childhood, is to teach children patterns of narrative structure that they will use throughout their lives in constructing causal ascriptions (along with many other kinds of stories).

...

Ahh ... I would love to improve this blog post with a bunch of concrete examples but that will need to wait for later ... I'm tired and need to wake up early in the AM ... at least "I" think that is the cause of me not wanting to improve it right now ;-D ...

AGI, Ethics, Cognitive Synergy and Ethical Synergy (from my Yale talk...)

2009-09-09T08:05:00.006-04:00

Earlier this year I gave a talk at Yale University titled "Ethical Issues Related to Advanced Artificial General Intelligence (A Few Small Worries)" ...

It was a verbal discussion focused rather than PPT focused talk but I did show a few slides (though I mostly ignored them during the talk): anyway the brief ugly slideshow is here for your amusement...

The most innovative point made during the talk was a connection between the multiple types of memory and multiple types of ethical knowledge and understanding.

I showed this diagram of different types of memory and the cognitive processes associated with them (click the picture to see a bigger, more legible version)

and then I showed this diagram

which associates different types of ethical intuition with different types of memory.

To wit:

Episodic memory corresponds to the process of ethically assessing a situation based on similar prior situations
Sensorimotor memory corresponds to "mirror neuron" type ethics, where you feel another person's feelings via mirroring their physiological emotional responses and actions
Declarative memory corresponds to rational ethical judgment
Procedural memory corresponds to "ethical habit" ... learning by imitation and reinforcement to do what is right, even when the reasons aren't well articulated or understood
Attentional memory corresponds to the existings of appropriate patterns guiding one to pay adequate attention to ethical considerations at appropriate times

I presented the concept that an ethically mature person should balance all these kinds of ethics.

This notion ties in with a paper that Stephan Bugaj and I delivered at AGI-08, called Stages of Ethical Development in Artificial General Intelligence Systems. In this paper we discussed, among other topics, Kohlberg's theory of logical ethical judgment and Gilligan's theory of empathic ethical judgment. In the present terms, I'd say Kohlberg's theory is declarative-memory focused whereas Gilligan's theory is focused on episodic and sensorimotor memory. We concluded there that to pass to the "mature" stage of ethical development, a deep and rich integration of the logical and empathic approaches to ethics is required.

The present ideas suggest a modification to this idea: to pass to the mature stage of ethical development, a deep and rich integration of the ethical approaches associated with the five main types of memory systems is required.

Why Bother to Vote?: A Novel Multiversal Answer

2009-09-08T10:02:00.004-04:00

Any sensible person with the choice of going to the polls to vote has almost surely asked themselves “Why should I bother voting when it’s incredibly unlikely my vote will make any difference, given the large number of people voting?”

[Note for non-US readers: in the US, unlike some countries, there is no legal requirement to vote; it's an option.]

I've discussed this issue with dozens of people and have never really heard any sensible answers.

I say "If I stay home and work or play Parcheesi instead of voting, then the election will proceed exactly the same way as if I had voted. The odds of me affecting this election are incredibly tiny."

They say: "Yeah, but if EVERYBODY thought that way, then democracy couldn't work." ... as if this were a counterargument.

Or: "Yeah, but if everyone intelligent enough to have that train of thought followed it and avoided voting, then only stupid people would vote and we'd have a government elected by the retarded.... Oh, wait ... would that be any different than what we actually have now?"

I've thought about this a lot, off and on, over the years, and finally I think I've come up with an interesting, novel answer to the question. To have a handy label, I'll call it the "multiversal answer."

This is a somewhat philosophically complex answer, which requires a deviation from our ordinary ways of thinking about our relationship between ourselves and the universe.

I'll run through some details and probability calculations, and then get back to philosophy and free will and such at the end.

Rational Agents

The multiversal answer pertains to agents who make choices based on expected utility maximization. That is, it pertains to agents who: Given a choice between two actions, will choose the one with the property that, after the choice is made, the agent’s utility will be highest. Or, to put it informally, it pertains to agents who follow the rule: “Choose the option that, in hindsight, you will wish you had made.”

Of course people don't always follow this sort of rule in determining their actions; people are complex dynamical systems and don't follow any simple rules. But, my point is to argue why voting might make sense for an agent following a simple rational decision-making procedure. I.e.: why voting might be a reasonable behavior even though, in the sense indicated above, the odds of your vote being decisive in an election are minimal.

Vote So That You'll Live in a Universe Where People Like You Vote

The conceptual basis of the multiversal answer is the principle that “you should vote because, if you vote, this means that after you’ve voted, you’ll know that you probably live in a universe where people similar to you vote.

On the other hand, if you don’t vote, this means you probably live in a universe where people similar to you don’t vote.”

Clearly, you would rather live in a universe where people similar to you vote.

(Yes, this could be formalized based on the degrees to which individuals with varying degrees of similarity to you vote. But we won’t worry about the math details for now.)

Your vote may not count much on its own, but it’s a bad thing if everyone similar to you (with the same preferences as you) doesn’t vote.

Note that it’s not good enough to intend to vote but then back out at the last minute. After all, if you do that, then probably everyone similar to you is going to do the same thing! So if you do that, it means you’re in a universe where people similar to you are likely to almost vote, rather than a universe where people similar to you are likely to vote.

Possible Worlds

Underlying this answer is a “possible worlds” philosophy, holding that there are many possible universes we could live in -- and we don’t know exactly which one we do live in, based on the limited data at our disposal.

So, given a predicate P like “the degree to which people similar to me vote,” we can estimate the truth value of P by a weighted average of the product

(degree that P holds in possible world W) * (probability that world W is the one I live in)

(or some similar formula).

Some Plausible Assumptions

So, based on the above, suppose we we assume that

P(people like me vote | I vote) > P(people like me vote | I don’t vote)

and

P(good world | people like me vote) > P(good world | people like me don't vote)

(where “good world” is shorthand for “I live in a possible world I like.” Again, this can be more fully formalized, but I won’t bother with that for now.)

Note that these probabilities are calculated across possible worlds. For instance,

P(people like me vote | I vote)

means

P(people like me vote in possible world W | I vote in possible world W)

A Critical Question

So, given the above, one has the question: From the above inequalities, can we derive that

P(good world | I vote) > P(good world | I don't vote)

which would imply that voting increases the probability of living in a good world (whether or not one wins the particular election one is voting in)???

The answer is: not quite.

But, sort of.

What the mathematics tells us is that this conclusion holds if

max[ P(gw|Iv & plmv) - P(gw|plmv), P(gw|Iv & ~plmv) - P(gw|~plmv) ]

<

.5 * [P(gw | plmv) - P(gw)] [ P(plmv|Iv) - P(plmv)]

where

gw = good world
Iv = I vote
plmv = people like me vote

Basically, if the left hand side of this equation is small, this means that the effect of me voting on the probability that I live in a good world, is almost entirely contained in the effect of people like me voting on this probability. But, this seems quite sensible.

So, if this condition holds, then voting increases the odds of being in a good world, so it makes some sense to vote to increase the odds of being in a good world.

There’s still a quantitative calculation to make, though. Voting has some cost, so one needs to estimate whether the increase in the expected goodness of {the world one estimates oneself to live in}, induced by voting, outweighs the cost of voting. This devolves into a bunch of algebra that I don’t feel like doing right now. But note that it’s a totally different calculation than the calculation as to whether one’s individual vote makes any difference in a particular election.

Free Will

Underlying the above perspective is an attitude toward "free will" which is different from the one conventional in the modern Western mindset.

In the conventional interpretation of "free will", a person can choose whether to vote or not, and this doesn't impact their estimate of what kind of universe they live in -- it's an independent, free choice.

In the interpretation used in the multiversal answer to the voting problem, a person can (in a sense) choose what to do, but then when they study their choices in hindsight, they can infer from the pattern of their choices something about the universe they live in.

Combining this with the "expectation maximization" approach, which says you should make the choices that you'll be happiest with in hindsight (after the choice is made) ... one comes up with the principle that you should make the choice that, in hindsight, will yield the most desirable implications about what kind of universe you live in.

And it's according to this principle that, in the multiversal answer, voting may be a sensible choice regardless of the small chance that your particular vote impacts the election.

The point is that, after voting, the fact that you voted will give you evidence that you live in a nice universe, where people like you vote and therefore things tend to go in a favorable way for you.

On the other hand, if you don't vote, then afterwards the fact that you didn't vote will give you evidence that you live in a universe where people like you don't vote, and therefore things tend to go against you.

So, I think the decision to not vote because your vote is very unlikely to impact the election, is based partly on a naive folk theory of "free will." In a more mature view of will and its relation to the universe, the decision to vote or not isn't exactly a "free and independent decision" ... but there is rationality in making the "not quite free or independent decision" to vote.

(Perhaps this is related to the intuition people have when they say things like "If everyone thought that way, then no one would vote." Statements like this may reflect some intuition about what it means to live in a good branch of the multiverse, which however conflicts with modern Western folk psychology intuition about free will.)

Will China Build AGI First?

2009-07-26T15:52:00.011-04:00

One of the reasons I spent 4 weeks in China this summer, organizing the First AGI Summer School and collaborating on research with my friend Hugo de Garis's Artificial Brain Lab at Xiamen University, was to qualitatively investigate Hugo's stories about the great potential for AGI R&D allegedly existent in the country of China.

One thing I learned about China is: the answer to almost any nontrivial question is some complex, multidimensional form of "maybe" or "sort of."

(Eventually the maybes and sort-ofs must collapse: Chinese do make hiring and firing decisions, get married, publish papers, and take other definitive actions ... but on the whole I found that in China there is a much greater willingness to embrace uncertainty than in the US, and a much smaller desire to make things clear and definite.)

In this spirit, I can't claim I came to a definite conclusion about the potential of China to lead the world in advanced AGI R&D.

But I can say that it's a definite possibility ;-) ....

Though the situation is complex, my gut feel is that Hugo is probably right, at least in the following sense: If a moderate number of AGI researchers (from the West and China both) apply their energy to pursuing the R&D opportunities that China offers, there is a strong potential that AGI research could advance there much faster than in other parts of the world.

Here are some relevant facts, militating in favor of China's role in AGI technology:

Even in the current dismal world economy, China's economy is still growing
Chinese students and researchers are willing to work long hours (of course some Americans and Europeans are too ... but my impression is that this willingness is greater in China)
The Chinese education system is very good at teaching advanced mathematics and algorithm theory, which are important for AGI
China is interested in pursuing advanced technologies -- both with practical applications in mind, and with a nationalistic motive of displaying their technological strength relative to other nations
Unlike the US, the Chinese research funding establishment has no "chip on its shoulder" about AI or AGI -- it has the same status as any other advanced technology. There was never an "AI winter" in China, nor is there dramatically more skepticism about AI than about other computer technologies
Due to the centralized system of government, if the central administration decides they value a certain technology, there is the potential for a massive amount of resources to be directed to that technology in a relatively rapid time-frame. Things often move very slowly in China, but sometimes they can also move much faster than in less centrally organized economies.
The cost of highly educated labor is low in China, so that if funding from outside China is found to help support China-based projects, this funding can go a long way!

Here are some other relevant facts, which are challenges China has to overcome if it wants to lead the world in AGI:

The Chinese university education system is narrowly disciplinary, whereas AGI requires copious interdisciplinary knowledge
Because China lacks a large, robust, cutting-edge software industry, there isn't that much "cultural knowledge" about how to manage complex software projects using "agile" methodologies. Yet, AGI is a complex software project that really demands an agile methodology. (Note that China has a load of great software engineers; the issue I'm pointing out regards software project management, not software engineering.)
Due to the nature of Chinese culture, it is fairly common for work environments to arise in which participants don't feel free to share their innovative ideas, and to point out problems with the ideas others are pursuing (especially if these "others" have higher social status according to Chinese tradition). However, some Chinese work environments are very friendly to innovation and criticism; so IMO this is best considered as a problem that can be overcome with attention to the personalities involved and the management mechanisms
Due to various factors including the restrictions on travel and Internet sites that the Chinese government places, China often feels "cut off" from the rest of the world, which results in a less-than-optimal degree of interaction with the international research community

Given the above factors, my conclusion is that IF China is able to attract sufficient foreign AGI experts, it may well be able to leapfrog ahead of other nations in the race to create AGI.

What the foreign experts would bring is not just AGI expertise and ideas, but expertise in allied areas like

interdisciplinary education
agile software project management
the management of innovation-friendly, "flat-hierarchy" research groups

Further, the presence of foreign experts in China full-time would result in other foreign experts more frequently traveling to China to speak, and in Chinese students more often traveling outside China for conferences and research visitation -- all of which would decrease the "China isolation" factor, and increase the intellectual potency of Chinese AGI research labs.

It's also the case that in Chinese academia right now, foreigners can get away with "shaking things up" a little more than Chinese nationals can. So even if a Chinese national showed up with exactly the same expertise and personality as a foreigner, they would have different strengths and weaknesses in the context. They would be able to get some things done more easily due to their Chinese-ness, but would also not be able to "get away with" as many disruptive methodologies.

So: the reason I think foreign AGI experts are critical is not that Chinese lack good ideas about AGI. (Yes, I think my own ideas about AGI are the best ones, but that's not my point right now!) It's that I don't think any one country has a monopoly on great AGI ideas and people, so the prize is likely to go to a country that can build an international AGI research community ... and also that there are certain organizational skills that are very useful for AGI, but not that well developed in China right now.

But there are serious challenges involved in recruiting foreign AGI experts to China:

The salaries are low by international standards. The salaries foreign faculty get at Chinese universities allow a very nice lifestyle in China -- but even so, they don't go that far in terms of international travel, purchase of electronics, or helping family members in the West.
Many AGI researchers have spouses and children who don't want to live in China. There are good international schools for researchers' children; but finding appropriate jobs for spouses can be difficult due to the language barrier and the different nature of the economy
The Chinese system of government is alien and off-putting to some foreigners. (As an example, this blog cannot be read by most Chinese Internet users right now, because blogger.com is blocked by the Chinese government. This sort of thing really bothers some foreign researchers and/or their families)
Some researchers will fear "career damage" if they go to a university without international name recognition (though, this factor would disappear quickly after a critical mass of researchers went to China)

So, the sixty-four trillion dollar question is to what extent these latter factors can be overcome.

I believe they could be overcome if Chinese universities or research agencies made very clear, very clearly research-friendly offers to foreign AGI researchers -- and then followed through on these offers once the researchers arrived. AGI researchers are a dedicated bunch, and many would put up with the problems cited above in order to have a good chance to lead a team of brilliant, qualified students at implementing powerful AGI systems.

As you may have guessed, one reason I'm cataloguing these factors so systematically is that I'm debating trying to rearrange my life to either move to China or (more likely) spend one semester per year in China.

At the moment, in my own discussions with Chinese universities about AGI research funding, I am finding things mildly confusing. The discussions are going interestingly, but I feel much less clarity than I would in a comparable discussion with a US university. This is nobody's fault -- it's a natural consequence of "cultural differences" -- but it's a factor that will have to be smoothed-out, IMO, if China is going to recruit a sizeable number of foreign AGI researchers.

So we now reach the conclusion of the above chain of thought. IF Chinese universities manage to fine-tune the art of recruiting foreign AGI researchers, then I think that China has a real chance of leading the world in the development of AGI.

I predict that, if China doesn't adopt the world lead in AGI, it will be because it fails at the things I cited above. They will fail to dominate in AGI if it turns out that the Chinese way of recruiting and retaining foreigners is too alien, causing a failure to accumulate a critical mass of foreign R&D leaders. This could certainly happen. Time will tell.

What Are the Risks if China Pulls Ahead in the AGI Race?

So China might plausibly take the lead in the AGI race.

As a citizen of the USA and Brazil (not China), does this worry me?

Not really.

Like Buckminster Fuller, I consider myself a "passenger on Spaceship Earth" (and I won't hesitate much to board another vessel when one becomes available -- or better yet I'd like to send multiple copies of myself on multiple vessels! But, I digress ;-).

One thing I'm being insistent on in my collaboration with Hugo de Garis's Artificial Brain Lab at Xiamen University, is that all our work be released as open-source code. The university folks there have no problem with this. So, in the case of my and Hugo's work, it's not a situation where we're trying to develop an AGI that will be exclusively owned by the Chinese government.

And, I feel strongly that anyone else doing AGI work in China -- or anywhere else! -- should take the same approach. The main reason I decided to open-source my own AGI project (OpenCog) (while keeping some valuable AGI-related technologies proprietary within Novamente LLC), is the intuition that AGI is a sufficiently big and thoroughgoingly important thing that it should be developed by the human collective mind as a whole, not by a small group or even a single nation.

Of course, if more and more AGI research gets done in China, then more and more of the world's AGI expertise will exist in China -- which will give China a substantial leadership position in AGI, regardless of whether the AGI code is open-source. But this really doesn't worry me much either, partly because I've been so impressed with the character and spirit of the younger generation of Chinese, who (in these scenarios we're discussing) will be doing most of the AGI work.

I met wonderful Chinese people of all ages during my visit to China. But there are huge generational differences among Chinese, and the Chinese who grew up with the Internet have a drastically different view of the world than the immediately prior generations. Most of the Chinese I met aged under 30 had a reasonably modern, international understanding of the world -- and some of the Chinese I met aged under 22 had such modern attitudes that they really could have been youth anywhere. The Internet is spreading international ideas and culture around the world, just like it's spreading the AGI meme around the world -- and may increasingly start spreading AGI researchers around the world ... we'll see.

Although I used a picture of Alfred E. Neuman above (like Hugo, I'm kind of a sucker for dramatic effect), I want to be clear that I don't have a cavalier attitude about the threat that could be posed if ANY government took control of the world's first AGI for their own parochial ends.

But I think this is a problem we need to work around, regardless of which country we do our research in.

By developing open-source code (made available on SourceForge, Launchpad, Google Code and so forth), and by carrying out our research in a way that emphasizes linkages with the international research community, we'll guarantee that AGI comes about as a product of the international collective mind of AGI researchers. This provides no grand guarantee of "AGI safety" (nothing can do that), but I strongly feel it's the best approach.

The First AGI Summer School in Xiamen, China (Retrospective and Random Related Musings)

2009-07-26T13:04:00.011-04:00

I returned home 2 weeks ago from the First AGI Summer School, which was held in the Artificial Brain Lab at Xiamen University in Xiamen, China at the end of June and the beginning of July.

Ever since I got back I've been meaning to write a proper summary of the summer school -- how it went, what we learned, and so forth -- but I haven't found the time, and it doesn't look like I'm going to; so, this blog post will have to suffice, for the time being at any rate.

First of all, I need to express my gratitude to Hugo de Garis

and Xiamen University for helping set up the summer school. Coming to Xiamen to do the Summer School was a great experience for me and the others involved -- so, thanks much!

Some photos I took in Xiamen are here (mixed up with a few that YKY took on the same trip). (Viewer beware: some of these are summer school photos, some are just "Ben's Xiamen tourism photos"....)

To get a sense of what was taught at the summer school -- and who was on the faculty -- you can go to the summer school website; I won't repeat that information here.

The first two weeks of the summer school were lecture-based, and the last week was a practical, hands-on workshop focused on the OpenCog AI system. Unfortunately I missed most of the hands-on segment, as I wound up spending much of that week meeting with various Chinese university officials about future possibilities for Chinese AGI funding (but I'll write another blog post about that), and demo-ing the Artificial Brain Lab robot to said officials.

See here for some videos of the above-mentioned robot, along with some "OpenCog virtual pet" demo videos that were shown at the summer school. (And, the OpenCog virtual pet was also gotten up and running "live" in Xiamen, of course....)

The number of students wasn't as large as we'd hoped -- but on the plus side, we did have a group of VERY GOOD students who learned a lot about AGI, which was after all the point.

(In fact, most conferences have found their attendance figures down this year, due to people wanting to save money on travel costs: an obvious consequence of the faltering world economy.) The majority of students were Chinese from Xiamen University and other universities in Fujian province, but there were also some overseas students from Europe, the US, Korea and Hong Kong (OK, well, Hong Kong isn't quite "overseas" ;-).

All the lectures were videotaped by Raj Dye (thanks Raj!!)

and will be put online once Raj gets time to edit them. I think these will form an extremely valuable resource, and will reach a lot more people than the summer school itself did. (Long live the Internet!!). Raj's active camera work captured a bunch of the dialogues during and after the talks as well, and I think these will make quite interesting viewing. As you might expect, there was some pretty intense give-and-take (especially, for example, during Allan Combs' talks on cognition and the brain).

I'm definitely interested to help organize some future AGI summer schools ... though the next one will be in a different location, as we've already done a pretty good job of spreading the word about AGI to the AI geeks of Fujian Province! Maybe the next one will even be back here in the boring old US of A ....

Random Observations on Chinese-ness in the AGI Context

I learned a lot about China in the course of doing the summer school (though I'm still pathetically ignorant about the place of course ... there's a lot to know) ... I won't try to convey 1% of what I learned here, but will just write down a few hasty and random semi-relevant observations.

First, I learned to speak verrrry slowly and clearly since Chinese students are more accustomed to written than spoken English! ;-)

More interestingly, I learned that the Chinese educational system is more narrowly disciplinary than the US system, and also more focused on memorization of declarative knowledge rather than practical "know how." Compared to their US counterparts, computer science graduates in China know an AWFUL LOT of computer science, yet don't have much advanced knowledge of areas beyond computer science, nor all that much software engineering knowledge or hands-on coding experience. ("Software Engineering" is a separate department in the Chinese university system, and I didn't get to know the Software Engineering students, only the Computer Science ones.) So one role the summer school served was just to introduce a bunch of Chinese AI students to some allied disciplines -- neuroscience, cognitive psychology, philosophy of mind -- that they hadn't seen much during their formal education so far.

(Actually, separately from the Summer School, I did give a talk to some undergrads in the Software Engineering School, on AI and Gaming, which contained one funny bit (unfortunately that talk was not videotaped). I wasn't sure if the students understood what I was talking about, so as a test I showed them this picture as part of my powerpoint

Normally I use this lovely picture as an example of "conceptual blending" (a cognitive operation that OpenCog and other AGI systems must carry out), but this time I announced it differently; I said: "Furthermore, the Artificial Brain Lab here at XMU has an ambitious backup plan, in case our computer science approach to AGI fails. We've devised a machine that can remove the head from a graduate student, and attach it to the body of a Nao humanoid robot, and thus create a kind of synergetic cyborg intelligence." I was curious to see if these Chinese undergrads understood my English well enough to tell that I was joking -- but from their reaction, I was unable to tell. They laughed because the picture looked funny, but I still don't know if they understood what I was saying! Fortunately the Summer School students were less inscrutable, and more reactive and communicative! And overall the AI in Games lecture went well in spite of this perplexing crosscultural joke experience....)

(As an aside within an aside, I also learned during various conversations that typical Chinese high school students spend from 7AM till 10PM or so at school, 6 days a week. Damn.)

Another thing that surprised me was the strength of knowledge the Chinese students had in neural nets, fuzzy logic, computer vision and other "soft computing" and robotics related AI, as compared to logic-based AI. By and large, they had a very strong mathematics background, and a good knowledge of formal logic -- but fairly little exposure to the paradigm in which logic is applied to create AI systems. Quite different from the typical American AI education.

All in all the Chinese seemed to have a lot less skepticism about "strong AI" than Americans. It's not that they had a great faith in its immediacy -- more that they lacked the egomaniacal confidence in its extreme difficulty or implausibility, which one so often finds in Westerners. Chinese culturally seem much more comfortable with accepting situations of great unconfidence, in which the evidence just doesn't exist to make a confident estimate.

I came to the summer school from the Toward a Science of Consciousness conference in Hong Kong, where I led a Machine Consciousness workshop -- which I won't write about here, because I wrote a summary of it for H+ magazine, which will appear shortly. Issues of machine consciousness came up now and then at the summer school, but interestingly, they seem to hold a lot less fascination for Chinese than for Westerners. When I put forth my panpsychist perspective in China (that the universe as a whole is conscious in a useful sense, and different systems -- like human brains and digital computers -- manifest this consciousness in different ways ... and our "theater of reflective consciousness" is one of the ways universal consciousness can manifest itself in certain sorts of complex systems), no one really bats an eye (and not just because the Chinese lack a taste for eye-batting). Not that Chinese scientists consider this panpsychist perspective wholly obvious or necessarily correct; but nor do they consider it outrageous -- and, most critically, very FEW Chinese seem to feel like many Westerners do, that "reductionism" or "materialism" is obviously correct. Once you remove the tendency toward dogmatic materialism, the whole topic and dilemma of "machine consciousness" loses its bite....

China versus California (A Semi-Digression)

(This section contains some ramblings on Oriental versus California culture, and the Singularity -- which are only semi-relevant to the summer school, but I'll put them here anyway, because I find them amusing! Hey, this is a blog, anything goes ;-)

In mid-July I voyaged from the Xiamen AGI summer school to California where I gave the keynote speech at the IJCAI workshop on Neural-Symbolic computing (a really interesting gathering, which I'll discuss some other time), and then gave a lecture on AGI at the Singularity University (at NASA Ames Lab, in Silicon Valley).

The contrast between the SU students and the Chinese AGI Summer School students couldn't have been more acute.

For one thing, there was a huge contrast of ego ... to phrase things dramatically: The SU students emanated an attitude that seemed to say "We know more than anyone on the planet!! We already knew almost everything we need to know to dominate the world as part of the techno-elite!"

The Chinese students were not actually more ignorant (though their knowledge bases had different strengths and weaknesses than those of the SU students), but they were dramatically more humble about their state of knowledge!

The SU students also seemed extremely eager to project everything I said about AGI into the world they knew best: Silicon Valley style Internet software. So, most of the questions during and after my talk centered around the theme: "Isn't it unnecessary to work on AGI explicitly ... won't AGI just emerge from the Internet after Silicon Valley startup firms create enough cool narrow-AI online widgets?" When I said I thought this was unlikely, then the questions turned to: "OK, but rather than writing an AGI that actually thinks on its own, shouldn't you just write a narrow-AI that figures out the best way to combine existing online widgets, and achieves general intelligence that way?" And so forth.

But I don't want to make it sound like the SU student body is "all of one mind" -- it's certainly a heterogeneous bunch. At the lunch following my talk at SU, one SU student surprised me with the following statement (paraphrased): "One reason I think AI systems may not achieve the same kind of ethical understandings or states of mind as humans, is that they lack one of the most important human characteristics: our humbleness. We humans have a lot of limitations in our bodies and minds, and these limitations have made us humble, and this humbleness is part of what makes us ethical and part of what makes us profoundly intelligent in a way that a mere calculating machine could ever be."

I laughed out loud and immediately said to the student: "OK, I'm onto you. You're not American." (The student did look Asian ... but I was guessing he was not Asian-American.)

He admitted to being from Korea ... and I noted that few Americans -- and especially no Silicon Valley techno-geek -- would ever identify humbleness as a central characteristic of humans or a key to human intelligence!

Then I couldn't help thinking of the saying "Pride comes before a fall" ... and Vinge's (correct) characterization of the Singularity as a point after which HUMANS WILL HAVE NO IDEA WHAT'S GOING ON ... i.e. no real ability to predict what happens next, as superhuman nonhuman intelligences will be dominating the scene.

Philosopher Walter Kauffmann coined the dorky but evocative term "humbition" to denote the combination of humility and ambition. There's not much humbition in Silicon Valley ... nor for that matter in the public trumpetings of the Chinese government ... but there was a LOT of humbition in the Chinese students at the AGI summer school and the Artificial Brain Lab. Perhaps this quality will serve them well as the world advances, and our knowledge and intuitions prove decreasingly adequate to comprehend our situation...

If you believe that AGI will be created from piecing together narrow-AI internet widgets, then yeah, mostly likely AGI will be created by the Silicon Valley techno-elite. But if (as I suspect) it requires fundamentally different ideas from the ones now underlying the world's technological infrastructure ... maybe it will be created by people who are more open to fundamentally new and different ideas.

But this leads into the next blog post I'm going to write, exploring the question of whether Hugo de Garis is right that AGI is going to get created in China rather than the West!

Musings on the Concept of a Systematic AGI Curriculum, and Lessons for Future AGI Summer Schools

Next, what did I learn this summer about the notion of an AGI summer school, and about teaching AGI altogether?

One big lesson that got reinforced in my mind is: Teaching AGI is very different than teaching Narrow AI!

There is basically no systematic AGI education in universities anywhere on the planet, and this fact certainly helps to perpetuate the current AGI research situation (in which there is very little AGI research going on). By and large, everywhere in the world, students graduate with PhD degrees in AI, without really knowing what "AGI research" means.

Another conclusion I came to is that a carefully crafted "AGI Summer School" curriculum could play a major role -- not only in providing AGI education, but in demonstrating how AGI material should be structured and taught.

However, creating a thorough, systematic AGI curriculum would be a lot of work ... and we didn't really attempt it for the First AGI Summer School. I think the lectures mostly went very well this time (well, you can judge when the videos come online!!), and the sequencing of the lectures made good didactic sense -- but, for the next AGI summer school, we'll put a little more thought into framing the curriculum in a systematic way. Now, having done the summer school once, it's more clear to me (and probably the other participants as well) what an AGI curriculum should be like.

First of all, it's obvious that to make a systematic AGI curriculum, one would need some systematic background curriculum in areas like

Neuroscience
Linguistics
Philosophy of Mind
Psychology (of Cognition, Perception, Emotion, etc.)

We didn't do enough of that this time. Allan Combs' lectures at the Xiamen summer school formed a nice start toward a Neuroscience background curriculum for AGI, but due to lack of time he couldn't do everything needed.

In this vein, one thing that became clear to me at the Xiamen summer school is: The standard "cognitive science" curriculum would certainly fill this need for background, but it's not exactly right, because it's not specifically focused on AGI ... AGI students really only need to digest a certain subset of the cognitive science curriculum, selected specifically with AGI-relevance in mind. But judiciously making this selection would be a nontrivial task in itself.

Next, as part of a thorough AGI curriculum, one would need a systematic review of different conceptions of what "general intelligence" is -- we did such a review at the Xiamen summer school, but not all that systematically. Pei Wang gave a nice talk on this theme, and then Joscha Bach and I presented our own conceptions of GI, and I also briefly reviewed the Hutter/Schmidhuber "universal intelligence" perspective.

Then there's the matter of reviewing the various AGI architectures out there. I think the Xiamen summer school did a fairly good job of that, with in-depth treatments of OpenCog, Pei Wang's NARS architecture, and Joscha Bach's MicroPsi ... and a briefer discussion of Hugo de Garis's neural net based Artificial Brain approach ... and then very quick reviews of other AGI architectures like SOAR and LIDA. Of course there are many, many architectures one could discuss, but in a limited time-frame one has to pick just a few and focus on them. (It would be nice if there were some more systematic way to review the various AGI architectures out there than taking a "laundry list" approach, but this isn't an education problem, it's a fundamental theory problem -- no such systematization exists, even in the research literature.)

There were a lot of OpenCog-related lectures at the Xiamen summer school, and one thing I felt was that it was both too much and too little! Too much OpenCog for a generic AGI Summer School, but too little for a real in-depth OpenCog education. At future summer schools we may split OpenCog stuff off to a greater extent: give a briefer OpenCog treatment in the main summer school lectures, and then do a separate one-week OpenCog lecture series after that, for students who want to dig deep into OpenCog.

Another educational issue is that each AGI architecture involves different narrow-AI algorithms, so that to really follow the architecture lectures fully, students needed to know all about forward and backward inference, attractor, feedforward and recurrent neural nets, genetic algorithms and genetic programming, and so forth. (Most of them did have this knowledge, so it wasn't a problem; actually this might be more of a problem in the US than in China, as China's education system is very strong on comprehensively teaching factual knowledge.) That is: even though AGI is quite distinct from narrow AI, existing AGI architectures make ample use of narrow-AI tools, so students need a good grounding in narrow AI to grok current AGI systems. It would be good to make a systematic list of tutorials on the most AGI-relevant areas of narrow AI, for students whose narrow-AI background is spotty. Again, we did some of this for the Xiamen summer school, but probably not enough.

Finally there's the terminology issue. There is no good "AGI glossary", and every researcher uses terms in slightly different ways. Updating and enlarging an online AGI glossary would be a great project for students at an AGI summer school to participate in!

Undramatic Non-Conclusion

So, the first AGI Summer School went pretty interestingly, and I'm really glad it happened. It was interesting to get to know China a little bit, and to get some experience teaching AGI in an intensive-course context. I learned a lot, and I guess the other faculty and the students did too.... I also made a number of excellent new friends, both among the Chinese and the foreign students. As with many complex real-world experiences, I don't really have any single dramatic summary or conclusion to draw ... but I'm looking forward both to future AGI summer schools, and to future experiences with "AGI in China"....

Quantum Logic Models of Classical Systems Give a New Twist on Quantum Consciousness

2009-06-22T08:53:00.004-04:00

A while ago I wrote a blog post suggesting that quantum logic should be applied more generally than to quantum physical systems ... that it should be applied to complex classical systems in some cases as well, if they are so complex that their states are unobservable to a certain observer.

This, I suggested, would require making the choice of logic observer-dependent: i.e., the system T might best be modeled by system S using quantum logic, but by system R using classical logic.

I didn't at the time see how to make this speculation rigorous but I've now found a related literature that helps a lot.

And by refining my previous idea, I've come up with an argument that possibly human consciousness may be effectively modeled using quantum logic, whether or not the human brain is a quantum system.

I may write a paper on this stuff at some point (in which process I'll probably figure out nicer ways to express the ideas), but wanted to write it down now while it's fresh in my mind.

Atmanspacher's Idea

Diederik Aerts and Liane Gabora have written some very nice papers related to this topic ... and I read their stuff years ago but didn't quite see how to connect it to my relevant intuitions.

What I discovered just recently was the related work of Harald Atmanspacher, which ties in more directly with the way I was thinking about these issues. (Some relevant papers by both of these guys are linked to at the end of this post.)

Put simply, Atmanspacher's view is that: In any case where two properties of a system cannot be simultaneously measured with high accuracy, you have a situation that should be modeled using quantum logic.

I.e., quantum logic should be applied to any case where there are incompatible observables ... whether or not this is due to quantum microphysics.

Making the Choice of Logic Observer-Dependent

My twist on Atmaspacher's idea is to suggest that quantum logic should be applied, by a cognitive system, to any situation that has two aspects which (perhaps by quantum microphysics, or perhaps simply due to its limitations as a cognitive system) the cognitive system cannot model simultaneously.

That is: If T has two aspects, and S cannot model these two aspects of T simultaneously without becoming non-S, then from the perspective of S, these aspects of T should be modeled using quantum logic rather than classical logic.

Note that S in this argument is not a specific physical system at a particular point in time, but rather a category of instantaneous physical systems, which are being considered as instantiations of a single abstract "system" (for example, "Ben Goertzel" is a category of instantaneous physical systems).

So, my suggestion is that whether T should be reasoned about by quantum or classical logic, must be determined by relativizing the reasoning to some category of instantaneous physical systems.

Possible Implications for Quantum Consciousness

What spurred me to start digging into these issues just now was a conversation with Stuart Hameroff, who believes consciousness to be a quantum phenomenon.

My suggestion is: It could be that a quantum model of human consciousness is the right one, even if the underlying physics of the brain is basically "classical" (and I don't claim to know for sure whether it is or not).

(Note that I referred above to a classical model of "human consciousness", not of consciousness in general -- I tend toward panpsychism, meaning I think everything is conscious and different systems just manifest universal consciousness in different ways.)

As a natural consequence of the above argument, I would suggest that each of us individually, due to our own processing limitations, cannot view ourselves in all aspects simultaneously.

If this is true, then perhaps we should model ourselves using quantum logic.

Being panpsychist I would not identify this with consciousness, but I would say that systems which are sufficiently complex that they implicitly model themselves using quantum logic, in predicting and analyzing their own dynamics, presumably have a distinctive character to the way they manifest universal consciousness.

Can We Tell the Cause of the Incompatibility?

An interesting question is: if I, as a cognitive system, am confronted with incompatible observables ... in what sense can I tell what the cause of this incompatibility is?

Can I tell a case where the incompatibility is caused by my own cognitive limitations, from a case where it is caused by fundamental indeterminacy such as is sometimes hypothesized to occur in quantum microphysics?

It would seem there is no direct way to make this determination, but we can induce general theories from observations of other system aspects, which lead us to hypotheses regarding the causes of an incompatibility.

Some Nice Quotes on Quantum Modeling of Classical Systems

These quotes come from

http://www.quantum-cognition.de/

Andrei Khrennikov:

“I propose to consider any system which produces quantum statistics as quantum (”quantum-like”). A possible test is based on the interference of probabilities. I was mainly interested in using such an approach to ”quantumness” to extend the domain of applications of quantum mathematical formalism and especially to apply it to cognitive sciences. There were done experiments on interference of probabilities for ensembles of students and a nontrivial interference was really found. … Yes, we might expect nonclassical statistics, but there was no reason to get the quantum one, i.e., cos-interference. But we got it!”

Diederik Aeerts & Liane Gabora:

"While some of the properties of quantum mechanics are essentially linked to the nature of the microworld, others are connected to fundamental structures of the world at large and could therefore in principle also appear in other domains than the micro-world."

Diederik Aeerts:

"The emergence of quantal macrostates does not necessarily require the reference to corresponding quantal microstates"

Harald Atmanspacher, Hans Primas & Peter beim Graben:

"A generalized version of the formal scheme of ordinary quantum theory, in which particular features of ordinary quantum theory are not contained, should be used in some non-physical contexts."

"Complementary observables can arise in classical dynamic systems with incompatible partitions of the phase space."

A Few Relevant References

Atmanspacher:

"Weak quantum theory"

http://www.igpp.de/english/tda/pdf/wqt.pdf

"Complementarity in Bistable Perception"

http://unjobs.org/authors/harald-atmanspacher

Aerts:

http://www.vub.ac.be/CLEA/aerts/publications/

see

Aerts, D. (1982). Example of a macroscopical situation that violates Bell inequalities. Lettere al Nuovo Cimento, 34, pp. 107-111.
Aerts, D. (1991). A mechanistic classical laboratory situation violating the Bell inequalities with 2sqrt(2), exactly 'in the same way' as its violations by the EPR experiments. Helvetica Physica Acta, 64, pp. 1-23.
Aerts, D. and Gabora, L. (2005). A theory of concepts and their combinations I: The structure of the sets of contexts and properties. Kybernetes, 34, pp. 167-191.
Aerts, D. and Gabora, L. (2005). A theory of concepts and their combinations II: A Hilbert space representation. Kybernetes, 34, pp. 192-221

How many transhumanists does it take to change a light bulb?

2009-05-25T09:16:00.003-04:00

Infinity.... None of them will touch the light bulb at all; they'll all just sit around talking amongst themselves and waiting for someone else to invent a self-changing cyber light bulb.

Reinforcement Learning: Some Limitations of the Paradigm

2009-05-20T23:33:00.005-04:00

(This email summarizes some points I made in conversation recently with an expert in reinforcement learning and AGI. These aren't necessarily original points -- I've heard similar things said before -- but I felt like writing them down somewhere in my own vernacular, and this seemed like the right place....)

Reinforcement learning, a popular paradigm for AI, economics and psychology, models intelligent agents as systems that choose their actions in such a way as to maximize their future reward. There are various ways of averaging future reward over various future time-points, but all of these implement the same basic concept.

I think this is a reasonable model of human behavior in some circumstances, but horrible in others.

And, in an AI context, it seems to combine particularly poorly with the capability for radical self-modification.

Reinforcement Learning and the Ultimate Orgasm

Consider for instance the case of a person who is faced with two alternatives

A: continue their human life as would normally be expected
B: push a button that will immediately kill everyone on Earth except them, but give them an eternity of ultimate trans-orgasmic bliss

Obviously, the reward will be larger for option B, according to any sensible scheme for weighting various future rewards.

For most people, there will likely be some negative reward in option B ... namely, the guilt that will be felt during the period between the decision to push the button and the pushing of the button. But, this guilt surely will not be SO negative as to outweigh the amazing positive reward of the eternal ultimate trans-orgasmic bliss to come after the button is pushed!

But the thing is, not all humans would push the button. Many would, but not all. For various reasons, such as love of their family, attachment to their own pain, whatever....

The moral of this story is: humans are not fully reward-driven. Nor are they "reward-driven plus random noise".... They have some other method of determining their behaviors, in addition to reinforcement-learning-style reward-seeking.

Reward-Seeking and Self-Modification: A Scary Combination

Now let's think about the case of a reward-driven AI system that also has the capability to modify its source code unrestrictedly -- for instance, to modify what will cause it to get the internal sensation of being rewarded.

For instance, if the system has a "reward button", we may assume that it has the capability to stimulate the internal circuitry corresponding to the pushing of the reward button.

Obviously, if this AI system has the goal of maximizing its future reward, it's likely to be driven to spend its life stimulating itself rather than bothering with anything else. Even if it started out with some other goal, it will quickly figure out to get rid of this goal, which does not lead to as much reward as direct self-stimulation.

All this doesn't imply that such an AI would necessarily be dangerous to us. However, it seems pretty likely that it would be. It would want to ensure itself a reliable power supply and defensibility against attacks. Toward that end, it might well decide its best course is to get rid of anyone who could possibly get in the way of its highly rewarding process of self-stimulation.

Not only would such an AI likely be dangerous to us, it would also lead to a pretty boring universe (via my current aesthetic standards, at any rate). Perhaps it would extinguish all other life in its solar system, surround itself with a really nice shield, and then proceed to self-stimulate ongoingly, figuring that exploring the rest of the universe would be expected to bring more risk than reward.

The moral of the above, to me, is that reward-seeking is an incomplete model of human motivation, and a bad principle for control self-modifying AI systems.

Goal-Seeking versus Reward-Seeking

Fortunately, goal-seeking is more general than reward-seeking.

Reward-seeking, of the sort that typical reinforcement-learning systems carry out, is about: Planning a course of action that is expected to lead to a future that, in the future, you will consider to be good.

Goal-seeking doesn't have to be about that. It can be about that ... but it can also be about other things, such as: Planning a course of action that is expected to lead to a future that is good according to your present standards.

Goal-seeking is different from reward-seeking because it will potentially (depending on the goal) cause a system to sometimes choose A over B even if it knows A will bring less reward than B ... because in foresight, A matches the system's current values.

Non-Reward-Based Goals for Self-Modifying AI Systems

As a rough indication of what kinds of goals one could give a self-modifying AI, that differ radically from reward-seeking, consider the case of an AI system with a goal G that is the conjunction of two factors:

Try to maximize the function F
If at any point T, you assess that your interpretation of the goal G at time T would be interpreted by your self-from-time-(T-S) as a terrible thing, then roll back to your state at time S

I'm not advocating this as a perfect goal for a self-modifying AI. But the point I want to make is this kind of goal is something quite different from the seeking of reward. There seems no way to formulate this goal as one of reward maximization. This is a goal that involves choosing a near-future course of action to maximize a certain function over future history -- but this function is not any kind of summation or combination of future rewards.

Limitations of the Goal-Seeking Paradigm

Coming at the issue from certain theoretical perspectives, it is easy to overestimate the degree to which human beings are goal-directed. It's not only AI theorists and engineers who have made this mistake; many psychologists have made it as well, rooting all human activity in goals like sexuality, survival, and so forth. To my mind, there is no doubt that goal-directed behavior plays a large role in human activity -- yet it also seems clear that a lot of human activity is better conceived as "self-organization based on environmental coupling" rather than as explicitly goal-directed.

It is certainly possible to engineer AI systems that are more strictly goal-driven than humans, though it's not obvious how far one can go in this direction without sacrificing a lot of intelligence -- it may be that a certain amount of non-explicitly-goal-directed self-organization is actually useful for intelligence, even if intelligence itself is conceived in terms of "the ability to achieve complex goals in complex environments" as I've advocated.

I've argued before for a distinction between the "explicit goals" and "implicit goals" of intelligent systems -- the explicit goals being what the system models itself as pursuing, and the implicit goals being what an objective, intelligent observer would conclude the system is pursuing. I've defined a "well aligned" mind as one whose explicit and implicit goals are roughly the same.

According to this definition, some humans, clearly, are better aligned than others!

Summary & Conclusion

Reward-seeking is best viewed as a special case of goal-seeking. Maximizing future reward is clearly one goal that intelligent biological systems work toward, and it's also one that has proved useful in AI and engineering so far. Thus, work within the reinforcement learning paradigm may well be relevant to designing the intelligent systems of the future.

But, to the extent that humans are goal-driven, reward-seeking doesn't summarize our goals. And, as we create artificial intelligences, there seems more hope of creating benevolent advanced AGI systems with goals going beyond (though perhaps including) reward-seeking, than with goals restricted to reward-seeking.

Crafting goals with reasonable odds of leading self-modifying AI systems toward lasting benevolence is a very hard problem ... but it's clear that systems with goals restricted to future-reward-maximization are NOT the place to look.

Science-synergetic philosophy: the religion of the future?

2009-05-13T10:03:00.004-04:00

(This may seem a hackneyed topic, but there are some moderately original points near the end here, if you bear with me ...)

As a card-carrying, future-thinking transhumanist, I take it as obvious that most of the particulars of current religions are relics of earlier eras in human cultural development, which currently do a lot of harm along with doing some good.

But I still find it interesting to ask what aspects of religion reflect underlying phenomena that are essential, meaningful and necessary -- and are likely to continue as humanity transcends the traditional "human condition" and enters its next phase of development....

Fish and Eagleton on the Wonders of Theology

What spurred this blog post was: My dad pointed out to me this New York Times blog post by Stanley Fish reviewing a book that extols the merits of religion (Reason, Faith and Revolution by Terry Eagleton).

The basic point Fish makes is that religion offers something science by its very nature cannot.

Eagleton acknowledges ... many terrible things have been done in religion’s name — but at least religion is trying for something more than local satisfactions, for its “subject is nothing less than the nature and destiny of humanity itself, in relation to what it takes to be its transcendent source of life.”

He notes that science cannot address what he calls "theological questions", where

By theological questions, Eagleton means questions like, “Why is there anything in the first place?”, “Why what we do have is actually intelligible to us?” and “Where do our notions of explanation, regularity and intelligibility come from?”

He also notes that the author is

... angry, I think, at having to expend so much mental and emotional energy refuting the shallow arguments of school-yard atheists like Hitchens and Dawkins.

I haven't read Eagleton's book and I'm unlikely to do so -- I have a long list of more interesting-looking reading material -- but Fish's summary did resonate with a paper I'm in the middle of writing (it's paused while I work on more urgent stuff) on the limits of science.

My basic point in that paper will be a simple one: science is based on finite sets of finite-precision observations. That is, all of scientific knowledge is based on some finite set of bits, comprising the empirical observations accepted by the scientific community.

To extrapolate beyond this bit-set, some kind of assumption is needed. To put it another way, some kind of "faith" is needed. Hume was the first one to make this point really clearly ... and we now understand the "Humean problem of induction" well enough to know it's not the kind of thing that can be "solved."

The Occam's Razor principle tries to solve it -- it says that you extrapolate from the bit-set of known data by making the simplest possible hypothesis. This leads to some nice mathematics involving algorithmic information theory and so forth. But of course, one still has to have "faith" in some measure of simplicity!

So: doing or using science requires, in essence, continual acts of faith (though these may be unconscious and routinized rather than conscious and explicit). To the extent that Dawkins, Hitchens or other anti-religion commentators de-emphasize this point, they're engaging in judicious marketing. (It's hard for me to feel too negative toward them about this, however, given the far more explicitly and dramatically dishonest marketing that religion has carried out over the last millennia.)

My paper will focus on what the limits of science tell you about AI, machine consciousness and so forth -- and I'll save that for another blog post, or the paper itself. (Don't worry though, my conclusion is not that scientifically enginering AGI is impossible ... I haven't lost the faith!)

Anyway, I certainly agree with Fish and Eagleton that religion addresses very important questions that science cannot, by its nature, answer.

But I find it rather screwy that Eagleton refers to

“Why is there anything in the first place?”, “Why what we do have is actually intelligible to us?” and “Where do our notions of explanation, regularity and intelligibility come from?”

and so forth as theological questions.

Surely, these are philosophical questions.

One can answer them in various ways without invoking any deities or demons!

"Why does God exist?" is a theological question ...

"Why does anything exist?" is philosophical...

(Though, for the record, I don't think "Why does anything exist?" is a very useful philosophical question. I'm more interested in questions like

"Why do separate objects exist, instead of just one big fluid cosmic mass?"
"In what sense could the universe be considered compassionate?"
"How much ethical responsibility should I feel toward (which) other minds?"
"Why does my mind perceive such a small subset of the space of all possible patterns?"
"How much can a mind grow and expand without losing its sense of self and becoming, experientially, a 'fundamentally different being'?"
"What is it like to be a rock?"
etc.

)

Theology is one way of providing answers to philosophical questions ... but by no means the only way.

I think that religion addresses some very important questions, that are beyond the scope of science -- and by and large provides these questions with extremely bad answers.

One of the many limitations of religion as conventionally conceived is indicated by the quote, given above, that religion's

“subject is nothing less than the nature and destiny of humanity itself....”

From a transhumanist perspective, the qualifier "nothing less than" is misplaced, as this is actually a very limiting subject. The nature and destiny of humanity are important; but one of the things that science has opened our minds to is the relative insignificance of humanity in the space of possible minds. I'm more interested in philosophies that address the nature and destiny of mind itself, rather than just the nature and destiny of one species on one planet.

It is of course a subtle matter to compare and judge different explanations to philosophical questions. You can't compare them using scientific or mathematical methods ... and of course the question of how to evaluate philosophical views becomes "yet another tough philosophical question", tied in with all the other ones.

A crude way to say it, is that it comes down to an intuitive judgment ... which leads into questions of how one can refine and improve one's intuition ... and these questions, of course, possess numerous answers that are philosophical- or religious- tradition -dependent...

Science-synergetic philosophy

It does seem to me, though, that there is an interesting notion of science-synergetic philosophy lurking somewhere in all this.

Suppose we take for granted that doing science -- just like other aspects of living life -- relies on a constant stream of acts of faith, which can't be justified according to science....

One may then note that there are various systems for mentally organizing these acts of faith.

Religions are among them. But religions are quite detached from the process of doing science.

It seems sensible to think about philosophical systems -- i.e. systems for organizing inner acts of faith -- that are intrinsically synergetic with the scientific process. That is, systems for organizing acts of faith, that

when you follow them, help you to do science better
are made richer and deeper by the practice of science

One can broaden this a little and think about philosophical systems that are intrinsically synergetic with engineering and mathematics as well as science.

Now, one cannot prove scientifically that a "scientifically synergetic philosophy" is better than any other philosophy. Philosophies can't be validated or refuted scientifically.

So, the reason to choose a scientifically synergetic philosophy has to be some kind of inner intuition; some kind of taste for elegance, harmony and simplicity; or whatever.

One prediction I have for the next century is that scientifically synergetic philosophies will emerge into the popular consciousness and become richer and deeper and better articulated than they are now.

Because Fish and Eagleton are right about some things: people do need more than science ... they do need collective processes focused on the important philosophical questions that go beyond the scope of science.

But my prediction is that we are going to trend more toward philosophical systems that are synergetic with science, rather than ones that co-exist awkwardly with science.

What will these future philosophical systems be like?

There's nothing extremely new about the concept of science-synergetic philosophy, of course.

Plenty of non-religious scientists and science-friendly non-scientists have created personal philosophies that don't involve deities or other theological notions, yet do involve meaningful approaches to personally exploring the "big questions" that religions address.

Among the many philosophers to take on the task of creating comprehensive science-synergetic philosophical systems, perhaps my favorite is Charles Peirce (who also developed a nice philosophy of science, though one that IMO is significantly incomplete ... but I've discussed that elsewhere.)

Building on work by Peirce and loads of others, I tried to lay out a science-synergetic philosophical system in my book The Hidden Pattern -- but like Peirce's writings, that is a fairly academic work, not an informal tract designed to inspire the common human in their everyday life.

My friend Philippe van Nedervelde likes to talk about this sort of thing as a "TransReligion/ UNReligion", but I confess to not finding that terminology very compelling.

Philippe is interested in (among many other things!) developing vaguely religion-like rituals that coincide with some sort of science-synergetic philosophy. There has been talk about formulating a "TransReligion/ UNReligion" as an outgrowth of the futurist group now called "The Order of Cosmic Engineers." Which I think is an interesting idea ... yet I'm not really sure it's the direction things will (or should) go.

I'm not sure there will emerge any one "Bible of science-synergetic transhumanist philosophy" ... nor any science-synergetic-philosophy analogues of speaking in tongues, kneeling at the altar, or consuming the simulated blood and flesh of the Savior the Son of God who gave his life for our sins. Perhaps, science-synergetic philosophy may wind up being something that pervades human culture in more of a broad-based, implicit way.

Time will tell!

Does Long-Lived Quantum Coherence Underlie Biological Processes?

2009-05-08T09:16:00.010-04:00

Is the human brain, at the levels directly relevant for analysis of cognition, best modeled as a classical or quantum system?

(For instance, a baseball in some sense needs to be modeled as a quantum system -- in the sense that the way its molecules hold together can be described only using quantum not classical physics; but classical physics can be used to explain the normally relevant aspects of its macroscopic behavior. So at the levels directly relevant for analysis of a baseball game, a baseball is best modeled as a classical system. OTOH, at the levels directly relevant for analysis of the electromagnetic behavior of a Superconducting Quantum Interference Device (SQUID) -- a small but macroscopic device, used in magnetoencephalography machines, and demonstrating macroscopic quantum coherence in its magnetic field -- the SQUID is best modeled as a quantum system. Classical physics models just won't explain why the SQUID, a device you can hold and pinch between your fingers (though it only works when supercooled, which would freeze your fingers!), makes MEG machines work.)

Current brain theory indicates that for understanding its role in giving rise to the mind, the brain is most effectively modeled as a classical system (i.e. the brain is more like a baseball than a SQUID) ... but of course current brain theory could be incomplete.

(Even if the brain is a macroscopic quantum system, this of course doesn't prove that quantum dynamics are necessary for intelligence or consciousness or anything like that. Those are bigger and deeper questions, and I've argued in the past that sufficiently complex "classical" systems might need to be treated using quantum logic ... but this gets into a lot of deep issues that I don't want to digress onto here.)

Stuart Hameroff is one of the more vocal proponents of the "quantum brain" idea, and he has a new paper reporting a new theory in this direction, arguing that dendro-dendritic synapses are mediated via macroscopic quantum dynamics, thus posing a quantum neural net that operates in complex coordination with the classical neural net formed by axonal-dendritic synapses.

I don't have a strong opinion on that particular theory of Hameroff's. I look forward to discussing it with him at the Toward a Science of Consciousness conference in Hong Kong next month.

But I was struck by one of the references at the end of his paper, a Nature paper entitled

Evidence for wavelike energy transfer through quantum coherence in photosynthetic systems

This is a 2007 paper that I had not noticed before, and it's interesting because it gives solid evidence of macroscopic quantum coherence in a biological process.

To quote part of the abstract:

Here we extend previous two-dimensional electronic spectroscopy investigations of the FMO bacteriochlorophyll complex, and obtain direct evidence for remarkably long-lived electronic quantum coherence playing an important part in energy transfer processes within this system. The quantum coherence manifests itself in characteristic, directly observable quantum beating signals among the excitons within the Chlorobium tepidum FMO complex at 77 K. This wavelike characteristic of the energy transfer within the photosynthetic complex can explain its extreme efficiency, in that it allows the complexes to sample vast areas of phase space to find the most efficient path.

In the comments to an earlier edit of this blog post, someone pointed out this more recent paper

Quantum Zeno Effect Underpinning the Radical-Ion-Pair Mechanism of Avian Magnetoreception

whose abstract says

The intricate biochemical processes underlying avian magnetoreception, the sensory ability of migratory birds to navigate using earths magnetic field, have been narrowed down to spin-dependent recombination of radical-ion pairs to be found in avian species retinal proteins. The avian magnetic field detection is governed by the interplay between magnetic interactions of the radicals unpaired electrons and the radicals recombination dynamics. Critical to this mechanism is the long lifetime of the radical-pair spin coherence, so that the weak geomagnetic field will have a chance to signal its presence. It is here shown that a fundamental quantum phenomenon, the quantum Zeno effect, is at the basis of the radical-ion-pair magnetoreception mechanism. The quantum Zeno effect naturally leads to long spin coherence lifetimes, without any constraints on the systems physical parameters, ensuring the robustness of this sensory mechanism. Basic experimental observations regarding avian magnetic sensitivity are seamlessly derived. These include the magnetic sensitivity functional window and the heading error of oriented bird ensembles, which so far evaded theoretical justification. The findings presented here could be highly relevant to similar mechanisms at work in photosynthetic reactions. They also trigger fundamental questions about the evolutionary mechanisms that enabled avian species to make optimal use of quantum measurement laws.

This of course is even more intriguing than the green sulphur bacteria stuff, because it has to do with perception in an intelligent macroscopic animal.

Hameroff's point in citing the paper on green sulphur bacteria (and it's a good one) seems to be: if long-lived quantum coherence can play an important role in photosynthesis, couldn't it also play a role in the brain somehow ... e.g. maybe via dendro-dendritic synaptic gap junctions?

The extrapolation from these other results to neuroscience is speculative, sure.... But this kind of result does make the possibility of quantum coherence impacting human cognition seem a bit less fanciful.

After all, I often recall that in the late 90's all the neuroscientists I talked to told me there was no neurogenesis nor synaptogenesis in adult mammals. Oops. Now they've got new data and changed their mind. My point isn't that quantum coherence is related to neuro or synapto genesis (though, who knows...), but rather that neuroscientists -- simultaneously with displaying the usual humility of biologists regarding the complexity of the systems they're studying -- have a long-standing habit of assuming the concept-set underlying their current understanding is much more adequate than it really is.

Our ignorance of the brain is why my own AI work is not based on trying to closely model the brain. Of course, it's possible that intelligence is fundamentally based on some freaky neuroquantum phenomenon, so that all digital-computer AI work is doomed by some intrinsic limitations ... but I doubt it. My own guess is that, even if the brain does involve macroscopic quantum coherence in some interesting sense, one can still make transhumanly intelligent systems using digital computers. And of course, if this doesn't work -- or if these transhumanly intelligent systems turn out to lack some crucial aspect of self-awareness as the quantum-consciousness advocates argue -- then we can always add some funky quantum computing chips into our AGI server farm!

"Transcendent Man" ... my brush with movie-stardom ... and my answer to "What is Your Goal?"

2009-05-05T21:32:00.005-04:00

My review of the Kurzweil-biopic/futurist-think-piece documentary Transcendent Man -- which features me mouthing off for 4-5 minutes in a zebra-striped cowboy hat -- in HPlus Magazine is here:

http://www.hplusmagazine.com/articles/ai/transcendent-man-film-about-kurzweil

In that article, as well as reviewing the film, I also recount some moderately interesting dialogue btw me and Ray Kurzweil that occurred in the moderated discussion at the end of the film's premiere at the Tribeca Film Festival...

After that conversation with Ray I discuss at the end of the article, the discussion-moderator asked me another question (which I didn't put in the review article): he asked me what my goal was. What was I trying to achieve?

What I said was something like this: "I would like me, and any other human or nonhuman animal who wants to, to be able to increase our intelligence and wisdom gradually ... maybe, say, 37.2% per year ... so that we can transcend to higher planes of being gradually and continuously, and feel ourselves becoming gods ... appreciate the process as it unfolds."

That's what I'm after, folks. Hope you'll come along for the ride!

Teaching Dolphins Lojban ... Giving Dolphins Prosthetic Hands

2009-04-26T11:51:00.004-04:00

A follow-up to my prior posts on cetacean intelligence...

I thought a bit about innovative ways we might be able to communicate better with our cetacean planet-mates...

1. Teach Dolphins Lojban

A couple decades ago, efforts were made to teach dolphins simple English, without dramatic success. Discussions were also had regarding creation of some sort of species-independent interlingua, which humans and dolphins could use to communicate with each other.

It occurred to me that using Lojban for that interlingua could make sense. Potentially, one could create special Lojbanic vocabulary for the shared human/dolphin environment. Lojban grammar is simple and unambiguous, and certainly has less species-specificity than any human natural language.

Also, one could create a form of Lojban "phonology" that generally follows the sound-production patterns habitually by dolphins, and speak to dolphins in this "Delphic Lojban" alongside the usual "human Lojban."

The biggest disadvantage of this approach is that it requires some human cetaceologists to learn Lojban.... But this cost seems worth paying, as the odds of success seem much higher than with human natural languages.

Note that there is no straightforward way to make a "phonologically Delphic" version of English. But because Lojban syntax is just a linearization of logical relationships, one could make a Delphic version of Lojban by translating those same logical relationships into sound in a wholly different way than is done in the human version of Lojban.

2. Give Dolphins Prosthetic Hands

Inside a dolphin's flippers, are bones that look like they should correspond to claws or fingers.

What if we created prosthetic fingers and thumbs for dolphins, and connected them to these bones ... and also connected them to the dolphin nervous system?

Admittedly, these modified dolphins would suffer impaired swimming ability, though one would hope the degree of this phenomenon could be palliated via appropriate design. (For instance, perhaps the fingers could be made retractable, so the dolphin could retract them when it wanted to swim, and extend them when it wanted to manipulate objects.)

This would be a highly experimental adventure in Brain-Computer Interfacing. But, as BCI research advances in the context of human-enhancement applications, I see no reason why it shouldn't advance in the context of dolphin-enhancement applications in parallel.

My thinking is that much of which distinguishes human intelligence from cetacean intelligence is our focus on complex manipulation of tools, and building things (including advanced phenomena like tools that make tools, etc.). If a dolphin brain self-reorganized to adapt to its prosthetic fingers, then the dolphin would have the capability to use tools in a more humanlike way.

Since the cetaceans' evolutionary progenitors had claws of some sort, there may be some vestigial neural wiring in the dolphin brain that will ease the self-reorganization that the dolphin brain needs to go through to make use of the prosthetic fingers.

Another possibility would be to build in the capability for human operators to periodically "take over" the dolphin fingers using remote control. This would serve to show the dolphin what to do with the fingers, both on the conscious reflective level, and on the level of unconscious habituation.

Of course discussions of what to build with the fingers, and how to use tools, could be carried out using Lojban (human or Delphic) ;-D

...

Ahhh ... all the really fascinating research that would get funded if I happened to receive a billion-dollar inheritance from some long-lost uncle ;-p

Why AGI Researchers Should Care about Cetaceans

2009-04-26T09:08:00.003-04:00

This is just a brief follow-up to my last post, and a prelude to the one that will follow, which is already brewing in my brain....

In case some of y'all are wondering why I ... whose main intellectual obsession is the creation of AGI systems with general intelligence at the human level and beyond ... have suddenly started ranting about cetacean intelligence, I suppose I should be more explicit about my research-related motivation for digging into the topic

Of course there's a personal motivation -- I love nonhuman animals ... at the moment as well as some humans I share my house with a parrot, 2 dogs and 5 bunnies; and there have been friends of a lot of other species in my life at various times.... In fact the parrot named Abaca might be classified as my best friend over the last few months ;-) ....

I've encountered wild dolphins up close in the shallows of the Indian Ocean when I lived in Western Australia 13-15 years ago, and was certainly struck by the experience, as brief and superficial as it was. I definitely wanted more (and would have sought out more if I hadn't left Western Oz to move to New York and start an AI company)

But what I want to focus on here is my intellectual motivation for, as an AGI researcher, finding cetacean intelligence important.

How often do I hear, among AGI researchers, words to the effect of "Of course we need to model our AGI systems on the human brain and mind, since after all it's the only example we have of a highly generally intelligent system."

I tend to resist this line of thinking ... I think we understand enough about the general mathematics and computer science of cognition that we can understand general intelligence in a manner going beyond the human-specific. My own AI work is an amalgam of aspects directly inspired by human intelligence, and aspects inspired by a broader understanding of intelligence.

But still, there is some point to the common observation that we only know one example of a highly generally intelligent system: the human brain.

But is it actually true?

What if the subset of cetacean intelligence researchers who believe cetaceans have general intelligence comparable to, or greater than, human intelligence ... are actually correct? (Which is my suspicion.)

Then in fact there's another example available -- and we're just not taking the trouble to study it as carefully and thoroughly as we should.

In my immediately previous blog post I gave some links into the cetacean intelligence literature, and some speculations as to what I think the broad nature of cetacean intelligence might possibly be.

In my next blog post I'll discuss some cutting-edge approaches that we might take over the next couple decades to more thoroughly understand cetacean intelligence.

I'm not suggesting that resources be taken from AGI and redirected to cetacean cognitive science: I think that both areas are distressingly underfunded.

In the case that we create AGI programs with superhuman general intelligence before we understand cetacean minds, I think we might still have something to learn from the minds of dolphins. Because cetacean minds may possess a quite different form of intelligence than either us or our AGI creations. And it's hard to tell what may be learned by studying some advanced, fundamentally different incarnation of intelligence.

There could possibly be interesting implications for the study of AGI ethics here, for instance. Cetacea are certainly not optimally ethical creatures ... they're capable of violence just like most other mammals ... but based on what we can understand today, it seems their social organization may have fewer egregious ethical issues than ours. As one example, they seem to have achieved a large-scale, global social organization without warfare. (Evidence of the global nature of their social organization is tentative, but provided by observations such as the way repeated phrases in whale "song" tend to arise in one part of the globe, then spread through whales in all the world's oceans, then die out after a time, replaced by others.)

I'm certainly not suggesting that study of cetacean society will magically provide the answer to the AGI ethics problem, or the problem of generally understanding general intelligence, etc. However, I think it would be very interesting to understand how a fundamentally different sort of general intelligence works, and how it has approached the society/ethics problem, as an additional body of evidence to utilize as we shape the minds of the future.

Cetacean consciousness...

2009-04-24T11:38:00.003-04:00

I've been reading many of the writings of John Lilly lately, and also poring through the literature on cetacean intelligence ... and I have to say it's fascinating stuff ....

I'm fascinated by Lilly's cetacean intelligence/communication work, his isolation tank work, even his obsessive (and, apparently, excessive) experiments with ketamine injection leading to long conversations with various hallucinated (?) extraterrestrials ;-)

(I read his stuff a couple decades ago but I've been through a lot of experiences since, and I can read it with different eyes now. I remember how inspirational his book "Programming and Metaprogramming the Human Biocomputer" was for me, when I read it at age 13 or 15 or whatever.)

Anyway ... plenty of scientists by now have followed up Lilly's intuitions about the deep intelligence of dolphins and other cetaceans. A bunch of research papers by various scientists (not under the influence of ketamine ;-) are here:

http://www.dolphin-institute.org/our_research/dolphin_research/dolphinresearchpublications.htm

For some relevant books by people less fringe-y than Lilly, but still quite insightful, see e.g.

Rachel Smolker's elegant scientific memoir
This edited volume on Dolphin Societies

but I've found no up-to-date comprehensive review book, so you really gotta read the journal literature and various books to understand what's known so far...

As of now there is no definitive scientific proof that cetaceans are extraordinarily intelligent ... though there's pretty solid proof that they're at least as clever as great apes, I would say (though different in mentality) ...

However, my qualitative impression from reviewing all the evidence is that they are, in some senses, dramatically more intelligent than great apes

I will write something systematic on this topic at some point, when I get more time and have read the literature more thoroughly (obviously this is just a background interest for me, so my reading is going pretty slowly...)

What got me musing about this topic right now was thinking about how the naive physics of our everyday world has impacted human intelligence, and what this might mean for engineering and educating AGI.

Last month Allan Combs and I wrote a paper for the NASA CONTACT workshop, discussing how the radically different environments of extraterrestrials might impact their mind-states and varieties of intelligence:

http://goertzel.org/papers/AlienMinds.pdf

(we'll academic-ize this and publish it somewhere, in time).

And this is also related to a paper I wrote a couple months back, musing about how the lack of fluids, powders, fabrics and other such substances in virtual worlds may impact their utility as homes for humanlike artificial minds:

http://goertzel.org/dynapsyc/2009/BlocksNBeadsWorld.pdf

(In that paper I also explored how it might be possible to enhance virtual worlds to largely remedy this shortcoming, using a special physics-engine technique I called "bead physics".)

In writing that NASA paper, I started wondering how it would impact a mind to evolve in an environment dominated by fluids rather than solids.

My speculation was that, in such a mind, notions of causing and building would be replaced by notions of flowing and shaping .... which would lead to all sorts of other differences.

Gino Yu then pointed out to me these fascinating speculations on the potential subjective experiences of cetaceans:

http://www.cbc.ca/ideas/features/ocean-mind/index.html

All this has spurred me to some of my own entertaining speculations (synthesizing various speculations of Lilly and others) ... to wit:

... what if (as Lilly speculated) the everyday states of mind of cetaceans are more like the states of mind that humans get into while on psychedelic drugs, than they are like our everyday consciousness?

After all, these creatures are breathing deeply and rhythmically ... they're floating in liquid ... generally they're living the sort of physical life that would put humans in a deep semi-meditative state ...

What if their big neocortices are devoted essentially to collaboratively composing and improvising music for each other to listen to?

... but perhaps something more advanced and subtle than human music, reflecting intricate patterns of social interaction, and holistic observations about the state of the underwater ecosystem, and emergences between these social and ecosystem patterns...

This would be a type of intelligence not focused on building tools or solving puzzles in the humanlike sense....

As with human intelligence, the main spur for the evolution of such intelligence would be social. Once the composition/improvisation of this kind of communicative/depictive music became a critical aspect of membership in cetacean society, then there would be evolutionary force to compose/improvise more and more appealing music....

In this hypothesis, the crux of dolphin communication might not be one-to-one conversation, but rather multi-player musical improvisation, with both spatial and temporal aspects. Dyadic conversation with practical import might occur, yet have vastly less complexity and subtlety than other aspects of the musical communication...

One interesting thing about this speculation is that, if it were true, it would mean that probing cetacean intelligence using concepts and methods developed for studying human intelligence, could push the researcher in badly wrong directions.

By analogy, imagine that a species whose main focus of intelligence was collaborative spatiotemporal music improvisation, tried to judge and explore human intelligence. Most humans would be judged as hopelessly moronic ... and then a few gifted musicians might be viewed as moderately intelligent. Due to the other species focusing on collaborative spatiotemporal music improvisation, they would miss what is really the crux of human intelligence: our dyadic linguistic communication, and our tool-building.

John Lilly wanted to probe cetacean communication with computer tech, back in the late 1970s and early 1980s. Computers are a lot better now, so someone could take a much better shot at it. But rather little research seems to be going on at the intersection of advanced AI pattern analysis and cetacean communication, at the moment. Too bad.

More ambitiously, one can envision creating an AI that shared both a humanlike body, and a dolphin-like body, and letting it exist in both worlds.

Lilly did make a good point, that we should probably take some of the $ we are spending on looking for alien lifeforms in space, and devote some of it instead to trying to communicate with these alien intelligences that apparently exist in our oceans. If we can't even communicate with the other intelligences on our own planet, cracking the codes of the minds and languages of beings on alien planets may not be realistic yet (though, of course, there is massive uncertainty in all these domains...).

There is some inordinately silly stuff written about cetacean intelligence -- I read one book on the theme that "Jesus was a dolphin"!! And Lilly certainly complicated his message about cetacean intelligence by mixing it up with some of his other messages, for instance about extraterrestrials whom he felt he contacted while in isolation tanks and on ketamine. But all that is really beside the point. When you look at the scope of existing qualitative evidence about cetacean intelligence, the picture is striking....

Whether the speculations I've made above are on-point or not, I'm convinced there is something very interesting going on in cetacean minds and societies -- which we are not putting nearly enough effort into understanding.

Instead, we are still killing them and making them into steaks.

Female mad scientists and creative nihilism (thoughts on Sofia Kovalevskaya)

2009-03-31T11:37:00.004-04:00

I've been thinking of making the protagonist of my next novel a female "mad scientist", but I wasn't sure how to write the character, so I started searching history for a good model.

I found damn few female mad scientists in recorded history.

The closest I found was the Russian mathematician Sofia Kovaleveskaya, whose name I knew from the Cauchy-Kovalevsky Theorem in partial differential equations. I hadn't known much about her before, so I did some reading and found she fit the bill pretty well:

World-class mathematician
Spent some time inventing weird new electrical machinery
Accomplished novelist
Also wrote plays and poetry
Participated in the Paris Commune, and generally schemed for revolutionary overthrow of governments
Helped her husband lose piles of money during a several year period devoted to "clever" real estate and financial speculation

A woman after my own heart -- wish I'd known her! She was also the first woman ever to get a math PhD (she lived in the mid-1800's).

Her biography is also full of nice tidbits, like

She first got passionate about advanced math when her attic happened to get wallpapered with lecture notes from a calculus class her father had taken years before. So she learned about limits and such from reading unordered pages of mathematical text pasted to a wall!
To study advanced math, she had to leave Russia (due to sexist regulations), and to do that she had to get married ... so she entered into a "fake marriage" with a platonic male friend with the sole purpose of escaping Russia to get to university in Western Europe (although, many years later, the fake marriage turned real...)

Her childhood memoir "A Russian Childhood" is a wonderful book and I'd recommend it to anyone who likes Russian literature.

This biography is also worth reading for the story it tells -- although the biographer's radical-feminist antimasculism is annoying, and appears to radically falsify Kovalevskaya's relationship with her husband, among other things

There's also a historical novel about her life, which I haven't read

Her novella "Nihilist Girl" has some greatness about it too, but feels first-draft-ish, like it needed a final edit to really become a work of art.

But the main thing I wanted to write about today was the revised idea of "nihilism" I got from reading "Nihilist Girl" and these other materials....

I've generally thought of "nihilism" as meaning "believing in nothing" ... or at least the attitude of Turgenev's character Bazarov, that nothing really matters much including one's own life....

But after reading Kovalevskaya, I realize that -- in thinking about Russian nihilism from the mid-1800s -- I was largely mistaking the parody for the real thing.

Kovalevskaya's brand of nihilism was significantly more interesting than that of the fictional character Bazarov.

It wasn't about absolutely rejecting everything and judging everything as meaningless and worthless. Rather, it was about rejecting any absolute values. It was about rejecting anything as sacred -- and opening everything up to question.

What was being rejected was a world-view in which there are certain absolute truths, in terms of which everything else must be assessed.

If you get rid of absolute truth, though, then what are you left with? Complete worthlessness and suicide, a la Bazarov? Or maybe not. What Kovalevskaya and her friends were after was something different.

One can think about it in terms of self-organization and strange attractors. Once one gets rid of absolute truth, and admits every single thing as open to question and revision, then one has a self-organizing dynamical system in which each thing gets potentially revised by each other thing. But the outcome of this doesn't need to be a homogeneous evaluation of everything as worthless. The outcome can be some other "strange attractor" in which each thing gets value from each other thing, according to a complex system of interdependencies and interactions.

Kovalevskaya-style nihilism, it seems, wasn't really about rejecting everything as equally worthless, but more about rejecting anything as absolutely valuable ... and letting the process of interactive, adaptive, mutual-value-adjustment spread through everything and lead to a productive evolution of new valuations and forms.

But I don't seem to have been the only one to get confused about nihilism.

Wikipedia says:

Nihilism (from the Latin nihil, nothing) is the philosophical position that values do not exist but rather are falsely invented. Most commonly, nihilism is presented in the form of existential nihilism which argues that life is without meaning, purpose or intrinsic value. Moral nihilists assert that morality does not exist, and subsequently there are no moral values with which to uphold a rule or to logically prefer one action over another.... The term nihilism is sometimes used synonymously with anomie to denote the general mood of despair at the pointlessness of existence that one has when they realize there are no necessary norms, rules, or laws.

and Nietzsche wrote in his notebooks (The Will to Power, section 585, translated by Walter Kaufmann)

A nihilist is a man who judges of the world as it is that it ought NOT to be, and of the world as it ought to be that it does not exist. According to this view, our existence (action, suffering, willing, feeling) has no meaning: the pathos of 'in vain' is the nihilists' pathos — at the same time, as pathos, an inconsistency on the part of the nihilists.

Nietzsche posited his own views as dramatically contradictory to nihilism -- and they certainly are wholly contradictory to Bazarov-style nihilism ... Nietzsche was all about creating your own values, rather than accepting any values as absolute, and rather than rejecting all values.

But it seems that Nietzsche was posing nihilism as a "straw man" to an even greater extent than I'd thought before ... and that his general views on trans-nihilist value-creation were not so fundamentally different than those of many of the Russian nihilists of the 1860s.

In Cities of the Red Night, Burroughs wrote "Nothing is true; everything is permitted" -- which on one reading reflects Bazarov-style nihilism ... and which Burroughs borrowed from Nietzsche, whose Zarathustra said:

“Nothing is true, all is permitted”: so said I to myself. Into the coldest water did I plunge with head and heart. Ah, how oft did I stand there naked on that account, like a red crab! –

But if one reads this Burroughs/Nietzsche aphorism as "Nothing is absolutely true; nothing is absolutely impermissible" then one has a Kovalevskaya-style nihilism, in which dogmatism is eliminated in favor of the creative self-organization of new value systems.

Dostoevsky also seems to have put a lot of energy into pillorying a straw-man version of Russian nihilism. Dostoevsky's nihilists are folks like Raskolnikov (the heartless, utilitarian murderer of Crime and Punishment) or Kirilov in The Possessed (a majorly hilarious character who preaches copiously about the worthlessness of existence and then suicides because he considers it the highest act of free will).

Kovalevskaya's "nihilist girl" character is about as far as you can get from Raskolnikov -- an extremely caring person, she marries a political prisoner (a man twice her age in whom she has no romantic interest) to save him from near-certain execution, even though this means her own exile to Siberia ... and generally decides to devote her life to helping Siberian prisoners, as a way of contributing to the common good. She doesn't lack values -- she just rejects having absolute values imposed on her, and wishes to create her own values based on her own intuitions and her engagement with the world.

And the narrator of Nihilist Girl makes a different choice -- she is a mathematician like Kovalevskaya, devoted to the life of science; she plainly states that she would not exile herself to save a political prisoner -- yet she just as plainly shares the same underlying philosophy of "creative nihilism" (my phrase, not Kovalevskaya's).

Dostoevsky courted Sofia Kovalevskaya's big sister Aniuta, as it happened. His story was that he broke off their engagement because she was too nihilistic. Her story (which has more ring of truth) was that she broke up with him, before they were formally engaged, because she didn't want to spend her life taking care of him and wanted more freedom to explore her own interests and passions.

But however the soap opera really went down, both Sofia and Aniuta were too nihilistic for Dostoevsky. The latter believed that only religious belief could save you from destructive nihilism of the form demonstrated by Bazarov, Raskolnikov or Kirilov. He didn't think new value systems could self-organize out of a pool of interacting non-absolutes ... to him value needed to begin with some absolute faith, some absolute assumptions.

All in all, I conclude that nihilism suffered from poor marketing, and an overly subtle and ironic name, which caused its more interesting variants to get forgotten, and its less interesting variants to get repeatedly parodied.

Long live creative nihilism, Kovalevskaya style!

Nothing is true; everything is permitted ;-)

When the Net Becomes Conscious

2009-03-10T21:03:00.005-04:00

A journalist emailed me today and asked me some questions about the possibility of the Internet becoming conscious. The questions and my answers follow:

> 1) Why do some people think it is possible for the internet (or internet plus humans) to become conscious? Is it to do with the network architecture?

Many scientists believe that consciousness is a property that will inevitably emerge from any complex system that has the right sort of internal dynamics, and the right sort of interaction with its environment.

Exactly what the right sort of dynamics and interactions are, different theorists disagree on. But it seems plausible that the Internet may have enough of them to develop its own sort of consciousness. The Internet perceives and acts on the world; it stores declarative, episodic and procedural memories; it recalls some information and forgets others; etc. In short it behaves a fair bit like a human mind, though there are a lot of differences too.

According to this perspective, the Internet might already have a degree of consciousness, though of a type quite different from human consciousness.

Neuroscientist Susan Greenfield views consciousness as consisting of "whole-brain activation patterns". In this sense one would say that the Internet of today has a more fragmented, dissociated consciousness than a human mind ... there aren't so many "whole-internet activation patterns", though there are intense patterns spanning large portions of the Internet.

Of course, there are many philosophies of consciousness. My own view of consciousness is a bit eccentric for the scientific world though rather commonplace among Buddhists (which I'm not): I think consciousness is everywhere, but that it manifests itself differently, and to different degrees, in different entities.

So to me the interesting question is whether the Internet has (or will develop) consciousness of the same type as humans, or maybe even of a more advanced and intricate type.

It seems that as the Internet expands and grows richer, it *could* develop a more human-like, more unified consciousness than it has now ... with more coherent "whole Internet activation patterns"...

> 2) What might be the consequences of such an event? Do you think it might be something that we should welcome?

The potential consequences of the Internet developing more coherent holistic activation patterns (ergo more humanlike consciousness) are rather difficult to predict, I find!

However, I personally am pessimistic about the future in the case that humans remain the most powerful minds on the planet. I don't trust us to use our increasingly advanced technologies in an ethical and nondestructive way.

So I think the outlook for humanity is probably better in the case that an emergent, coherent and purposeful Internet mind develops, than in the case where it doesn't.

But there is a lot of uncertainty in either case!

> 3) If it were possible, what would be needed to make the internet conscious? How far away from that situation are we?

My guess is that humanlike consciousness is not going to spontaneously evolve from the Net. However, I think someone could engineer it, by specifically creating an AI system on a server farm, oriented toward serving as a kind of "central cognition engine" for the Internet as a whole.

This central cognition engine wouldn't need to control everything on the Net; it would just need to read a lot of the information out there on the Net, and then insert information of its own creation in appropriate locations (posting to email lists, creating web pages, buying and selling things, etc.).

The engine might be created with some other primary purpose (e.g. as an artificial scientist aimed at making new discoveries via collaborating with human scientists online), or it might be created specifically with the goal of transforming the Internet into a more coherent, more humanlike intelligence. Either way the effect might be the same.

This is the scenario I described in my 2001 book "Creating Internet Intelligence," and I still think it is a plausible one.

Once Science Eliminates Pain ...

2009-02-23T13:44:00.001-05:00

The bored or overly curious may check out my latest neurological dysfunction (aka work of fiction) "The Last Aphrodisiac", at

http://goertzel.org/fiction.htm#pain

What happened was, I was driving late at night listening to a Morphine CD in the car, then got home, lay in bed and fell asleep with the song "Cure for Pain" in my head.

I had a number of dreams on the theme (what if pain were really eliminated, in some interesting sense? what would life be like? what if it were rediscovered?) and woke up plagued by this story. On a cross-country flight to a weekend workshop on "Evaluation and Metrics for Human-level AI", I decided to write it down...

At first I thought it would take a single page to write down, but it wound up 15 pages, and the punchline doesn't start to unfold till page 7 or 8.

This is the first story I've written in a long time that doesn't involve AI in any serious way. Rather, it uses future tech like uploading-to-superhuman-form and cranial jacks to enlarge upon certain aspects of human relationships, especially romantic ones. It's probably the closest thing to a maudlin love story I'll ever write (well, I hope so).

Ahh, the things that can transpire between a man, a woman, and an illicit cranial jack modification device... ;-)

Freaky Russian Paranormal Biology (plus irrelevant personal-history rambling)

2009-02-13T18:15:00.003-05:00

A friend pointed me to this paper

http://www.emergentmind.org/gariaev06.htm

which I found provoking and interesting, though nowhere near fully convincing....

The authors argue that much "junk DNA" actually serves as an interface to some sort of energy-informational field reminiscent of Sheldrake's "morphogenetic field"...

Their comments on traditional biology are a bit naive: in the last 5 years a lot of functions of formerly-known-as-junk-DNA have been found. So it's no longer true that ordinary biology says 95%+ of DNA is useless. We're finding this DNA serves a lot of regulatory functions, even though it doesn't code for proteins.

On the other hand, that oversight doesn't make their theory wrong; it just makes them out of date regarding traditional biology.

The experimental results they describe are certainly compelling and intriguing. Fraud is always a possibility, yet, I'm wary to dismiss results as likely fraud just because they violate currently standard scientific theories. (After all, every scientific paradigm prior to the current ones has been overthrown, right?)

As an aside in their discussion, they mention some old results involving Kirlian photography. I tried to build a Kirlian camera in my basement once, in Randolph NJ in the late 1990s, with a view toward trying to replicate the "phantom leaf effect" (see link below) ... but wound up setting fire to part of the basement instead. (I did build a Tesla coil from a neon sign transformer, and it worked a few times, but eventually it caught fire before I finished making it into a Kirlian camera apparatus.)

A recent attempt to replicate the phantom leaf effect, with intriguing results, is described here:

http://shadowboxent.brinkster.net/lemurkirlian2.html

It's hard to be anywhere near certain, but I'm intuitively inclined to feel there might be some truth somewhere in the vicinity of these guys' wacky theory.

Obviously it's close to my own "glocal theory of psi" ... in the language of my "glocal memory theory" what they're saying is that living organisms have "keys" (the observed physical substance) and "maps" (the correlated energy-information field).

I'm hesitant to use the word "energy" in this context as these Russian authors do, because "energy" has a specific meaning in physics, and this (even if real) may be something different. I have thought of their energy-informational field as a kind of "pattern space", more like Sheldrake's idea of a morphogenetic field....

But, without some hypothesis regarding the dynamical laws of this posited morphogenetic field, their theory remains more philosophical than even "speculatively scientific."

It seems that conceivably -- if there's any reality here -- one could learn something about the laws of this field via systematically varying parameters in Kirlian photography experiments.

Yet another line of research I'd probably fund if I were super-wealthy ;-p

P.S.

(warning: loosely related personal-history rambling below...)

Wow, I remember now that when I was 18 years old and in my senior year of college (at Simon's Rock, in western Massachusetts) and visited CalTech, where I was hoping/considering to go to grad school, I mentioned to one of the math profs there that I was interested in doing a PhD thesis on using partial differential equations to model bioelectromagnetic fields as had been discovered in some strange Russian experiments.

His reply was something like "Well, you certainly know a lot of big words, but do you know any math?" We talked a bit and he discovered that I did; but I didn't get admitted to CalTech anyway.... It was obvious that the math department there did not like that potential thesis topic!!

On that same visit to California I visited Berkeley's Logic and Methodology PhD program, where I expressed my interest in writing a thesis on using hypersets (non-well-founded sets) to model consciousness (which now is one of the themes of one of the handful of half-finished books on my hard drive). This was better received, and they were likely to admit me with funding (or so they said verbally) but I wound up not completing my application because the students there told me that the department inevitably made its students take 7-9 years to finish their PhDs. I didn't want to be in school that long, so I wound up at NYU's Courant Institute instead, which I liked because it combined math, theoretical physics and computer science in one department...

On that trip I also visited a girl at UCSD whom I had a crush on (from when she'd attended college with me in Massachusetts), and was disturbed to find she'd become a fundamentalist Christian, handing out Jesus brochures on the street. (I saw her 11 years ago and she seemed to have gotten over that phase long ago, fortunately... though it was a bad visit as I'd been up nearly the entire night, insomniac due to too much arguing with my wife-at-the-time Gwen, and was completely bleary-eyed and -minded for the whole visit ... that was a few days before the birth of my daughter Scheherazade and Gwen was mad at me for repeatedly getting stoned out of my mind with one of my friends at a time when the baby might pop out at any minute ... well, at least being totally stoned helped the fighting go down easier!!)

In hindsight, that was a rather entrepreneurial trip for a college senior to make (esp. an 18 year old one): I basically invited myself on interviews to grad schools I was interested in, prior to even completing my formal applications to the places ... to go talk to the profs and students there, sit in on classes, and get a feel for the departments. I didn't realize at the time that this was a fairly eccentric thing to do. But it was a good idea.... Although I had only one set of clothes for the whole trip, because People Express Airlines (which featured a great $99 cross country flight) sent my luggage to Europe by mistake.

(Unfortunately that trip caused me to get fired from my job as a math homework grader, because I just took off from college for a week and flew to California without giving any notice to my boss, so all the papers went ungraded while I was gone. I wasn't too conscientious back then. I'm still a slob with paying bills for the house and such, but I try not to be a mess like that in professional life anymore! Still it's hard to focus on reality and not be an absent-minded professor sometimes ;-)

All that probably has something to do with Kirlian photography, auras and Russian theories of morphogenetic fields ... but since I don't get stoned hardly ever these days, I'm not in the right state of mind to find the connecting thread; and I'll go cook dinner instead and then get down to some useful work (oh yeah, and Scheherazade and I are going to watch the film Baghead tonight... which unfortunately has nothing to do with Buckethead...)

Hmmm... that reminds me of the Timothy Leary Family Reunion I attended in San Francisco last week ... what a wonderful assortment of old hippies with cosmic looks in their eyes and wild ideas in their brains; along with various sympathetically resonating youngsters like me ... but, I won't go there right now...

Nietzschean Nonlinear Economics?

2009-02-02T06:37:00.005-05:00

I was thinking a little more about the current economic situation and why it seems so confusing to people, including experts.

Part of the problem is just that the systems involved are so complex, of course -- the economy has complexified and the human brain has not kept up.

Another part of the problem, though, is that none of the traditional economic theories (neither the mathematical ones or the qualitative ones) embrace this complexity in principle.

I.e., the very fact of the increased complexity of the economy has direct implications, which seem to require introduction of new approaches for economic analysis.

One way to look at this is in terms of what I semi-seriously call "Nietzschean economics," where a more abstract notion of economic power replaces the traditional notion of money.

Put simply: it seems that the modern economy has somewhat obsoleted the concept of money, which has economists (and many others) confused...

For instance, China has a lot of US dollars right now, in principle ... but they can't really spend most of it, because if they traded their dollars for goods on the open international market, then the dollar would collapse in the international currency markets, the US economy would tank, demand for Chinese goods would collapse, and China would risk massive internal unrest etc.

It's not exactly that the Chinese government has money they can't use; but they have money that comes with severe restrictions on the ways it can be used. Having this money confers great economic power on them; but, this economic power can't necessarily be used in the traditional manner of buying stuff.

In general: with the world so interconnected, the notion of a unit of currency as being something that can be exchanged for a certain amount of stuff, anywhere, doesn't really apply...

More complexly, the same sorta phenomenon applies among and within investment banks and other financial institutions. E.g. a large bank holds some portfolio of financial instruments ... but if they sell a lot of one kind of instrument, this impacts the markets in a way that affects the value of the others, etc. So one can hardly assign each instrument they hold a value independently of the others.... The values of their various portfolio items are interdependent in the manner of a system of simultaneous nonlinear equations.

What this growing, rampant interdependence means is that the traditional concepts and tools of economics don't closely apply to the international biz/finance world anymore...

A lot of this was foreseen, on a qualitative level, in Galbraith's book "Economics and the Public Purpose" from the 1970s ... he talked about the "market economy" versus the "technostructure" and pointed out that the latter (being a complex of large corporations, governments and other institutions) follows quite different rules.

Well, now the nonlinear dynamics of the technostructure is running rampant -- but economists are still mainly studying it with tools designed for studying markets.

One step I think that needs to be taken, on the theory level, is to view actual buying power as Level 1 of a hierarchy of types of economic power. I'm thinking of:

Level 1 = power to buy goods or services [right now, or at some future point(s) in time]
Level 2 = power to influence others' Level 1 power
Level 3 = power to influence others' Level 2 power
.. etc. ...

In principle, one could boil down Level k power into Level 1 power. But in practice, with the economy so complex, this involves calculations that are infeasible to do. So, economic agents are in effect seeking Level k power without a clear picture of how it will in the future boil down into Level 1 power. This might be thought of as Nietzschean economics (as Nietzsche viewed the "will to power" as the essential dynamic of the universe).

The ultimate extreme of all this of course would be

Level infinity = power to influence others' Level infinity power

which was basically Nietzsche's view of the driving force of the universe ... and I do think that international economics and politics boils down to this sometimes: power for it's own sake, rather than being tied to ultimately influencing the acquisition of goods and services.

Traditional economics is based on the notion that everything boils down to buying power ... but in a world where no one is smart enough to calculate what their decisions will imply in terms of buying power, this sort of economics seems to have limited applicability...

What we need is a nice, elegant, pragmatically applicable theory of the nonlinear dynamics of Level k economic power under conditions where

the computational complexity of recognizing important high-level patterns in an economy

vastly exceeds

the computational capability of even the smartest individual participants in the economy

This might be a fun thing to work on, but I've got a thinking machine to build, so hopefully somebody else will do it ... or we'll have to wait for the AI to solve the problem. (Of course, an appropriately constructed AI could also palliate the problem, due to having increased capability to recognize economic patterns, either due to possessing greater-than-human general intelligence, or due to combining human-level general intelligence with specialized capabilities for economic analysis.)

The Subtle Structure of the Everyday Physical World = The Weakness of Abstract Definitions of Intelligence

2008-12-27T00:56:00.009-05:00

In my 1993 book "The Structure of Intelligence" (SOI), I presented a formal definition of intelligence as "the ability to achieve complex goals in complex environments." I then argued (among other things) that pattern recognition is the key to achieving intelligence, due to the algorithm

Recognize patterns regarding which actions will achieve which goals in which situations
Choose a goal that is expected to be good at goal achievement in the current situation

The subtle question in this kind of definition is: How do you average over the space of goals and environments? If you average over all possible goals and environments, weighting each one by their complexity perhaps (so that success with simple goals/environments is rated higher), then you have a definition of "how generally intelligent a system is," where general intelligence is defined in an extremely mathematically inclusive way.

The line of thinking I undertook in SOI was basically a reformulation in terms of "pattern theory" of ideas regarding algorithmic information and intelligence that originated with Ray Solmonoff; and Solomonoff's ideas have more recently been developed by Shane Legg and Marcus Hutter into a highly rigorous mathematical definition of intelligence.

I find this kind of theory fascinating, and I'm pleased that Legg and Hutter have done a more thorough job than I did of making a fully formalized theory of this nature.

However, I've also come to the conclusion that this sort of approach, without dramatic additions and emendations, just can't be very useful for understanding practical human or artificial intelligence.

What is Everyday-World General Intelligence About?

Let's define the "everyday world" as the portion of the physical world that humans can directly perceive and interact with -- this is meant to exclude things like quantum tunneling and plasma dynamics in the centers of stars, etc. (though I'll also discuss how to extend my arguments to these things).

I don't think everyday-world general intelligence is mainly about being able to recognize totally general patterns in totally general datasets (for instance, patterns among totally general goals and environments). I suspect that the best approach to this sort of totally general pattern recognition problem is ultimately going to be some variant of "exhaustive search through the space of all possible patterns" ... meaning that approaching this sort of "truly general intelligence" is not really going to be a useful way to design an everyday-world AGI or a significant components of one. (Hutter's AIXItl and Schmidhuber's Godel Machine are examples of exhaustive search based AGI methods.)

Put differently, I suspect that all the AGI systems and subcomponents one can really build are SO BAD at solving this general problem, that it's better to characterize AGI systems

NOT in terms of how well they do at this general problem

but rather

in terms of what classes of goals/environments they are REALLY GOOD at recognizing patterns in

I think the environments existing in the everyday physical and social world that humans inhabit are drawn from a pretty specific probability distribution (compared to say, the "universal prior," a standard probability distribution that assigns higher probability to entities describable using shorter programs), and that for this reason, looking at problems of compression or pattern recognition across general goal/environment spaces without everyday-world-oriented biases, is not going to lead to everyday-world AGI.

The important parts of everyday-world AGI design are the ones that (directly or indirectly) reflect the specific distribution of problems that the everyday world presents an AGI system.

And this distribution is really hard to encapsulate in a set of mathematical test functions. Because, we don't know what this distribution is.

And this is why I feel we should be working on AGI systems that interact with the real everyday physical and social world, or the most accurate simulations of it we can build.

One could formulate this "everyday world" distribution, in principle, by taking the universal prior and conditioning it on a huge amount of real-world data. However, I suspect that simple, artificial exercises like conditioning distributions on text or photo databases don't come close to capturing the richness of statistical structure in the everyday world.

So, my contention is that

the everyday world possesses a lot of special structure
the human mind is structured to preferentially recognize pattern related to this special structure
AGIs, to be successful in the everyday world, should be specially structured in this sort of way too

To encompass this everyday-world bias (or other similar biases) into the abstract mathematical theory of intelligence, we might say that intelligence relative to goal/environment class C is "the ability to achieve complex goals (in C) in complex environments (in C)"

And we could formalize this by weighting each goal or environment by a product of

its simplicity (e.g. measured by program length)
its membership in C, considering C as a fuzzy etc

One can create a formalization of this idea using Legg and Hutter's approach to defining intelligence also.

One can then characterize a system's intelligence in terms of which goal/environment sets C it is reasonably intelligent for.

OK, this does tell you something.

And, it comes vaguely close to Pei Wang's definition of intelligence as "adaptation to the environment."

But, the point that really strikes me lately is how much of human intelligence has to do, not with this general definition of intelligence, but with the subtle abstract particulars of the C that real human intelligences deal with (which equals the everyday world).

Examples of the Properties of the Everyday World That Help Structure Intelligence

The propensity to search for hierarchical patterns is one huge example of this. The fact that searching for hierarchical patterns works so well, in so many everyday-world contexts, is most likely because of the particular structure of the everyday world -- it's not something that would be true across all possible environments (even if one weights the space of possible environments using program-length according to some standard computational model).

Taking it a step further, in my 1993 book The Evolving Mind I identified a structure called the "dual network", which consists of superposed hierarchical and heterarchical networks: basically a hierarchy in which the distance between two nodes in the hierarchy is correlated with the distance between the nodes in some metric space.

Another high level property of the everyday world may be that dual network structures are prevalent. This would imply that minds biased to represent the world in terms of dual network structure are likely to be intelligent with respect to the everyday world.

The extreme commonality of symmetry groups in the (everyday and otherwise) physical world is another example: they occur so often that minds oriented toward recognizing patterns involving symmetry groups are likely to be intelligent with respect to the real world.

I suggest that the number of properties of the everyday world of this nature is huge ... and that the essence of everyday-world intelligence lies in the list of these abstract properties, which must be embedded implicitly or explicitly in the structure of a natural or artificial intelligence for that system to have everyday-world intelligence.

Apart from these particular yet abstract properties of the everyday world, intelligence is just about "finding patterns in which actions tend to achieve which goals in which situations" ... but, this simple meta-algorithm is well less than 1% of what it takes to make a mind.

You might say that a sufficiently generally intelligent system should be able to infer these general properties from looking at data about the everyday world. Sure. But I suggest that would require a massively greater amount of processing power than an AGI that embodies and hence automatically utilizes these principles? It may be that the problem of inferring these properties is so hard as to require a wildly infeasible AIXItl / Godel Machine type system.

Important Open Questions

A couple important questions raised by the above:

What is a reasonably complete inventory of the highly-intelligence-relevant subtle patterns/biases in the everyday world?
How different are the intelligence-relevant subtle patterns in the everyday world, versus the broader physical world (the quantum microworld, for example)?
How accurate a simulation of the everyday world do we need to have, to embody most of the subtle patterns that lie at the core of to everyday-world intelligence?
Can we create practical progressions of simulations of the everyday world, such that the first (and more crude) simulations are very useful to early attempts at teaching proto-AGIs, and the development of progressively more sophisticated simulations roughly tracks the development of progress in AGI design and development.

The second question relates to an issue I raised in a section of The Hidden Pattern, regarding the possibility of quantum minds -- minds whose internal structures and biases are adapted to the quantum microworld rather than to the everyday human physical world. My suspicion is that such minds will be quite different in nature, to the point that they will have fundamentally different mind-architectures -- but there will also likely be some important and fascinating points of overlap.

The third and fourth questions are ones I plan to explore in an upcoming paper, an expansion of the AGI-09 conference paper I wrote on AGI Preschool. An AGI Preschool as I define it there is a virtual world defining a preschool environment, with a variety of activities for young AI's to partake in. The main open question in AGI Preschool design at present is: How much detail does the virtual world need to have, to support early childhood learning in a sufficiently robust way? In other words, how much detail is needed so that the AGI Preschool will posssess the subtle structures and biases corresponding to everyday-world AGI? My AGI-09 conference paper didn't really dig into this question due to length limitations, but I plan to address this in a follow-up, expanded version.

The Increasing Value of Peculiar Intelligence

2008-11-26T20:48:00.002-05:00

One more sociopolitical/futurist observation ...

If you're not aware of David Brin's Transparent Society concept, you should read the book, and start with the Web page

http://www.davidbrin.com/tschp1.html

His basic idea is that, as surveillance technology improves, there are two possibilities:

The government and allied powers watch everybody, asymmetrically
Everybody watches everyone else, symmetrically (including the government and allied powers being watched)

He calls the latter possibility sousveillance ... all-watching ...

What occurs to me is that in a transparent society, there is massive economic value attached to peculiar intelligence

This is because, if everyone can see everything else, the best way to gain advantage is to have something that nobody can understand even if they see it

And it's quite possible that, even if they know that's your explicit strategy, others can't really do anything to thwart it...

Yes, a transparent society could decide to outlaw inscrutability. But this would have terrible consequences, because nearly all radical advances are initially inscrutable....

Inscrutability is dangerous. But it's also, almost by definition, the only path to radical growth.

I argued in a recent blog post that part of the cause of the recent financial crisis is the development of financial instruments so complex that they are inscrutable to nearly everyone -- so that even if banks play by the rules and operate transparently, they can still trick shareholders (and journalists) because these people can't understand what they see!

But it seems that this recent issue with banks is just a preliminary glimmering of what's to come....

Yes, maybe there is something a bit self-centered or self-serving in this theory, since I seem to personally have more differential in "peculiar intelligence" than in most other qualities ... but, call it what you will, my peculiar intelligence is definitely pushing me to the conclusion that peculiar intelligence is going to be a more and more precious commodity as the Age of Transparency unfolds...

Obviously, you can also see this phenomenon in financial trading even in a non-crisis time. It's not enough to be smart at predicting the markets to make a LOT of money ... because if you're just smart, but in non-peculiar way, then once you start trading a lot of money, others will observe your trades and pick up the pattern, and start being smart in the same way as you ... and then you'll lose your edge. The way to REALLY make a lot of money in the markets is to be very smart, and in a very peculiar way, so that even if others watch your trades carefully, they can't understand the pattern, and they can't imitate your methodology....

The best trader I know personally is Jaffray Woodriff who runs quantitative.com, and he exemplifies this principle wonderfully: very intelligent, very peculiar (though in an extremely personable, enjoyable way), and very "peculiarly intelligent" ;-)

Democratic market socialism redux

2008-11-26T20:05:00.002-05:00

All in all, the conclusion I'm coming to lately ... as reflected in my last two blog posts, as well as in some other thinking ... is that government is going to need to do some rather careful and specific things to guide society toward a positive Singularity.

Yes, if someone creates a hard-takeoff with a superhuman AGI, then the government and other human institutions may be largely irrelevant.

But if there is a soft takeoff lasting say 1-3 decades ... before the hard takeoff comes along ... then, my view is increasingly that the market system is going to screw things up, and lead to a situation where there are a lot of unhappy and disaffected people ... which increases the odds of some kind of nasty terrorist act intervening and preventing a positive Singularity from occurring.

It seems we may need to review the general line of thinking (though not many of the specific proposals) from old democratic-socialism-style texts like Economics and the Public Purpose ...

Perhaps one positive consequence of the current economic crisis is that it may cause the US public to understand the value of well-directed government spending....

And yes, I'm well aware that most of my colleagues in the futurist community tend to be libertarian politically. I think they're just wrong. I am all in favor of getting rid of victimless crimes ... legalizing drugs and prostitution and so forth ... but, given the realities of the next century and the risks of a negative Singularity, I don't think we can afford to leave things up to the unpredictable self-organizing dynamics of the market economy ... I think we as a society will need to reflect on what we want the path to Singularity to be, and take specific concerted governmental actions to push ourselves along that path...

This is not the political dialogue for 2008 ... different issues are occupying peoples' minds right now ... but it may be the political dialogue for 2012 or 2015 ... or at latest, I'd guess, 2020 ...

Why the average workweek isn't decreasing faster ... and what we can do about it

2008-11-26T18:34:00.005-05:00

This is another post on political, economic and futurist themes ... starting out with a reflection on a bogus patent, and winding up with a radical social policy proposal that just might improve life in the near future and also help pave the way to a positive Singularity... (yeah, I know I know, lack of ambition has never been one of my numerous faults ;-)

Let's start with the simple, oft-asked question: Why isn't the average workweek decreasing faster ... given all the amazing technology recently developed?

One clue is found in some news I read today: IBM has patented the idea of a specialized electronic device that makes it handier to split your restaurant bill among several people at the table:

http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=7,457,767.PN.&OS=PN/7,457,767&RS=PN/7,457,767

The patent application is really quite hilarious reading ... and the number of references to prior, equally bullshitty patents for related inventions is funny too.

What strikes me most here is the amount of effort expended by lawyers, patent examiners and judges in dealing with this crap. Not to mention their paralegals, secretaries, and on and on and on.

Why does this part of the contemporary human workload exist?

One could argue that some wasted work is a necessary part of a complex economic system, and that it would be hard to eliminate the crap without throwing out a lot of useful stuff as well.

I'm sure this is part of the truth, but I don't think it's the whole story.

Another part of the answer, I think, is: This kind of BS work exists because people have time to do it.

If people didn't have time to do this shit, because they all had to be occupied gathering food or making shelter or defending themselves against attackers -- or conceiving or manufacturing truly original and interesting inventions -- then the legal system would rapidly get adjusted so as to make bullshit patents like this illegal.

But, because we have plenty of extra resources in our economy ... due to the bounty created by advances in technology ... we (collectively and without explicitly intending to) adjust various parameters of our economy (such as the patent system) in such a way as to create loads of bullshit work for ourselves.

But there's a little more to it than this. Granted that we have the time ... but why do we choose to spend our time on this kind of crap? Instead of, say, doing less useless work and having more free time?

One important, relevant fact to grasp is that people like working.

Yes, it's not just that we're ambitious or greedy, it's that we actually like working -- in spite of the fact that we also like complaining about working. (Don't blame me -- humans are just fucked-up creatures....).

There is loads of data supporting this idea; I'll cite just a smattering here.

Americans work more and are happier than Europeans:

http://sayanythingblog.com/entry/americans_work_harder_happier_than_europeans/

Old folks who work longer are happier:

http://ideas.repec.org/p/pra/mprapa/5606.html

I can't remember the link, but studies show that people are on average happier at work than at home (if asked at random times during the day), but will when asked SAY they are happier at home than at work.... I think this study was done by Mihaly Csikszentmihalyi, who you can find out about at

http://www.time.com/time/health/article/0,8599,1606395,00.html

http://www.amazon.co.uk/Flow-Classic-Work-Achieve-Happiness/dp/0712657592

He is the pioneer of the study of "flow", the state of supreme happiness that many humans enter into when carrying out some highly absorbing, satisfying activity, like creating an artwork, participating in a sport, acting in a play or doing scientific or other professional work ... or gardening ... or chopping down a tree ... actually the activity can be almost anything, the real point is in the way the mind interacts with the activity: it has to be enough to fully occupy the mind without causing too much frustration, so that the self is continually lost and rediscovered within the ongoing dynamism of the interactive process....

Among many more interesting observations, he notes that:

"
Those at work generally report that they wish they were at home, but when they're home they often feel passive, depressed or bored. "They have in mind that free time at home will make them feel better, but often it doesn't,"
"

But, the plot thickens ... because, although we like working ... this doesn't necessarily mean we like working long hours.

People who work part-time are happier than those working full-time (84 per cent are happy versus 79 per cent):

http://www.arboraglobal.com/documents/Happiness%20at%20Work%20Index%202007.pdf

So where we seem to fail is in creating more part-time jobs...

This seems to be because, for an employer, it's always more cost efficient to hire one full time worker than two part-timers ;-(

So, instead of making use of the bounty provided by technology by creating part-time jobs and also creating more opportunities for creative, growth-producing, flow-inducing leisure ... we make use of it by creating more and more full-time jobs doing more and more bullshit work, like processing patents for "inventions" like the one I referenced above.

Given the way our brains work, we're better off working full-time than we would be not working at all.

But there is at least some evidence to suggest we'd be even better off working fewer hours....

But, given the way the market system works, there is almost never any incentive for any employer to allow part-time workers. It just isn't going to be rational from their perspective, as it will decrease their economic efficiency relative to competitors.

The market system, it seems, is going to push toward the endless creation of BS work, because no individual company cares about reducing the workweek ... whereas individual companies DO in many cases profit by creating BS work ... if they can bill other companies (or the government) for this bullshit work....

So, this leads to the idea that the government of France may have had the right idea at heart, in creating the 35 hour maximum workweek. Which they have since rescinded, because in a global context, it made them uncompetitive with other countries lacking such a restriction.

But anyway, outlawing long work hours is obviously not the answer. People should have the freedom to work long hours and get paid for them, if they want to.

So, an interesting question, policy-wise, is how the government can create incentives for reduced workweeks, without introducing punitive and efficiency-killing regulations.

One possibility would be, instead of just government projects aimed at paying people to build infrastructure, to launch government projects aimed at paying people to do interesting, creative, growth-inducing stuff in their spare time.

Basically, what I'm suggesting is: massively boosted government funding for art and science and other not-directly-economically-productive creative work ... but without making people jump through hoops of fire to get it (i.e. no long, tedious, nearly-sure-to-fail grant applications) ... and specifically designed NOT to discriminate against people who do not-directly-economically-productive creative work only part-time.

This would make it harder for companies to find full-time employees, because it wouldn't be all that much more lucrative to work full-time than to work part-time plus earn a creative-activity stipend on the side. But, unlike France's previous restrictive laws, it would enable companies that REALLY need full-time employees to hire them, so long as they were able to pay the required premium salaries ... or recruit workers who preferred the offered work to paid on-the-side creative-activity....

I suspect this would actually boost the economy, by shifting effort away from BS make-work like processing bogus patents, and toward creative work that would end up having more indirect, ancillary economic value in all sorts of ways.

This may seem a funny thing to think about in the current economic context, when the economy is in such trouble globally. But I consider it obvious that the current economic troubles are "just" a glitch (albeit an extremely annoying one) on the path to a post-scarcity economy. And even so, the government is still giving away money right now to people who are out of work. What if the payment were decreased to people who didn't engage in creative activities, but increased to people who did. Peoples' lives would become richer, as more of them would be more involved in creative activities. And, the human world as a whole would become richer because of all of these new creations.

And this sort of thing will become all the more interesting once robots and AI software eliminate more and more jobs. At that point the market system, unrestricted, would probably create an insane overgrowth of bullshit full-time jobs ... but a part-time creative-activity incentive as I've described could interfere with this dynamic, and nudge things toward a more agreeable situation where people divide their time between genuinely economically useful work, and non-directly-economically-useful but personally-and-collectively-rewarding creative activity.

Furthermore, this would create a culture of creative activity, that would serve us well once robots and AIs eliminate jobs altogether. It would be great if we could greet the Singularity with a culture of collective creativity rather than one characterized by a combination of long hours on useless bullshit jobs, combined with leisure time spent staring at the tube (whether the TV or the YouTube) or playing repetitive shoot-em-up video games ... etc. etc.

OK -- now ... time for me to get back to work ;-)