M O B J E C T I V I S T

The Stretched Exponential

2009-11-23T20:43:00.000-08:00

I would not use the stretched exponential because its derivation involves a disordered superposition of damped exponentials where only the time constant gets varied. In the dispersion that I care about, I take the velocity as the primary disperser, and only disperse the time constants if we have an additional waiting time distribution to consider.

I will toss a diagram up here as an interesting experiment. For example the idea of the human mobility plot that I refer to in my recent cross-posted TOD article has an exceedingly simple rationalization. I have the derived equation that gives the probabilities of how far people have moved in a certain time, based on the dispersion principle.

P(x,t) = beta/(beta + x/t)

To simulate this behavior is really beyond simple. You simply have to draw from a uniform random distribution for distance (x) and then draw another number for a random time span (t). Or you can do it from two inverted MaxEnt exponential draws (doesn't really matter apart from the statistics). You then divide the two to arrive at a random velocity, i.e. v=x/t. Nothing more simple than this formula or formulation. The ranked histogram for the Monte Carlo simulation of 10,000,000 trials of independent draws looks like the following points with the dispersion formula in red:

The random draws definitely converge to the derived Maximum Entropy dispersion derivation. The program takes a few lines of source code, too trivial too even post.

Where else does the stretched exponential get some top billing? Well, it turns out by taking the complementary cumulative of the stretched exponential, you get the Weibull distribution. And the reason I don't use the Weibull follows from the same basic concern. The Weibull when applied in such areas as bathtub curve for reliability analysis also deals with a superposition of time constants, and not velocity, the latter which I think makes much more pragmatic sense. I suggest that the active velocity of creep and heat and other factors affect failure much more than passive abstractions such as a varying set of time constants.

http://mobjectivist.blogspot.com/2009/10/creep-failure.html
http://mobjectivist.blogspot.com/2009/10/failure-is-complement-of-success.html

I became motivated to write this post up due to the 1998 paper by J.H.Laherrère and D.Sornette, "Stretched exponential distributions in nature and economy: fat tails with characteristic scales", European Physical Journal B 2, April II, p525-539 : in arXiv. They have some very interesting ideas and useful heuristic approximations, but I want to make a claim and drive a stake in the ground that I use a completely inverted analysis than the stretched exponential and parabolic fractal crowd apply to their own interpretation. Velocity, not time, drives these behaviors, and no one seems to want to pursue that angle.

You can describe both he stretched exponential and the dispersive formulation as fat-tailed distributions, but the dispersive remains the phattest. At least until something better comes along.

Information and Crude Complexity

2009-11-20T06:36:00.001-08:00

(this article of mine has been sitting in the queue at TheOilDrum for more than a week so I decided to post it here)

Information and Crude Complexity

Abstract (please read this as a set of squished-together PowerPoint bullet points)

People become afraid when you mention theory. Everyone talks about entropy without actually understanding it. Simplicity can come out of complexity. "Knowledge" remains a slippery thing. We think that science flows linearly as previous knowledge get displaced with new knowledge. Peak oil lies in this transition much like plate tectonics at one time existed outside of the core knowledge. We define knowledge by whatever the scientific community currently believes. “Facts are not knowledge. Facts are facts, but how they form the big picture, are interconnected and hold meaning, creates knowledge. It is this connectivity, which leads to breakthroughs …” You will either think you understand the following post, or know for a fact that you don't.

Scientific theories get selected for advancement much like evolution promotes the strongest species to survive. New theories have to co-exist with current ones, battling with each other to prove their individual worth [Ref 1]. That may partly explain why the merest mention of "theory" will tune people out, as it will remind them of the concept of biological evolution, which either they don't believe in, or consider debatable at best. Generalize this a bit further and you could understand why they could also reject the scientific method. If we admit to this as a chronic problem, not soon solved, the idea of accumulating knowledge seems to hold a kind of middle ground, and doesn't necessarily cause a knee-jerk reaction like pushing a particular theory would.¹

So, what kinds of things do we actually want to know? For one, I will assert that all of us would certainly want to know that we haven't unwittingly taken a sucker's bet, revealing that someone has played us. I suspect that many of the diehard TOD readers, myself included, want to avoid this kind of situation. In my mind, knowledge remains the only sure way to navigate the minefield of confidence schemes. In other words, you essentially have to know more than the next guy, and the guy after that, and then the other guy, etc. TOD does a good job of addressing this as we constantly get fed the unconventional insights to explain our broader economic situation.

Ultimately we could consider knowledge as a survival tactic -- which boils down to the adage of eat or be eaten. If I want to sound even more pedantic, I would suggest that speed or strength works to our advantage in the wild but does not translate well to our current reality. It certainly does not work in the intentionally complex business world, or even with respect to our dynamic environment, as we cannot outrun or outmuscle oil depletion or climate change without putting our thinking hats on.

This of course presumes that we know anything in the first place. Nate Hagens had posted on TOD earlier this year the topic "I Don't Know". I certainly don't profess to have all the answers, but I certainly want to know enough not to get crushed by the BAU machine. So, in keeping with the traditions of the self-help movement, we first admit what we don't know and build from there. That becomes part of the scientific method, which a few of us want to apply.

As a rule, I tend to take a nuanced analytical view to the way things may play out. I will use models of empirical data to understand nagging issues and stew over them for long periods of time. The stewing is usually over things I don't know. Of course, this makes no sense for timely decision making. If I morphed into a Thompson's gazelle with a laptop cranking away on a model under a shady baobob tree on the Serengeti, I would quickly get eaten. I realize that such a strategy does not necessarily sound prudent or timely.

Nate suggests that the majority of people use fast and frugal heuristics to make day-to-day decisions (the so-called cheap heuristic that we all appreciate). He has a point in so far as not always requiring a computatonal model of reality to map our behaviors or understanding. As Nancy Cartwright noted:

This is the same kind of conclusion that social-psychologist Gerd Gigerenzer urges when he talks about “cheap heuristics that make us rich.” Gigerenzer illustrates with the heuristic by which we catch a ball in the air. We run after it, always keeping the angle between our line of sight and the ball constant. We thus achieve pretty much the same result as if we had done the impossible—rapidly collected an indefinite amount of data on everything affecting the ball’s flight and calculated its trajectory from Newton’s laws.

This points out the distinction between conventional wisdom and knowledge. A conventionally wise person will realize that he doesn't have to hack some algorithm to catch a ball. A knowledgeable person will realize that he can (if needed) algorithmically map a trajectory to know where the ball will land. So some would argue that, from the point of timely decision making, whether having extra knowledge makes a lot of sense. In many cases, if you have some common sense and pick the right conventional wisdom, it just might carry you in your daily business.

But then you look at the current state of financial wheeling-dealings. In no way will conventional wisdom help guide us through the atypical set of crafty financial derivatives (unless you stay away from it in the first place). Calvin Trillin wrote recently in the NY Times that the prospect of big money attracted the smartest people from the Ivy Leagues to Wall Street during the last two decades, thus creating an impenetrable fortress of opaque financial algorithms, with the entire corporate power structure on board. Trillin contrasted that to the good old days, where most people aiming for Wall St careers didn't know much and didn't actually try too hard.

I reflected on my own college class, of roughly the same era. The top student had been appointed a federal appeals court judge — earning, by Wall Street standards, tip money. A lot of the people with similarly impressive academic records became professors. I could picture the future titans of Wall Street dozing in the back rows of some gut course like Geology 101, popularly known as Rocks for Jocks.

I agree with Trillin that the knowledge structure has become inverted; somehow the financial quants empowered themselves to create a world where no one else could gain admittance. And we can't gain admittance essentially because we don't have the arcane knowledge of Wall Street's inner workings. Trillin relates:

"That’s when you started reading stories about the percentage of the graduating class of Harvard College who planned to go into the financial industry or go to business school so they could then go into the financial industry. That’s when you started reading about these geniuses from M.I.T. and Caltech who instead of going to graduate school in physics went to Wall Street to calculate arbitrage odds."
“But you still haven’t told me how that brought on the financial crisis.”

“Did you ever hear the word ‘derivatives’?” he said. “Do you think our guys could have invented, say, credit default swaps? Give me a break! They couldn’t have done the math.”

If you can believe this, it appears that the inmates have signed a rent-controlled lease on the asylum and have created a new set of rules for everyone to follow. We have set in place a permanent thermocline that separates any new ideas from penetrating the BAU of the financial industry.

I need to contrast this to the world of science, where one can ague that we have more of a level playing field. In the most pure forms of science, we accept, if not always welcome, change in our understanding. And most of our fellow scientists won't permit intentional hiding of knowledge. Remarkably, this happens on its own, largely based on some unwritten codes of honor among scientists. Obviously some of the financial quants have gone over to the dark side, as Trillin's MIT and Caltech grads do not seem to share their secrets too readily. By the same token, geologists who have sold their soul to the oil industry have not helped our understanding either.

Given all that, it doesn't surprise me that we cannot easily convince people that we can understand finance or economics or even resource depletion like we can understand other branches of science. Take a look at any one of the Wilmott papers featuring negative probabilities or Ito calculus, and imagine a quant using the smokescreen that "you can't possibly understand this because of its complexity". The massive pull of the financial instruments, playing out in what Steve Ludlum calls the finance economy, does often make me yawn in exasperation out of the enormity of it all. Even the domain of resource depletion suffers from a sheen of complexity due to its massive scale -- after all, the oil economy essentially circles the globe and involves everyone in its network.

Therein lies the dilemma: we want and need the knowledge but find the complexity overbearing. Thus the key to applying our knowledge: we should not fear complexity, but embrace it. Something might actually shake out.

COMPLEXITY

The word complexity, in short order, becomes the sticking point. We could perhaps get the knowledge but then cannot breech the wall of complexity.

I recently came across a description of the tug-of-war between complexity and simplicity when I happened across a provocative book called "The Quark and the Jaguar : Adventures in the Simple and the Complex" by the physicist Murray Gell-Mann. I discovered this book while researching the population size distribution of cities. One population researcher, Xavier Gabaix, who I believe has a good handle on why Zipf's law holds for cities, cites Gell-Mann and his explanation of power laws. Gell-Mann's book came out fifteen years ago but it contains a boat-load of useful advice for someone that wants to understand how the world works (pretentious as that may sound).

I can take a couple of bits of general advice from Gell-Mann. First, when a behavior gets too complex, certain aspects of the problem can become more simple. We can rather counter-intuitively actually simplify the problem statement, and often the solution. Secondly, when you peel the onion, everything can start to look the same. For example, the simplicity of many power-laws may work to our advantage, and we can start to apply them to map much of our current understanding². As Gell-Mann states concerning the study of the simple and complex in the preface to the book:

It carries with it a point of view that facilitates the making of connections, sometimes between facts or ideas that seem at first glance very remote from each other. (Gell-Mann p. ix)

He calls the people that practice this approach "Odysseans" because they "integrate" ideas from those who "favor logic, evidence, and a dispassionate weighing of evidence", with those "who lean more toward intuition, synthesis, and passion" (Gell-Mann p. xiii). This becomes a middle ground for Nate's intuitive cognitive (belief system) approach and my own practiced analysis. Interesting in how Gell-Mann moved from Caltech (one of Trillin's sources for wayward quants) to co-founding the Santa Fe Institute where he could pursue out-of-the-box ideas³. He does caution that at least some fundamental and basic knowledge underlines any advancements we will achieve.

Specialization, although a necessary feature of our civilization, needs to be supplemented by integration of thinking across disciplines. One obstacle to integration that keeps obtruding itself is the line separating those who are comfortable with the use of mathematics from those who are not. (Gell-Mann p.15)

I appreciate that Gell-Mann does not treat the soft sciences as beneath his dignity and he seeks an understanding as seriously as he does deep physics. He sees nothing wrong with the way the softer sciences should work in practice, he just has problems with the current practitioners and their methods (some definite opinions that I will get to later).

For now, I will describe how I use Gell-Mann and his suggestions as a guide to understand problems that have confounded me. His book serves pretty well as a verification blueprint for the way that I have worked out my analysis. As it turns out, most of what Gell-Mann states regarding complexity I happily crib from, and allows me to use an appeal to authority card to rationalize my understanding. (For this TOD post I was told to not use math and since Gell-Mann claims that his book is "comparatively non-technical", I am obeying some sort of transitive law⁴ here) As a warning, since Gell-Mann deals first and foremost in the quantum world, his ideas don't necessarily come out intuitively.

That becomes the enduring paradox -- simplicity does not always relate to intuition. This fact weighs heavily on my opinion that cheap heuristics likely will not provide the necessary ammunition that we will need to make policy decisions.

BAU (business as usual) ranks as the world's most famous policy heuristic. A heuristic describes some behavior, and a simple heuristic describes it in the most concise language possible. So, BAU says that our environment will remain the same (as Nate would say "when NOT making a decision IS making a decision"). Yet we all know that this does not work. Things will in fact change. Do we simply use another heuristic? Let's try dead reckoning instead. This means that we plot the current trajectory (as Cartwright stated) and assume this will chart our course for the near future. But we all know that that doesn't work either as it will project CERA-like optimistic and never-ending growth.

Only the correct answer, not a heuristic, will effectively guide policy. Watch how climate change science works in this regard, as climate researchers don't rely on the Farmer's Almanac heuristics to predict climate patterns. Ultimately we cannot disprove a heuristic -- how can we if it does not follow a theory? -- yet we can replace it with something better if it happens to fit the empirical data. We only have to admit to our sunk cost investment in the traditional heuristic and then move on.

In other words, even if you can't "follow the trajectory" with your eye, you can enter a different world of abstraction and come up with a simple, but perhaps non-intuitive, model to replace the heuristic. So we get some simplicity but it leaves us without a perfectly intuitive understanding. The most famous example that Gell-Mann provides involves Einstein's reduction of Maxwell's four famous equations in complexity by 1/2 to two short concise relations; Einstein accomplishes this by invoking the highly non-intuitive notion of the space-time continuum. Gell-Mann specializes in these abstract realms of science, yet uses concepts such as "coarse graining" to transfer from the quantum world to the pragmatic tactile world, with the name partially inspired by the idea of a grainy photograph (Gell-Mann p.29). In other words, we may not know the specifics but we can get the general principles, like we can from a grainy photograph.

Hence, when defining complexity it is always necessary to specify a level of detail up to which the system is described, with finer details being ignored. (Gell-Mann p.29)

The non-intuitive connection that Gell-Mann triggers in me involves the use of probabilities in the context of disorder and randomness. Not all people understand probabilities, and in particular how we apply them in the context of statistics and risk (except for sports betting of course) , yet they don't routinely get used in the practical domains that may benefit from their use. How probabilities work in terms of complexity I consider mind-blowingly simple, primarily due to our old friend Mr. Entropy.

Never mind that entropy ranks as a most anti-intuitional concept.

SIMPLICITY

Reading Gell-Mann's book, I became convinced that applying a simple model should not immediately raise suspicions. Lots of modeling involves building up an artifice of feedback-looped relationships (see the Limits to Growth system dynamics model for an example), yet that should not provide an acid test for acceptance. In actuality, the large models that work consist of smaller models built up from sound principles, just ask Intel how they verify their microprocessor designs.

My approach consists of independent research and then forays into what I consider equally simple connections to other disciplines, essentially the Odyssean thinking that Gell-Mann supports.

I would argue that the fundamental trajectory of oil depletion provides one potentially simplifying area to explore. I get the distinct feeling that no one has covered this, especially in terms of exactly why the classical heuristic, i.e. Hubbert's logistic curve, often works. So I have merged that understanding with the fact that I can use it to also understand related areas such as:

I collectively use these to support the oil dispersive discovery model -- yet it does bother me that no one has happened across this relatively simple probability formulation. You would think someone would have discovered all the basic mathematical principles over the course of the years, but apparently this one has slipped through the cracks.

Gell-Mann predicted in his book that this unification among concepts would occur if you continue to peel the onion. To understand the basics behind the simplicity/complexity approach, consider the complexity of the following directed graphs of interconnected points. Gell-Mann asks us which graphs we would consider simple and which ones we would consider complex. His answer relates to how compactly or concisely we can describe the configurations. So even though (A) and (B) appear simple and we can describe them simply, the graph in (F) borders on ridiculously simple, in that we can describe it as "all points interconnected". So this points to the conundrum of a complex, perhaps highly disordered system, that we can fortunately describe very concisely. As humans, the fact that we can do some pattern recognition allows us to actually discern the regularity from the disorder.

Figure 1: Gell-Mann's connectivity patterns.

However, what exactly the pattern means may escape us. As Gell-Mann states:

We may find regularities, predict that similar regularities will occur elsewhere, discover that the prediction is confirmed, and thus identify a robust pattern: however, it may be a pattern that eludes us. In such a case we speak of an "empirical" or "phenomenological" theory, using fancy words to mean basically that we see what is going on but do not yet understand it. There are many such empirical theories that connect together facts encountered in everyday life. (Gell-Mann p.93)

That may sound a bit pessimistic, but Gell-Mann gives us an out in terms of always considering the concept of entropy and applying the second law of thermodynamics (the disorder in an isolated system will tend to increase over time until it reaches an equilibrium). Many of the pattens such as the graph in Figure 1(F) have their roots in disordered systems. Entropy essentially quantifies the amount of disorder, and that becomes our "escape hatch" in how to simplify our understanding.

In fact, however, a system of very many parts is always described in terms of only some of its variables, and any order in those comparatively few variables tends to get dispersed, as time goes on, into other variables where it is no longer counted as order. That is the real significance of the second law of thermodynamics. (Gell-Mann p.226)

One area that I have recently applied this formulation to has to do with the distribution of human travel times. Brockmann et al reported in Nature a few years ago a scalability study that provoked some scratching of heads (one follow-on paper asked the questions "Do humans walk like monkeys?"). The data seemed very authentic as at least one other group could reproduce and better it, even though they could not explain the mechanism. The general idea, which I have further described here, amounts to nothing more than tracking individual travel times over a set of distances, and thus deriving statistical distributions of travel time by either following the cookie trails of paper money transactions (Brockmann) or cell phone calls (Gonzalez). This approach provides a classic example of a "proxy" measurement; we don't measure the actual person with sensors but we use a very clever approximation to it. Proxies can take quite a beating in other domains, such as historical temperature records, but this set of data seems very solid. You will see this in a moment.

Figure 2: Human travel connectivity patterns, from Brockmann, et al.

Note that this figure resembles the completely disordered directed graph shown by Figure 1(f). This gives us some hope that we can actually derive a simple description of the phenomenon of travel times. We have the data, thus we can hypothetically explain the behavior. As the data has only become available recently, likely no one has thought of applying the simplicity-out-of-complexity principles that Gell-Mann has described.

So how to do the reduction⁵ to first principles? Gell-Mann brings up the concept of entropy as ignorance. We actually don't know (or remain ignorant of) the spread or dispersion of velocities or waiting times of individual human travel trajectories, so we do the best we can. We initially use the hint of representing the aggregated travel times -- the macro states -- as coarse-grained histories, or mathematically in terms of probabilities.

Now suppose the system is not in a definite macrostate, but occupies various macrostates with various probabilities. The entropy of the macrostates is then averaged over them according to their probabilities. In addition, the entropy includes a further contribution from the number of bits of information it would take to fix the macrostate. Thus the entropy can be regarded as the average ignorance of the microstate within a macrostate plus the ignorance of the macrostate itself. (Gell-Mann p.220)

In the way Gell-Mann stated it, I interpret it to mean that we can apply the Maximum Entropy Principle for probability distributions. In the simplest case, if we only know the average velocity and don't know the variance we can assume a damped exponential probability density function (PDF). Since the velocities in such a function follow a pattern of many slow velocities and progressively fewer fast velocities, but with the mean invariant, the unit normalized distribution of transit probabilities for a fixed distance looks like the figure to the right (see link for derivation). To me it actually looks very simple, although people virtually never look at exponentials this way, as it violates their intuition. What may catch your eye in particular is how slowly the curve reaches the asymptote of 1 (which indicates a power-law behavior). If normal statistics acted on the velocities, the curve would look much more like a "step" function, as most of the transits would complete at around the mean, instead of getting spread out in the entropic sense.

Further since the underlying exponentials describe specific classes of travel, such as walking, biking, driving, and flying, each with their own mean, the smearing of these probabilities leads to a characteristic single parameter function that fits the data as precisely as one could desire. The double averaging of the microstate plus the macrostate effectively leads to a very simple scale-free law as shown by the blue and green maximum entropy lines I added in Figure 3.

Figure 3: Dispersion of mobility for human travel. The green line indicates agreement with a truncated Maximum Entropy estimate, and the blue dots indicate no truncation

I present the complete derivation here and the verification here. If you decide to read in more depth, keep in mind that it really boils down to a single-parameter fit -- and this over a good 5 orders of magnitude in one dimension and 3 orders in the other dimension. Consider this agreement in the face of someone trying to falsify the model; they would essentially have to disprove entropy of dispersed velocities.

It has often been empasized, particularly by the philosopher Karl Popper, that the essential feature of science is that its theories are falsifiable. They make predictions, and further observations can verify those predictions. When a theory is contradicted by observations that have been repeated until they are worthy of acceptance, that theory must be considered wrong. The possibility of failure of an idea is always present, lending an air of suspense to all scientific activity. (Gell-Mann p.78)

Further, this leads to a scale-free power law that looks exactly like the Zipf-Mandelbrot law that Gell-Mann documents, which also describes ecological diversity (the relative abundance distribution) and the distribution of population sizes of cities, from which I found Gell-Mann in the first place.

Since we invoke the name of Mandelbrot, we need to state that the observation of fractal self-similarity on different scales applies here. Yet Gell-Mann states:

Zipf's law remains essentially unexplained, and the same is true of many other power laws. Benoit Mandelbrot, who has made really important contributions to the study of such laws (especially their connection to fractals), admits quite frankly that early in his career he was successful in part because he placed more emphasis on finding and describing the power laws than on trying to explain them (In his book The Fractal Geometry of Nature he refers to his "bent for stressing consequences over causes."). (Gell-Mann p.97)

Gell-Mann of course made this statement before Gabaix came up with his own proof for city size, and obviously before I presented the variant for human travel (not that he would have read my blog or this blog in any case).

Barring the fact that it hasn't gone through a rigorous scientific validation, why does this formulation seem to work so well at such a concise level? Gell-Mann provides an interesting sketch showing how order/disorder relates to effective complexity, see Figure 4 below. At the left end of the spectrum, where minimum disorder exists, it takes very little effort to describe the system. As in Figure 1(a), "no dots connected" describes that system. In contrast, at the right end of the spectrum, where we have a maximum disorder, we can also describe the system very simply -- as in Figure 1(f), "all dots connected". The problem child exists in the middle of the spectrum, where enough disorder exists that it becomes difficult to describe and thus we can't solve the general problem easily.

Figure 4: Gell-Mann's complexity estimator. "the effective complexity of the observed system (can have) more to do with the particular observer's shortcomings than with the properties of the system observed." (Gell-Mann p.56)

So in the case of human transport, we have a simple grid where all points get connected (we can't control where cell phones go) and we have a maximum entropy in travel velocities and waiting times. The result becomes a simple explanation of the empirical Zipf-Mandelbrot Law [wiki]. The implication of all this is that through the use of cheap oil for powering our vehicles, we as humans have dispersed almost completely over the allowable range of velocities. It doesn't matter that we have one car that is of a particular brand and that an airliner is prop or jet, the spread in velocities while maximizing entropy is all that matters. Acting as independent entities, we have essentially reached an equilibrium where the ensemble behavior of human transport obeys the second law of thermodynamics concerning entropy.

Entropy is a useful concept only when a coarse graining is applied to nature, so that certain kinds of information about the closed system are regarded as important and the rest of the information is treated as unimportant and ignored. (Gell-Mann p.371)

Consider one implication of the model. As the integral of the distance-traveled curve in Figure 3 relates via a proxy to the total distance traveled by people, the only direction that the curve can go in an oil-starved country is to shift to the left. Proportionally more people moving slowly means that fewer proportionally will move quickly -- easy to state but not necessarily easy to intuit. That is just the way entropy works.

Figure 5: Assuming that human travel statistics follows the maximum entropy velocity dispersion model, a reduction in total travel will likely result in a shift as shown by the dotted blue curve.

But that does not end the story. Recall that Gell-Mann says all these simple systems have huge amounts of connectivity. Since one disordered system can look like another, and as committed Odysseans, we can make many analogies to other related systems. He refers to this process as "peeling the onion". Figuratively as one can peel a particular onion, another layer can reveal itself that looks much like the surrounding layer. I took the dispersive travel velocities way down to the core of the onion in a study I did recently on anomalous transport in semiconductors.

Often in physics, experimental observations are termed "anomalous" before they are understood.

-- Richard Zallen, "The physics of amorphous solids", Wiley-VCH, 1998

If you can stomach some serious solid-state physics take a peek at the results -- it's not like you will see the face of Jesus, but the anomalous behavior does not seem so anomalous anymore. Like Gell-Man states, these simple ideas connect all the way through the onion to the core.

SCALING

The big sweet Vidalia onion that I want to peel is oil depletion. All the other models I work out indirectly support the main premise and thesis. They range from the microscopic scale (semiconductor transport) to the human scale (travel times) and now to the geologic scale. I assert that in the Popper sense of falsifiability, one must disprove all the other related works to disprove the main one, which amounts to a scientific form of circumstantial evidence, not quite implying certainty but substantiating much of the thought process. It also becomes a nerve-wracking prospect; if one of the models fails, the entire artifice can collapse like a house of cards. Thus the "air of suspense to all scientific activity" that Gell-Mann refers to.

So consider rate dispersion in the context of oil discovery. Recall that velocities of humans become dispersed in the maximum entropy sense. Well, the same holds for prospecting for oil. I suggest that like human travel, all discovery rates have maximum dispersion subject to an average current-day-technology rate.

A real eye-opener to me occurred when I encountered Gell-Mann's description of depth of complexity. I consider this a rather simple idea because I had used it in the past, actually right here on TOD (see the post Finding Needles in a Haystack where I called it "depth of confidence"). It again deals with the simplicity/complexity duality but more directly in terms of elapsed time. Gell-Mann explains the depth of complexity by invoking the "monkeys typing at a typewriter" analogy. If we set a goal for the monkeys to type out the compleat works of Shakespeare, one can predict that due solely to probability arguments they would eventually finish their task. It would look something like the following figure with the depth D representing a crude measure of generating the complete string of letters that comprises the text.

Figure 6: Gell-Mann's Depth (d) is the cumulative Probability (P) that one can
gain a certain level of information within a certain Time (T).

No pun-intended, Gell-Mann coincidentally refers to D as a "crude complexity" measure; I use the same conceptual approach to arrive at the model of dispersive discovery of crude oil. The connection invokes the (1) dispersion of prospecting rates (varying speeds of monkeys typing at the typewriters) and (2) a varying set of sub-volumes (different page sizes of Shakespeare's works). Again, confirming the essential simplicity/complexity duality, the fact that we see a connectivity lies more in the essential simplicity in describing the disorder than anything else.

The final connection (3) involves the concept of increasing the average rate of speed of the typewriting monkeys over a long period of time. We can give the monkeys faster tools without changing the relative dispersion in their collective variability⁶. If this increase turned out as an exponential acceleration in typing rates (see Figure 10), the shape of the Depth curve would naturally change. This idea leads to the dispersive discovery sigmoid shape -- as our increasing prospecting skill analogizes to a speedier version of a group of typewriting monkeys. See the figure to the right for a Monte Carlo simulation of the monkeys at work [link].

It doesn't matter that we have one oil reservoir that has a particular geology and that this somehow deflects the overall trajectory, as we would have to if we considered a complete bottom-up accounting approach. I know this may disturb many of the geologists and petroleum engineers who hold to the conventional wisdom about such pragmatic concerns, but that essentially describes how a thinker such as Gell-Mann would work out the problem. The crude complexity suggests that we turn technology into a coarse grained "fuzzy" measurement and accelerate it to see how oil depletion plays out. So if you always thought that the oil industry essentially flailed away like various monkeys at a typewriter, you would approximate the reality more so than if you believed that they followed some predetermined Verhulst-generated story-line. So this model embraces the complexity inherent of the bottom-up approach, but ignoring the finer details and dismissing out of hand that determinism plays a role in describing the shape.

Luis de Sousa gives a short explanation of how the deterministic Verhulst equation leads to the Logistic here and it remains the conventional heuristic wisdom that one will find on wikipedia concerning the Hubbert Peak Oil curve. However, Verhulst generated determinism does not make sense in a world of disorder and fat-tail statistics, as only stochastic measures can explain the spread in discovery rates. This becomes the mathematical equivalent of "not seeing the forest for the trees". Pragmatically, the details of the geology do not matter, just like the details of the car or bicycle or aircraft you travel in does not matter for modeling Figure 3.

This approach encapsulates the gist of Gell-Mann's insights on gaining knowledge from complex phenomena. His main idea is the astounding observation that complexity can lead to simplicity. I am starting to venture onto very abstract ice here, but the following figure represents where I think some of the models reside on the complexity mountain.

Figure 7 : Abstract representation of our understanding of resource depletion.

Notice that I place the "Limits to Growth" System Dynamics model right in the middle of the meatiest complexity region. That model has perhaps too many variables and so will mine the swamps of complexity without adding much insight (or in more jaded terms, any insight that you happen to require). Many people assume that the Verhulst equation, used to model predator-prey relationships and the naive Hubbert formula of oil depletion, is complex since it describes a non-linear relation. However the Verhulst actually proves too simple, as it includes no disorder, and doesn't really explain anything but a non-linear control law. The only reason that it looks like it works is that the truly simple model has a fortuitous equivalence to the simplified-complex model⁷, which exists as the dispersive discovery model on the other right-hand side of the spectrum. On the other hand, consider that the export land model (ELM) remains simple and starts to include real complexity, approaching the bottom-up models that many oil depletion analysts typically use.

Further to the left, I suggest that the naive heuristics such as BAU and dead reckoning don't fit on this chart. They assume an ordered continuance of the current state, yet one can't argue heuristics in the scientific sense as they have no formal theory to back them up⁸. The complementary effect way to the right suggests enough disorder that we can't even predict what may happen, the so-called Black Swan theory proposed by Taleb.

On the bulk of the right side, we have all the dispersive models that I have run up the flag-pole for evaluation. These all basically peel the onion, and follow Gell-Mann's suggestion that all reductive fundamental behaviors will show similarities at a coarse graining level. This includes one variation that refer to as the dispersive aggregation model for reservoir sizing. This has some practicality for estimating URR and it comes with its own linearization technique along the same lines as Hubbert Linearization (HL). You may ask if this is purely an entropic system, why would reservoirs become massive?

Sometimes people who for some dogmatic reason reject biological evolution try to argue that the emergence of more and more complex forms of life somehow violates the second law of thermodynamics. Of course it does not, any more than the emergence of more complex structures on a galactic scale. Self-organization can always produce local order. (Gell-Mann p.372)

Gell-Mann used the example of earthquakes and the relative scarcity of very large earthquakes to demonstrate how phenomenon can appear to "self-organize". Laherrere has used a parabolic fractal law, a pure heuristic to model the sizing of reservoirs (and eathquakes), whereas I use the simple dispersive model as shown below.

Figure 8 : Dispersed velocities suggests a model of aggregation, much like Gabaix suggests for aggregation of cities. Very few large reservoirs and many small ones, just as in the distribution of cities.

These dispersive forms all fit together tighter than a drum. That essentially explains why I think we can use simple models to explain complex systems. I admit that I have tried to take this to some rather unconventional analogies, yet it seems to still work. I keep track of these models on this here blog.

Figure 9 : Popcorn popping kinetics follows the same dispersive dynamics [link].

DISCUSSION

I found many other insights in Gell-Mann's book that expand the theme of this post and so seem worthwhile to point out. I wrote this post with the intention of referencing Gell-Mann heavily because many of the TOD comments in the past have criticized not incorporating a popular science angle to the discussion. I consider Gell-Man close to Carl Sagan in this regard (w/o the "billions" of course). I essentially used the book as an interactive guide, trying to follow his ideas by comparing them to models that I had worked on.

Evidently, the main function of the book is to stimulate thought and discussion.

Running through the entire text is the idea of the interplay between the fundamental laws of nature and the operation of chance. (Gell-Mann p.367)

The role of chance, and therefore probabilities, seems to rule above all else. Not surprising from a quantum mechanic.

Gell-Mann has quite a few opinions on the state of multi-disciplinary research, with interesting insight in regards to different fields of study. He treats the problems seriously as he believes certain disciplines have an aversion to accommodating new types of knowledge. And these concerns don't sit in a vacuum, as he spends the last part of the book discussing sustainability and ways to integrate knowledge to solve problems such as resource depletion.

The lnformational Transition

Coping on local, national, and transnational levels with environmental and demographic issues, social and economic problems, and questions of international security as well as the strong interactions among all of them, requires a transition in knowledge and understanding and in the dissemination of that knowledge and understanding. We can call it the informational transition. Here natural science, technology behavioral science, and professions such as law, medicine, teaching, and diplomacy must all contribute, as, of course, must business and government as well. Qnly if there is a higher degree of comprehension, among ordinary people as well as elite groups, of the complex issues facing humanity is there any hope of achieving sustainable quality.

It is not sufficient for that knowledge and understanding to be specialized. Of course, specialization is necessary today But so is the integration of specialized understanding to make a coherent whole, as we discussed earlier. It is essential, therefore, that society assign a higher value than heretofore to integrative studies, necessarily crude, that try to encompass at once all the important features of a comprehensive situation, along with their interactions, by a kind of rough modeling or simulation. Some early examples of such attempts to take a crude look at the whole have been discredited, partly because the results were released too soon and because too much was made of them. That should not deter people from trying again, but with appropriately modest claims for what will necessarily be very tentative and approximate results.

An additional defect of those early studies, such as Limits to Growth, the first report to the Club of Rome, was that many of the critical assumptions and quantities that determined the outcome were not varied parametrically in such a way that a reader could see the consequences of altered assumptions and altered numbers. Nowadays, with the ready availability of powerful computers, the consequences of varying parameters can be much more easily explored. (Gell-Mann p. 362)

Gell-Mann singles out geology, archaelogy, cultural anthropology, most parts of biology for criticism, and many of the softer sciences, not necessarily because the disciplines lack potential, but because they suffer from some massive sunk-cost resistance to accepting new ideas. He gives the example of distinguished members of the geology faculty of Caltech "contemptuosly rejecting the idea of continental drift" for many years into the 1960's (Gell-Mann p. 285). This extends to beyond academics, as I recently I came across some serious arguments about whether geologists actually understand the theory behind geostatistics and the use of a technique called "kriging" to estimate mineral deposits from bore-hole sampling (just reporting the facts). And then Gell-Mann relates this story on practical modeling within the oil industry:

Peter Schwartz, in his book "The Art of the Lonq Wew", relates how the planning team of the Royal Dutch Shell Corporation concluded some years ago that the price of oil would soon decline sharply and recommended that the company act accordingly The directors were skeptical, and some of them said they were unimpressed with the assumptions made by the planners. Schwartz says that the analysis was then presented in the form of a game and that the directors were handed the controls, so to speak, allowing them to alter, within reason, inputs they thought were misguided. According to his account, the main result kept coming out the same, whereupon the directors gave in and started planning for an era of lower oil prices. Some participants have a different recollections of what happened at Royal Dutch Shell, but in any case the story beautifully illustrates the importance of transparency in the construction of models, As models incorporate more and more features of the real world and become correspondingly more complex, the task of making them transparent, of exhibiting the assumptions and showing how they might be varied, becomes at once more challenging and more critical. (Gell-Mann p. 285)

Trying to understand why some people tend to a very conservative attitude, Gell-Mann has an interesting take on the word "theory" and the fact that theorists in many of these fields get treated with little respect.

"Merely Theoretical" -- Many people seem to have trouble with the idea of theory because they have trouble with the word itself, which is commonly used in two quite distinct ways. On the one hand, it can mean a coherent system of rules and principles, a more or less verified or established explanation accounting for know facts or phenomena. On the other hand, it can refer to speculation, a guess or conjecture, or an untested hypothesis, idea or opinion. Here the word is used with the first set of meanings, but many people think of the second when they hear "theory" or "theoretics". (Gell-Mann p.90)

Unfortunately, I do think that this meme that marginalizes peak oil "theory" will gain momentum over time. Particularly, in terms of whether peak oil theory has any real formality behind it, as certainly no one in academic geology besides Hubbert⁹has really addressed the topic. Gell-Mann suggests that many disciplines simply believe that they don't need theorists. TOD commenter SamuM provided some well-founded principles to consider when mounting a theoretical approach, especially in responding to countervaling theories, i.e in debunking the debunkers. I am all for continuing this as a series of technical posts. ¹⁰

In the field of economics, Barry Ritholtz has also recently suggested a more scientific approach in The Hubris of Economists, yet he doesn't think that modeling necessarily works in economics (huh?). He might well consider that economics and finance modeling assumes absolutely no entropic dispersion. Taleb suggests that they should include fat-tails. The amount of effort placed in applying normal statistics has proven out as a colossal failure. We get buried daily in discussions on how to best to generate a course-correction within our economy, balanced between a distinct optimism and a bleak pessimism. At least part of the pessimism stems from the fact that we think the economy will forever stay coveniently complex beyond our reach. I would suggest that simple models may help just as well and that it allows us to understand when non-cheap heuristics and complex models woork against our best interests (i.e. when we have been played).

The "cost of information" addresses the fact that people may not know how to make reasonable free market decisions (for instance about purchases) if they don't have the necessary facts or insights. (Gell-Mann p.325)

Above all, Gell-Man asks the right questions and provides some advice on how to move forward..

If the curves of population and resource depletion do flatten out, will they do so at levels that permit a reasonable quality of human life, including a measure of freedom, and the persistence of a large amount of biological diversity, or at levels that correspond to a gray world of scarcity, pollution, and regimentation, with plants and animals restricted to a few species that co-exist easily with mankind? (Gell-Mann p.349)

We are all in a situation that resembles a fast vehicle at night over unknown terrain that is rough, full of gullies, with precipices not far off. Some kind of headlight, even a feeble and flickering one, may help to avoid some of the worst disasters. (Gell-Mann p.366)

I hope that I have illustrated how I have attempted to separate the simple from the complex. If this has involved too much math, I apologize.

I admit that we still don't understand econ though.

References

Murray Gell-Mann, "The Quark and the Jaguar : Adventures in the Simple and the Complex", 1995, Macmillan

Calvin Trillin, "Wall Street Smarts", NY Times, October 13, 2009

Nassim Nicholas Taleb, "The Black Swan: The Impact of the Highly Improbable", 2007, Random House

D. Brockmann, L. Hufnagel & T. Geisel, "The scaling laws of human travel", Nature, Vol 439|26, 2006.

Marta C. González, César A. Hidalgo & Albert-László Barabási,, "Understanding individual human mobility patterns", Nature, Vol 453, 779-782 (5 June 2008).

Figure 10: A damped exponential contains a maximum entropy amount of information, such as the decay of radioactive material. The rising exponential usually occurs due to a degree of feedback reinforcing some effect, such as technology advances.

Footnotes

¹The TV pundit Chris Matthews regularly asks his guests to "tell me something I don't know". That sounds reasonable enough until you realize that it would require beyond mind reading.

²Power laws are also fat-tail laws, which has importance wrt Black Swan theory.

³See this Google video of Gell-Mann in action talking about creative ideas. Watch the questions at the end where he does not suffer fools gladly.

⁴Unfortunately the transitive law is a mathematical law which means that we can never escape math.

⁵In terms of coarse graining, explaining the higher level in terms of the lower is often called "reduction".

⁶In marathon races, the dispersion in finishing times has remained the same fraction even as the winners have gotten faster

⁷See this link with regard to Fermi-Dirac statistics. That also looks similar but comes about through a different mechanism.

⁸Excepting perhaps short-term Bayes. Bayesian estimates use prior data to update the current situation. BAU is a very naive Bayes (i.e. no change) whereas dead reckoning is a first order update, the derivative.

⁹Who was more of a physicist and did it more out of curiosity than anything else.

¹⁰I also see many TOD comments that show analogies to other phenomena that basically don't hold any water at all. We do need to continue to counter these ideas if they don't go anywhere, as they just add to the noise

How did we ever survive?

2009-11-13T23:11:00.000-08:00

Geostatistics a fraud?

2009-10-28T19:08:00.000-07:00

The academic field of "economic geology" seems to cover the exploitation of resources rather than stewardship of the resources [TOD link]. If somebody in this research area had actually wanted to or had the charter to, they could have done an impartial study of resource depletion based on some rather simple models. The fact that a branch of this field is called "geostatistics" makes one think that somebody must be doing original research. But looking at the Wiki entry for geostatistics and the Wiki entry for the "father" of geostatistics, Matheron, you find lots of controversy.

And then, following the links, you also find that a web-site exists with the URL http://www.geostatscam.com, subtitled "Geostatistics: From human error to scientific fraud" . The engineer who runs the site, Jan Merks, makes the strong claims that the specialty of geostatistics consists of "voodoo statistics" and "scientific fraud". Granted, it looks like this criticism resides mainly in the context of mineral mining and perhaps petroleum extraction is not really a part of this field, but it makes you wonder what exactly constitutes research in geostatistics. A sample of Merks' charges:

Degrees of freedom fighters amongst professional engineers and geoscientists are addicted to Matheron's junk science of geostatistics. Hardcore krigers and cocksure smoothers turned mineral exploration into a game of chance with the stakes of mining investors.

From what I understand about the technique known as "kriging", it seems to provide a way to interpolate potential mineral deposits from sampled spatial measurements in surrounding areas. The criticism leveled on this technique rests on the observation that geostatisticians never want to provide a sound estimate of the variance of their interpolation. According to Merks, they at best create a phony variance to justify their estimate. If this is indeed so, I would agree that we should seriously look into what weird interpolations geostatistics considers as science.

As a counter-example, in all the models I use for oil depletion, I always use a huge variance consistent with the Maximum Entropy Principle. An assumed variance such as this maximizes the disorder, and provides the best estimate for reasoning under uncertainty. In other words, if you don't know the variance, you have too assume the worst case. The bad kriging estimates seem to basically say that if point A has X mineral grade and point B has Y mineral grade, then point C halfway between A and B has (X+Y)/2 mineral grade. From what I understand, unless you have a real spatial correlation between the points (which apparently no one verifies) no way can you do a naive interpolation like this. I have written papers on spatial correlation and always start by looking for the presence or lack of statistical independence between separated points.

So if geostatistics treats probability and statistics like a cook interpolates half-a-cup of bleached flour, the field has serious problems. It brings up the rather obvious question, why do they teach this? Do they realize the sad fact that exploration works at best as a pure crap-shoot and you might as well get the fresh eager graduates out in the field with nothing more than a pair of dice? Are the rockhead geologists that jaded?

H G Wells predicted that statistical thinking would become as important as the ability to read and write. Would Wells have embraced the nouveau pseudo science of interpolation without justification with the same unbridled passion as the world's mining industry did? [from Merks website]

Kernel of Logistic for Dispersive Discovery

2009-10-27T19:11:00.000-07:00

The derivation of the dispersive discovery curve builds from a premise that individual discovery trajectories act on an ensemble of subvolumes within a larger volume (for example, the earth). Although the average rate of these arcs show an exponential acceleration over time, the individual rates get dispersed according to the Maximum Entropy Principle. The subvolumes also show an exponential PDF of sizes corresponding to the disorder expected in the geological search sizes of varying oil-bearing regions around the earth. Once the cumulative search in a subvolume completes, it reaches an asymptotic value proportional to the amount of oil in that region.

The following figure shows several Monte Carlo (MC) simulations of the build-up of the logistic Dispersive Discovery profile. This set of charts features only 100 sample traces accumulated per solid black line, yet the characteristic S-shape shows up clearly. The dashed curve behind the MC cumulative represents the logistic sigmoid function corresponding to theory.

In each of these cases, a gradual build-up of the individually colored profiles lead to nearly identical characteristic cumulative curves (save for counting noise).

I used a spreadsheet for this simulation as the MC algorithm reduces to a short algorithm. In the following formula, two exponential random variates sit in cells D$1 (growth dispersion) and D$2 (subvolume dispersion). An incremental time runs along column $A. The SIGN function truncates the individual cumulatives to truncate when the search reaches the subvolume limit. The term $A$1 represents the scaling term for the characteristic dispersion velocity spread. The last term acts like an integrator for $D$4 from the previous time step value D3.

+D$1*(SIGN(D$2-(D$1*$A4)/$A$1)+1)/2/$A$1+D3

If the time/$A series shows a linear growth in time, the summation of a series of these results in a hyperbolic asymptotic growth. If the time accelerates, we get the logistic form.

Each of the subvolume traces represent a discovery kernel function similar to a shocklet kernel. The physical realization of this becomes very intuitive, as prospecting for oil requires a complete survey of an entire subvolume, with the search accelerating over long periods of time as new technologies and tools become available. On the other hand, searches over very small regions may not have the aid of accelerated search and so the hyperbolic asymptotic growth results. This gives us the characteristic creaming curve in smaller basins. If you can understand when accelerating search applies and when linear search applies, it becomes very easy to predict what type of profile will occur in a region.

One thing I notice in the MC simulation is that the long term noise becomes greater as fewer discoveries contribute to the tail end of the curve. This may get reflected in the real statistics we will see as rather sparse fat-tail discoveries contribute to a long term decay.

A commenter at TOD recommended this visual as an aid to understanding how dispersion works.

Creep Failure

2009-10-24T08:41:00.000-07:00

As a follow-up to the dispersive failure analysis I found a potential behavior that illustrates the early failure profile in a less abstract and perhaps more realistic manner.

Due to dispersion, a slow linear rate of growth in the breakdown process will lead to an early spike in the failure (or hazard) rate. This becomes the characteristic leading downward slope in the bathtub curve.

Figure 1: Linear into accelerating growth function

The enhanced early probability of failure arises purely through the spread in the failures through disorder mechanisms just as in the popcorn popping experiment. This in general leads to a bathtub shape with an analogy to the life-span of the human body (infant mortality, maturity, old age). For mechanical equipment, the shape in (b) provides a more realistic portrayal according to some.

Figure 1: Empirically observed bathtub curves
(a) electronics components (b) mechanical components

I earlier had abstracted away the velocity of the failure stimulus and integrity of the component to generate a parametric expression for the failure rate.

r(t) = dg(t)/dt / (tau + g(t))

In a real situation the velocity/integrity abstraction might occur as a stress/strain pairing particularly in a mechanical component.

We scale the integrity of the component as a physical dimension; it could be a formally defined measure such as strain, but we leave it as an abstract length for the sake of argument. The process acting on this abstraction becomes a velocity; again this could be a real force, such as the real measure of stress. [link]

In this case, a mechanism called creep can play a big role in determining the failure dynamics. Creep happens to a load under a constant stress condition over a period of time. This leads to a curve as shown below, which demonstrates a relatively quick rise in strain (i.e. deformation) before entering a linear regime and then an exponential as the final wear-out mechanism becomes too great.

Figure 3: Creep curve which physically realizes the growth function of Figure 1.

Although the stress/strain relationship can get quite complex, to first-order, we can compare the curve in Figure 3 to the general curve in Figure 1. The monotonic increase in the abstract growth term, g(t), remains the same in both cases, with both a linear and exponential regime noticeable in the middle (secondary) and late (tertiary) periods. The big difference lies in the early (primary) part of the curve, where due to elastic and plastic deformation (particularly in a viscoelastic material) the growth increases rapidly before settling into the linear regime. This fast "settling-in" regime intuitively provides a pathway to an earlier failure potential than a purely accumulating process would.

One can approximate the general trend in the primary part of the growth either by a rising damped exponential or by a parabolic/diffusive growth that rises with the n'th root of time. The following figure uses a combination of a square root and the exponential growth to model the creep growth:

g(t)= A*sqrt(kt) + B*(e^-ct -1)

Figure 4: Modeled creep growth rate

Applying this growth rate to the dispersive failure rate, g(t), we get the following bathtub curve.

Figure 5: Bathtub curve for growth rate exhibiting physical creep has a sharper initial failure rate fall-off due to earlier deformation.

Clearly, the early part of the curve becomes accentuated relative to the linear growth mechanism, and a more asymmetric v-shape results, characteristic of the mechanical failure mode of Figure 1(b).

References:

The physics of creep: creep and creep-resistant alloys, F. Nabarro, H. De Villiers, CRC Press,1995.

Verifying Dispersion in Human Mobility

2009-10-22T21:42:00.001-07:00

Another article in Nature supports the model I put together for human mobility patterns
http://mobjectivist.blogspot.com/2009/10/scaling-laws-of-human-travel.html

My model seems to match the observed trends even more precisely and further reinforces the fundamental idea of entropic dispersion of travel velocities. Instead of using a paper money tracking system as in the previous Brockmann article, the authors (Gonzalez et al) used public cell-phone calling records -- this seems to perform more directly rather than the indirect mechanism of proxy records of bill tracking to monitor human mobility.

Given that money is carried by individuals, bank note dispersal is a proxy for human movement, suggesting that human trajectories are best modeled as a continuous time random walk with fat tailed displacements and waiting time distributions.
...
Contrary to bank notes, mobile phones are carried by the same individual during his/her daily routine, offering the best proxy to capture individual human trajectories.
...
Individuals display significant regularity, as they return to a few highly frequented locations, like home or work. This regularity does not apply to the bank notes: a bill always follows the trajectory of its current owner, i.e. dollar bills diffuse, but humans do not.

Even though the proxy records give the same general fat-tail trends, the essential problem with the bank note process remains the transaction process. The very likely possibility exists that a dollar bill exchanges hands among three unique individuals at a minimum between reporting instances, yet the cell-phone records an individual at more randomized and therefore less deterministic intervals.

So I don't expect the average rates of travel to necessarily agree between the two data-sets.

The following fit uses the same model as I used previously, with data sampled at 1 week intervals. Notice that the data fits the Maximum Entropy Dispersion (the green curve) even better than bill tracking at 10 day intervals.

dP/dr = beta*t/(beta*t+r)^2

The value of beta for this data set is 0.36 instead of 1 for the bill dispersion data set. I placed a cutoff on the dispersion by preventing a smearing into faster rates of 400 km/day and above, but this seems fairly reasonable as the model otherwise works over 5 orders of magnitude. It actually works so well that it detects a probability offset in the original data calibration; the probability PDF should sum to one over the entire interval yet the Gonzalez data exhibits a bias as it creeps up slightly over the normalized curve. This is real as their own heuristic function (when I took the time to plot it) also shows this bias.

Another figure that Gonzalez plots mines the data according to a different sampling process, yet the general trend remains.

Moreover, the researchers almost got the Maximum Entropy dispersion function right by doing a blind curve fit, but ultimately could not explain it. Instead of the predicted power-law exponent of -2, they use -1.75. Yet since they do not normalize their curve correctly with the beta parameter that a probability distribution requires, they think this value of -1.75 holds some significance. Instead the -1.75 power law is likely an erroneous fit and -2 works better -- while not violating Occam's law.

Entropy always wins out on these phenomena and it really tells us how (in the sense of having no additional information, i.e. a Jaynesian model of entropy) people will statistically use different forms of transportation. The smearing occurs over such a wide range because people will walk, bicycle, residentially drive, freeway drive, or take air transportation. The entropy of all these different velocities serves to generate the dispersion curve that we empirically observe. The fact that it takes such little effort to show this with a basic probability model truly demonstrates how universal the model remains. The bottom line is that a single parameter indicating an average value of dispersive velocity is able to map over several orders of magnitide; only a second-order correction having as much to do with the constrained physical breadth of the USA and how fast people can ultimately travel in a short period of time prevents a complete "single parameter" model fit.

Ultimately, what I find interesting is how the researchers in the field seem to flail about trying to explain the data with non-intuitive heuristics and obscure random walk models. Gonzalez at al have gotten tantalizingly close to coming up with a good interpretation, much closer than Brockmann in fact, yet they did not quite make the connection. If I could tell them, I would hint that their random walk is random but the randomness itself is not randomized. That explains so many phenomena yet they can't quite grasp it.

References:

Understanding individual human mobility patterns, Marta C. González, César A. Hidalgo & Albert-László Barabási, Nature 453, 779-782 (5 June 2008)

There is an Addendum (12 March 2009) associated with this document.

The Scaling Laws of Human Travel

2009-10-17T07:23:00.000-07:00

I spotted a fairly recent-vintage paper on the statistics of human travel published in Nature magazine. This had the novelty of using the internet to collect indirect travel information by compiling the spread of money, via the bill tracking system http://www.whereisgeorge.com.

I can't argue the basic premise behind the approach. The authors, Brockmann, et all, assume that people disperse throughout a geographic region and make the claim that you can essentially track their motion (indirectly) by following the trajectory of the money they carry. They also make the correct interpretation that dispersion via random walk mechanisms plays a big role in the spread. That part seems quite reasonable. And the utility of understanding this phenomena holds great promise. For example, one can use it to understand the implications of reducing the travel overhead as we enter an energy-scarce era, as well as understanding the dynamics of pandemics. Yet, considering how important the concept is and the prestige of the journals that publish the work, they completely hose up the mathematics behind the phenomena. By that I mean they made the result horribly complex and opaque.

If they had used the correct formulation of dispersion, the agreements to their premise would have shown a very simple universality; yet they invoke some sophisticated notion of Levy flights and other esoteric models of stochastic mathematics to derive an overly complex result. Eventually they come up with a scaling law exponent which they affix with the value 0.59 and 1.05. They claim these odd numbers holds some notion of "universality" and claim that this results in some new form of "ambivalent" process. It seems a bit pretentious and I will use this post to derive some actual, much more practical, laws. For context, my own arguments use some of the same concepts that I have used for Dispersive Discovery of oil.

The statistics of what they want to model stems from sets of collected data from timestamped geographical locations. The figure below shows typical vector traces of travel across the USA.

Figure 1 : Typical bill travel traces due to Brockmann

The collected information appears very comprehensive and the fact that everyone uses money, attests to the fact that the approach should show little bias with minimal sample error. I don't have the data directly in my hands so some parts I may have misinterpreted, but so far so good.

We part company from that point. I don't really care to know the sophisticated random-walk model they use but it appears pretty much derived from the Scher-Montrose Continuous Time Random Walk (CTRW) model that I have looked at previously (see
http://mobjectivist.blogspot.com/2009/06/dispersive-transport.html and notice how that problem on a microscopic scale is similarly overly complicated)

Instead of overt formality, I would set up the mathematical premise straightforwardly. We have a dispersion of velocities described by a maximum entropy probability distribution function (PDF).

p(v) = alpha*e^-alpha*v

The PDF above describes a mean velocity with a standard variance equal to the mean. This places all moments as finite values and becomes an intuitive minimally biased estimate considering no further information is available. We next assume that the actual distance traveled happens over a period of time.

The authors first describe a PDF of money traversing a distance r in less than 4 days.

p(r | t less than 4 days) = probability that the bill traveled r distance in less than 4 days

This becomes very easy to solve (if we assume that the distance traveled is uniformly distributed across the interval -- another maximum entropy estimator for that constraint). So for any one time, the cumulative distance traveled at any one time is expressed by the following cumulative distribution function (CDF), with r scaled as a distance:

P(r,t) = e^-alpha*r/t

The term alpha has the units of time/distance. As we haven't considered the prospects of waiting time variation yet, this point provides a perfect place to add that to the mix. Intuitively, money does not constantly stay in motion, but instead may sit in a bank vault or a piggy bank for long periods of time. So we smear the term alpha again with a maximum entropy value.

p(alpha) = beta*e^-beta*alpha

But we still have to integrate this over all smeared values, with respect to P(r,t). This gives the equation

P(r,t) = beta/(beta + x/t)

To generate the final PDF set, we take the derivative of this equation with respect to time, t, to get back the temporal PDF, and with respect to r, to get the spatial PDF (ignoring any radial density effects) .

dP/dt = beta*r/(beta*t+r)^2
dP/dr = beta*t/(beta*t+r)^2

So we can generate a PDF for r for a specific value of T=4 days and then fit the curve to a value for beta. So beta becomes the average velocity/waiting time for a piece of paper money (the bank note the authors refer to) between the times it makes trips from place to place. This approach only differs from the use of dispersion for oil discovery in the fact that velocity shows a greater dispersion than it would otherwise; the fact that time acts as the dispersive parameter and this goes in the denominator of velocity=r/t, implies that the fat tails appear in both the spatial dimension and the temporal dimension.

Figure 2: Contour profile in spatial/temporal dimensions for
dispersion of money. High levels and hot colors are high probability.

To verify the model, the authors plot the results in two orthogonal dimensions. First against time:

P(t | r less than 20 kilometers) = The probability that a bill has traveled 20 km or less in a set time.

The constrained curve fits are shown below with beta set to 1 kilometer per day. The various sets of data refer to 3 classes of initial entry locations (metropolitan areas, intermediate cities, and small towns).

Figure 3a and 3b: Profiles of probability along two orthogonal axis.
The red lines are fits for dispersion with beta = 1.

The short-distance spatial curve has some interesting characteristics. From the original paper, a plot of normalized probabilities shows relative invariance of distance traveled with respect to time for short durations of less than 10 days.

Figure 4: Contour profile from Brockmann.

Time appears bottle-necked for around 10 days on the average. This is understandable as the waiting time between transactions can contain 5 distinct stages. As most of these transactions may take a day or two at the minimum, it is easy to conceive that the total delay is close to T=10 days between updates to the bill reporting web site. So this turns into a weighted hop invariant to time but scaled to reflect the average distance that the money would actually travel.

Figure 5: Latency at short time intervals has to go through 5 processing steps.
This is similar to what occurs in oil processing, see here.

If we plot our theory on a similar color scale, it looks like the 2-dimensional profile below. :

Figure 6: Contour plot of scaled probability values.
This shows good agreement with the Brockmann data in Figure 4.

Note that these look different than the first contour color scale in Figure 2 since the authors multiplied the probability values by r to maintain a similar dynamic range across all values. Otherwise the probability values would diminish at large r or t.

Figure 7: Alternate perspective of Figure 6. Note that the scaling by r
keeps the dynamic range intact, in comparison to Figure 2.

The bottom-line is that the actual model remains simple and both CTRW and the scaling-law exponent do not come into play. They also curiously called the process ambivalent (what the heck does that even mean?). I consider it the opposite of ambivalent and the solution remains rather straightforward. The authors Brockmann and company simply made a rookie mistake and did not consider Occam's razor, in that the simplest explanation works the best. Consider this par for the course when it comes to understanding dispersion.

As far as using the simple model for future policy decisions, I say why not. It essentially features a single parameter and covers the entire range of data. One can use it for modeling the propagation of infectious diseases (a depressing topic) and trying to minimize travel (an optimistic view).

I would almost say that case closed and problem solved, yet this is such a simple result that I have concerns I might have missed something. I definitely ignored the diffusional aspects and the possibility of random walk in 2-dimensions, yet I believe these largely get absorbed in the entropic smearing and the USA is not as much of a random two-dimensional stew as one may imagine. But as with all these simple dispersion arguments, I get the feeling that somehow this entire analysis approach has been unfortunately overlooked over the years.

References

"The scaling laws of human travel", D. Brockmann, L. Hufnagel & T. Geisel, Nature, Vol 439|26, 2006.

Failure is the complement of success

2009-10-08T23:11:00.000-07:00

Alternate title: Solving the slippery nature of the bathtub curve.

To reset the stage, I think I have a fairly solid model for oil discovery. The basic premise involves adding a level of uncertainty to search rates and then accelerating the mean through a volume of search space. This becomes the Dispersive Discovery model.

As I started looking at dispersion to explain the process of oil discovery, it seemed likely that it would eventually lead to the field of reliability. You see, for every success you have a failure. We don't actively seek a failure, but they lie ready to spring forth at some random unforeseen time. In other words, we can never predict when a failure occurs; just as when we look for something at random -- like an oil reservoir, we will never absolutely know when we will find it.

So the same dispersion in search rates leading to a successful oil find also leads to the occurrence -- in that same parallel upside-down universe -- of a failure. As we saw in the last post, what is the seemingly random popping of a popcorn kernel but a failure to maintain its hard shell robustness? And by the same line of reasoning, what is a random discovery but a failure by nature to conceal its inner secrets from an intrepid prospector?

The Classic Failure Premise: The classic approximation for a random failure involves a single parameter, the failure rate r. This gets derived at least empirically from the observation that if you have a pile N of working components, then the observed failure rate goes as:

(EQ 1)
dN/dt = -rN

so the rate of loss relative to the number operational remains a constant throughout the aggregated lifetime of the parts. The solution to the differential equation is the classic damped exponential shown below:

Figure 1: The classical failure rate gives a damped exponential over time.

Now this works very effectively as a first-order approximation and you can do all sorts of reliability studies with this approximation. For example it matches the conditions of a Markov process, and the fact that it lacks memory means that one can solve large large sets of coupled equations (this has application in the oil shock model, but that is another post).

Deviation from the Classical Premise: However, in reality the classic approximation doesn't always hold. As often observed, the failure rate of a component does not remain constant if measured empirically over a population. Instead the shape over time ends up looking something like a bathtub.

Figure 2: The Bathtub Curve showing frequent early failures and late failures.

One can see three different regimes over the life-cycle of a component. The component can either fail early on as a so-called "infant mortality", or it can fail later on randomly (as a lower probability event), or eventually as a process of wear-out. Together, the three regimes when pieced together form the shape of a bathtub curve. Curiously, a comprehensive theory for this aggregated behavior does not exist (some even claim that a unified theory is impossible) and the recommended practice suggests one create an analysis bathtub curve in precisely a piece-wise fashion. Then the analyst can predict how many spares one would need or how much money to spend on replacements for the product's life-cycle.

Although one can get by with that kind of heuristic, one would think that someone has unified a concept that doesn't require a piece-wise approximation. As it turns out, I believe that no one has really solved the problem of deriving the bathtub curve simply because they haven't set up the correct premise with a corresponding set of assumptions.

The Dispersive Failure Premise: Instead of going directly to Equation 1, let's break the failure mechanism down into a pair of abstractions. First, recall the classic description of the irresistible force meeting the immovable object (wiki). Let's presume the battle between the two describes the life-cycle of a component. In such a situation we have to contend with modeling the combination of the two effects, as eventually the irresistible force of wear and tear wins out over the seemingly immovable object as its integrity eventually breaks down. In other words, failure arises from a process governed by a time rate of change (of the irresistible force) which operates against a structure that maintains some sense of integrity of the component (the immovable object).

To set this up mathematically, consider the following figure. We scale the integrity of the component as a physical dimension; it could be a formally defined measure such as strain, but we leave it as an abstract length for the sake of argument. The process acting on this abstraction becomes a velocity; again this could be a real force, such as the real measure of stress. Now when something breaks down, the irresistible force has been applied for a certain length of time against the immovable object. The amount of time it takes to cover this distance is implicitly determined by the integral of the velocity over the time. However, due to the fact that real-life components are anything but homogeneous in both (1) their integrity and (2) the applied wear-and-tear, we have to apply probability distributions to their nominal values. Pictorially it looks like a range of velocities trying to reach the effective breakdown dimension over the course of time.

Figure 3: Abstraction for the time dependence of a failure occurrence.

Some of the trajectories will arrive sooner than others, some will arrive later, but a mean velocity will become apparent. This variation has an applicable model if we select an appropriate probability density function for the velocities, denoted by p(v) (and justified later for the integrity of the structure, a corresponding p(L)). Then we can devise a formula to describe what fraction of the velocities have not reached the "breakdown" length.

Probability of no breakdown as a function of time =
integral of p(v) over time for those velocities not reaching the critical length, L

For the maximum entropy PDF of p(v)=alpha*exp(-alpha*v) this mathematically works out as

P(t) = 1-e^-alpha*L/t

for a set of constant velocities probabilistically varying in sample space. This becomes essentially a dispersion of rates that we can apply to the statistical analysis of failure. If we then apply a maximum entropy PDF to the set of L's to model randomness in the integrity of the structure

p(L) = beta*exp(-beta*L)

and integrate over L, then we get

P(t) = 1-1/(1+alpha/(beta*t))

This has a hyperbolic envelope with time. The complement of the probability becomes the probability of failure over time. Note that the exponential distributions have disappeared from the original expression; this results from the alpha and beta densities effectively canceling each other out as the fractional term alpha/beta. The alpha is a 1/velocity constant while the beta is a 1/length constant so the effective constant is a breakdown time constant, tau=alpha/beta.

P(t) = 1-1/(1+tau/t)

The assumption with this curve is that the rate of the breakdown velocities remains constant over time. More generally, we replace the term t with a parametric growth term

t -> g(t)
P(t) = 1-1/(1+tau/g(t))

If you think about the reality of a failure mode, we can conceivable suspend time and prevent the breakdown process from occurring just by adjusting the velocity frame. We can also speed up the process, via heating for example (as the popcorn example shows). Or we can imagine placing a working part in suspended animation, nothing can fail during this time so time essentially stands still. The two extreme modes roughly analogize to applying a fast forward or pause on a video.

A realistic growth term could look like the following figure. Initially, the growth proceeds linearly, as we want to pick up failures randomly due to the relentless pace of time. After a certain elapsed time we want to speed up the pace, either due to an accelerating breakdown due to temperature or some cascading internal effect due to wear-and-tear. The simplest approximation generates a linear term overcome by an exponential growth.

Figure 4: Accelerating growth function

or written out as:

g(t) = a*t + b*(e^ct -1)

This becomes a classic example of a parametric substitution, as we model the change of pace in time by a morphing growth function.

Now onto the bathtub curve. The failure rate is defined as the rate of change in cumulative probability of failure divided by the fraction of operational components left.

r(t) = -dP(t)/dt / P(t)

this results in the chain rule derivation

r(t) = dg(t)/dt / (tau + g(t))

for the g(t) shown above, this becomes

r(t) = (a+b*c*e^ct) / (tau + a*t + b*(e^ct -1))

which looks like the bathtub curve to the right for a specific set of parameters, a=1, b=0.1, c=0.1, tau=10.0. The detailed shape will change for any other set but it will still maintain some sort of bathtub curvature. Now, one may suggest that we have too many adjustable parameters and with that many, we can fit any curve in the world. However, the terms a,b,c have a collective effect and simply describe the rate of change as the process speeds up due to some specific physical phenomena. For the popcorn popping example, this represents the accelerated heating and subsequent breakdown of the popcorn kernels starting at time t=0. The other term, tau, represents the characteristic stochastic breakdown time in a dispersive universe. For a failed (i.e. popped) popcorn kernel, this represents a roll-up of the dispersive variability in the internal process characteristics of the starch as it pressurizes and the dispersive variability of the integrity of the popcorn shell at breakdown (i.e. popping point). We use the maximum entropy principle to estimate these variances since we have no extra insight to the quantitative extent of this variance. As a bottom-line for the popcorn exercise, these parameters do exist and have a physical basis and so we can obtain a workable model for the statistical physics. I can assert a similar process occurs for any bathtub curve one may come across, as one can propose a minimal set of canonical parameters necessary to describe the transition point between the linear increase and accelerated increase in the breakdown process.

The keen observer may ask: whatever happened to the classical constant failure rate approximation as described in Equation 1? No problem, as this actually drops out of the dispersion formulation if we set b=tau and a=0. This essentially says that the acceleration in the wear and tear process starts immediately and progresses as fast as the characteristic dispersion time tau. This is truly a zero-order approximation useful to describe the average breakdown process of a component.

So the question remains, and I seem to always have these questions; why hasn't this rather obvious explanation become the accepted derivation for the bathtub curve? I can find no reference to this kind of explanation in the literature; if you read "A Critical Look at the Bathtub Curve" by Klutke et al [1], from six years ago, you will find them throwing their hands up in the air in their attempt to understand the general bathtub-shaped profile.

Next, how does this relate to oil discovery? As I stated at the outset, a failure is essentially the flip-side of success. When we search for oil, we encounter initial successes around time=0 (think 1860). After that, as more and more people join the search process and we gain technological advances the accelerated search takes over. Eventually we find all the discoveries (i.e. failures) in a large region (or globally) and something approaching the classic logistic results. In this case, the initial downward slope of the oil discovery bathtub curve becomes swamped by the totality of the global search space. The mathematics of dispersive failures and the mathematics of dispersive discovery otherwise match identically. Thus you see how the popcorn popping statistical data looks a lot like the Hubbert peak, albeit on a vastly different time scale.

As a side observation, a significant bathtub curve could exist in a small or moderately sized region. This may occur if the initial discovery search started linearly with time, with a persistent level of effort. If after a specific time, an accelerated search occurred the equivalent of a bathtub curve could conceivably occur. It would likely manifest itself as a secondary discovery peak in a region. So, in general, the smaller exploration regions show the initial declining part of the bathtub curve and the larger global regions show primarily the upswing in the latter part of the bathtub curve.

As I continue to find physical process that one can model with the dispersion formulation, I start to realize that this explains why people don't understand the bathtub curve ... and why they don't understand popcorn popping times ... and why they don't understand anomalous transport ... and why they don't understand network TCP latencies ... and why they don't understand reserve growth ... and why they don't understand fractals and the Pareto law ... and finally why they don't understand oil discovery. No one has actually stumbled on this relatively simple stochastic formulation (ever?). You would think someone would have discovered all the basic mathematical principles over the course of the years, but apparently this one has slipped through the cracks. For the time being I have this entire field to myself and will try to derive and correct other misunderstood analyses until someone decides to usurp the ideas (like this one).

The finding in this post also has a greater significance beyond the oil paradigm. We need to embrace uncertainty, and start to value resiliency [2]. Why must we accept products with built-in obsolescense that break down way too soon? Why can't we take advantage of the understanding that we can glean from failure dispersion and try to make products that last longer? Conservation of products could become as important as conservation of energy, if as things play out according to a grand plan and oil continues to become more and more expensive.

References

Klutke, et al, "A Critical Look at the Bathtub Curve", IEEE Transactions on Reliability, Vol.53, No.1, 2003. [PDF]
Resilience: the capacity to absorb shocks to the system without losing the ability to function. Can whole societies become resilient in the face of traumatic change? In April 2008 natural and social scientists from around the world gathered in Stockholm, Sweden for a first-ever global conference applying lessons from nature's resilience to human societies in the throes of unprecedented transition.

Popcorn Popping as Discovery

2009-10-02T15:44:00.000-07:00

In the search for the perfect analogy to oil depletion that might exist in our experiential world, I came across a most mundane yet practical example that I have encountered so far. Prompted by a comment by Memmel at TOD who bandied about the term "simulated annealing" in trying to explain the oil discovery process, I began pondering a bit. At the time, I was microwaving some popcorn and it struck me that the dynamics of the popping in some sense captured the idea of simulated annealing, as well as mimiced the envelope of peak oil itself.

So I suggested as a reply to Memmel's comment that we should take a look at popcorn popping dynamics. The fundamental question is : Why don't all the kernels pop at the same time?

It took me awhile to lay out the groundwork, but I eventually came out with a workable model, the complexity of which mainly involved the reaction kinetics. Unsurprisingly, the probability and statistics cranked out straightforwardly as it parallels the notion of dispersion that I have worked in terms of the Dispersive Discovery Model. First, we define the basic premise with the aid of Figure 1

Figure 1: Popcorn popping kinetics. Each bin has a width of 10 seconds, and the first kernel popped at 96 seconds. So the overall width is quite large in comparison to the first pop time. Graph taken from Statistics with Popcorn [1]. The curve itself looks similar to a oil discovery peak.

In obvious ways, the envelope makes sense as experience and intuition tells us that a maximum popping rate exists which corresponds to the period of the loudest and densest popping noise. Certainly, if you stuck a thermometer in the popcorn medium you would find the average temperature rising fairly uniformly until it reaches some critical temperature. Naively, you could them imagine everything popping at once or within a few seconds of one another -- after all, water in a pop seems to boil quite suddenly. But from the figure above, the spread seems fairly wide for popcorn. The key to the range of popping times lies in the dispersive characteristics of the popcorn. This dispersion essentially shows up because of the non-uniformity among the individual kernels. Intuitively this may get reflected as variations in the activation barrier of the kernels or in the micro-variability in the medium.

Some may suggest that the temperature spread may occur simply because of the effects of the aggregation of the popcorn kernels interacting with each other as they pop. For example, one kernel popping may jostle the environment enough to effectively cool down the surrounding medium, thus delaying the effects of the next kernel. However, that remains a rather insignificant effect. I thought about doing the experiment myself until I ran across an impressively complete study executed by a team of food scientists. Measured painstakingly against temperature, the cereal scientists placed individual kernels in a uniformly heated pot of oil, and tabulated each kernel's popping time. They then plotted the cumulative times and determined the rate constants, trying to make sense of the behavior themselves (curiously, this experiment was only completed for the first time 4 years ago).

Kinetics of Popping of Popcorn, J. E. Byrd and M. J. Perona (PDF)

Anyone who has made popcorn knows that in a given sample of kernels, the kernels fortunately do not all pop at the same time. The kernels seemingly pop at random and exhibit a range of popping times (Roshdy et al 1984; Schwartzberg 1992; Shimoni et al 2001). The model described above does not explain this observation. It explains why popcorn pops, but has nothing to say about when a kernel will pop. The goal of this work was to use the methods of chemical kinetics to explain this observation. More specifically, we performed experiments in which the number of unpopped kernels in a sample was measured as a function of time at a constant bath temperature. This type of experiment has not been reported in the literature, and the data are amendable to the methods of chemical kinetics. In addition, we formulated a quantitative kinetic model for the popping of popcorn and have used it to interpret our results. The literature contains no kinetic model for the popping of popcorn.

If you read the article you note that the idea of a rate constant comes into play. In fact, you can think of the individual kernels obeying the laws of physics as they plot their own trajectory until they reach a critical internal pressure and ultimately pop. In other words, they pop at a rate that does not depend on their neighbors (since they have no neighbors in the experiment). I suggest that two mechanisms come into play with respect to the internal dynamics of the popcorn kernel. First, we have the mechanism of the starchy internal kernel which heats up at a certain rate and starts to build up in pressure over time. This happens at an average rate but with an unknown variance; let us say that has a maximum entropy such that the standard variance equals the mean, see the maximum entropy principle. Second, we have the average rate itself accelerating over time, so that the pressure both builds up and breaks down the underlying medium at a faster and faster pace. Finally, we have variations in the kernel shell (i.e. the thickness of the pericarp layer) which acts as an activation barrier, thus defining the effective exit criteria for the kernel to pop. However, this latter mechanism does not exhibit as large of a variance, as that would trigger more immediate popping and not as explosive an effect [2]. The overall process has analogies to the field of study known as "physics of failure"; each popcorn kernel eventually fails to maintain its rigidity as it pops, and just like real failure data, this is dispersed over time. Physics of failure relies on understanding the physical processes of stress, strength and failure at a very detailed level, and I apply this at a level appropriate for understanding popcorn.

Given this premise, the math works out exactly to the approach formulated for Dispersive Discovery. I use the generalized version which applies the Laplace transform technique to generate a cumulative envelope of the fraction of unpopped popcorn over time=t and oil temperature=T.

P(t,T) = 1 - exp(-B/f(t,T))/(1+A/f(t,T))

the term f(t,T) represents the mean value of the accelerating function, and the terms A and B reflect the amount of dispersion in the shell characteristics; if B=1 and A=0 the shell has a fixed breakthrough point and if B=0 and A=1 the shell has an exponentially damped breakthrough point (i.e. lots of weaker kernels) . The latter set defines the complement of the logistic sigmoid, if f(t,T) accelerates exponentially.

f(t,T) = exp(R(T,t)) - exp(R(T,0))
R(t,T) = k*(T-T_c)²*t - c*(T-_Tc)

The basic physics of failure is encompassed in the exponential term f(t,T) which states that the likelihood of failing increases exponentially over time, as the internal structure of the popcorn starts to compound its failure mechanisms. At T=T_c and below that temperature nothing ever pops. At higher temperatures than the critical temperature, the rate correspondingly increases according to a power law (see right). Temperature as related by the parameter k is the dial of the accelerator. The parameter c acts as the delay time for how long it takes the kernel to reach the equilibrium temperature of the oil.

The set of data from the Byrd and Perona experiment is shown below, along with my temperature dependent dispersive model shown as the colored lines for A=0.5, B=0.5, c=0.1/degree, T_c=170 degrees, and k=0.00008/degree²/second. The solid lines black lines are the fit to their model which essentially adjusts each parameter for each temperature set. I left the model minimally parametric over the temperature range and adjusted only the oil temperature for each curve -- in other words, they all share the base parameters.

Figure 2: Measurements of the fraction of unpopped popcorn remaining as a function of time over a range in oil cooking temperatures (in degrees C). Two timescales are shown. These can be compared to the complementary sigmoid functions.

The authors of the study don't use the same formulation as I do because the theorists don't tend to apply the fat-tail dispersion math that I do. Therefore they resort to a first-order approximation which uses a Gaussian envelope to generate some randomness. They essentially do the equivalent of setting the B term to 1 and the A term to 0. This really shows up in the better fit at low temperatures at early popping times -- see the green curve at T=181.5 degrees in Figure 2 which tends to flatten out at earlier times than their model. Overall the fit is extremely good over the range of curves, as I contend that any deviations occur because of the limited sample size of the experiments. It gets awfully tedious to measure individual popping times and the fluctuations from the curve will surely arise. You can see this in a magnification of the low-temperature data set shown to the right. If you notice some of the 200 degree data points pop later than the 190 degree data points. These occur solely due to statistical probability artifacts due to the small sample size (between 50 and 100 kernels per experiment at the temperature measured). If they had at least 500 measurements per data set the fluctuations would have decreased by at least a factor of two. You can also observe that the statistical fluctuations show up very well if plotted as a frequency histogram in Figure 3 below. Compare this set against Figure 1, which smooths out the curve via larger bins.

Figure 3: The set of "Hubbert Curves" for popping popcorn at several temperatures.

Each one of the curves gets transformed into something approaching a logistic curve as the temperature accelerates in temperature and then keeps rising. You can think of the individual pops as those kernels meeting the criteria for activation. The laggards include those kernels that haven't caught up, either intrinsically or because of the non-uniformity of the medium (as a counter-example, water is uniform and it mixes well).

The upshot is that this popcorn popping experiment stands as an excellent mathematical analogy for dispersive discovery. If we consider the initial pops that we hear as the initial stirrings of discrete discoveries, then the analogy holds as the popping builds to a crescendo of indistinguishable pops as we reach peak. After that the occasional pop makes its way out. This happens with oil discovery as well, as the occasional discovery pop can occur well down the curve, as we have defined the peak in the early 1960's.

Figure 4: A discrete Monte Carlo simulation of dispersive oil discovery which shows both the statistical fluctuations and the sparseness of the finds down the tail of the curve. With more data sample the fluctuations diminish in size.

So contrary to recent activity of huge new discoveries as reported by the NY Times, the days of sustained discoveries remain behind us and we are seeing the occasional pop of the popcorn.

Ugo Bardi at TOD had posted a suggestion that we come up with a "mind-sized model" to described the Hubbert Peak. This makes a lot of sense as we still have the problem of trying to convince politicians and ordinary citizens of what constitutes the issue of peak oil and more generally peak oil.

Why we try to do this has some psychological basis. This last week, I worked on some ideas pertaining to the technique of Design of Experiments, and a colleague had turned me on to an in-house course on the topic. There, in the course notes introduction, the instructor had laid out the rules essentially describing the three stages of learning. The bullet points pointed out that understanding did not come about instantaneously, and instead progressed through various psychological stages. I figured that he did this as a precaution to not wanting to frighten his students at the level of effort required in learning the subject material. This explanation basically amounted to a cone of experience conceptual model, with a progression through the stages of enactive, iconic, and symbolic learning. Enactive amounts to a type of hands-on learning through first-hand experience; iconic boils it down to a psychological connection to the material via shared experience; and symbolic brings it to the level of analogies and potentially mathematics for the sophisticated learner.

The fundamental problem with understanding peak oil is that the enactive first-person learning and ultimately iconic shared experience only occurs on the most abstract level. We just cannot comprehend the global scope of the problem and can only reduce it to our day-to-day interactions and what we see at a local level. Therefore, only when we experience, or hear about something like gas station queues and price hikes do we make a connection to the overriding process at work. But these observations serve merely as side-effects and do not ultimately explain the bigger picture of oil depletion. In Bardi's view, it does not represent a good mind's eye view of peak oil.

Bardi tried to do this by comparing oil depletion to a Lotka-Volterra (L-V) model of predator-prey interactions. This works on all three levels, culminating in an abstraction that demonstrates the Hubbert Curve in mathematical curves. That would work perfectly ... if it actually modeled the reality of the situation. Unfortunately Bardi's L-V doesn't cut it and it will actually misrepresent our continued understanding of the situation. In other words, we cannot use Bardi's analysis as a tool for practicing depletion management. Our learning basically ends up in an evolutionary dead-end, and even though it effectively works on an emotional and psychological level to fill in our cone of experience, it misses the mark completely on mathematical correctness.

Other people try to make the symbolic connection by comparing the oil discovery process as searching through a pile of peanuts or cashews for edible pieces. This works on a qualitative level, but until we have some quantitative aspects, it ends up short on the symbolic mathematical level. I myself have used the analogy of finding needles in the haystack, which brings in the formal concepts of dispersion. The idea of popcorn popping fills in more of the enactive and iconic levels of our experience (who hasn't waited in agonizing expectation of the furtive last few pops?) and the experiments themselves demonstrate the symbolic math involved -- the process of accelerating discovery as our reserve bag of popcorn inflates, followed by a peak in maximum discoveries, followed by a slow decline as the most stubborn kernels pop. The production cycle culminates as we consume the popped popcorn at a later date.

To compare this understanding against Bardi's Lotka-Volterra or of the Verhulst formulations shows the limitations of the the predator prey model, as it simply does not take into account the stochastic environment of exploration. For example, what if we had applied an L-V formulation to the popcorn experiment, what would we get? Since it is deterministic, it would come out like a spike with the cumulative showing as a step function. All the L-V can do is work on the mean value, so that it misses the real dynamics of random effects. With L-V, all the popcorn would pop at the same time. This is as ridiculously simplistic an assumption as the spherical cow.

Moreover, the L-V model also assumes that some collective feedback plays a large role, as if the amount we have found or the amount remaining (as to in the Verhulst equation) determines the essentials of how depletion comes about. Yet, as the single kernel popcorn experiment bears out, the feedback term does not really exist. All we really have to understand is that certain stubborn pockets will remain and we need to keep accelerating our search if we want to maintain the Hubbertian Logistic-like shape. In other words, the natural shape does not decline as some intrinsic collective property. On an iconic level, the popcorn kernels that have popped don't tell their buddy kernels to stop popping once the bag is nearing completion. Yet, the predator-prey model tells us that is the reality and Bardi wants us to believe that interaction happens. That essentially what I want to get across: a real understanding of the situation.

So in effect, with the correct mathematical model, I could easily work out the popcorn dynamics from first principles and use that as a predictive tool for some future need (maybe I could go work for Orville Redenbacher). Or on a symbolic level, I would tell a politician that I could accurately gauge how long popcorn would take to pop and use that information to convince him that we could gauge future oil depletion dynamics. Unless he had never seen a popcorn popper in action (GHWB perhaps?) he would say, yeah that would make sense -- and then I could relate that to how it makes sense in how oil depletion dynamics plays out.

This is the kind of mind-sized model that we have just worked out.

Notes
[1] Apparently high school and college science teachers like to use popcorn popping experiments as a way to introduce the procedures of statistical data collection and the scientific method.

[2] A fixed exit criteria is like the finish line in a race (see below). A random exit criteria is where the finish line varies.

Krugman, Cities, and Oil

2009-09-16T21:56:00.000-07:00

As an economist, Paul Krugman uses mathematical concepts in his research as necessary. In a recent article, he reviewed the state of economic research and found many deficiencies in how mainstream economists (at the Fed, etc) apply theories to the real world. Some people took exception to what they deemed an attack on the use of math in economics by Krugman. He subsequently wrote a short blog piece to set the record state.

In the economic geography stuff, for example, I started with some vague ideas; it wasn’t until I’d managed to write down full models that the ideas came clear. After the math I was able to express most of those ideas in plain English, but it really took the math to get there, and you still can’t quite get it all without the equations.

What Krugman says serves as a useful piece of advice. In shorter terms, the mechanism of mathematics provides the insight to our understanding. Unless you work the math, the insight may never come, simply because it captures and retains the bookkeeping and juggling of ideas for our over-taxed brains. Often times you would never know if certain effects canceled out, compounded, or came out negligible unless you took the time to formalize the arguments.

Take a look at one of Krugman's books on economics and geography. One area of interest he had concerns the size and spatial distribution of city and town populations, and specifically the organization of edge cities. Many had observed a general behavior of very few large populations and progressively more smaller cities. This has the moniker of Zipf's Law -- a heuristic that seems to match the data from a rank histogram of the USA:

  P(Size) = k/Sizeⁿ, where n=1

Granted some people might consider this completely intuitive from the start and require no insight beyond this point to make more elaborate arguments. Or you can take this empirical result and try to understand why it occurs by diving further into the math (Mitzenmacher and Gabaix). In one of his textbooks ("The Self-Organizing Economy"), Krugman had invoked a preferential attachment argument due to Herbert Simon to explain the historical city population growth.

My own understanding derives from a straightforward application of dispersion to growth rates of the measure of interest. For the case of oil reservoirs, a maximum entropy dispersion of material drift velocity during its formation can generate such a dispersion. The solution to this leads to a probability distribution function

  P(Size) = 1/(1+c/Size)

This looks something like the Zipf-Mandelbrot variation which includes a constant limiting term to prevent a singularity at the origin (in ecology this is also known as a relative abundance distribution).

We can illustrate the similarity between the distribution of city population sizes (from USA census data) with the distribution of discovered oil reservoir sizes in the USA (from the Baker and Gehman data). See to the right for a ranked histogram of the two sets of measures. I fit a dispersive aggregation model for each set of data with the single parameter c describing the characteristic aggregation size. Most interesting in the aligned data set is the multi-level correspondence between the number of cities and the number of reservoirs at varying densities. First note that the number of very large cities and very large reservoirs is comparable, about 9 cities (not urban areas) over a million people and about 13 or 14 oil reservoirs over a billion barrels in recoverable reserves. From there, the ratio holds relatively steady, for every city of a specific size numbered in thousands, you find an oil reservoir with that same size in million barrels that you can "attach" to that city. So you find that around 200 cities have 100K or more in population and about 200 reservoirs have 100 MB or more in oil. This works for awhile until the number of small cities starts to overtake the number of small reservoirs as counted by Baker.

Applying a sanity check to the 1000x ratio, means that each "reigning" citizen of a large city will use about 1000 barrels (100 MB/100K) of natively-supplied oil over the course of time. I use the term "reigning" to indicate that each city's population has followed a slowly growing equilibrium that will asymptotically reach some carrying capacity with births/immigration balanced by deaths/emmigration, whereas oil has a finite limit not at all related to a carrying capacity. So reigning essentially means a steady-state citizen, and that equivalent person has used 1000 barrels or 42,000 gallons of American-sourced unrefined oil over the past 150 years.

I find this exercise useful, if for nothing else, that it can give people a feel for how many reservoirs that we have remaining. Think of how many sizable cities we have, and that amounts to how many equivalently-sized reservoirs that we have to essentially feed those cities. Eventually, all these reservoirs will turn into ghost towns as they deplete and become shut-in while the cities will remain. Most people do not have a feel for what 300 million people means yet they can start to comprehend thousands of reservoirs and cities, and the significance of those numbers. It also helps to put the "Drill, Baby,Drill" nonsensical mantra into context: think in terms of how many reservoirs we would need to discover to replace the ghost reservoirs that will crop up -- essentially one for every city if we want to depend on a captive native resource.

As influential as Krugman remains among rational economists, I wonder when he will really start hitting the real problems of constrained resources that this country (and the world) faces. Economists have the habit of theoretically deferring to the substitution of one resource for another as soon as we reach a constraint. However, without clear technological alternatives, this constraint looks ominous and would make Krugman look even more like a contrarian than he normally appears to the supply-siders. I know Krugman could comprehend the math, but I don't know whether he wants to go there.

Deviations from Hubbert Linearization

2009-08-23T09:28:00.000-07:00

If dispersive discovery proceeds with an exponentially accelerated search over a spatially dispersed volume then we see the classic Hubbert curve -- a Logistic sigmoid for cumulative growth. Applying the technique of Hubbert Linearization to that formulation, we plot a straight line, as shown to the right (plotting cumulative production U against fractional production P/U).

What happens if at some point in the accelerating search we apply an even more aggressive search policy? Say that we super-accelerate by applying a Gompertz-like growth term exp(kt²) instead of the linear exponential in the classic Logistic. The previously straight line develops a bulge that initially looks like a shallower HL slope but which eventually slopes downward to the URR cumulative intersection U₀. Note that the URR gets normalized to 1 in the figure and amounts to a geological limit.

Conversely, what happens if the accelerating search stabilizes and transforms into a steady, linear growth? In this case, the HL linearization plummets before asymptotically approaches the same URR.

This deviation from the linear HL behavior may have happened already, but the noise in the discovery history likely has obscured the effect. In terms of the deviation direction, this could go either way. An aggressive search acceleration would come about if oil prospectors had confidence that they could apply a huge, albeit transient, investment into their infrastructure. On the other hand, the deceleration would obviously come about if the oil industry collectively started to give up and thus either reduce their search effort or resort to maintaining their previous rate.

As a key to understanding classical economics one really only has to understand the concept of continual growth. An implicit Ponzi scheme exists within our system, but remains largely unobservable since the acceleration occurs over decades. In the absence of alternative resources to fuel the engine of growth, we can observe directly the hoisting of our own petard as economic growth starts to mutate. Two possible outcomes will prevail: either a feverish last attempt to maintain pace in the face of a wall looming ahead, or a resignation to the idea that continuous growth can not sustain itself.

dispersive transport

2009-06-28T22:18:00.001-07:00

Why is dispersion anomalous?

This post touches on the nature of theoretical and
experimental research and illustrates how a fundamental idea can take quite a circuitous route before it lodges in a remotely related application area. The acceptance of the original idea tends to create a momentum that makes it difficult to dislodge from the conventional wisdom and impenetrable to anyone but the cognoscenti.

First off, prep yourself for some solid-state physics. But don't worry about the math as the commentary makes up for the potential MEGO. The extended narrative traverses the scale of applicability from statistical mechanics to environmental geology picking up arcs of connectivity along the way. I also buried a valuable nugget in here, presenting a surprisingly powerful analytical result that has laid dormant for over 30 years, perhaps even 20 years prior to that, and has huge implications for the analysis of solar cells, MOS technology, and quantum electronics. To put it another way, if I had to do another graduate thesis, I could easily defend this argument, and on top of that, I would have fun doing it. It all fits together tighter than a Peyton Manning spiral. The fact that it also connects across disparate domains of science makes it frankly mind-blowing.

I call it mind-blowing from the fact that the actual argument derives from such a simple premise, and I have to seriously wonder why no one has picked up on this before. I actually question some of the belief systems inherent in these fields of study and assert the likelihood of a sunk cost effect getting in the way of a fundamental understanding. If this sounds familiar to those following our oil predicament, it should, as I have definitely seen such oversight play out before.

Often in physics, experimental observations are termed "anomalous" before they are understood. Once theory succeeds in explaining and illuminating the observations, they are no longer "anomalous" and instead come to be regarded as "obvious". A crucial paper can trigger such an "anomalous => obvious" transition, and in the present case that key role was played by a 1975 paper by Scher and Montroll. That landmark paper has become basic to our understanding of a striking characteristic of carrier motion (now called dispersive transport) which is a common occurence in amorphous semiconductors, though foreign to our experience with crystals.
-- Richard Zallen, "The physics of amorphous solids", Wiley-VCH, 1998

Hmm ... Reserve growth also considered anomalous...¹

The term anomalous in scientific code-speak essentially means "dunno". We have to admit that we don't understand lots of things, largely due to issues of complexity, observability, or just too much noise. Yet that doesn't prevent us from trying to extract a fundamental meaning of some strange behavior that we observe.

The applied mathematicians Harvey Scher & Elliott Montroll originally tried to explain the concept of anomalous behavior of photo-conductivity in amorphous semiconductors. Scientists had long understood the complementary non-anomalous behavior in non-amorphous materials. To set the stage see the schematic figure to the right.

In a crystalline semiconductor with a contact electrode at each end, a pulse of light incident at one electrode will, upon the effect of an electric field or potential drop between the electrodes, generate an almost immediate flow of current across the load lasting as long as the transit time of the photo-induced carriers.² The carriers, either electrons or holes depending on the polarity of the electric field bias relative to the absorbing electrode, will drift from the site of the photon-induced excitation, to the opposite contact. Experimentalists consider the behavior well-characterized; the mobility of the carriers at temperature, the strength of the electric field, and the contact separation, d, determine the transit time, t_T, of the output current pulse. Because the carriers scatter against lattice imperfections, the speed does not continuously accelerate but instead achieves a bounded drift velocity, v₀. This drift mobility has an intuitive real-world analog -- think in terms of a drag coefficient for the analogous situation of a falling body under gravitational forces, which eventually achieve what we call terminal velocity. For all practical purposes, the equivalent terminal velocity occurs almost immediately in a semiconductor (a short relaxation or quenching time) and sets the bound for the transit time duration.

Ideally, the pulse looks like a perfect square wave with temporal duration t_T. In terms of a mathematical expression, the current behaves as I(t) = K*[u(t)-u(t-t_T)] where u(t) is the unit step operator with magnitude K. See the figure to the right.

Because of carrier diffusion, the actual drop-off in current has rounded leading and trailing edges as the charged carrier pulse spreads out a bit into a Gaussian packet as it propagates. This relatively innocuous but well-understood form of dispersion, known as diffusion, occurs from random walk excursions as the carrier makes its way across the transit width. In the ideal case, the diffusion constant varies linearly with the mobility (or drag for particle systems) according to the Einstein relation. For high mobilities and small contact separations, the amount of diffusion that occurs does not appreciably round the pulse edges. The prized high mobility solid-state material allows device manufacturers to fabricate ultra-high speed photo-detectors as the sharp transition and short transit time generates an excellent and well-characterized frequency response.

This class of semiconductor has an ordered structure due to the crystalline lattice structure and it has properties such as carrier mobility which remain uniform through the sample. Such behavior shows little dispersion, either through diffusion or disorder. The narrower the distribution of velocities, the sharper the transition. Scientists have generally understood this for years and no one raises the spectre of anomalous behavior.

The truly anomalous behavior observed occurs in amorphous versions of certain semiconductors. The narrow pulse of carriers seen in an ordered sample now shows a huge spread in its concentration profile as it makes its way between the contacts. Obviously dispersion plays some role in this behavior, as it goes by the name "dispersive transport". Scientists had known about this "anomalous dispersion" since 1957 but it took nearly two decades before Scher and Montroll presented the solution to the problem mathematically.³

Figure 1: (a) Normal transport shows a drifting packet of carriers. As
long as the packet travels, we can detect a current proportional to the
amount of carriers active. As they reach the opposing contact, the
current rapidly declines to zero. The Gaussian-shaped packet widens
slightly as it travels due to diffusion about the mean. A few extra
fast carriers reach the far contact sooner than the bulk of the
carriers, while a few stragglers take up the rear. (b) In dispersive
transport, the velocity of the carriers varies over a wide range so
that the original narrow impulse of carriers quickly spreads out as it
drifts and diffuses across the width. This gives a long tail to the
photo-response profile.⁴
The
stragglers keep arriving in this "fat tail" world, with progressively
fewer in number as though they had joined and tried to complete a long marathon race (see Marathon Dispersion).

Figure 2: The typical measurement in the photo-response current starts with a
spike followed by a soft plateau, then a shoulder or transition region,
followed by the ubiquitous long tail⁵.
Shapes of the current profile taken over a range of experimental
conditions show invariance in the general shape with respect to the
electric field and specimen thickness. Importantly, this profile does not
follow from the expected spread of the Gaussian packet. Scale
invariance or universality manifests itself in statistics if one can
first transform the ordinates into dimensionless quantities while conserving the moments.⁶ Transport of holes and electrons
near the absorption region electrode contributes to the initial spike,
which has no impact on the longer tail due to the complementary carrier
type (this transient spike is also known as prompt transport⁷)

Figures 1 and 2 give a qualitative view of the dispersive transport that occurs in a disordered semiconductor. The first figure describes what we think happens internally and the second figure provides a view of the observable result. Figures 3 and 4 illustrate a couple of experimental results of widely studied amorphous (a-As₂Se₃) and organic materials (TNF-PVK).


Figure 3: Typical experimental Time-Of-Flight curve shows a set of superimposed measurements from an early Scher-Montroll behavior which exhibited the "universality" property of the scaling across different measurement conditions (the applied voltage in this case).	Figure 4: The TOF curve of an organic semiconductor illustrates the characteristic knee that Scher and Montroll had predicted; the two slopes differ but must sum to -2 according to their theory.

The anomaly in the title of the Scher-Montroll paper⁸ referred to
the fact that no one previously could formally explain the long tails in
the response (so called "time of flight") measurements. Other researchers
clearly had an inkling that it had something to do with the high
amounts of disorder leading to greater amounts of diffusion and dispersive spread than in an
ordered material. Amorphous materials naturally have many inhomogeneities, defects, and
carrier traps that can lead to varying delays in transit time. Scher and Montroll derived a statistical formulation of random walk
called the Continuous Time Random Walk (CTRW) that they then applied to
the experimental results.

Most experimentalists around that
time got good results using the CTRW formulation so that it has become
fairly well accepted in semiconductor circles for the last 30 years.
The math gets fairly hairy in spots, and soon experimentalists
began to simply use the empirical sloped lines to get at the Scher-Montroll disorder parameter (ɑ = alpha). High values of alpha indicate more order and low values more disorder (0 < ɑ < 1).

Research continues in understanding dispersive transport as new electronic materials come on line. Physicists have had a long-standing interest in disorder, as the finding of an order/disorder transition easily classifies as a type of "holy grail" discovery, certain to elicit oohs and aahs from their colleagues. As the figure to the right shows, one can add a controlled amount of disorder to a sample and observe the results of diffusive transport. The upper TOF trace shows linear transport while the lower trace shows the effects of dispersive transport. In the latter case the disorder comes in the way of the intentional introduction of impurities, apparently forming electronic traps which slow down the carrier motion as it traverses the width of the sample.⁹

Generally I have noticed that the basis for much of the current research has to do with finding some novel aspect of dispersion relating somehow to material properties. In reality, the rather mundane effect of randomness due to heterogeneity likely plays a far more important role. For many of these materials, we simply can't control the distribution of defects and traps and the disorder evolves into a garden variety randomness, with which we have a single mean rate, say average drift velocity, to characterize the behavior.

If you look at the curve to the right, the red line shows my simple assumption for maximum disorder. I had noticed this same shaped curve in my studies of dispersive discovery that I posted to TOD [], and had a hunch that I could use the same formulation in the semiconductor case. After all, as I assert that dispersion is just dispersion, and I have enough experience dealing with semiconductor physics that I didn't expect any gotchas. As for the ideas of Scher and Montroll, I turn their formulation upside down and don't even consider a random walk premise, as this leads to overly complex math.

Breakthrough

I started to look at this problem because I had a nice intuitive way of modeling dispersive behavior in oil production [google links]
and figured that I could try applying my general dispersive model to dispersive
transport.¹⁰ And that I could do it much more simply than the approach by Scher and Montroll. At certain places in their seminal papers and review articles, I find passages that amount to "... and then a miracle occurs" and knew that this meant some messy first-principles work had gone missing. The way I turned their model on its head basically amounted to working in the rate domain, corresponding to velocities, instead of the time domain that corresponded to random-walk hopping (see figure).¹¹ The latter derives from the classical work used to describe everything from Brownian motion to large scale diffusion. The former relates to a more or less pragmatic view of the world which relies on entropy considerations instead of the statistics of hopping over energy barriers with small probabilities.

As a basic premise, I use the Maximum Entropy Model (MEM) to select a stochastic rate Probability Density Function (PDF or more precisely PMF for probability mass function) in which I can then derive dispersive transport.

One way to choose the “right” distribution p is by
using the principle of maximum entropy. This principle states that the
least biased probability assignment is that which maximizes the system
entropy subject to the constraints supplied by the available
information.¹²

For the constrained system of interest, all we really know is the mean carrier transport velocity. If we don't know the higher order moments, the MEM says to use a damped exponential as the PDF to maximize entropy. In a general sense, this maximizes the amount of disorder that exists in this quasi-equilibrium system. It says that many slow carriers exist, with an exponentially diminishing supply of fast carriers. For the fixed geometry shown in the schematics at the top, the normalized expression for the time dependence of dispersed current reaching the far contact derives as follows:

I(t) = I₀ [1 - e^-1/t (1 + 1/t)] (EQ 1)

This essentially describes the integral over a PDF of normalized velocities

p(v) = (1/v₀)exp(-v/v₀) (EQ 2)

for carriers that have not yet swept through the transport layer. We assign t_T = d/v₀= 1 to show the scale invariance of the result of Equation 1. Crucially the formulation maintains the moments of the distribution. If the velocity distribution becomes dispersed as a damped exponential then the cumulative position distribution of a particle/carrier also advances by a damped exponential. Nothing more to it than that!

I pulled out the fundamental transport coefficients such as mobility and the diffusion constant for the time-being as this assumes that a uniform drift plays the prominent role. In the normalized case, I show the response profile below superimposed on the figure from Kao. At a subjective level, it follows the qualitative plateau/decline behavior quite well.

Figure 5: The maximum entropy dispersion for time of flight according to the normalized (EQ 1)

Since we know that dispersion plays a role in the transport, we just have to figure out how to use the much simpler dispersion formulation instead of the hideous Scher-Montroll derivation. The key to understanding physics is to keep it simple, but not too simple (quoting Einstein I believe). In fact the maximum entropy formulation that I had used previously in the dispersion analysis for oil field sizes, discovery, and reserve growth, I retain in this analysis¹³. Also known as the "method of least information", it essentially relies on using common sense in not trying to under- or over-estimate the variance of the dispersive spread. In one sense, the interpretation I make looks similar to the schematic at the right. I assume the equivalence of multiple mobility pathways through the device.

For any one pathway, the advance in the particles motion has a diffusive component as well as a drift component. This leads to an expression involving time as shown below¹⁴

<x> = sqrt( Dt + (v₀t)² ) (EQ 3)

This essentially incorporates the concurrent diffusion component along with the drift component of the velocity and we can make an implicit transform into the actual timeline. The drift velocity v₀ relates to the electric field by v₀= uE, where u is the carrier mobility (pronounced "mu") and E is the electric field strength. I would consider this a routine parametrization into a Hilbertian space where we can maintain moments of the distributions across dimensions, <t>/t_T = <x>/w.

I(t) = I₀ [1 - exp(-w/sqrt(Dt + (vt)²)) (1 + w/sqrt(Dt + (vt)²))] (EQ 4)

Qualitatively the constant (drift) velocity drops as t^-2 while diffusional velocity drops by t^-1. I am not certain whether the formulation by Scher and Montroll take this into account. They simply say that long-range correlations go as t^-a-1 when they set up their CTRW model. I believe this step links my exponentially damped rate dispersion to their long range time correlations.

Many experimental results show the knee in the curve of Scher and Montroll, but with usually not much dynamic range. I looked at a few material studies done fairly recently to see how well the simple theory works.

Transport in SiO₂

For verifying any theoretical formulation, you usually want to match
the behavior to as wide a dynamic range as experimentally feasible. The
larger the dynamic range in the measured quantities, the more
confidence that you have in its worth or value.


Figure 6: Dispersive transport via the MEM model compared to SiO2 measurements. This shows mainly diffusion with the drift catching up at longer times.	Figure 7: The effect of changing the width of the transport layer. The Montroll-Scher knee does not show up prominently.

The case of carrier transport across SiO₂ insulating layers for MOS devices provides some cases of amazing dynamic range, up to 8 orders of magnitude in current. I took data from the text "Ionizing radiation effects in MOS devices and circuits" by Ma and Dressendorfer. The fundamental idea remains the same in this situation as the photo-response experiment, although a different form of ionizing radiation supplies the pulse of carriers -- in this case holes become the charge carrier instead of electrons. Otherwise, the same diffusive transport occurs, with the authors trying to explain the results by applying the same unwieldy Scher-Montroll formulation. As a side note, these kinds of measurements need a delicate touch as the dose of the radiation can actually effect the field due to space charge formation. I did some pioneering work on a similar experiment years ago where I tried to force dopant concentrations via ion bombardment into a growing junction and the bias of the junction alone pulled the mobile dopants from one side of the junction to the other. The key is that even though you see weird stuff happen, you can always explain it via some rather elementary considerations.

In any case the fits to the data using the simple diffusive transport model works over a large dynamic range in ordinates. The sharp bend near the top indicates the potential start to the plateauing, and one can observe that some of the pairs of data indeed do flatten out. The other gradual bend indicates the transition between diffusion transport and drift transport. The universality of this bend does not scale perfectly as drift does depend on the electric field whereas the diffusion doesn't. And as we will see in the next example, the temperature may not play a big role in deviations from universality.

Transport in a-Si:H

Amorphous
semiconductors have a huge influence on the solar cell and photovoltaic
industry. In general, it costs much less to manufacture amorphous
materials as the fabrication facilities do not have to follow as strict a material process. Unfortunately the performance characteristics of the amorphous silicon in comparison to its crystalline brethren leaves lots of room for improvement. Although not as important for solar cells, the photo-response time for an incident light stimulus shows the long tails characteristic of diffusive transport.

I culled the data from a 2005 paper by Emelianova, et al studying the photo-response of amorphous hydrogenated Silicon (a-Si:H). This material was undoped and the investigation looked at hole carriers. I found this study very comprehensive and it leans toward questioning the applicability of Scher and Montroll's original formulation in terms of an alternate model that they formulate.

For the curves fit to the right, I used the simple expression in Equation 4 and plugged in the coefficients as stated in the legend and the table below. I have never seen a spanking new model that popped out with such obvious agreement in my life. This basically should set the hairs on end; I really believe that after 50 years of first discovering the anomalous dispersive transport that a simple equation, suitable for spreadsheet entry, would agree so well.

Mobility (u)	Width (w)	Diffusion constant (D)	Temperature (T)	Current Scaling (C)	Electric Field (E)
0.00193 cm²/V/s	2.4 microns	0.69 * u	264	1.2e-13	varies as in figure (V/cm)

t/t₀ = normalized time = sqrt (2Dt + (uEt)²)/w (EQ 5)

I(t) = I₀ [1 - exp (-t₀/t)(1 + t₀/t)] (EQ 6)

where I₀ = C*E*E

The idealness of the fit should preclude me from over-analyzing the results but a few interesting issues remain.

(1) For one, in this case the relation between mobility and diffusion constant does not obey the
Einstein relation but this rarely happens in non-ideal and disordered materials as the energy states get sufficiently smeared across the bandgap. The general Einstein relation relates the diffusion constant D to the energy distribution of the carrier states. ¹⁵

D = u/q [N(E_c)/N'(E_c)]

Diffusion exists in the absence of an electric field and so thermal energy acts as the only stimulus to allow a carrier to move to an adjacent site. For a narrow variation of E_c around the Boltzmann distribution¹⁶, the relation D=u/q*kT holds as an invariant, but as E_c spreads out -- and in the maximum entropy case of a large variance knowing only the mean -- the diffusion constant tracks E_c more than it does temperature, T. I worked it out and D=u/q*(kT+E_c) in that case. Since E_ctypically exceeds the statistical value of thermal energy, kT, we will see a higher diffusivity than one would expect from an ordered solid (see Schiff).

(2) Also, the initial transient spike has to do with the collection of complementary carriers at the near electrode, and has no influence on the results (undoped material generates equal number of oppositely charged carriers). As a probability exercise, the results also show that the integrated area under each curve is identical to within 0.2% for each voltage bias. In fact the cumulative charge collection based on Equation 1 becomes the following simple formula:

Q(t) = I₀ t*(1 - e^-1/t) (EQ 7)

The build-up of charge starts linearly and then converges asymptotically to a value proportional to the total number of carriers generated during the pulse duration (excepting recombination and other losses).

The authors apply their own model to the results and suggest that the dispersion is wider than gaussian as the figure to the right shows, yet they also curiously indicate that is a gaussian non-dispersive transport. Much of the confusion arises from the original Scher-Montrose formulation which demarcates the curves into ordered or non-disordered instead of what I would like to see -- a dispersed diffusion-dominated regime versus a dispersed drift-dominated regime.

The upshot of the good agreement of my fundamental model with the results means that any smart electrical engineer can start using the simple formulation right now, and should that engineer want to calculate frequency response or impulse response of an amorphous material device, they just have to use Equation 6. They can do FFT or Laplace transforms or anything they want since they have an analytical result which they can plop into their notebook or spreadsheet or Matlab and work out. I guarantee no one would want to mess with the Montroll-Scher result as it gets way too unwieldy and I dare say that no one actually understands it. I consider this simplicity a huge benefit.

The only caveat: you need a disordered material to apply this to .... but, of course, that goes with the premise.

Quantum Dots

Scientists have looked to unique materials including a variety of organic semiconductors in the hope of creating structures suitable for quantum dot devices. This paper Charge Carrier Transport in Poly(N-vinylcarbazole):CdS Quantum Dot Hybrid Nanocomposite provides a few time-of-flight curves in terms of a completely different material system. These TOF's appear to obey the same simple maximum entropy model for dispersive transport as you can see in Figure 8 and Figure 9.


Figure 8: TOF traces taken at different applied electric fields. The original diagram did not have dimensions on the axis so I guessed on the scaling based on the inset. I show the simple dispersive transport model as symbols with the electric field dependence as in Equation 6	Figure 9: All curves plotted on a universal scale. The t^-2 drift dependence extends beyond the range of the data.

Of course good agreement means that the disorder in the systems has to agree with the maximum entropy model. Nothing precludes different diffusion mechanisms or even further disorder, implying even fatter tails than t^-1. Some systems likely exist with a mix of order and disorder, such as crystalline semiconductors with many defects. In that case, one could conceivably separate out the effects.

The Connection

"When the weird gets going, the weird turn pro" -- Hunter S. Thompson

I got sidetracked into the dispersive transport behavior of carriers in disordered solid-state materials as I searched for ideas that might substantiate the oil depletion models that I had worked on. I have long asserted that everything about the behavior of oil, from reservoir sizes, to oil discovery, and on to reserve growth has as a basis the effects of dispersion¹⁷. Just as the disorder in amorphous semiconductors causes a dispersion in carrier velocities, so too does the randomness and disorder in aspects of the fossil fuel process. Whether the randomness has to do with varying velocities in the drift of oil over eons or the variance of human search efforts (see figure at right), these all lead to the same fundamental formulation for dispersive analysis. Moreover, any chaotic or complex behavior gets smoothed out by the filter of dispersion. I essentially derived a new math shorthand to describe oil, and stumbled across the fact that this same derivation applies equally well to a field totally removed from the macroscopic. I essentially went from the macroscopic to the microscopic, and then back again to substantiate what I had earlier conjectured.

Recall again the two scientists Scher and Montroll, who originally formulated the CTRW theory to explain dispersive carrier transport. They essentially worked as applied mathematicians and have gained quite a bit of recognition for their ideas. Montroll, arguably the more well-known of the two has since died¹⁸, but Harvey Scher has continued on applying the same formalism to other application areas.

Guess where he has applied it?

Answer: Transport of materials underground via porous structures ... as you may have guessed, pretty much the same life-cycle that petroleum operates under.
So Scher essentially transitioned from the
microscopic world of semiconductors to the macroscopic world of the earth. Currently Scher works as a consultant for a group of geologists and environmental scientists that use the CTRW theory to explain the way that contamination and other solutes spread over time via diffusive transport.¹⁹

I have problems with the CTRW theory in that at a certain step in the derivation, the authors invoke the legendary "and then a miracle occurs" argument into the proof. This turns into the essential observation that long-range correlations go as 1/t^alpha.²⁰ Well, I can generate that just by invoking the Maximum Entropy assumption on the variance of velocities. In that case, the inverse time power-law behavior naturally takes over and an integral exponent depending on the mean velocity from the specific type of motion occurring -- either diffusion (t^-1) or drift (t^-2). If a combination of the behaviors occurs, just solve the classical equations of motion assuming Fick's Law and calculus, and the characteristic dispersive formula appears, just as for dispersive transport in amorphous semiconductors.


*Figure 10*: Application of the dispersive transport to the motion of solute. This experiment showed a transition as the solute migrated over time. Analysis of Tracer Test Breakthrough Curves in Heterogeneous	Figure 11: The illustration shows some of the causes of pore-scale dispersion. Solute traveling more tortuous pathways between sediment grains will move more slowly than that moving along more direct pathways. Diverging pathways will also cause the contaminant to spread perpendicular to the aquifer flow direction.

Double Breakthrough

Figure 12: Uranine dye moving downstream in
Fisher Creek after
injection to trace the
destination of the water as it disappears.
(Groundwater Tracing in the Woodville Karst Plain)

Most of the solute transport measurements use something called breakthrough curve analysis. "Breakthrough curves" enable a researcher to estimate the amount
of dispersion occurring in a flow of solute (or contamination or
whatever) in a media. For a non-dispersive flow, the breakthrough
curve looks like a unit step where the tracer material is detected
abruptly at a specific time at a certain point downstream. This has an analog to the Time-of-Flight measurements used in photo-response studies described earlier²¹. But due to
randomness and variability in the media due to pore structures (for
example), the dispersion smears the breakthrough curve over a broad
time window.

A very simple model for a breakthrough curve
involves solving the equation for a maximum entropy spread in
velocities (for a given mean velocity) at a specific distance L. Given
the average time taken is T=L/v, and a random variate would take
time = t, then the breakthrough curve looks like exp(-L/vt). If you plot
this curve it looks like what some people refer to as a "reciprocal
exponential". It isn't the classic exponential because the time
parameter goes in the denominator. That happens because we are dealing
with rates, and not time for the stochastic parameter.

Next, we must realize that an idealized breakthrough curve assumes a fixed separation, L, in a very controlled experimental environment.
In reality, the distance L's become spread out over space and for a
maximum entropy PDF of L, an uncontrolled "breakthrough curve" will
have a temporal behavior that looks like 1/(1+L/vt), where L becomes the mean
separation. This looks exactly like the formula for enigmatic reserve growth in oil discoveries that I derived before [ref].

I am satisfied with using a maximum entropy estimator for the dispersion because the effects could be due to many different possibilities. So, in a sense, variability overrules complexity and if we can concentrate on understanding the mean value, we have a very simple way to characterize the system. That is my premise and I have to be able to defend it from many angles.

Take a look at the figure to the right from a hydrogeology experiment.²² Granted, I do not know anything about the particulars of the particular experiment, yet I assert that I can do a better job of fitting to the results solely because I do not place a bias on my estimator. I simply apply maximum entropy to randomize the effect, making the only assumption the mean transport rate.

I see some indication that Scher and his colleagues have at least considered this simple premise. From the following extract, note that they imply that some sort of ensemble average acts as a precondition to further analysis. In other words, they make the presupposition that geology is random and uncontrollable, just like amorphous semiconductors and human processes.

The point average of v and D can be very sensitive to small
changes in the local volume used to determine the average. Conversely,
if one fixes the volume to a practical pixel size (e.g., 10 m3) the use
of a local average v and D in each volume can be quite limited, i.e.,
the spreading effects of unresolved residual heterogeneities are
suppressed [e.g., Dagan, 1997]. We will return to this issue in a
broader context in section 4. It essentially involves the degrees of
uncertainty and its associated spatial scales. We start, at first, with an ensemble average of the entire medium and discuss the role of this approach in the broader context.
-- "Physical Pictures of Transport in Heterogeneous Media:
Advection-Dispersion, Random Walk and Fractional Derivative
Formulations." Brian Berkowitz, Joseph Klafter, Ralf Metzler, and Harvey
Scher

Back to Oil

As a very general
technique we can apply the equivalent of breakthrough analysis across many domains. The usual problem remains that different application domains use different terminology. I never used breakthrough analysis terminology because no one does controlled experiments when they look for or extract oil. Oil exploration is a commercial enterprises after all and oil prospectors get what they can, while they can, and don't necessarily ponder any deeper meaning. Yet, I view the over-riding dispersion analysis as a very general concept and I simply apply the same technique in oil depletion by making the analogy to dispersion in human-aided discovery search rates. The fact that it also occurs for physical
processes such as contaminant flow in groundwater, carrier transport in amorphous semiconductors, or TCP dispersion should not surprise anyone.²³

Over 50 years have lapsed since the day that Hubbert first sketched a Logistic curve to model oil depletion, and I think science has had a mental block on the dispersion problem all this time. We can easily and simply explain the dynamics of the oil production curve by using these same ideas from dispersion analysis.²⁴

Getting meta for a moment, to
you I exist only as a blogger. I don't know if this analysis will go
anywhere. As you may realize, I have some good ideas on the way we can analyze oil
depletion. Yet, I have no credentials in that field. A pseudonymous
writer can only way sway an argument based on the logic of his
arguments. If you can follow the argument in this post, and believe it
applies, and that all experimental evidence backs up the theory, my
credibility builds. Someone will then say, "well, he got that part
right, maybe this other part makes some sense". As far as I can tell,
no one has documented a similar simple approach to what I have formulated via my blog postings. It takes a bit of intuition to determine the situations where disorder and diversity rules and where it does not. I say that where you can
appropriately apply these arguments you can start to understand the
dynamics. I can certainly understand the dynamics of the Hubbert curve via dispersion just as I can understand the transient of an amorphous semiconductor time-of-flight experiment by applying dispersion. The fact that no one else sees it this way turns my task into one of salesmanship, unfortunate, since no one likes a salesman.

Putting that aside, I assert that bottom-line we really can use fundamental concepts to understand the dynamics of these behaviors. Absolutely nothing about any of these empirical observations I would consider anomalous. No one resorts to calling the variability in rain "anomalous" and why we haven't universally figured out simple solutions as described here remains the real mystery. Or that is indeed the real anomaly.
</meta>

WHT
http://mobjectivist.blogspot.com

FOOTNOTES

Or "enigmatic" as some have referred to the oil situation, see http://mobjectivist.blogspot.com/2008/07/solving-enigma-of-reserve-growth.html
Figure from "Physics of amorphous semiconductors", Kazuo Morigaki
"Transit-time measurements of charge carriers in disordered silicons: amorphous, nanocrystalline, and porous", E. A. Schiff
"The physics of amorphous solids" Richard Zallen, Wiley-VCH, 1998
Dielectric phenomena in solids, Kwan-Chi Kao, Elsevier,2004
On a log-log
plot, as long as the orders of magnitude scale is maintained, one can
fit a curve simply by translation
"Ionizing radiation effects in MOS devices and circuits", T. P. Ma, Paul V. Dressendorfer
Anomalous transit-time dispersion in amorphous solids, H. Scher, E. W. Montroll, Phys. Rev. B 12, 2455 - 2477 (1975)
Computer simulation of photocurrent transients for charge transport in disordered organic materials containing traps

Proc. SPIE, Vol. 3799, 94 (1999)

Sergey V. Novikov, David H. Dunlap, Vasudev M. Kenkre, Anatoly V. Vannikov
One can use the Laplace transform to characterize the dimension of the disorder, see
http://mobjectivist.blogspot.com/2008/08/general-dispersive-discovery-laplace.html
"Excess electrons in dielectric media", C. Ferradine, J-P Jay-Gerin.
A MAXIMUM ENTROPY ANALYSIS OF SINGLE SERVER QUEUING SYSTEM WITH SELF-SIMILAR INPUT TRAFFIC

A. Asars, E. Petersons
I describe a comprehensive model for oil depletion and apply dispersion to three aspects of the model.
http://mobjectivist.blogspot.com/2008/11/comprehensive-oil-depletion-model-life.html
<img src="http://img224.imageshack.us/img224/448/comprehensivemodelsi5.gif">
"Diffusion with drift on a finite line", M.Khantha, V. Balakrishnan, Journal Physics C: Solid State Physics, 16 (1983)
Hydrogenated amorphous silicon, R. A. Street
The Maxwell-Boltzmann is an approximation to the actual Fermi-Dirac distribution at higher temperatures.
See theshock model for exceptions to this rule.
Montroll held advanced research director positions at IBM Research, Institute for Defense Analysis, and Office of Naval Research.
In the context of geological materials, CTRW theory has been developed and applied extensively. For an extensive review, see B. Berkowitz, A. Cortis, M. Dentz and H. Scher, Modeling non-Fickian transport in geological formations as a continuous time random walk, Reviews of Geophysics, 44, RG2003, doi:10.1029/2005RG000178, 2006.
"Dispersion in heterogeneous geological formations" Brian Berkowitz
They also refer to the alpha disorder parameter as beta.
One significant difference is that solute does not induce a charge of current that we can measure. Instead the solute is measured directly as a concentration density.
Colloid Mediated Transport of Contaminants in Shallow Aquifers, 2009 Institute of
Hydrochemistry,
Technische Universität München
Dispersion exists in the case of Network TCP latencies where collisions may occur. The
velocities disperse from the maximum according to the nominal transit
time and the mean latencies are known. Since transit times are related
to the inverse of speed for a fixed separation, the distribution of
times goes like T*exp(-T/t)/t². The breakthrough curve for TCP dispersion should look like exp(-T/t) beyond the fixed latency due to wave propagation. see http://mobjectivist.blogspot.com/2008/09/network-dispersion.html
Just apply an accelerating electric field to the the TOF experiment and I guarantee the output will start looking like a Logistic.

Secretary of Energy

2009-06-25T00:08:00.000-07:00

This article by an organic farmer who used to work under Secretary of Energy Steven Chu has an interesting aside:

Cheer Up, It's Going to Get Worse

Fridley also believes assistance will not come from the world's leaders. Transition can only be a grass-roots revolution. He points out that Secretary of Energy Steven Chu was previously the director of Lawrence Berkeley National Laboratory, where Fridley has done much of his thinking about peak oil and Transition.

"[Chu] was my boss," Fridley says. "He knows all about peak oil, but he can't talk about it. If the government announced that peak oil was threatening our economy, Wall Street would crash. He just can't say anything about it."

Yet he does talk about painting roofs white, and arguing with boneheads such as Rep. Joe Barton.

Resource Substituition

2009-06-09T23:39:00.000-07:00

As a sad ironies go, this one takes the cake. We can actually observe a classic case of resource substitution taking place in real time. The island of Palau, formally mined for phosphate deposits, may now act as a resource for prisoner detainment.

HuffPo: Pacific Island Nation Agrees To Take Chinese Gitmo Detainees

The even more phophate-exploited island of Nauru may also hold some possibilities for prisoner detainment. These nations are desperate for money now that they have run out of natural resources.

Future Consequences

2009-06-04T22:33:00.000-07:00

Still Digging Up Exxon Valdez Oil, 20 Years Later

Are any of these the same scientists Exxon Mobil hired to deny AGW? It is well-documented you guys (assuming you really are a spokesman for ExMob) have spent loads of money on fake science and propaganda in that area - why wouldn't you spend heaps of money hiring pet "scientists" to muddy the waters (pun intended) on the long-term consequences of your careless spill? TOD

A careless spill and even more consistently careless accounting illustrating to us all that oil companies never show any foresight in projecting future consequences. The spill occurred as an accident, the aftermath borne of bad PR, but the lack of good depletion models came about by intent. Boardrooms at Exxon and elsewhere made conscious decisions to keep the peak oil message from the public, with the excuse that they had too much on their mind while looking for new oil.
Whocoodanode. That should really scare us the most.

The Role of Geology

2009-06-02T19:29:00.000-07:00

Several scientists have reviewed a book by Ian Pilmer called "Heaven and Earth. Global warming: the missing science"

"The most disappointing aspect of this book is the wide use of subjective and often emotive text, unbecoming of a scientific treatise, and this is despite a tirade in Chapter 1 suggesting that Ian is the only scientist, or geology the only field of science that understands the scientific method, is rigorous in the use of observations, and the setting and testing of hypotheses. To suggest that the discipline of geology is the framework in which to analyse the climate-change issue, is as indefensible as suggesting that climate models alone are the basis for determining human response to this issue." -- Dr Graeme Pearman, Honorary Senior Research Fellow at Monash University in Melbourne

Pilmer teaches geology at University of Adelaide, and from this review seems to claim that "geology the only field of science that understands the scientific method". I find that kind of hard to believe given the forthrightness of all the expert geologists warning us about oil depletion in the past (said with a sardonic grin).

As a reality check, we know that geology has a role, yet we need fresh perspectives and viewpoints to get to the truth of the matter. Whether one should add subjective or emotive narrative to the text, I could go one way or another. If it takes some opinionated tirades to get people interested, it may turn out worthwhile. I think of Richard Feynman, a very colorful character, who backed up his musings with tremendous insight.

I do wonder if someone has a Pilmer-like book critical of Peak Oil in the works.

Deltoid: http://scienceblogs.com/deltoid/2009/05/plimer_and_arctic_warming.php

Econ 101 and Resource Depletion

2009-05-25T04:51:00.000-07:00

We cannot easily pin down human and group decision making, yet the object acted on -- that of constrained resources -- we have a solid chance of quantifying. I really think we can make some progress understanding this and I think it has great importance for planning; I know many people believe that a collapse is a collapse and so why bother. Fine, but as long as policy forms the backbone of politics, we might as well present something something formal to our leaders besides anguish.

At one time I probably had some respect for economists, sensing that they must have brilliant theories. I once took a macroeconomics class from Walter Heller, who served as an economic advisor to both Kennedy and Johnson. Heller would show up for the lectures and say something profound (I assume, I can't remember), but mostly TA's taught the class. One TA sketched out a long equation on the chalkboard, let us ponder over it for a few seconds, and then promptly erased it. He said something to the effect, "that is the last time you will see an equation". I don't know why, but that action had always bothered me; it seemed like a resignation to failure.

Recently someone on TOD pointed to a couple of short articles by Robert Nadeau concerning the origins of economic theory in the 19th century

Short article : Long article

Nadeau asserts that not only does the formation of classical economics rely on strange analogies to physics, but that the analogies that the early economists chose came out of physics theories that nobody had verified yet. They basically consisted of vague postulations on the balance of energy existing in the "ether", in other words not the concrete Newtonian stuff that everyone agreed on, but ideas of the more abstract mysterious origins of electrical or magnetic energy that started to gain traction around that time. And to top it off, many of the early economists had strong religious beliefs and thought god had something to do with the economy (i.e. the idea of the invisible hand). This all occurred slightly before physicists like Maxwell figured out the practical formulation that has served us well. I got from Nadeau that the only perturbation to the original ideas came by way of Keynes and others who allowed that we could subtly guide economies, and this provided a not-so-invisible hand.

So, the analytic battle exists on several fronts. We have an economic theory that went down the toilet last year. We have the quants on Wall Street who have basically made a mess of the situation and caused many people to distrust any math. We have chaos believers who immediately put up red flags warning that dangers lurk in any analysis. We have other mathematicians that blow smoke by creating impressive formalisms that show purity but lack any connection to practicality. And finally we have a somewhat MEGO public that can't get interested.

I stand by the side of using practical probability and statistics and don't get cowed by the nay-sayers that say we can't and will never understand any of this. A few days ago, I found a book in my stacks called "The Earth's Dynamic Systems" written by Hamblin in 1975. This guy had definite concerns about our resources and comes across as the opposite of a cornucopian, yet he did write the following:

We basically know the extent of our mineral resources and the rates of our consumption. It is not difficult to project how long they will last.

So why is it that 30 years later, we still have no fundamental frameworks and argue over the reliability of heuristics such as HL? And still no one knows how much remains in SA reserve? Do we really believe that we can get by with back-of-the-envelope estimates alone? Yet as we look at our current predicament, a few valid theories of economics and of resource depletion dynamics certainly wouldn't have hurt.

Not online yet, but I found an interesting opinion piece in the latest Time magazine called "Excluding the Extremist" by Justin Fox. In the column he discusses the financial advice of Peter Schiff and others who don't toe the mainstream line.

"But there's a thriving line of academic research showing that including divergent opinions and models of how the world works makes groups better at solving problems"

That puts it fairly succinctly and I still think a quantitative analysis when applied to resource depletion holds promise. Unfortunately, the financial quants have given the term quantitative a bad rap.

Justin Fox has a book coming out next month called "The Myth of the Rational Market".

Bicycling takes place in the context of constant near misses
http://www.guardian.co.uk/politics/2009/may/23/boris-cycling

Hazards of biking

2009-04-30T20:16:00.000-07:00

This is basically what I go through on a weekly basis:
out of control

The Dutch biker made a nice maneuver at the end.

I currently use a fixed-year-gear with a reversible hub that you can convert to "suicide" mode by flipping the wheel (a flip-flop hub).

I should have gotten one of these bikes long ago. This chrome-moly "model" from Cross Lake Sales has no brand but it doesn't matter because they likely all get built in the same factory. I had a weird situation where the freewheel lost its ratchet during a period of near-freezing weather and I had to partly coast it home. But then it fixed itself a day later. It coasts very quietly so I think the freewheel has a tight margin on how efficiently it grabs. For under $300 these should be everywhere.

Cross Lake Sales

Innovative Bailout Machines

2009-03-20T20:34:00.000-07:00

From an IBM television commercial currently making the rounds:

“Math can do anything … it can fix the economy.”

Why it is hard to move forward

2009-03-13T20:42:00.000-07:00

White House: Greed will help

“In the past few years, we’ve seen too much greed and too little fear; too much spending and not enough saving; too much borrowing and not enough worrying,” Summers said Friday in a speech to the Brookings Institution. “Today, however, our problem is exactly the opposite.”

In remarks to a private dinner at the U.S. Chamber of Commerce on Wednesday, Summers was even blunter, according to an attendee: “Before, we had too much greed and too little fear. Now, we have too much fear and too little greed.”

Unfortunately this does not work with regards to conserving oil and getting off the oil spigot. Greed remained the only constant from the first moment we struck oil in the 1800's. It provided a virtually untapped source of stimulus to extract every last bit of oil we could access. Not that I think greed will ever disappear, but I thought perhaps that we can modulate this effect somehow. The fear of increasing scarcity of oil may have that effect due to demand destruction, but due to the engine of the economy and its requirements for energy to propel increasing GDP, we will likely see greed once again drag energy along, one way or another.

Lack of Data

2009-03-03T22:56:00.000-08:00

Does this not sound familiar?
Bernie Sanders wants Ben Bernanke to name names.

The money Sanders is referring to is loans the Fed has made outside the TARP program. Bernanke says the loans are "over-collateralized," but opted not to disclose anything more about them, citing the "stigma" attached to receiving such loans. To that, I have only two comments.

First, what stigma? The market is now assuming that every financial firm is in deep doo-doo. If anything, knowing which firms are receiving help and which are not removes one of the biggest uncertainties out there. It might actually improve the markets.

The lack of information in the world's finances directly parallels the lack of knowledge we have in oil reserve accounting.

What you cannot measure you cannot control. And that is why we have to model and simulate and analyze this stuff by our lonesome. The corporatocracy will never lift a finger to help.

Recipe for Disaster

2009-02-26T18:45:00.000-08:00

The limits of easy oil availability clearly caused the massive economic downturn that started last year.

From an article "Recipe for Disaster: The Formula that Killed Wall Street", the author tries to convince us that a single equation devised by David X. Li caused the financial crisis.

But we really can't blame David X. Li.

I went and hunted down Li's original paper and came to the conclusion that he tried his best to actually take into account the risk of dependent events. It just turned out that the math he invoked gave some simple metrics which people flocked to because it boiled down to a single number. His Gaussian copula function transforms a pair of independent probabilities to something that has a dependency, thus raising the risk of default from a multiplicative probability of small numbers to something orders of magnitude higher. This is actually a proper move. Somebody would sooner or later discover this heuristic, and Li happened to get there first.

However, the underlying math of invoking hazard functions for deducing the default risk has even more severe implications, and that concept existed before Li even tried to apply his own set of heuristics. Anyone that understands stationary processes knows that economics has a wicked time dependence and that huge risks are waiting in the wings since hazard rates are not static or stationary in the financial markets.

So what we are seeing with the Wired article and some other articles that preceded this, is an attempt to lay the blame on a single person, where we know that the actual blame needs to be placed on all the stooges that thought that they could get away with squirelling away and deferring the possibilities of ultimately risky strategies. We are seeing the annointment of Li as the "O-ring" failure mode of the financial sector. (The infamous O-ring of the first space shuttle disaster provided the convenient scapegoat to that whole affair, even though probably more endemic problems were at the root of the explosion)

Predicting failures on physical processes and actuarial types of things is much more refined than applying the same concepts to financial instruments. Likewise, the math, probability, and statistics behind oil depletion is as sound as grade-school arithmetic compared to the assumptions that the "Wall Street" quantitative analysts have used to make money. And the soundness of oil depletion analysis that I have done is not that I use a necessarilly different kind of math or work out the physical characteristics, but that I rely on limits and concrete measurable parameters.

The blame put on Li is scapegoating to the highest order, but the wheels are in motion. All I can say is that it is probably better to put the blame on a Wall Street type than on the Community Reinvestment Act. The seemingly well-educated do indeed deserve most of the scorn, not some working stiffs or low-income people trying to make ends meet.

The idea of CDS behind many of the bond investments is to insure against bond defaults. As good actuarians, the insurance analysts estimated the probabilities of failure/default and balanced those against the cost of "credit default swaps", i.e. an insurance agreement. The fact that clients could purchase an excess of these agreements, means that they could actually make money should the default occur. This, in contrast to conventional insurance where you need to guard solely against loss. So the effect of making money by the equivalent of "shorting" looms large.

No matter, the premise that the bond actuarians could actually predict bond default times deluded everyone. Instead of assuming fixed probabilities for time-before-default (the survival time), I could have predicted (had I cared) that the hazard functions modeled would not look anything like a constant stationary failure rate. They would all blow up as soon as resource depletion hit hard. This becomes a case of poor estimates of probabilities and the syncronicity of contrary economic events.

In the oil depletion analysis, I use dispersion on rates. Hazard functions act like rate functions as well, with a fixed Markovian probability of failure per unit of time defined in most cases. This produces a nominal damped exponential probability of survival for a bond (the bond dies, or euphemistically does not survive when it defaults). Yet we dare not use dispersion on failure rates for these bonds. The failures disperse most obviously on time constants, not on rates. Economic crises shouldn't really map like a random breakdown of a part with a slowly accumulating probability of failure over the part's lifetime. It just goes bang at any time and the whole thing falls apart -- that turns into a much more realistic model.

If we do the dispersion on time constants (a range in tau in the following figure), you can see that failures/defaults can occur much earlier than using a fixed tau (leading to the damped exponential).

As Taleb noted, the amount of charlatanism in all this is incredible. I really don't know how these "quants" could even consider their analyses robust at all ... unless they wanted to snooker everyone and earn the payout from the CDS's and rolled up CDO's when the bonds ultimately defaulted. It is entirely possible they had this in mind all along. Then it would all make sense. Otherwise, it makes economics and finance more an more like a dismal science.

Then why do not economists write or analyze anything about very simple economic matters such as resource (e.g. oil, natural gas, etc) depletion ?

Resource depletion is so simple and has nothing to do with belief systems or money-making opportunities, more like bean-counting, yet they can't be bothered to confront one of the most challenging problems of our times, and one that has huge economic implications?

The religion of economics is based on infinite resources and a forever expanding growth potential. Without that assumption, macroeconomic theory basically collapses on itself. And the economists (who ironically invented the sunk-cost model) do not want to see all their theoretical work go down the drain. This is indeed pseudo-science cloaked by the religious belief in maintaining tenure and subscribing to the comfort of group-think. In other words, don't veer too off-course and miss the chance for a Nobel prize. The fact that Wired even brought up the idea that David X. Li was potentially a candidate for an economics Nobel makes me ill.

USA Field Size Distribution Update

2009-02-12T18:33:00.000-08:00

I asked for some help in fleshing out some points on this reservoir field size distribution.

USA Field Size Distribution
A few days ago David N kindly sent me a copy of the Baker paper, and I transcribed some of the data points here.

The Baker paper collected statistics up to 1986, and consisted of data from about 14,000 fields. Robelius with more recent data put it at 34,500 fields.

From the data, it looks like between 1986 and now that many more of the smaller fields became developed and therefore got counted in the statistics. So in the last 20 years, we probably have gained substantial mileage from the low volume reservoirs, explained by either of these possibilities:

Smaller fields get deferred for production due to economic reasons
Smaller fields have a smaller cross-section for discovery so therefore show up later in the historical process. This second-order effect plays a smaller role in dispersive discovery than one would intuit -- i.e. not as if all big fields get found first, instead the probability weighting has a slight bias toward bigger fields.

Since 1986, I eyeballed the breakpoint at around 20 million-barrel fields. But now, if we try to generate a new transition breakpoint for the next 20 years, it would appear at less than 1 MB (C=0.6) as suggested by the dispersive aggregation model. This gets us close to physical limits and a real point of diminishing returns (but you probably knew that already).

Boyle's Law and proprtionate natural gas extraction

2009-01-15T19:09:00.000-08:00

I think I got this insight from commenter ElwoodElmore at TOD. He mentioned the use of the ideal gas law, also known as Boyle's Law to understand depletion from a reservoir of gaseous resources. Only material in the gas phase (such as natural gas) can compress with the following relationship between pressure (P) and volume (V):

PV = nRT

This basically says that when pressure increases, volume decreases proportionately, all other factors remaining equal. In other words, this basically states mathematically what we all intuitively understand in terms of compression -- we can compress gas but not liquids.

The other terms in the ideal gas law:

n = the number of moles of gas
R = the universal gas constant
T = the absolute temperature

form a constant only if the individual terms remain constant. Yet in reality, through the process of extraction, we do remove material from a pressurized reservoir. This causes the number of moles (n) to decrease; a mole defining a unit of dimension corresponding to about 6x10²³ molecules of gas.

As pressure (P) defines the rate of release from a natural gas reservoir:

P = nRT/V

and the volume (V) stays constant in the cavern, then the pressure must decrease as material gets removed from the reservoir.

This gives us the proportionality, P = k*n, whereby we draw down from any reservoir a linear fraction of the amount (n) left. This forms an alternative basis for the proportionate extraction of the Oil Shock model, this time applying it to natural gas.

This brings up another interesting observation. Another commenter at TOD, Kalle, posted a link to this PDF paper The evolution of giant oil field production behaviour. The authors have gotten on the right track with a depletion rate approach. They essentially observe a characteristic depletion rate value at peak production for a range of oil fields. The variance of this value remains relatively small.

They refer to a "The Maximum Depletion Rate Model" paper in press which I can't get a hold of. I bet that it uses the same principles as I use in the Oil Shock model. The characteristic rate becomes a more-or-less constant factor across a range of fields, making it eminently suitable and a verification for the Markovian basis of the shock model.

So we can substantiate that both oil and natural gas follow this proportionate draw-down behavior, but not necessarily for the same reasons. But that's what happens with the typical probabilistic model.

As Simon-Pierre Laplace once said:

The theory of probabilities is at bottom nothing but common sense reduced to calculus.

Ref: Practical Enhanced Reservoir Engineering: Assisted with Simulation : This came out in 2008 and covers Boyle's Law, among other things in 600+ pages.

M O B J E C T I V I S T

The Stretched Exponential

Information and Crude Complexity

Information and Crude Complexity

COMPLEXITY

SIMPLICITY

SCALING

DISCUSSION

References

How did we ever survive?

Geostatistics a fraud?

Kernel of Logistic for Dispersive Discovery

Creep Failure

Verifying Dispersion in Human Mobility

The Scaling Laws of Human Travel

Failure is the complement of success

Popcorn Popping as Discovery

Krugman, Cities, and Oil

Deviations from Hubbert Linearization

dispersive transport

Why is dispersion anomalous?

Hmm ... Reserve growth also considered anomalous...1

Breakthrough

Transport in SiO2

Transport in a-Si:H

Quantum Dots

The Connection

Double Breakthrough

Back to Oil

FOOTNOTES

Secretary of Energy

Resource Substituition

Future Consequences

The Role of Geology

Econ 101 and Resource Depletion

Hazards of biking

Innovative Bailout Machines

Why it is hard to move forward

Lack of Data

Recipe for Disaster

USA Field Size Distribution Update

Boyle's Law and proprtionate natural gas extraction

Hmm ... Reserve growth also considered anomalous...¹

Transport in SiO₂