In a conversation with a friend the other day, I mentioned this video by Greg Chaitin. Being the nerd I am, I re-watched it for about the 5th time. Doing so brought to mind ideas I had been meaning to express, but then forgot, but then remembered again on watching the video. It’s basically a meta-theory of knowledge…
Table of Contents:
Part 1 : Structuralism, Functionalism, Post-structuralism & Hinduism
Part 2 : The Graspable and the Ungraspable
Part 3 : A Toy Model Of Epistemology
My main sources are the work of Greg Chaitin and Stephen Wolfram (whom I blogged about already), but mostly I use Chaitin’s ideas. A summary of the ideas are as follows:
Some aspects of life and experience can be codified and formalized, which means we can mentally grasp these. Other aspects cannot be formalized and are ungraspable. Those aspects that can be formalized we can think of as “science”. The remaining aspects, although we can describe them, we cannot make a theory of them. Chaitin provides a wonderful formalism for how this can be conceptualized in terms of his algorithmic information theory (don’t be put off by the big name. The concepts we use below are easy). I go one step further here and apply the same ideas I used to describe life and death to describe the interplay between these two aspects of knowledge. The result describes all possible states of knowledge, and shows the interplay of the graspable and ungraspable aspects of knowing.
The recognition that some concepts are graspable and others are ungraspable is very old and has parallels through the whole history of human thought. Before presenting my ideas, let’s do a super-quick review of the historical ideas.
The Ungraspable and The Graspable
The ancient Greeks knew clearly the difference between the infinite and the finite, as did the ancient Hindus. This distinction is reflected in the difference between the continuous and the discreet, which led Leibniz to calculus. It is also the difference between the Relative and the Absolute, which are also called the Many and The One. It is what Nietzsche called Dionysian versus Apollonian. What Michael Moorcock called Chaos versus Law. It is the meaning versus the form. It is Romanticism versus Rationalism.
It always helps to make a table:
|Ungraspable, U||Graspable, G|
|The Absolute||The Relative|
|The One||The Many|
Many more pairs could be added, but I think you get the idea. These are all pairs of opposites. All generate what could be called “essential tensions”. All have played fundamental roles in the history of the human intellect. Contemplating these essential tensions is at the heart of the most important of Mankind’s endeavors.
Science is based on the ever-recurring tension between the inductive discovery and description of the facts of our experience, versus our deductive theoretical frameworks for understanding the facts of experience.
The common thread between the left and right columns is precisely as they are labeled. The things on the left are those things that are ungraspable in any fundamental sense. Our mind slips and slides as if on ice, or as if sloshing around on a slippery, greasy surface in the futile attempt to cage, describe, or codify these things: infinity, Absoluteness, Oneness, meaning, consciousness, etc. All of these, and other such, we can symbolize simply as “U” for “ungraspable”.
The things on the right are the things we can grasp, either in our hands or in our minds, but mainly in our minds. These are things we can hold, and turn around in our mind, and study and understand: the finite, the discreet, forms and laws, relative differences, and so on. These things and their cousins we can symbolize one and all by the letter “G” for that which we can grasp.
So we can speak of an essential tension in our experience between U and G. U and G are another way to speak of Yin and Yang.
Now we would like a framework for understanding U and G, and for that we turn to Professor Chaitin.
What Is A Theory?
Chaitin presents a set of mathematical ideas that are easiest to understand as a theory about scientific theories. It goes like this. On one hand you have a bunch of data. On the other hand you have some way to understand the data, call it a theory. You measure the number of bits in the data and the number of bits of the theory and you get two extreme possibilities:
- # of bits of data ≈ # of bits of the theory
- # of bits of data >>> # of bits of theory
We can use a popular scale metaphor to illustrate this idea:
In Case 1, the “theory” (green) has the same amount of information (would weigh the same on an “information scale”) as the data (orange). According to Chaitin, this is the very definition of “complexity”: we cannot reduce the data to a more compact form. We cannot compress the data. In this case, the theory is not a theory at all but is just an alternative way to express or to describe the data.
In Case 2, the information content of the theory is much less (<<) than the information content of the data. Here there is massive compression of information (like the way a ZIP or RAR file reduces the size of a file). The theory is, effectively, a compressed form of the data. The theory allows one to understand the data in a massively simplified form. Often in science, this is called “elegant”: a theory is elegant when a simple expression captures a large body of specific examples.
We see that Chaitin was able to generate an effective notion of complexity vs simplicity by simply considering the relationship between the size of the data in bits and the size of the theory in bits. There is much more to his model, but my simplifications retain the essence of Chaitin’s ideas. You can hear him discuss this model in the video linked above.
Before applying Chaitin’s idea, let’s just look at a few examples to cement the point.
Examples of Theories and Not-Theories
Example 1: The number pi (π) (example of case 2). The ratio of a circle’s circumference to its diameter equals π. The number π is an infinite bit decimal. We can think of the digits of π as the “data” and see that there are infinite bits of data. However, we can express π by the simple formula c/d (circumference/diameter). One imagines writing either: (1) a computer programing that solves the formula c/d to generate the digits of π, or (2) a program that simply contains all the digits of π and prints them one by one.
Which program is smaller? Obviously the program for the formula is much, much smaller. In principle, the program storing the digits would have infinite bits (which is impossible, but we are just being theoretical at the moment).
So, the formula for π could be considered a “theory” and the concept of π is “simple” and “not complex” because there is a way to represent π that is very small in bit size compared to the decimal representation of π.
[Side note: To me, when seen from this point of view, the whole thing looks like magic: we are here converting between an infinite and a finite thing with an equal sign between them!]
Example 2: Newton’s Law of Gravitation (also case 2). This can be used to calculate the orbits of the planets.
Granted it is not perfect. Einstein’s theory of gravity is better. Further, the real world application of any law of gravity is a many-bodied problem, which causes chaotic trajectories. Again, however, we are being theoretical here and can dispense with these issues for the moment.
The point is, we can have long lists of numbers that describe the time and position of the planets. Such a thing is called an ephemeris. To illustrate Chaitin’s point, all we need to do is (a) write Newton’s law on a piece of paper and (b) have a real ephemeris. Now weigh both of them on a scale. Newton’s law obviously weighs less. So, Newton’s law, in Chaitin’s terms is a real theory. It is a compression of a vast amount of data that can be calculated from the formula, a small part of which is contained in an ephemeris.
Example 3: Natural Things (3 examples of case 1): Consider the following pictures of: (a) finger prints, (b) rocks, and (c) leopard spots:
Can we find some theory that will generate the specific finger prints of the person in the top row but not the person in the bottom row? Or can we formulate a specific theory of why the upper right-hand rock has exactly the shape and color pattern it does, and why each of the rocks has the specific shape and color pattern it has? Can we write a specific theory of why the top leopard has exactly the patterns of spots on its face that it does?
The very attempt to formulate such specific theories would seem absurd. What is the point? In these cases we might hope to have some general theory of finger prints, or rocks, or leopard spots that can output, perhaps in a statistical sense, all of the possible specific variations. But the idea of a specific theory for each specific individual case doesn’t really make sense, and you will not find anybody undertaking such endeavors.
Nonetheless, these are real items in nature, and if we wish to be “scientific” about them, what are we supposed to do?
In each of these cases then, when it comes to the specific item, there is no general theory of that specific item. Now, we are looking at phenomena that have irreducible complexity, as Chaitin calls it. The best we can do is to describe each specific instance. In these types of cases, the description will have about the same information content in bits as the original system. There is no compression, there is no theory.
Two Types of Science
Now we can see that there is a parallel between systems that Chaitin considers “simple” and those that have “irreducible complexity” and the items in the table above that are “graspable” and “ungraspable”. The parallel, made plain, is:
Ungraspable, U ⇔ “complex” (case 1)
Graspable, G ⇔ “simple” (case 2)
There are basically two types of scientific description: (1) those that are mere descriptions, alternative ways to express phenomena for which no general law exists, and (2) those that can find general laws that are simple relative to the scope of phenomena they describe.
For case 1, perhaps the descriptions are given in ordinary language, or maybe in computer code, or perhaps in formalism designed specifically to capture a given phenomenon (e.g. the notation of chemistry for example). For case 2, the theories are generally expressed in the language of mathematics.
So really, by any accepted standard, case 2—those things that are graspable in terms of simple laws whose information content is much less than the data they describe—is what most people think of when they think of a “scientific theory”.
On the other hand, things like biology, the tree of life, the classification of the species; psychology, classifications of languages, personality types, or of the collective unconscious (ala Jung); these types of sciences are merely descriptive.
Not that I want to dwell on it too much here, but it is an instructive example. We can use the above criteria to gauge Darwin’s theory of evolution. Can we use Darwin’s theory to calculate the data of biology in the same way we can use Newton’s formulas to calculate the trajectories of the planets?
Not really; actually not at all. Darwin’s theory is qualitative and doesn’t allow us to calculate anything. At best, we might be able to convert it to algorithms that give broad statistical possibilities for things like gene distributions in a population. But as a theory, Darwinian evolution will never allow the calculation of all the specific life forms and their genealogy through time in the vast history of the Earth.
In contrast, Newton’s laws allow us to calculate the specific trajectory of our planet Earth around our specific Sun.
What this implies is that life and evolution, and by this I mean the exact specific instances of life and evolution on Earth, are irreducibly complex. To imagine that some theory will come along and calculate the exact form of life and its exact trajectory seems like a pipe dream in terms of Chaitin’s framework.
For the last few Isaac Asimov fans out there, this line of thought suggests that Hari Seldon’s theory of “psychohistory” is a complete pipedream as portrayed in Asimov’s (what used to be) famous Foundation trilogy.
What I am saying is that large swaths of Nature fall under the “U” heading in the table above. It would seem that only a limited range of phenomena can be captured under the “G” heading.
In other words, most of the phenomena of Nature are ungraspable by our mind in the sense that we will find no easier way to understand them than by simply describing them, by taking them at face value.
This all reflects an old philosophical dilemma of the contingent versus the necessary, something Leibniz discussed a lot. An example of “necessary” is “2 + 2 = 4”, this is just necessarily true. The idea of “contingent” is the pattern of stripes on my cat’s fur. There is no necessity for this that one can imagine, but it is a fact, it is real. It would seem we can map Chaitin’s “simple” theories into the necessary, and the irreducibly complex into the contingent, like the leopard spots, rocks and finger prints illustrated above.
This is getting longer than I thought, so we will continue the tale in part 2. Let’s summarize what was said above. We can identify two broad categories of things our minds contemplate: the graspable and the ungraspable, with different examples of each provided in the table above. We can then map these onto Chaitin’s idea of “simple” and “complex” theories based on the relative information content of data and our ability to simplify and understand the data.
Next time, we will mix this with some nonlinear dynamics and get a really weird picture that also harkens back to Leibniz, who also spent a lot of time talking about all the possible worlds, and why ours is the best of all possible worlds.