Metarationality: a messy introduction

In the last couple of years, David Chapman’s Meaningness site has reached the point where enough of the structure is there for a bunch of STEM nerds like me to start working out what he’s actually talking about. So there’s been a lot of excited shouting about things like ‘metarationality’ and ‘ethnomethodology’ and ‘postformal reasoning’.

Not everyone is overjoyed by this. There was a Less Wrong comment by Viliam back in January which I thought made the point clearly:

How this all feels to me:

When I look at the Sequences, as the core around which the rationalist community formed, I find many interesting ideas and mental tools. (Randomly listing stuff that comes to my mind: Bayes theorem, Kolmogorov complexity, cognitive biases, planning fallacy, anchoring, politics is the mindkiller, 0 and 1 are not probabilities, cryonics, having your bottom line written first, how an algorithm feels from inside, many-worlds interpretation of quantum physics, etc.)

When I look at “Keganism”, it seems like an affective spiral based on one idea.

I am not saying that it is a wrong or worthless idea, just that comparing “having this ‘one weird trick’ and applying it to everything” with the whole body of knowledge and attitudes is a type error. If this one idea has merit, it can become another useful tool in a large toolset. But it does not surpass the whole toolset or make it obsolete, which the “post-” prefix would suggest.

Essentially, the “post-” prefix is just a status claim; it connotationally means “smarter than”.

To compare, Eliezer never said that using Bayes theorem is “post-mathematics”, or that accepting many-worlds interpretation of quantum physics is “post-physics”. Because that would just be silly. Similarly, the idea of “interpenetration of systems” doesn’t make one “post-rational”.

In other words, what have the metarationalists ever done for us? Rationality gave us a load of genuinely exciting cognitive tools. Then I went to metarationality, and all I got were these lousy Kegan stages.

This would be a very fair comment, if the only thing there was the Kegan idea. I’m one of the people who does find something weirdly compelling in that, and I was thinking about it for months. But I have to admit it’s a total PR disaster for attracting the people who don’t, on the level of equating torture with dust specks. (At least PR disasters are one thing that rationalists and metarationalists have in common!)

‘You don’t understand this because you haven’t reached a high enough stage of cognitive development’ is an obnoxious argument. People are right to want to run away from this stuff.

Also, as Viliam points out, it’s just one idea, with a rather unimpressive evidence base at that. That wouldn’t warrant a fancy new word like ‘metarationality’ on its own. [1]

Another idea I sometimes see is that metarationality is about the fact that a particular formal system of beliefs might not work well in some contexts, so it’s useful to keep a few in mind and be able to switch between them. This is point 2 of a second comment by Viliam on a different Less Wrong thread, trying to steelman his understanding of metarationality:

Despite admitting verbally that a map is not the territory, rationalists hope that if they take one map, and keep updating it long enough, this map will asymptotically approach the territory. In other words, that in every moment, using one map is the right strategy. Meta-rationalists don’t believe in the ability to update one map sufficiently (or perhaps just sufficiently quickly), and intentionally use different maps for different contexts. (Which of course does not prevent them from updating the individual maps.) As a side effect of this strategy, the meta-rationalist is always aware that the currently used map is just a map; one of many possible maps. The rationalist, having invested too much time and energy into updating one map, may find it emotionally too difficult to admit that the map does not fit the territory, when they encounter a new part of territory where the existing map fits poorly. Which means that on the emotional level, rationalists treat their one map as the territory.

Furthermore, meta-rationalists don’t really believe that if you take one map and keep updating it long enough, you will necessarily asymptotically approach the territory. First, the incoming information is already interpreted by the map in use; second, the instructions for updating are themselves contained in the map. So it is quite possible that different maps, even after updating on tons of data from the territory, would still converge towards different attractors. And even if, hypothetically, given infinite computing power, they would converge towards the same place, it is still possible that they will not come sufficiently close during one human life, or that a sufficiently advanced map would fit into a human brain. Therefore, using multiple maps may be the optimal approach for a human. (Even if you choose “the current scientific knowledge” as one of your starting maps.)

I don’t personally find the map/territory distinction all that helpful and will talk about that more later. Still, I think that this is OK as far as it goes, and much closer to the central core than the Kegan stage idea. To me it’s rather a vague, general sort of insight, though, and there are plenty of other places where you could get it. I’m not surprised that people aren’t falling over themselves with excitement about it.

I think people are looking for concrete, specific interesting ideas, along the lines of Viliam’s list of concepts he learned from rationality. I very much have this orientation myself, of always needing to go from concrete to abstract, so I think I understand a bit of what’s missing for a lot of people.

(Example: My first experience of reading Meaningness a few years ago was to click around some early posts, read a lot of generalities about words like ‘nebulosity’ and ‘pattern’ and ‘eternalism’, and completely miss the point. ‘Maybe this is some kind of Alain de Botton style pop philosophy? Anyway, it’s definitely not something I care about.’ There are a lot more specifics now, so it’s much easier to follow a concrete-to-abstract path and work out what’s going on. If you also tend to learn this way, I’d advise starting with the most recent parts, or something from the metablog, and working your way back.)

I do think that these concrete, specific ideas exist. I wrote a little bit in that first Less Wrong thread about what I found interesting, but it’s pretty superficial and I’ve thought about it a lot more since. This is my first attempt at a more coherent synthesis post. I’ve written it mostly for my own benefit, to make myself think more clearly about some vague parts, and it’s very much work-in-progress thinking out loud, rather than finalised understanding. This the kind of thing I enjoy writing, out on the ‘too early’ side of this ‘when to write’ graph. (This blog is mostly for talking about about things I’m still trying to understand myself, rather than polished after-the-fact explanation.)

There’s a lot there, it’s real stuff rather than vague generalities about how systems sometimes don’t work very well, and it’s very much worth bothering with. That’s what I want to try and get across in this post!

Also, this is just my idea of what’s interesting, and I’ve stuck to one route through the ideas in this post because otherwise the length would get completely out of control. Maybe others see things completely differently. If so, I’d like to know. I’ve filled this with links to make it easy to explore elsewhere.


This post is pretty long, so maybe a summary of what’s in it would be a good idea. Roughly, I’m going to cover:

  • How we think we think, vs. how we actually think: if you look closely at even the most formal types of reasoning, what we’re actually doing doesn’t tend to look so rigorous and logical.
  • Still, on its own that wouldn’t preclude the brain doing something very rigorous and logical behind the scenes, in the same way that we don’t consciously know about the image processing our brain is doing for our visual field. Some discussion of why the prospects for that don’t look great either.
  • Dumb confused interlude on the unreasonable effectiveness of mathematics.
  • The cognitive flip: queering the inside-the-head/outside-the-head binary. Sarah Perry’s theory of mess as a nice example of this.
  • Through fog to the other side: despite all this confusion, we can navigate anyway.

How we think we think, vs. how we actually think

I’ve got like a thousand words into this without bringing up my own weirdo obsession with mathematical intuition, which is unusually restrained, so let’s fix that now. It’s something I find interesting in itself, but it’s also this surprisingly direct rabbit hole into some rather fundamental ideas about how we think.

Take mathematical proofs, for example. In first year of a maths degree, everyone makes a massive deal out of how you’re going to be learning to write proofs now, and how this is some extra special new rigorous mode of thought that will teach you to think like a real mathematician. This is sort of true, but I found the transition incredibly frustrating and confusing, because I could never work out what was going on. What level of argument actually constitutes a proof?

I’d imagined that a formal proof would be some kind of absolutely airtight thing where you started with a few reasonable axioms and some rules of inference, and derived everything from that. I was quite excited, because it did sound like that would be a very useful thing to learn!

We did learn a little bit of formal logic stuff, basic propositional and predicate calculus. But most of the proofs we saw were not starting there, along with say the axioms of the real numbers, and working up. (If they had, they’d have been hundreds of pages long and completely unilluminating.)

Instead we proved stuff at this weird intermediate level. There were theorems like ‘a sequence of numbers that always increases, but is bounded by some upper value, will converge’, or ‘a continuous function f on the interval from a to b takes on all the values between f(a) and f(b)’. We weren’t deriving these from basic axioms. But also we weren’t saying ‘yeah that’s obviously true, just look at it’, like I’d have done before going to the class. Instead there was this special domain-specific kind of reasoning with lots of epsilons and deltas, and a bunch of steps to include.

How do you know which steps are the important ones to include? I never really worked that out at the time. In practice, I just memorised the proof and regurgitated it in the exam, because at least that way I knew I’d get the level right. Unsurprising, this didn’t exactly ignite a fire of deep intellectual excitement inside me. In the end I just gave up and started taking as many applied maths and physics courses as possible, where I could mostly continue doing whatever I was doing before they tried to introduce me to this stupid proofs thing.

If I went back now I’d probably understand. That weird intermediate level probably is the right one, the one that fills in the holes where your intuition can go astray but also avoids boring you with the rote tedium of obvious deductive steps. [2] Maybe seeing more pathological examples would help, cases where your intuitive ideas really do fail and this level of rigour is actually useful. [3]

An interesting question at this point is, how do you generate these intermediate-level proofs? One answer would be that you are starting from the really formal level, thinking up very careful airtight proofs in your head, and then only writing down some extracted key steps. I think it’s fairly clear you’re not doing that, at least at the level of conscious access (more on the idea that that’s what we’re ‘really’ doing later).

The reality seems to be messier. Explicitly thinking through formal rules is useful some of the time. But it’s only one method among many.

Sometimes, for example, you notice that, say, an equation admits an algebraic simplification you’ve used many times before, and mechanistic formula-churning takes over for a while. This may have required thinking through formal rules when you first learned it, but by now your fingers basically know the answer. Sometimes the resulting expression looks messy, and some rather obsessive part of the mind is not happy until like terms are collected tidily together. Sometimes you realise that part of the complexity of the problem ‘is just book-keeping’, and can be disposed of by, for example, choosing the origin of your coordinate system sensibly. The phrase ‘without loss of generality’ becomes your friend. Sometimes a context-specific trick comes to mind (‘probably it’s another one of those thingies where we sandwich it between two easier functions and show that they converge’).

There’s no real end to the list of things to try. Generating proofs is a fully general education in learning to Think Real Good. But some fundamental human faculties come up again and again. The mathematician Bill Thurston gave a nice list of these in his wonderful essay On proof and progress in mathematics. This is the sort of essay where when you start quoting it, you end up wanting to quote the whole lot (seriously just go and read it!), but I’ve tried to resist this and cut the quote down to something sensible:

(1) Human language. We have powerful special-purpose facilities for speaking and understanding human language, which also tie in to reading and writing. Our linguistic facility is an important tool for thinking, not just for communication. A crude example is the quadratic formula which people may remember as a little chant, “ex equals minus bee plus or minus the square root of bee squared minus four ay see all over two ay.” …

(2) Vision, spatial sense, kinesthetic (motion) sense. People have very powerful facilities for taking in information visually or kinesthetically, and thinking with their spatial sense. On the other hand, they do not have a very good built-in facility for inverse vision, that is, turning an internal spatial understanding back into a two-dimensional image. Consequently, mathematicians usually have fewer and poorer figures in their papers and books than in their heads …

(3) Logic and deduction. We have some built-in ways of reasoning and putting things together associated with how we make logical deductions: cause and effect (related to implication), contradiction or negation, etc. Mathematicians apparently don’t generally rely on the formal rules of deduction as they are thinking. Rather, they hold a fair bit of logical structure of a proof in their heads, breaking proofs into intermediate results so that they don’t have to hold too much logic at once …

(4) Intuition, association, metaphor. People have amazing facilities for sensing something without knowing where it comes from (intuition); for sensing that some phenomenon or situation or object is like something else (association); and for building and testing connections and comparisons, holding two things in mind at the same time (metaphor). These facilities are quite important for mathematics. Personally, I put a lot of effort into “listening” to my intuitions and associations, and building them into metaphors and connections. This involves a kind of simultaneous quieting and focusing of my mind. Words, logic, and detailed pictures rattling around can inhibit intuitions and associations.

(5) Stimulus-response. This is often emphasized in schools; for instance, if you see 3927 × 253, you write one number above the other and draw a line underneath, etc. This is also important for research mathematics: seeing a diagram of a knot, I might write down a presentation for the fundamental group of its complement by a procedure that is similar in feel to the multiplication algorithm.

(6) Process and time. We have a facility for thinking about processes or sequences of actions that can often be used to good effect in mathematical reasoning. One way to think of a function is as an action, a process, that takes the domain to the range. This is particularly valuable when composing functions. Another use of this facility is in remembering proofs: people often remember a proof as a process consisting of several steps.

Logical deduction makes an appearance on this list, but it’s not running the show. It’s one helpful friend in a group of equals. [4]

Mathematics isn’t special here: it just happened to be my rabbit hole into thinking about how we think about things. It’s a striking example, because the gap between the formal stories we like to tell and the messy reality is such a large one. But there are many other rabbit holes. User interface design (or probably any kind of design) is another good entry point. You’re looking at what people actually do when using your software, not your clean elegant theory of what you hoped they’d do. [5]

Apparently the general field that studies what people actually do when they work with systems is called ‘ethnomethodology’. Who knew? Why does nobody tell you this??

(Side note: if you poke around Bret Victor’s website, you can find this pile of pdfs, which looks like some kind of secret metarationality curriculum. You can find the Thurston paper and some of the mathematical intuition literature there, but overall there’s a strong design/programming focus, which could be a good way in for many people.)

After virtue epistemology

On its own, this description of what we do when we think about a problem shouldn’t necessarily trouble anyone. After all, we don’t expect to have cognitive access to everything the brain does. We already expect that we’re doing a lot of image processing and pattern recognition and stuff. So maybe we’re actually all running a bunch of low-level algorithms in our head which are doing something very formal and mathematical, like Bayesian inference or something. We have no direct access to those, though, so maybe it’s perfectly reasonable that what we see at a higher level looks like a bunch of disconnected heuristics. If we just contented ourselves with a natural history of those heuristics, we might be missing out on the chance of a deeper explanatory theory.

Scott Alexander makes exactly this point in a comment on Chapman’s blog:

Imagine if someone had reminded Archimedes that human mental simulation of physics is actually really really good, and that you could eyeball where a projectile would fall much more quickly (and accurately!) than Archimedes could calculate it. Therefore, instead of trying to formalize physics, we should create a “virtue physics” where we try to train people’s minds to better use their natural physics simulating abilities.

But in fact there are useful roles both for virtue physics and mathematical physics. As mathematical physics advances, it can gradually take on more of the domains filled by virtue physics (the piloting of airplanes seems like one area where this might have actually happened, in a sense, and medicine is in the middle of the process now).

So I totally support the existence of virtue epistemology but think that figuring out how to gradually replace it with something more mathematical (without going overboard and claiming we’ve already completely learned how to do that) is a potentially useful enterprise.

Chapman’s response is that

… if what I wrote in “how to think” looked like virtue ethics, it’s probably only because it’s non-systematic. It doesn’t hold out the possibility of any tidy answer.

I would love to have a tidy system for how to think; that would be hugely valuable. But I believe strongly that there isn’t one. Pursuing the fantasy that maybe there could be one is actively harmful, because it leads away from the project of finding useful, untidy heuristics.

This is reasonable, but I still find it slightly disappointing, in that it seems to undersell the project as he describes it elsewhere. It’s true that Chapman isn’t proposing a clean formal theory that will explain all of epistemology. But my understanding is that he is trying to do something more explanatory than just cataloguing a bunch of heuristics, and that doesn’t come across here. In other parts of his site he gives some indication of the sorts of routes to better understanding of cognition he finds promising.

Hopefully he’s going to expand on the details some time soon, but it’s tempting to peek ahead and try and work out the story now. Again, I’m no expert here, at all, so for the next section assume I’m doing the typical arrogant physicist thing.

The posts I linked above gave me lots of pieces of the argument, but at first I couldn’t see how to fit them into a coherent whole. Scott Alexander’s recent predictive processing post triggered a few thoughts that filled in some gaps, so I went and pestered Chapman in his blog comments to check I had the right idea.

Scott’s post is one of many posts where he distinguishes between ‘bottom-up’ and ‘top-down’ processing.

Bottom-up processing starts with raw sensory data and repackages it into something more useful: for vision, this would involve correcting for things like the retinal blind spot and the instability of the scene as we move our gaze. To quote from the recent post:

The bottom-up stream starts out as all that incomprehensible light and darkness and noise that we need to process. It gradually moves up all the cognitive layers that we already knew existed – the edge-detectors that resolve it into edges, the object-detectors that shape the edges into solid objects, et cetera.

Top-down processing is that thing where Scott writes ‘the the’ in a sentence and I never notice it, even though he always does it. It’s the top-down expectations (‘the word “the” isn’t normally repeated’) we’re imposing on our perceptions.

This division makes a lot of sense as a general ordering scheme: we know we’re doing both these sorts of things, and that we somehow have to take both into consideration at once when interpreting the scene. The problem is working out what’s relevant. There’s a gigantic amount of possibly relevant sense data, and a gigantic amount of possible relevant existing knowledge. We need to somehow extract the parts that are useful in our situation and make decisions on the fly.

On the bottom-up side, there are some reasonable ideas for how this could work. We can already do a good job of writing computer algorithms to process raw pixel data and extract important features. And there is often a reasonably clearcut, operational definition of what ‘relevant’ could possibly mean.

Relevant objects are likely to be near you rather than miles away; and the most salient objects are likely to be the ones that recently changed, rather than ones that have just sat there for the last week. These sort of rules reduce the pressure to have to take in everything, and push a lot of the burden onto the environment, which can cue you in.

This removes a lot of the work. If the environment can just tell us what to do, there’s no need to go to the effort of building and updating a formal internal model that represents it all. Instead of storing billions of Facts About Sense Data you can have the environment hand them to you in pieces as required. This is the route Chapman discusses in his posts, and the route he took as an AI researcher, together with his collaborator Phil Agre (see e.g. this paper (pdf) on a program, Pengi, that they implemented to play an arcade game with minimal representation of the game surroundings).

In the previous section I tried to weaken the importance of formal representations from the outside, by looking at how mathematical reasoning occurs in practice as a mishmash of cognitive faculties. Situated cognition aims to weaken it from the inside instead by building up models that work anyway, without the need for too much representation.

Still, we’re pushing some way beyond ‘virtue epistemology’, by giving ideas for how this would actually work. In fact, so far there might be no disagreement with Scott at all! Scott is interested in ideas like predictive processing and perceptual control theory, which also appear to look at changes in the sense data in front of you, rather than trying to represent everything as tidy propositions.

However, we also have to think about the top-down side. Scott has the following to say about it:

The top-down stream starts with everything you know about the world, all your best heuristics, all your priors, everything that’s ever happened to you before – everything from “solid objects can’t pass through one another” to “e=mc^2” to “that guy in the blue uniform is probably a policeman”.

This looks like the bit where the representation sneaks in. We escaped the billions of Facts About Sense Data, but that looks very like billions of Facts About Prior Experience to me. We’d still need to sort through them and work out what’s relevant somehow. I haven’t read the Clark book, and Scott’s review is very vague about how this works:

Each level receives the predictions from the level above it and the sense data from the level below it. Then each level uses Bayes’ Theorem to integrate these two sources of probabilistic evidence as best it can.

My response is sort of an argument from incredulity, at this point. Imagine expanding out the list of predictions, to cover all the things you know at the same level of specificity as ‘that guy in the blue uniform is probably a policeman’. That is an insane number of things! And you’re expecting people to sort through these on the fly, and compute priors for giant lists of hypotheses, and keep them reasonably consistent, and then run Bayesian updates on them? Surely this can’t be the story!

Arguments from incredulity aren’t the best kinds of arguments, so if you do think there’s a way that this could plausibly work in real time, I’d love to know. [6]

Coping with the complexity would be much plausible if we could run the same situational trick as with the bottom-up case, finding some way of avoiding having to represent all this knowledge by working out which parts are important in the current context. But this time it’s far harder to figure out operationally how that would work. There’s no obvious spatial metric on our thoughts such that we can determine ‘which ones are nearby’ or ‘which ones just changed’ as a quick proxy for relevance. And the sheer variety of types of thought is daunting – there are no obvious ordering principles like the three dimensions of the visual field.

Chapman’s reply when I asked him about it was:

It was when we realized we had no idea how to address this that Phil [Agre] and I gave up on AI. If you take a cognitivist approach—i.e. representing knowledge using something like language or logic—the combinatorics are utterly impossible. And we had no good alternative.

So it’s not like I can suggest anything concrete that’s better than the billions of Facts About Prior Experience. That’s definitely a major weakness of my argument here! But maybe it’s enough to see that we didn’t need this explicit formal representation so far, and that it’s going to be a combinatorial disaster zone if we bring in now. For me, that’s enough clues that I might want to look elsewhere.

Brief half-baked confused interlude: the mathematical gold standard

Maybe you could go out one stage further, though. OK, so we’re not consciously thinking through a series of formal steps. And maybe the brain isn’t doing the formal steps either. But it is true that the results of correct mathematical thinking are constrained by logic, that to count as correct mathematics, your sloppy intuitive hand-waving eventually has to cash out in a rigorous formal structure. It’s somehow there behind the scenes, like a gold standard backing our messy exchanges.

Mathematics really is unreasonably effective. Chains of logic do work amazingly well in certain domains.

I think this could be part of the MIRI intuition. ‘Our machines aren’t going to do any of this messy post-formal heuristic crap even if our brains are. What if it goes wrong? They’re going to actually work things out by actual formal logic.’ (Or at the very least, they’re going to verify things afterwards, with actual formal logic. Thanks to Kaj Sotala for pointing this distinction out to me a while ago.)

I don’t understand how this is possibly going to work. But I can’t pretend that I really know what’s going on here either. Maths feels like witchcraft to me a lot of the time, and most purported explanations of what it is make no sense to me. Philosophy of mathematics is bad in exactly the same way that moral philosophy is bad, and all the popular options are a mess. [7]

The cognitive flip

I think I want to get back to slightly more solid ground now. It’s not going to be much more solid though, because I’m still trying to work this out. There’s a bit of Chapman’s ‘Ignorant, irrelevant, and inscrutable’ post that puzzled me at first:

Many recall the transition from rationalism to meta-rationalism as a sudden blinding moment of illumination. It’s like the famous blotchy figure above: once you have seen its meaning, you can never unsee it again. After you get meta-rationality, you see the world differently. You see meanings rationalism cannot—and you can never go back.

This is probably an illusion of memory. The transition occurs only after years of accumulating bits of insight into the relationship between pattern and nebulosity, language and reality, math and science, rationality and thought. At some point you have enough pieces of the puzzle that the overall shape falls into place—but even then there’s a lot of work left to fill in the holes.

Now first of all, I don’t recall any such ‘blinding moment of illumination’ myself. That possibly doesn’t mean much, as it’s not supposed to be compulsory or anything. (Or maybe I just haven’t had it yet, and everything will make sense tomorrow…)

What was more worrying is that I had no clear idea of what the purported switch was supposed to be. I’ve thought about this a bit more now and I think I’m identifying it correctly. I think that the flip is to do with removing the clean split between having a world outside the head and a representation inside it.

In the ‘clean split’ worldview, you have sensory input coming in from the outside world, and your brain’s job is to construct an explicit model making sense of it. Here’s a representative quote summarising this worldview, taken from the Less Wrong wiki article on ‘The map is not the territory’:

Our perception of the world is being generated by our brain and can be considered as a ‘map’ of reality written in neural patterns. Reality exists outside our mind but we can construct models of this ‘territory’ based on what we glimpse through our senses.

This worldview is sophisticated enough to recognise that the model may be wrong in some respects, or it may be missing important details – ‘the map is not the territory’. However, in this view there is some model ‘inside the head’, whose job is to represent the outside world.

In the preceding sections we’ve been hacking away at the plausibility of this model. This breakdown frees us to consider different models of cognition, models that depend on interactions between the brain and the environment. Let’s take an example. One recent idea I liked is Sarah Perry’s proposed theory of mess.

Perry tackles the question of what exactly constitutes a mess. Most messes are made by humans. It’s rare to find something in the natural world that looks like a mess. Why is that?

Maybe this sounds like an obscure question. But it’s exactly the kind of question you might sniff out if you were specifically interested in breaking down the inside-the-head/outside-the-head split. (In fact, maybe this is part of the reason why metarationality tends to look contentless from the outside. Without the switch everything just looks like an esoteric special topic the author’s interested in that day. You don’t care about mess, or user interface design, or theatre improv, or mathematical intuition, or whatever. You came here for important insights on reality and the nature of cognition.)

You’re much better off reading the full version, with lots of clever visual examples, and thinking through the answer yourself. But if you don’t want to do that, her key thesis is:

… in order for mess to appear, there must be in the component parts of the mess an implication of extreme order, the kind of highly regular order generally associated with human intention. Flat uniform surfaces and printed text imply, promise, or encode a particular kind of order. In mess, this promise is not kept. The implied order is subverted.

So mess is out in the world – you need a bunch of the correct sort of objects in your visual field to perceive a mess. But mess is not just out in the world – you also have to impose your own expectation of order on the scene, based on the ordered contexts that the objects are supposed to appear in. Natural scenes don’t look like a mess because no such implied order exists.

Mess confuses the neat categories of ‘in the world’ and ‘in my head’:

It is as if objects and artifacts send out invisible tendrils into space, saying, “the matter around me should be ordered in some particular way.” The stronger the claim, and the more the claims of component pieces conflict, the more there is mess. It is these invisible, tangled tendrils of incompatible orders that we are “seeing” when we see mess. They are cryptosalient: at once invisible and obvious.

In the language of the previous section, we’re getting in bottom-up signals from our visual field, which is resolved into the bunch of objects. And then by some ‘magic’ (invisible tendrils? a cascade of Bayesian updates?) the objects are recognised from the top-down side as implying an incompatible pile of different ordering principles. We’re seeing a mess.

Here’s some of my mess:

2017-09-02 21.34.17

I can sort of still fit this into the map/territory scheme. Presumably the table itself and the pile of disordered objects are in the territory. And then the map would be… what? Some mental mess-detecting faculty that says ‘my model of those objects is that they should be stacked neatly, looks like they aren’t though’?

There is still some kind of principled distinction here, some way to separate the two. The territory corresponds pretty well to the bottom-up bit, and is characterised by the elements of experience that respond in unpredictable, autonomous ways when we investigate them. There’s no way to know a priori that my mess is going to consist of exercise books, a paper tetrahedron and a kitten notepad. You have to, like, go and look at it.

The map corresponds better to the top-down bit, the ordering principles we are trying to impose. These are brought into play by the specific objects we’re looking at, but have more consistency across environments – there are many other things that we would characterise as mess.

Still, we’ve come a long way from the neat picture of the Less Wrong wiki quote. The world outside the head and the model inside it are getting pretty mixed up. For one thing, describing the remaining ‘things in the head’ as a ‘model’ doesn’t fit too well. We’re not building up a detailed internal representation of the mess. For another, we directly perceive mess as mess. In some sense we’re getting the world ‘all at once’, without the top-down and bottom-up parts helpfully separated.

At this point I feel I’m getting into pretty deep metaphysical waters, and if I go much further in this direction I’ll make a fool of myself. Probably a really serious exploration in this direction involves reading Heidegger or something, but I can’t face that right now so I think I’ll finish up here.

Through fog to the other side

A couple of months ago I had an idea for a new blog post, got excited about it and wrote down this quick outline. That weekend I started work on it, and slowly discovered that every sentence in the outline was really an IOU for a thousand words of tricky exposition. What the hell had I got myself into? This has been my attempt to do the subject justice, but I’ve left out a lot. [8]

I hope I’ve at least conveyed that there is a lot there, though. I’ve mostly tried to do that through the methods of ‘yelling enthusiastically about things I think are worth investigating’ and ‘indicating via enthusiastic yelling that there might be a pile of other interesting things nearby, just waiting for us to dig them up’. Those are actually the things I’m most keen to convey, more even than the specifics in this post, but to do that I needed there to be specifics.

I care about this because I feel like I’m surrounded by a worrying cultural pessimism. A lot of highly intelligent people seem to be stuck in the mindset of ‘all the low-hanging fruit’s been plucked, everything interesting requires huge resources to investigate, you’re stuck being a cog in an incredibly complicated system you can barely understand, it’s impossible to do anything new and ambitious by yourself.’

I’ve gone through the PhD pimple factory myself, and I understand how this sort of constricting view takes hold. I also think that it is, to use a technical phrase, total bollocks.

My own mindset, which the pimple factory didn’t manage to completely destroy, is very different, and my favourite example to help explain where I’m coming from has always been the theory of evolution by natural selection. The basic idea doesn’t require any very complicated technical setup; you can explain the it in words to a bright ten-year-old. It’s also deeply explanatory: nothing in biology makes sense except in the light of it. And yet Darwin’s revolution came a couple of hundred years after the invention of calculus, which requires a lot more in the way of technical prerequisites to understand.

Think of all those great mathematicians — Gauss, Lagrange, Laplace — extending and applying the calculus in incredibly sophisticated ways, and yet completely clueless about basic questions that the bright ten-year-old could answer! That’s the situation I expect we’re still in. Many other deep ways of understanding the world are probably still hidden in fog, but we can clear more of it by learning to read new meaning into the world in the right ways. I don’t see any reason for pessimism yet.

This is where the enthusiastic yelling comes from. Chapman’s Meaningness project attacks the low-hanging-fruit depressive slump both head on by explaining what’s wrong with it, and indirectly by offering up an ambitious, large-scale alternative picture full of ideas worth exploring. We could do with a lot more of this.

It may look odd that I’ve spent most of this post trying to weaken the case for formal systems, and yet I’m finishing off by being really excitable and optimistic about the prospects for new understanding. That’s because we can navigate anyway! We might not think particularly formally when we do mathematics, for example, but nothing about that stops us from actually getting the answer right. A realistic understanding of how we reason our way through messy situations and come to correct conclusions anyway is likely to help us get better at coming up with new ideas, not worse. We can clear some more of that fog and excavate new knowledge on the other side.


1. I don’t really love the word ‘metarationality’, to be honest. It’s a big improvement on ‘postrationality’, though, which to me has strong connotations of giving up on careful reasoning altogether. That sounds like a terrible idea.

‘Metarationality’ sounds like a big pretentious -ism sort of word, but then a lot of the fault of that comes from the ‘rationality’ bit, which was never the greatest term to start with. I quite like Chapman’s ‘the fluid mode’, but ‘metarationality’ seems to be sticking, so I’ll go with that. (back)

2. There’s also a big social element that I didn’t get at the time. If you’re a beginner handing in homework for your first analysis course, you may need to put a lot of steps in, to convince the marker that you understand why they’re important. If you’re giving a broad overview to researchers in a seminar, you can assume they know all of that. There’s no one canonical standard of proof.

At the highest levels, in fact, the emphasis on rigour is often relaxed somewhat. Terence Tao describes this as:

The “post-rigorous” stage, in which one has grown comfortable with all the rigorous foundations of one’s chosen field, and is now ready to revisit and refine one’s pre-rigorous intuition on the subject, but this time with the intuition solidly buttressed by rigorous theory. (For instance, in this stage one would be able to quickly and accurately perform computations in vector calculus by using analogies with scalar calculus, or informal and semi-rigorous use of infinitesimals, big-O notation, and so forth, and be able to convert all such calculations into a rigorous argument whenever required.) The emphasis is now on applications, intuition, and the “big picture”. This stage usually occupies the late graduate years and beyond.

(Incidentally, this blog post is a good sanitised, non-obnoxious version of the Kegan levels idea.) (back)

3. I recently went to a fascinating introductory talk on divergent series, the subject that produces those weird Youtube videos on how 1 + 2 + 3 + … = -1/12. The whole thing was the most ridiculous tightrope walk over the chasm of total bullshit, always one careful definition away from accidentally proving that 1 = 0, and for once in my life I was appreciating the value of a bit of rigour. (back)

4. The list isn’t supposed to be comprehensive, either. I would definitely add aesthetics as an important category… sometimes an equation just looks unpleasant, like a clunky sentence or some badly indented code, and I feel a compulsion to tidy it into the ‘correct’ form. (back)

5. My current job is as a relative noob programmer supporting and extending existing software systems. I’ve been spending a lot of time being dumped in the middle of some large codebase where I have no idea what’s going on, or getting plonked in front of some tool I don’t understand, and flailing around uselessly while some more experienced colleague just knows what to do. There’s now a new guy who’s an even bigger noob on the project than me, and it’s fun to get to watch this from the other side! He’ll be staring blankly at a screen packed with information, fruitlessly trying to work out which bit could possibly be relevant, while I’ll immediately just see that the number two from the end on the right hand side has got bigger, which is bad, or whatever. (back)

6. When I griped about this in the SSC comments I was advised by Eli to read about Bayesian nonparametrics. Which is what people also say to nostalgebraist, and I really should learn what this stuff is. (back)

7. Has anyone else noticed the following?

Platonism virtue ethics
formalism deontology
logicism utilitarianism

I don’t know what this means but I’m pretty sure it’s all just bad. (back)

8. The worst omission is that I’ve only glancingly mentioned the difference between epistemic uncertainty and ontological ambiguity, the subject I started to hint about in this post. This is an extremely important piece of the puzzle. I don’t think I could do a good job of talking about it, though, and David Chapman is currently writing some sort of giant introduction to this anyway, so maybe it made sense to focus elsewhere. (back)

no better state than this

I’m writing a Long Post, but it’s a slog. In the meantime here are some more trivialities.

  1. I realised that the three images in my glaucoma machine post could be condensed down to the following: “Glaucoma, the ox responded / Gaily, to the hand expert with yoke and plough.”

    This is really stupid and completely impenetrable without context, and I love it.

  2. I’ve been using the fog-clearing metaphor for the process of resolving ambiguity. It’s a good one, and everyone else uses it.

    It’s probably not surprising that we reach for a visual metaphor, as sight is so important to us. It’s common to describe improved understanding in terms of seeing further. Galileo named his scientific society the Academy of Lynxes because the lynx was thought to have unparalleled eyesight, though unfortunately that finding seems not to have replicated. (That was the high point of naming scientific institutions, and after that we just got boring stuff like ‘The Royal Society’.)

    I’m more attached to smell as a metaphor, though. We do use this one pretty often, talking about having a ‘good nose’ for a problem or ‘sniffing out’ the answer. Or even more commonly when we talk about good or bad taste, given that taste is basically smell.

    I’m probably biased because I have atrocious eyesight, and a good sense of smell. I’d rather join an Academy of Trufflehogs. I do think smell fits really well, though, for several reasons:

    • It’s unmapped. Visual images map into a neat three-dimensional field; smell is a mess.
    • The vocabulary for smells is bad. There’s a lot more we can detect than we know how to articulate.
    • It’s deeply integrated into the old brain, strongly plugged into all sorts of odd emotions.
    • It’s real nonetheless. You can navigate through this mess anyway! Trufflehogs actually find truffles.

3. An even better metaphor, though, is this beautiful one I saw last week from M. John Harrison on Twitter. ‘You became a detector, but you don’t know what it detects’:

This mental sea change is one of my weird repetitive fascinations that I keep going on about, here and on the old tumblr. Seymour Papert’s ‘falling in love with the gears’, or the ‘positive affective tone’ that started attaching itself to boring geology captions on Wikipedia. The long process of becoming a sensitive antenna, and the longer process of finding out what it’s an antenna for. There is so absolutely NO BETTER STATE THAN THIS.

Three replies

These are responses to other people’s posts. They’re all a bit short for an individual post but a bit long/tangential/self-absorbed for a reply, so I batched them together here.

1. Easy Mode/Hard Mode inversions

I spend a lot of time being kind of confused and nitpicky about the rationalist community, but there’s one thing they do well that I really really value, which is having a clear understanding of the distinction between doing the thing and doing the things you need to do to look like you’re doing the thing.

Yudkowsky was always clear on this (I’m thinking about the bit on cutting the enemy), and people in the community get it.

I appreciate a lot this having done a PhD. In academia a lot of people seem to have spent so long chasing after the things you need to do to look like you’re doing the thing that they’ve forgotten how to do the thing, or even sometimes that there’s a thing there to do. In parts, the cargo cults have taken over completely.

Zvi Mowshowitz gives doing the thing and doing the things you need to do to look like you’re doing the thing the less unwieldy names of Hard Mode and Easy Mode (at least, I think that’s the key component of what he’s pointing at).

It got me thinking about cases where Easy Mode and Hard Mode could invert completely. In academia, Easy Mode involves keeping up with the state of the art in a rapidly moving narrow subfield, enough to get out a decent number of papers on a popular topic in highly ranked journals during your two year postdoc. You need to make sure you’re in a good position to switch to the new trendy subfield if this one appears to run out of steam, though, because you need to make sure you get that next two year postdoc on the other side of the world, so that …

… wait a minute. Something’s gone wrong here. That sounds really hard!

Hard Mode is pretty ill-defined right now, but I’m not convinced that it necessarily has to be any harder than Easy Mode. I have a really shitty plan and it’s still not obviously worse than the Easy Mode plan.

If there was a risk of a horrible, life-ruining failure in Hard Mode, I’d understand, but there isn’t. The floor, for a STEM PhD student with basic programming skills in a developed economy, is that you get a boring but reasonably paid middle class job and think about what you’re interested in in your spare time. I’m walking along this floor right now and it’s really not bad here. It’s also exactly the same floor you end up on if you fail out of Easy Mode, except you have a few extra years to get acquainted with it.

If there is a genuine inversion here, then probably it’s unstable to perturbations. I’m happy to join in with the kicking.

2. ~The Great Conversation~

Sarah Constantin had the following to say in a recent post:

… John’s motivation for disagreeing with my post was that he didn’t think I should be devaluing the intellectual side of the “rationality community”. My post divided projects into into community-building (mostly things like socializing and mutual aid) versus outward-facing (business, research, activism, etc.); John thought I was neglecting the importance of a community of people who support and take an interest in intellectual inquiry.

I agreed with him on that point — intellectual activity is important to me — but doubted that we had any intellectual community worth preserving. I was skeptical that rationalist-led intellectual projects were making much progress, so I thought the reasonable thing to do was to start fresh.


‘Doubted that we had any intellectual community worth preserving’ is strong stuff! Apparently today is Say Nice Things About The Rationalists Day for me, because I really wanted to argue with it a bit.

I may be completely missing the point on what the ‘rationality community’ is supposed to be in this argument. I’m only arguing for the public-facing, internet community here, because that’s all I really know about. I have no idea about the in-person Berkeley one. Even if I have missed the point, though, I think the following makes sense anyway.

Most subcultures and communities of practice have a bunch of questions people get really exercised about and like to debate. I often internally think of this as ~The Great Conversation~, with satiric tumblr punctuation to indicate it’s not actually always all that great.

I’ve only been in this part of the internet for a few years. Before that I lurked on science blogs (which have some overlap). On science blogs ~The Great Conversation~ includes the replication crisis, alternatives to the current academic publishing system, endless identical complaints about the postdoc system (see part 1 of this post), and ranting about pseudoscience and dodgy alternative therapies.

Sometimes ~The Great Conversation~ involves the big names in the field, but most of the time it’s basically whoever turns up. People who enjoy writing, people who enjoy the sound of their own voice, people with weird new ideas they’re excited about, people on a moral quest to fix things, grumpy postdocs with an axe to grind, bored people, depressed people, lonely people, the usual people on the internet.

If you go to the department common room instead, the academics probably aren’t talking about the things on the science blogs. They’re talking about their current research, or the weird gossip from that other research group, or what the university administration has gone and done this time, or how shit the new coffee machine is. ~The Great Conversation~ is mostly happening elsewhere.

This means that the weirdos on the internet have a surprisingly large amount of control over the big structural questions in the field. This often extends to having control over what those questions are in the first place.

The rationalist community seems to be trying to have ~The Great Conversation~ for as much of human intellectual enquiry as it can manage (or at least as much as it takes seriously). People discuss the replication crisis, but they also discuss theories of cognition, and moral philosophy, and polarisation in politics, and the future of work, and whether Bayesian methods explain absolutely everything in the world or just some things.

The results are pretty mixed, but is there any reasonably sized group out there doing noticeably better, out on the public internet where anyone can join the conversation? If there is I’d love to know about it.

This is a pretty influential position, as lots of interesting people with wide-ranging interests are likely to find it and get sucked in, even if they’re mostly there to argue at the start. Scott Aaronson is one good example. He’s been talking about these funny Singularity people for years, but over time he’s got more and more involved in the community itself.

The rationalist community is some sort of a beacon for something, and to me that ought to count for ‘an intellectual community worth preserving’.

3. The new New Criticism

I saw this on nostalgebraist’s tumblr:

More importantly, the author approaches the game like an art critic in perhaps the best possible sense of that phrase (and with M:TG, there are a lot of bad senses). He treats card design as an art form unto itself (which it clearly is!), and talks about it like a poetic form, with various approaches to creativity within constraints, a historical trajectory with several periods, later work exhibiting a self-consciousness about that history (in Time Spiral, and very differently in Magic 2010), etc.

That is, he’s taking a relatively formal, “internal,” New Criticism-like approach, rather than a historicist approach (relate the work to contemporary extra-artistic phenomena) or an esoteric/Freudian/high-Theory-like approach (take a few elements of the work, link them to some complex of big ideas, uncover an iceberg of ostensibly hidden structure). I don’t think the former approach is strictly better than the latter, but it’s always refreshing because so much existing games criticism takes the latter two approaches.

I know absolutely nothing about M:TG beyond what the acronym stands for, but reading this I realised I’m also really craving sources of this sort of criticism. I recently read Steve Yegge’s giant review of the endgame of Borderlands, a first person shooter that I would personally hate and immediately forgot the name of. Despite this I was completely transfixed by the review, temporarily fascinated by tiny details of gun design, enjoying the detailed explorations of exactly what made the mechanics of the game work so well. This is exactly what I’m looking for! I’d rather have it for fiction or music than games, but I’ll take what I can get.

I kind of imprinted on the New Critics as my ideal of what criticism should be, and although I can see the limitations now (snotty obsession with narrow Western canon, tone deaf to wider societal influences) I still really enjoy the ‘internal’ style. But it’s much easier now to find situated criticism, that wants to relate a piece of art to, say, Marxism or the current political climate. And even easier to find lists of all the ways that that piece of art is problematic and you’re problematic for liking it.

Cynically I’d say that this is because the internal style is harder to do. Works of art are good or bad for vivid and specific internal reasons that require a lot of sensitivity to pinpoint, whereas they’re generally problematic for the same handful of reasons that everything else is problematic. But probably it’s mostly just that the internal style is out of fashion. I’d really enjoy a new New Criticism without the snotty high culture focus.

Two cultures: tacit and explicit

[Epistemic status: no citations and mostly pulled straight out of my arse, but I think there’s something real here]

While I was away it looks like there was some kind of Two Cultures spat on rationalist-adjacent tumblr.

I find most STEM-vs-the-humanities fight club stuff sort of depressing, because the arguments from the humanities side seem to me to be too weak. (This doesn’t necessarily apply this time – I haven’t tried to catch up on everyone’s posts.) Either people argue that the humanities teach exactly the same skills in systematic thinking that the sciences do, or else you get the really dire ‘the arts teach you to be a real human being‘ arguments.

I think there’s another distinction that often gets lost. There are two types of understanding I’d like to distinguish, that I’m going to call explicit and tacit understanding in this post. I don’t know if those are the best words, so let me know if you think I should be calling them something different. Both are rigorous and reliable paths to new knowledge, and both are important in both the arts and sciences. I would argue, however, that explicit understanding is generally more important in science, and tacit understanding is more important in the arts.

(I’m interested in this because my own weirdo learning style could be described as something like ‘doing maths and physics, but navigating by tacit understanding’. I’ve been saying for years that ‘I’m trying to do maths like an arts student’, and I’m just starting to understand what I mean by that. Also I feel like it’s been a bad, well, century for tacit understanding, and I want to defend it where I can.)

Anyway, let’s explain what I mean by this. Explicit understanding is the kind you come to by following formal logical rules. Scott Alexander gives an example of ‘people who do computer analyses of Shakespeare texts to see if they contain the word “the” more often than other Shakespeare texts with enough statistical significance to conclude that maybe they were written by different people’. This is explicit understanding as applied to the humanities. It produces interesting results there, just as it does in science. Also, if this was all people did in the humanities they would be horribly impoverished, whereas science might (debatably) just about survive.

Tacit understanding is more like the kind you ‘develop a nose for’, or learn to ‘just see’. That’s vague, so here are some examples:

  • Taking a piece of anonymised writing and trying to guess the date and author. This is a really rigorous and difficult thing my dad had to do in university (before pomo trashed the curriculum, [insert rant here]). It requires very wide-ranging historical reading, obviously, but also on-the-fly sensitivity to delicate tonal differences. You’re not combing through the passage saying ‘this specific sentence construction indicates that this passage is definitely from the late seventeenth century’. There might be some formal rules like this that you can extract, but it will take ages, and while you’re doing the thing you’re more relying on gestalt feelings of ‘this just looks like Dryden’. You don’t especially need to formalise it, because you can get it right anyway.

  • Parody. This is basically the same thing, except this time it’s you generating the writing to fit the author. Scott is excellent at this himself! Freddie DeBoer uses this technique to teach prose style, which sounds like a great way to develop a better ear for it.

  • Translation. I can’t say too much about this one, because I’ve never learned a foreign language :(. But you have the problem of matching the meaning of the source, except that every word has complex harmonic overtones of different meanings and associations, and you have to try and do justice to those as well as best as you can. Again, it’s a very skilled task that you can absolutely do a better or worse job at, but not a task that’s achieved purely through rule following.

I wish these kinds of tacit skills were appreciated more. If the only sort of understanding you value is explicit understanding, then the arts are going to look bad by comparison. This is not the fault of the arts!

Crackpot time 2: cargo culting hard

I’m back from another weird foundations-of-physics workshop in the middle of nowhere, this one even smaller and more casual than the last. Also much more relaxed, schedule-wise, so there was plenty of time to think idly about various rubbish in my head.

Last time I was inspired to write my crackpot plan, so it feels like a good time to revisit it a bit, but mostly this is just a large braindump to get various things out of temporary memory before I lose them.

Continue reading “Crackpot time 2: cargo culting hard”

glaucoma machine yoked to a plough

I’m going to start reintroducing a few tumblr-style posts without much editing, as this thing is starting to develop a stodgy Real Blog atmosphere where I feel like I need to post Proper Serious Writing.

This is supposed to be more of a workbook, and I think I’ll learn faster if I up the percentage of experimental/embarrassing/badly-thought-out posts.

There’s this mindset I sometimes kick myself into, which is roughly ‘I’m going to work hard at this thing, and I’m going to like it.

It’s got a very specific emotional tone and a specific range of application. I wouldn’t bother trying to use it for utterly dull stuff like the washing up where I do not care at all. On the other hand, there’s a definite theatrical aspect (the ‘and I’m going to like it’ bit) where I’m kind of faking up the enthusiasm in the hope that some genuine enthusiasm will follow. Getting up early in the winter and working when it’s still dark outside is the right sort of situation for it.

I’d never really thought consciously about this before, but I noticed the other morning that it’s got three different images attached to it in my head. By ‘image’ I don’t mean a vivid mental picture (I don’t have much of a visual imagination at all, oddly for someone who loves geometry), just vague sort-of-images or bits of phrases that cluster round the thing.

The first one is a half-remembered line from The Waste Land, something about ‘the boat responded gaily to the slightest touch’. The real version turns out to be ‘Damyata: The boat responds / Gaily, to the hand expert with sail and oar’.

This is highly relevant. It contains the right sort of things: precision and responsiveness and genuine enjoyment. I’m actually impressed with whatever part of my brain came up with that, apparently without much conscious supervision.

After that the images go downhill fast. The second is something to do with oxen yoked to a plough. Which is not imaginative at all, and also a pretty miserable vision of the potential rewards of hard work. I guess there’s a kind of stoic, stubborn element that’s useful here.

Also, I have never in my life thought clearly about what oxen yoked to a plough actually look like (though of course I’m googling it now). The words in my head are something about oxen yoked to a plough, but the image is more like an old-fashioned heavy leather horse harness.

The third part, ludicrously, is something like one of those glaucoma testing machines you get at the opticians. I can’t quickly find a public domain image, just google it if you don’t know what I’m talking about. I’m not imagining the bit that puffs an unpleasant jet of air into your eye, but the bar bit you push your forehead against in order to keep your face aligned properly.

Apparently this is how I’m taking the image of the harness, which is attached to the ox, and applying it to myself. Sticking my forehead against this machine is how I’m yoking myself to the badly-imagined plough – it’s the crucial image that makes the other two apply to me specifically.

I had no idea I was imagining something so specific and weird. Imagine trying to consciously think up this crap! But somehow it kind of hangs together as something inspiring, if you don’t look at the component parts: fluid delicacy mixed with stolid determination, joined (of course) at the forehead. By a glaucoma machine!

It’s hopeless to try and write about this sort of thing accurately, because so much of what’s going on is not language-based. (And there’s a mess of other associations when I start thinking about this. I’m not sure the process stops, there’s just more and more of this nonsense.) But it’s also fun to try, because it’s all so entertainingly stupid!

Self-similar procrastination

Sierpinski triangle
Your to do list should look like this. Or something. I’m still working out the finer details.

[image source]

I had a gigantic insight on my walk home tonight, which is clearly going to make me millions, but before I start on my self-help book empire I needed to write this rushed crappy blog post explaining it. And before that, I needed to do the dishes. Because my grand theory requires self-similar competence at all scales.

OK, so really this thing is not very profound or original at all. It’s pretty much the same as John Perry’s structured procrastination, but with a slightly different emphasis. (And probably this emphasis appears elsewhere too.)

If you somehow haven’t come across the structured procrastination essay before, it’s wonderful and you should read it. The key part:

Procrastinators often follow exactly the wrong tack. They try to minimize their commitments, assuming that if they have only a few things to do, they will quit procrastinating and get them done. But this approach ignores the basic nature of the procrastinator and destroys his most important source of motivation. The few tasks on his list will be, by definition, the most important. And the only way to avoid doing them will be to do nothing. This is the way to become a couch potato, not an effective human being.

I’m highly susceptible to this particular bad idea and end up doing nothing too often. Partly this is because I tend to have overambitious crackpot plans, so there’s always a good supply of ‘most important’ tasks to put at the top of the list. And partly it’s because I don’t seem to get bored as easily as most people and am unusually good at sitting around doing nothing very much, so it’s easy to slump into the couch potato ground state.

I do think sitting around doing nothing very much is highly underrated by a lot of people, but that would be a different post. In my case, I definitely need nudging towards actually getting shit done, instead of thinking idly about things I could do.

John Perry advocates avoiding this low-energy stuck state by filling up your to do list with ‘a hierarchy of the tasks you have to do, in order of importance from the most urgent to the least important’. The main mechanism he advances for why this works is that fear of the big intimidating tasks at the top drives you down the list, pushing you into actually completing many of the lower-ranked items.

I think this effect is somewhat important, but my thesis is that the most important mechanism is actually going the other way, from the bottom of the list up. Deadline fear is definitely useful, but I think that the energy and confidence created by completing the small tasks is the most important bit.

My intuition is that energy tends to be created at the microscale and bubbles up from there. I definitely use this principle to try and build momentum at work when I have a seriously boring task to do. I don’t want to do the actual task, but maybe I can be bothered to open up the password manager and get out the password I need for the server, and then maybe I can be bothered to open up a terminal and log on. Then maybe I’ll type in some trivial command to get myself used to the fact that I’m going to be typing in some commands. I don’t actually care what files are in that particular directory, but listing them has enough of the flavour of ‘doing work’ that it’s often enough to push me over the threshold into doing real work.

I think this is all pretty uncontroversial at the microscale. Any grumpy old fart who writes a weekly column for the Telegraph on how The Kids These Days Have No Discipline could tell you that sitting up straight and making your bed in the morning (or whatever) will propagate through to getting more done in general.

Where it gets interesting is at the mid-scale – projects you’re spending weeks to months on, but that aren’t all that important, at least in comparison with Big Intimidating Project at the top of the list. These are the ones I find myself wanting to cross off the list, because they’re ‘wasting time’.

But I’m coming to realise that they play an extremely important role in the task ecosystem. In some sense these are the largest-scale projects that you know you can actually pull off. Really big intimidating projects tend to have some sort of ‘research’ type element, where you don’t know what would even constitute a solution when starting out. Mid-sized projects, on the other hand, take a considerable amount of effort but are much more well defined. You more-or-less know how you’re going to tackle them, and what a successful outcome will look like. Successful mid-sized projects give you the confidence and energy to keep going, gradually allowing you to push further and further up the scale.

(I think it’s also important that at least some of these are self-contained projects in their own right, rather than subtasks of Big Intimidating Project. Lopping chunks off of Big Intimidating Project and tackling them separately is an excellent strategy, but my intuition is that this can’t be the only thing. Probably this is something to do with needing a supply of new ideas to keep bubbling up at all scales, but I haven’t thought about it very carefully.)

Here’s an example of an idea bubbling up. Last August I declared a Shitty Projects Month, as I could tell I needed some sort of break from more focussed work. It’s the kind of idea you can’t really fail at, and I did indeed do some shitty work on a couple of shitty projects, but I didn’t feel too pleased with how it went at the time. I suppose I was hoping that the results would be, well, less shitty.

Somehow, though, I found myself coming back to one of the projects a couple of months ago. I’d had an idea for a toy project I could try and do using the d3.js visualisation library – nothing useful, but it would look pretty if I got it right. I spent most of my time fighting my poor understanding of the library, and indeed of Javascript in general, and didn’t get very far.

Eventually it came back into my head, though, and this time I had the bright idea of prototyping in Inkscape. Once I could see something visually I was a lot more excited about the project and made rapid progress. I haven’t finished yet because there’s other stuff I have to do this month, but it looks likely that it’s going to be a major component in the visual design of the proper website I’m finally going to make, which will be my next mid-range non-physics project. If I don’t run into any more weird distractions.

And of course, the best example of a mid-range project bubbling up from triviality is this blog itself. I got the shittiest possible blog, a basic tumblr with the default design, and started writing with no particular plan. It turned out that what I wanted to write about was mathematical intuition (and chalk!? no idea about that one) so I went with that. And then got it off tumblr and turned it into something approximating a proper blog.

I haven’t run out of ideas yet, so hopefully this one can keep bubbling up. That’s my excuse for writing essentially the same post over and over again. Self-similar blogging at all scales!

Everything And Its Discontents

I was trying to dig up some blog I read ages ago that I thought used a variation on ‘X and its Discontents’ in its tagline. I haven’t found it, but I have dug up a pretty extensive list of things people on the internet are discontented about (who knew?).

Main conclusions are that everyone hates -isms and Europe:

The New Turkey
The China Boom
The Crowded Public Sphere
Data Nationalism
Late Neoliberalism
Late Style
Art Direction
Nordic Art
Ceramics as a Medium
Method Acting
Scholarly Publishing
Political Rule
Multidimensional Poverty
Hearts and Minds
Social Media
The Enlightenment
European Integration
The European Court of Human Rights
The Dollar
Film Adaptation
The United Nations
The Social Imaginary
Modern Water
Grand Coalition Politics
Class Size
Urban Computing
Urban Governance
Urban Tourism
The 21st Century Urban Housing Crisis
The Bacteriological City
The Internet
Open Science
The Modern University
The Nobel Prize
The Singularity
Spooky Interaction
Communal Living
Mission Creep
Patent Alienability
The Standard Model of Talent Development
The Jobless World
Conspiracy Theory
Curatorial Education
Climate Based Daylight Modelling
Continental Realism
Preservative Realism
Realist Criminology
Situational Crime Prevention
The Achievement Society
Technological Progress
The Technological Body
Strategic Philanthropy
Transitional Justice
Cosmopolitan Justice
Judicial Engagement
International Law
Family Law
The Lawn-Chemical Economy
The World’s Course

Research debt, double distilled

I said a few weeks ago that I was going to talk more about this article by Chris Olah and Shan Carter on the idea of research debt, but every time I went back to it I felt that the original article made the point so clearly and elegantly that there was very little I wanted to add. (Other than ‘I want this thing in physics too please!’)

So I started to pull out my favourite bits, intending to do a very lazy quotes-and-comments sort of post, and realised that with a couple of additions they made a coherent summary of the original post on their own. In the process I discovered that there were a few things I wanted to say, after all.

If you just read straight down the blockquotes you get a double-distilled microversion of the original essay. Or you can also read the bits I’ve stuck in between.

For centuries, countless minds have climbed the mountain range of mathematics and laid new boulders at the top. Over time, different peaks formed, built on top of particularly beautiful results. Now the peaks of mathematics are so numerous and steep that no person can climb them all.

This has always saddened me. In Men of Mathematics, E. T. Bell labels Henri Poincaré as the ‘The Last Universalist’, the last person to be able to range freely across all fields of mathematics as they existed in his time. Now Bell did have a tendency to over-dramatise things, but I think this is basically right.

Probably this is unavoidable; probably the expansion wave has accelerated too fast, and those days will not return. I’m temperamentally susceptible to millenarian dreams of the return of the once and future universalists, but I accept that this is unlikely.

Still, there is a lot of compression that is within reach:

The climb is seen as an intellectual pilgrimage, the labor a rite of passage. But the climb could be massively easier. It’s entirely possible to build paths and staircases into these mountains. The climb isn’t something to be proud of.

The climb isn’t progress: the climb is a mountain of debt.

The analogy is with technical debt in programming, which is all the awkward stuff thrown to the side in an effort to get software into production quickly. Eventually you have to go back and deal with the awkward stuff, which has an unfortunate tendency to compound over time.

The insidious thing about research debt is that it’s normal. Everyone takes it for granted, and doesn’t realize that things could be different. For example, it’s normal to give very mediocre explanations of research, and people perceive that to be the ceiling of explanation quality. On the rare occasions that truly excellent explanations come along, people see them as one-off miracles rather than a sign that we could systematically be doing better.

People who are truly excellent at explaining research are probably rare. But ‘better explanations than we have currently’ seems like a very, very easy target to hit, once people are persuaded to put resources into hitting it.

I plan to finally start taking my own advice soon, and start putting whatever notes and bits of intuition I’ve gathered online. I’m not too convinced that they’ll be especially great, but the current floor is pretty low.

Research distillation is the opposite of research debt. It can be incredibly satisfying, combining deep scientific understanding, empathy, and design to do justice to our research and lay bare beautiful insights.

Distillation is also hard. It’s tempting to think of explaining an idea as just putting a layer of polish on it, but good explanations often involve transforming the idea. This kind of refinement of an idea can take just as much effort and deep understanding as the initial discovery.

Distillation is fundamentally a different sort of activity to the types of research that are currently well supported by academia. Distillers aren’t mountain climbers; they engage with their subject by criss-crossing the same ground over and over again, following internally-generated trails of fascination that can be hard to interpret from the outside. They want to understand!

An aspiring research distiller lacks many things that are easy to take for granted: a career path, places to learn, examples and role models. Underlying this is a deeper issue: their work isn’t seen as a real research contribution. We need to fix this.

Distillers generally have little interest in who can get to the top of the mountain fastest, and anyway it certainly won’t be them. In an environment that rewards no other activity, they tend to disappear quickly. They require different infrastructure.

None of this infrastructure currently exists, but it easily could do. Research distillation doesn’t intrinsically need to cost huge amounts of money. It’s not like we need to spend billions on a gigantic high-energy collider to smash our current explanations together. This is an area where transitioning from moaning about academia to actually doing something about it looks to be pretty straightforward.

It’s one of the nice sorts of problems where small efforts at the margins are already useful. It certainly helps if you have Google’s resources behind you, but you can also just polish up any half-decent notes you have lying around on a topic that’s currently poorly explained and put them online, and you’ve made a tiny contribution towards fixing the problem.

If you are excited to distill ideas, seek clarity, and build beautiful explanations, we are letting you down. You have something precious to contribute and we aren’t supporting you the way we should.

I’ve been saying ‘they’ throughout this post, but, I mean, it’s obvious why I care about this thing. This is my old tumblr ‘about me’ page:


It’s amusing self-deprecation, but unfortunately I also meant it a lot of the time. (I still believe the programmer bit, but I’m starting to have some optimism about improvement there too.) My standard line after finishing my thesis was ‘I love physics but I’m bad at research’.

I had a poorly understood but strongly felt sense of what I wanted instead, academia was clearly not going to provide it, and I just wanted to get out. ‘Research distillation’, however, is a reasonably close fit. (Maybe not an exact one. I feel the ‘criss-crossing existing territory’ approach goes deeper than just refining existing ideas, and is a valid route to original research in itself. But it’s an ecosystem I think I would have been able to cope with, and succeed in.)

So I’ll admit my enthusiasm for the idea of research distillation is mostly pure self-interest. But I’m pretty sure that a thriving ecosystem of distillers would also help academia. After all, you only criss-cross the territory for love of the subject. The external rewards are currently too poor for any other motivation to make sense.

Reading into the landscape

Written quickly and probably not very clear – it’s a workbook post not a polished-final-thoughts post. Vaguely inspired by this exchange between Julia Galef and Michael Nielsen.

One of my favourite things is the point in learning a new topic where it starts to get internalised, and you begin to be able to see more. You can read into a situation where previously you had no idea what was going on.

Sometimes the ‘seeing’ is metaphorical, but sometimes it’s literal. I go walking quite a lot, and this year I’m seeing more than before, thanks to an improved ability to read into the landscape.

I got this from Light and Colour in the Outdoors, a classic 30s book on atmospheric phenomena by the physicist Marcel Minnaert. It’s really good, and I’m now regretting being cheap and getting the Dover version instead of the fancy coffee-table book (note to self: never buy a black-and-white edition of a book with the word ‘colour’ in the title).

I’ve only read a few sections, but already I notice more. Last weekend I got the coach to London, and on the way out I saw a sun dog I’d probably have missed before. And then on the way back it was raining with the sun shining onto the coach windscreen in front, and I thought to myself, ‘I should probably look behind me’. I turned, and right on cue:

2017-05-20 20.23.49

This is entry-level reading into the landscape, but still quite satisfying. Those with exceptional abilities seem to have superpowers. George Monbiot in Feral talks about his friend Ritchie Tassell:

… he has an engagement with the natural world so intense that at times it seems almost supernatural. Walking through a wood he will suddenly stop and whisper ‘sparrowhawk’. You look for the bird in vain. He tells you to wait. A couple of minutes later a sparrowhawk flies across the path. He had not seen the bird, nor had he heard it; but he had heard what the other birds were saying: they have different alarm calls for different kinds of threat.

This is the kind of learning that fascinates me! You can do it with maths as well as with sparrowhawks…

This has been on my mind recently as I read/reread Venkatesh Rao’s posts on ambiguity and uncertainty. I really need to do a lot more thinking on this, so this post might look stupid to me rather rapidly, but it’s already helping clarify my thoughts. Rao explains his use of the two terms here:

I like to use the term ambiguity for unclear ontology and uncertainty for unclear epistemology…

The ambiguity versus uncertainty distinction helps you define a simpler, though more restricted, test for whether something is a matter of ontology or epistemology. When you are missing information, that’s uncertainty, and an epistemological matter. When you are lacking an interpretation, that’s ambiguity, and an ontological matter.

Ambiguity is the one that maps to the reading-into-the-landscape sort of learning I’m most fascinated by, and reducing it is an act of fog-clearing:

20/ In decision-making we often use the metaphors of chess (perfect information) and poker (imperfect information) to compare decision-makers.

21/ The fog of intention breaks that metaphor because the game board /rules are inside people’s heads. Even if you see exactly what they see, you won’t see the game they see.

22/ Another way of thinking about this is that they’re making meaning out of what they see differently from you. The world is more legible to them; they can read/write more into it.

I think this is my main way of thinking about learning, and probably accounts for a fair amount of my confusion when interacting with the rationalist community. I’m obsessed with ambiguity-clearing, while the rationalists are strongly uncertainty-oriented.

For example, here’s Julia Galef on evaluating ‘crazy ideas’:

In my experience, rationalists are far more likely to look at that crazy idea and say: “Well, my inside view says that’s dumb. But my outside view says that brilliant ideas often look dumb at first, so the fact that it seems dumb isn’t great evidence about whether it will pan out. And when I think about the EV here [expected value] it seems clearly worth the cost of someone trying it, even if the probability of success is low.”

I’ve never thought like that in my life! I’d be hopeless at the rationalist strategy of finding a difficult, ambitious problem to work on and planning out high-risk steps for how to get there, but luckily there are other ways of navigating. I mostly follow my internal sense of what confusions I have that I might be able to attack, and try to clear a bit of ambiguity-fog at a time.

That sounds annoyingly vague and abstract. I plan to do a concrete maths-example post some time soon. In the meantime, have a picture of a sun dog: