I’m at @ssica3003‘s Sensemaker Workshop today, and thought it would be fun to get a blog post out while I’m here, so I dug out this draft I wrote back in August for the newsletter. I wasn’t sure I liked it much at the time, but reading back it’s better than I remembered and works as a first stab at the the idea, at least. Hopefully I can get it a little further down the pipeline. There are some questions at the end, so let me know if you have any thoughts.
Anyway, the rough idea is… there’s an extraordinary explosion of creative idea generation going on online. And there’s this fascinating kind of pipeline where people will start feeling out the vague beginnings of an idea through twitter threads and dumb throwaway posts and blog comments and email conversations, and then if something looks promising they’ll discuss more and pull in bits of other people’s ideas, and gradually build up to more thought out, polished work.
I’m excited about this culture for a lot of reasons. It’s a kind of online version of the casual, unobservable ‘dark matter’ part of academia, the part you can’t access by looking at published work – all the throwing around wild claims over coffee in the common room and in the pub on Friday evenings, bits of ‘yeah, that paper is unreadable, but this is what they’re really talking about’ insight from people in the know, standing round the whiteboard trying to figure something out, group meeting gossip, and the like. And it’s an incredibly vivid and alive version, at a time when large parts of normal academia have become rigid and bureaucratised and plain boring. This seems important to me: it’s the shitposting engine that produces the raw generative power that can drive more focussed work further down the line.
There’s a kind of wild energy; people aren’t afraid to go after big topics. We’ve got ourselves free of the constraints of the academic pimple factory:
An example of Little History is an essay by Matt Might (clearly a Marvel superhero in a counterfactual universe) titled The Illustrated Guide to a Phd. Go read it. It’ll only take a minute. It frames the sum of all human knowledge as a big circular bubble, and your PhD as a little pimple on the surface of it. I’ll call this the Mighty Diagram. It gets passed around in graduate student circles with depressing frequency.
Instead of a dent in the universe, you get a pimple on a uncritically proceduralist conceptualization of the frontier of knowledge as the sum of all the peer-reviewed academic literature in the world.
What makes this essay utterly horrifying is that it is actually an accurate description of what a PhD is; it calibrates academic career expectations correctly and offers an accurate sense of perspective on the peer-reviewed life. I suspect Matt Might sincerely intended the essay as a helpful guide to academic survival, but its effect is to put aspiring scholars in their place, rather than help them find a sense of place in the universe. It’s a You Are Here map for your intellectual journey at the end of a PhD, you disgusting little pimple, you. Kneel before this awe-inspiring edifice of knowledge that you’re lucky to be allowed to add a pimple to.
This rings very true with my own experience of academia, and the mindset it got me into. I personally found that after a couple of years out of there my thinking kind of cleared and became more expansive, and I was able to have good ideas again.
Unfortunately, this pipeline only goes so far. Currently, I think we’re in something like this situation:
It currently tends to dump ideas out somewhere around the ‘insight porn’ point – ideas that you read, think ‘oh that’s clever’, hit the like button, maybe comment on or talk about for a bit, then completely forget a week later. In the best case, a fragment of the idea or a bit of new jargon escapes into the local thought soup and can be combined with other ideas that are currently percolating. Sometimes this can be quite a powerful effect on its own. But there are a lot of places that academia still goes to that just can’t be reached in this way.
One of my favourite examples of this dynamic is Sarah Perry’s theory of mess. This is a genuinely great idea, and it’s not just a vague ‘insight’ – it’s an initial sketch of a satisfying explanatory theory of what mess is, complete with some very convincing examples and thought experiments (put a kaleidoscope filter on your mess, and it’s no longer mess!). But as far as I can tell, it got the same treatment as everything else that goes down the pipe – we all liked it and moved on. No real discussion (that I know of) of how to test it, or what has already been done in this line, or probing to see where it might fall down. Does it work? Who knows! On to the next idea!
Now, there’s an obvious explanation for why this happens. Most of us are not doing this as a full time job. We’re fitting this into the spare time we get, alongside paid work or other responsibilities. So we’re only really interested in doing the enjoyable parts of the idea generation process. Chucking around ideas is easy and fun, whereas checking whether they actually work is hard and boring. It’s not a big surprise that people prefer easy and fun work to hard and boring work.
There’s a lot of truth to this, but I think it’s slightly too cynical, in that it both makes the first part of the pipeline sound too easy and the second half of the pipeline too hard. Chucking around ideas is easy, but to be able to do that we need to have some good ones to chuck around, and that’s not exactly trivial. We have some advantage in being able to go after very broad, vague, ambiguous, undeveloped topics, and slowly clear fog. There’s no pressure to quickly get to a point where we can publish something. And at the same time, polishing up ideas is hardly some unrelenting tedious grind. Calculating can be fun, testing can be fun, writing up can be fun. If your eventual aim is to publish in traditional academia then there are some definite unfun parts, like altering your conversational blog post style to fit a more academic register, but this is only one part of the process.
My own experiments
For me, at least, it just feels unsatisfying to leave ideas at the insight porn stage. There’s a natural pull in the direction of getting further down the pipeline, rather than a tedious sense of duty. I’ve been playing around with some haphazard experiments of my own, and I think I’ve got past the insight porn stage too with some of them, but nowhere near as far as I’d like. I’ll go through a couple of examples.
A few years ago, I wrote a tumblr post called stupid bat and ball, title all lower case, 700 words of low-effort writing not far above the shitpost level. I wasn’t really expecting it to go anywhere further. But it did contain a small core of insight – the bat and ball question of the Cognitive Reflection Test is different to the other two questions in some respect, and so the questions don’t really form a natural set. When I got the wordpress blog I reposted it, and eventually it attracted some really good comments that probed the mechanics of the bat and ball question much more deeply than I had. So I realised that this idea probably was worth investigating and that I should up my game a bit, and I started reading some of the literature. I discovered that the bat and ball question came first and the others were picked ‘to be like it’, with no elaboration of the process for picking them, which confirmed my suspicion that not much work went into question validation. And I found a fascinating follow-up paper showing how ridiculously sticky the wrong answer is.
The comments to this post pushed things further again, coming up with more detailed explorations of how the difficulty relates to the way the problem maps numbers to an abstract quantity (the difference in price), but fools you into mapping it to a concrete one (the price of the bat). @_awbery pointed out that this abstract/concrete confusion is completely missing from the other two questions, where all the quantities map to concrete objects. And anders devised a set of ‘similar questions’ that turn up the level of abstractness one step at a time. These comments point towards something like rat-running experiments for the Cognitive Reflection Test, getting an understanding of how the tools we’re trying to use actually work before using them to make inferences about abstractions like ‘cognitive reflection’. I do think a potentially valuable contribution could be made here.
But… I’m not really the person to do it. (Even if I cared more about this specific question than I do. I’d pretty much used up my remaining store of shits-to-give on writing the blog post, and didn’t even have enough left to engage with the comments as fully as I’d have liked to.) Doing psych research without fooling yourself sounds like an absolute minefield even if you know what you’re doing, and I have no expertise at all.
So I guess in this case I quit the pipeline at the level of having a sort of slapdash lit review with some pointers to interesting ways to take it further. Not the most impressive result. But the interesting bit for me was the distance I travelled from the original tumblr post, which I’d put no effort into at all, and the way the project took on a life of its own, with other people helping to propel this considerably further than I’d ever thought to take it myself.
My other example is all my thinkingabout negative probability in the last year and a half. Although it sounds superficially like a kind of a crackpot topic, there are deep links to quantum mechanics on phase space, and I’ve been using my fascination with this as a serious starting point to learn all kinds of interesting things in quantum foundations/quantum information. I’ve been experimenting with the discipline of using a single paper as my focus, and this has been incredibly helpful for keeping me on track, and damping down my normal habit of wandering from subject to subject too quickly to pick up anything useful.
I’m more serious about this project than the bat and ball one – it actually connects to an enduring deep interest rather than something I blundered into by accident. Again, I’m not yet as far down the pipeline as I want to be, but I’ve got past the vague insight level. My last couple of posts explored an intriguing decomposition of the Wigner function for a qubit that I found myself, and that I can see some potential use for in interpreting negative probabilities. Since then had quite a few more ideas that I want to investigate, and I’ve started to link things into a more coherent picture. There’s also a lot more I could be doing in terms of making contact with people in academia and asking questions (something I’m rather bad at). I can definitely see how to push further.
It’s still really funny to me that I’m cheerfully crashing about between cognitive psych and quantum foundations, with a few clueless forays into reading Derrida for good measure. Whereas in academia I’d have felt daring if I tried to pivot from burst to continuous sources of gravitational waves from neutron stars, or something. Obviously this is too scattered for me to get anything done, and I need to get better at idea triage. But there’s something really psychologically healthy about this mindset of just taking a direct run at whatever I feel like, instead of thinking ‘oh, that’s outside my field, I can’t think about that.’ I want to keep this even as I hopefully learn to focus my efforts more usefully.
Right, I want to push this out now so I can stop being antisocial at the workshop. I’ll end with some questions:
What are examples of people navigating the whole shitpost-to-scholarship pipeline successfully on the public internet? I’m particularly interested in people who are trying for academia-style focussed research on specific object-level questions, rather than big-picture synthesis or popularisation.
Is there any kind of institutional support out there, or is it all just individual weird nerds pursuing individual weird research programs?
Has anyone written about this well already? For a start, Venkatesh Rao had a couple of excellent threads here and here on a similar topic. It’s a much more pessimistic take, which actually fits my current drizzle-soaked winter-brain opinions better than these cheery ramblings from last summer – for example, in most of my experiments I haven’t managed to get much further than this sort of ‘reading published literature and blogging a few derivative observations’ stuff. I’d like to hear about anything else relevant that people have liked.
At the end of my last post, I talked about Brian Cantwell Smith’s idea of ‘the middle distance’ – an intermediate space between complete causal disconnectedness and rigid causal coupling. I was already vaguely aware of this idea from a helpful exchange somewhere in the bowels of a Meaningness comments section but hadn’t quite grasped its importance (the whole thread is worth reading, but I’m thinking about the bit starting here). Then I blundered into my own clumsy restatement of the idea while thinking about cognitive decoupling, and finally saw the point. So I started reading On the Origin of Objects.
It’s a difficult book, with a lot more metaphysics than I realised I was signing up for, and this ‘middle distance’ idea is only a small part of a very complex, densely interconnected argument that that I don’t understand at all well and am not even going to attempt to explain. But the examples Smith uses to illustrate the idea are very accessible without the rest of the machinery of the book, and helpful on their own.
I was also surprised by how little I could find online – searching for e.g. “brian cantwell smith” “middle distance” turns up lots of direct references to On The Origin of Objects, and a couple of reviews, but not much in the way of secondary commentary explaining the term. You pretty much have to just go and read the whole book. So I thought it was worth making a post that just extracted these three examples out.
Example 1: Super-sunflowers
Smith’s first example is fanciful but intended to quickly give the flavour of the idea:
… imagine that a species of “super-sunflower” develops in California to grow in the presence of large redwoods. Suppose that ordinary sunflowers move heliotropically, as the myth would have it, but that they stop or even droop when the sun goes behind a tree. Once the sun re-emerges, they can once again be effectively driven by the direction of the incident rays, lifting up their faces, and reorienting to the new position. But this takes time. Super-sunflowers perform the following trick: even when the sun disappears, they continue to rotate at approximately the requisite ¼° per minute, so that the super-sunflowers are more nearly oriented to the light when the sun appears.
A normal sunflower is directly coupled to the movement of the sun. This is analogous to simple feedback systems like, for example, the bimetallic strip in a thermostat, which curls when the strip is heated and one side expands more than the other. In some weak sense, the curve of the bimetallic strip ‘represents’ the change in temperature. But the coupling is so direct that calling it ‘representation’ is dragging in more intentional language that we need. It’s just a load of physics.
The super-sunflower brings in a new ingredient: it carries on attempting to track the sun even when they’re out of direct causal contact. Smith argues that this disconnected tracking is the (sunflower) seed that genuine intentionality grows from. We are now on the way to something that can really be said to ‘represent’ the movement of the sun:
This behaviour, which I will call “non-effective tracking”, is no less than the forerunner of semantics: a very simple form of effect-transcending coordination in some way essential to the overall existence or well-being of the constituted system.
Example 2: Error checking
Now for a more realistic example. Consider the following simple error-checking system:
There’s a 32 bit word that we want to send, but we want to be sure that it’s been transmitted correctly. So we also send a 6-bit ‘check code’ containing the number of ones (19 of them in this instance, or 010011 in binary). If these don’t match, we know something’s gone wrong.
Obviously, we want the 6-bit code to stay coordinated with the 32-bit word for the whole storage period, and not just randomly change to some other count of ones, or it’s useless. Less obviously (“because it is such a basic assumption underlying the whole situation that we do not tend to think about it explicitly”), we don’t want the 6-bit code to invariably be correlated to the 32-bit word, so that a change in the word always changes the code. Otherwise we couldn’t do error checking at all! If a cosmic ray flips one of the bits in the word, we want the code to remain intact, so we can use it to detect the error. So again we have this ‘middle distance’ between direct coupling and irrelevance.
Example 3: File caches
One final real-world example: file caches. We want the data stored in the cache to be similar to the real data, or it’s not going to be much of a cache. At the same time, though, if we make everything exactly the same as the original data store, it’s going to take exactly as long to access the cache as it is to access the original data, so that it’s no longer really a cache.
Flex and slop
In all these examples, it’s important that the ‘representing’ system tries to stay coordinated with the distant ‘represented’ system while they’re out of direct contact. The super-sunflower keeps turning, the check code maintains its count of ones, the file cache maintains the data that was previously written to it:
In all these situations, what starts out as effectively coupled is gradually pulled apart, but separated in such a way as to honor a non-effective long-distance coordination condition, leading eventually to effective reconnection or reconciliation.
For this to be possible, the world needs to be able to support the right level of separation:
The world is fundamentally characterized by an underlying flex or slop – a kind of slack or ‘play’ that allows some bits to move about or adjust without much influencing, and without being much influenced by, other bits. Thus we can play jazz in Helsinki, as loud as we please, without troubling the Trappists in Montana. Moths can fly into the night with only a minimal expenditure of energy, because they have to rearrange only a tiny fraction of the world’s mass. An idea can erupt in Los Angeles, turn into a project, capture the fancy of hundreds of people, and later subside, never to be heard of again, all without having any impact whatsoever on the goings-on in New York.
This slop makes causal disconnection possible – ‘subjects’ can rearrange the representation independently of the ‘objects’ being represented. (This is what makes computation ‘cheap’ – we can rearrange some bits without having to also rearrange some big object elsewhere that they are supposed to represent some aspect of.) To make the point, Smith compares this with two imaginary worlds where this sort of ‘middle distance’ representation couldn’t get started. The first world consists of nothing but a huge assemblage of interlocking gears that turn together exactly without slipping, all at the same time. In this world, there is no slop at all, so nothing can ever get out of causal contact with anything else. You could maybe say that one cog ‘represents’ another cog, but really everything is just like the thermostat, too directly coupled to count interestingly as a representation. The second world is just a bunch of particles drifting in the void without interaction. This has gone beyond slop into complete irrelevance. Nothing is connected enough to have any kind of structural relation to anything else.
The three examples given above – file caches, error checking and the super-sunflower – are really only one step up from the thermostat, too simple to have anything much like genuine ‘intentional content’. The tracking behaviour of the representing object is too simple – the super-sunflower just moves across the sky, and the file cache and check code just sit there unchanged. Smith acknowledges this, and says that the exchange between ‘representer’ and ‘represented’ has to have a lot more structure, with alternating patterns of being in and out of causal contact, and some other ‘stabilisation’ patterns that I don’t really understand, that somehow help to individuate the two as separate objects. At this point, the concrete examples run completely dry, and I get lost in some complicated argument about ‘patterns of cross-cutting extension’ which I haven’t managed to disentangle yet. The basic idea illustrated by the three examples was new to me, though, and worth having on its own.
Last year I wrote a post which used an obscure term from cognitive psychology and an obscure passage from The Bell Jar to make a confused point about something I didn’t understand very well. I wasn’t expecting this to go very far, but it got more interest than I expected, and some very thoughtful comments. Then John Nerst wrote a much clearer summary of the central idea, attached it to a noisily controversial argument-of-the-month and sent it flying off around the internet. Suddenly ‘cognitive decoupling’ was something of a hit.
If I’d known this was going to happen I might have put a bit more effort into the original blog post. For a start, I might have done some actual reading, instead of just grabbing a term I liked the sound of from one of Sarah Constantin’s blog posts and running with it. So I wanted to understand how the term as we’ve been applying it differs from Stanovich’s original use, and what his influences were. I haven’t done a particularly thorough job on this, but I have turned up a few interesting things, including a surprisingly direct link to a 1987 paper on pretending that a banana is a phone. I also learned that the intellectual history I’d hallucinated for the term based on zero reading was completely wrong, but wrong in a way that’s been strangely productive to think about. I’ll describe both the actual history and my weird fake one below. But first I’ll briefly go back over what the hell ‘cognitive decoupling’ is supposed to mean, for people who don’t want to wade through all those links.
Roses, tripe, and the bat and ball again
Stanovich is interested in whether, to use Constantin’s phrase, ‘rational people exist’. In this case ‘rational’ behaviour is meant to mean something like systematically avoiding cognitive biases that most people fall into. One of his examples is the Wason selection task, which involves turning over cards to verify the statement ‘If the card has an even number on one face it will be red on the reverse’. More vivid real-world situations, like Stanovich’s example of ‘if you eat tripe you will get sick’, are much easier for people to reason about than the decontextualised card-picking version. (Cosmides and Tooby’s beer version is even easier than the tripe one.)
A second example he gives is the ‘rose syllogism’:
Premise 1: All living things need water
Premise 2: Roses need water
Therefore, Roses are living things
A majority of university students incorrectly judge this as valid, whereas almost nobody thinks this structurally equivalent version makes sense:
Premise 1: All insects need oxygen
Premise 2: Mice need oxygen
Therefore, Mice are insects
The rose conclusion fits well with our existing background understanding of the world, so we are inclined to accept it. The mouse conclusion is stupid, so this doesn’t happen.
A final example would be the bat and ball problem from the Cognitive Reflection Test: ‘A bat and a ball cost $1.10. The bat costs $1 more than the ball. How much does the ball cost?’. I’ve already written about that one in excruciating detail, so I won’t repeat myself too much, but in this case the interfering context isn’t so much background knowledge as a very distracting wrong answer.
Stanovich’s contention is that people that manage to navigate these problems successfully have an unusually high capacity for something he calls ‘cognitive decoupling’: separating out the knowledge we need to reason about a specific situation from other, interfering contextual information. In a 2013 paper with Toplak he describes decoupling as follows:
When we reason hypothetically, we create temporary models of the world and test out actions (or alternative causes) in that simulated world. In order to reason hypothetically we must, however, have one critical cognitive capability—we must be able to prevent our representations of the real world from becoming confused with representations of imaginary situations. The so-called cognitive decoupling operations are the central feature of Type 2 processing that make this possible…
The important issue for our purposes is that decoupling secondary representations from the world and then maintaining the decoupling while simulation is carried out is the defining feature of Type 2 processing.
(‘Type 2’ is a more recent name for ‘System 2’, in the ‘System 1’/’System 2’ dual process typology made famous by Kahneman’s Thinking, Fast and Slow. See Kaj Sotala’s post here for a nice discussion of Stanovich and Evan’s work relating this split to the idea of cognitive decoupling, and other work that has questioned the relevance of this split.)
I don’t know how well this works as an explanation of what’s really going on in these situations. I haven’t dug into the history of the Wason or rose-syllogism tests at all, and, as with the bat and ball question, I’d really like to know what was done to validate these as good tests. What similar questions were tried? What other explanations, like prior exposure to logical reasoning, were identified, and how were these controlled for? I don’t have time for that currently. For the purposes of this post, I’m more interested in understanding what Stanovich’s influences were in coming up with this idea, rather than whether it’s a particularly good explanation.
Context, wide and narrow
Constantin’s post is more or less what she calls a ‘fact post’, summarising research in the area without too much editorial gloss. When I picked this up, I was mostly excited by the one bit of speculation at the end, and the striking ‘cognitive decoupling elite’ phrase, and didn’t make any effort to stay close to Stanovich’s meaning. Now I’ve read some more, I think that in the end we didn’t drift too far away. Here is Nerst’s summary of the idea:
High-decouplers isolate ideas from each other and the surrounding context. This is a necessary practice in science which works by isolating variables, teasing out causality and formalizing and operationalizing claims into carefully delineated hypotheses. Cognitive decoupling is what scientists do.
To a high-decoupler, all you need to do to isolate an idea from its context or implications is to say so: “by X I don’t mean Y”. When that magical ritual has been performed you have the right to have your claims evaluated in isolation. This is Rational Style debate…
While science and engineering disciplines (and analytic philosophy) are populated by people with a knack for decoupling who learn to take this norm for granted, other intellectual disciplines are not. Instead they’re largely composed of what’s opposite the scientist in the gallery of brainy archetypes: the literary or artistic intellectual.
This crowd doesn’t live in a world where decoupling is standard practice. On the contrary, coupling is what makes what they do work. Novelists, poets, artists and other storytellers like journalists, politicians and PR people rely on thick, rich and ambiguous meanings, associations, implications and allusions to evoke feelings, impressions and ideas in their audience. The words “artistic” and “literary” refers to using idea couplings well to subtly and indirectly push the audience’s meaning-buttons.
Now of course, Nerst is aiming at a much wider scope – he’s trying to apply this to controversial real-world arguments, rather than experimental studies of cognitive biases. But he’s talking about roughly the same mechanism of isolating an idea from its surrounding context.
There is a more subtle difference, though, that I find interesting. It’s not a sharp distinction so much as a difference in emphasis. In Nerst’s description, we’re looking at the coupling between one specific idea and its whole background context, which can be a complex soup of ‘thick, rich and ambiguous meanings, associations, implications and allusions’. This is a clear ‘outside’ description of the beautiful ‘inside’ one that I pulled from The Bell Jar, talking about how it actually feels (to some of us, anyway) to drag ideas out from the context that gave them meaning:
Botany was fine, because I loved cutting up leaves and putting them under the microscope and drawing diagrams of bread mould and the odd, heart-shaped leaf in the sex cycle of the fern, it seemed so real to me.
The day I went in to physics class it was death.
A short dark man with a high, lisping voice, named Mr Manzi, stood in front of the class in a tight blue suit holding a little wooden ball. He put the ball on a steep grooved slide and let it run down to the bottom. Then he started talking about let a equal acceleration and let t equal time and suddenly he was scribbling letters and numbers and equals signs all over the blackboard and my mind went dead.
… I may have made a straight A in physics, but I was panic-struck. Physics made me sick the whole time I learned it. What I couldn’t stand was this shrinking everything into letters and numbers. Instead of leaf shapes and enlarged diagrams of the hole the leaves breathe through and fascinating words like carotene and xanthophyll on the blackboard, there were these hideous, cramped, scorpion-lettered formulas in Mr Manzi’s special red chalk.
In this description, the satisfying thing about the botany classes is the rich sensory context: the sounds of the words, the vivid images of ferns and bread mould, the tactile sense of chopping leaves. This is a very broad-spectrum idea of context.
Now, Stanovich does seem to want cognitive decoupling to apply in situations where people access a wide range of background knowledge (‘roses are living things’), but when he comes to hypothesising a mechanism for how this works he goes for something with a much narrower focus. In the 2013 paper with Toplak he talks about specific, explicit ‘representations’ of knowledge interfering with other explicit representations. (I’ll go into more detail later about exactly what he means by a ‘representation’.) He cites an older paper, Pretense and Representation by Leslie, as inspiration for the ‘decoupling’ term:
In a much-cited article, Leslie (1987) modeled pretense by positing a so-called secondary representation (see Perner 1991) that was a copy of the primary representation but that was decoupled from the world so that it could be manipulated — that is, be a mechanism for simulation.
This is very clearly about being able to decouple one specific explicit belief from another similarly explicit ‘secondary representation’, rather than the whole background morass of implicit context. I wanted to understand how this was supposed to work, so I went back and read the paper. This is where the banana phones come in.
Pretending a banana is a phone
The first surprise for me was how literal this paper was. (Apparently 80s cognitive science was like that.) Leslie is interested in how pretending works – how a small child pretends that a banana is a telephone, to take his main example. And the mechanism he posits is… copy-and-paste, but for the brain:
As in, we get some kind of perceptual input which causes us to store a ‘representation’ that means ‘this is a banana’. Then we make a copy of this. Now we can operate on the copy (‘this banana is a telephone’) without also messing up the banana representation. They’ve become decoupled.
What are these ‘representations’? Leslie has this to say:
What I mean by representation will, I hope, become clear as the discussion progresses. It has much in common with the concepts developed by the information-processing, or cognitivist, approach to cognition and perception…
This is followed by a long string of references to Chomsky, Dennett, etc. So his main influence appears to be, roughly, computational theories of mind. Looking at how he uses the term in the paper itself, it appears that we’re in the domain of Good Old-Fashioned AI: ‘representations’ can be put into a rough correspondence with English propositions about bananas, telephones, and cups of tea, and that we then use them as a kind of raw material to run inference rules on and come to new conclusions:
Leslie doesn’t talk about how all these representations come to mean anything in the real world — how do we know that the string of characters ‘cups contain water’, or its postulated mental equivalent, has anything to do with actual cups and actual water? How do we even parse the complicated flux of the real world into discrete named objects, like ‘cups’, to start with? There’s no story in the paper that tries to bridge this gap — these representations are just sitting there ‘in the head’, causally disconnected from the world.
Well, OK, maybe 80s cognitive science was like that. Maybe Leslie thought that someone else already had a convincing story for how this bit works, and he could just apply the resulting formalism of propositions and inference rules. But this same language of ‘representations’ and ‘simulations’ is still being used uncritically in much more recent papers. Stanovich and Toplak, for example, reproduce Leslie’s decoupling diagram and describe it using the same terms:
For Leslie (1987), the decoupled secondary representation is necessary in order to avoid representational abuse — the possibility of confusing our simulations with our primary representations of the world as it actually is… decoupled representations of actions about to be taken become representations of potential actions, but the latter must not infect the former while the mental simulation is being carried out.
There’s another strange thing about Stanovich using this paper as a model to build on. (I completely missed this, but David Chapman pointed it out to me in an earlier conversation.) Stanovich is interested in what makes actions or behaviours rational, and he wants cognitive decoupling to be at least a partial explanation of this. Leslie is looking at toddlers pretending that bananas are telephones. If even very young children are passing this test for ‘rationality’, it’s not going to be much use for discriminating between ‘rational’ and ‘irrational’ behaviour in adults. So Stanovich would need a narrower definition of ‘decoupling’ that excludes the banana-telephone example if he wants to eventually use it as a rationality criterion.
So I wasn’t very impressed with this as a plausible mechanism for decoupling. Then again, the mechanism I’d been imagining turns out to have some obvious failings too.
Rabbits and the St. Louis Arch
When I first started thinking about cognitive decoupling, I imagined a very different history for the term. ‘Decoupling’ sounds very physicsy to me, bringing up associations of actual interaction forces and coupling constants, and I’d been reading Dreyfus’s Why Heideggerian AI Failed, which discusses dynamical-systems-inspired models of cognition:
Fortunately, there is at least one model of how the brain could provide the causal basis for the intentional arc. Walter Freeman, a founding figure in neuroscience and the first to take seriously the idea of the brain as a nonlinear dynamical system, has worked out an account of how the brain of an active animal can find and augment significance in its world. On the basis of years of work on olfaction, vision, touch, and hearing in alert and moving rabbits, Freeman proposes a model of rabbit learning based on the coupling of the brain and the environment…
The organism normally actively seeks to improve its current situation. Thus, according to Freeman’s model, when hungry, frightened, disoriented, etc., the rabbit sniffs around until it falls upon food, a hiding place, or whatever else it senses it needs. The animal’s neural connections are then strengthened to the extent that reflects the extent to which the result satisfied the animal’s current need. In Freeman’s neurodynamic model, the input to the rabbit’s olfactory bulb modifies the bulb’s neuron connections according to the Hebbian rule that neurons that fire together wire together.
In many ways this still sounds like a much more promising starting point to me than the inference-rule-following of the Leslie paper. For a start, it seems to fit much better with what’s known about the architecture of the brain (I think – I’m pretty ignorant about this). Neurons are very slow compared to computer processors, but make up for this by being very densely interconnected. So getting anything useful done would rely on a huge amount of activation happening in parallel, producing a kind of global, diffuse ‘background context’ that isn’t sharply divided into separate concepts.
Better still, the problem of how situations intrinsically mean something about the world is sidestepped, because in this case, the rabbit and environment are literally, physically coupled together. A carrot smell out in the world pulls its olfactory bulb into a different state, which itself pulls the rabbit into a different kind of behaviour, which in turn alters the global structure of the bulb in such a way that this behaviour is more likely to occur again in the future. This coupling is so direct that referring to it as a ‘representation’ seems like overkill:
Freeman argues that each new attractor does not represent, say, a carrot, or the smell of carrot, or even what to do with a carrot. Rather, the brain’s current state is the result of the sum of the animal’s past experiences with carrots, and this state is directly coupled with or resonates to the affordance offered by the current carrot.
However, this is also where the problems come in. Everything is so closely causally coupled that there’s no room in this model for decoupling! The idea behind ‘cognitive decoupling’ is to be able to pull away from the world long enough to consider things in the abstract, without all the associations that normally get dragged along for free. In the olfactory bulb model, the rabbit is so locked into its surroundings that this sort of distance is unattainable.
At some point I was googling a bunch of keywords like ‘dynamical systems’ and ‘decoupling’ in the hope of fishing up something interesting, and I came across a review by Rick Grush of Mind as Motion: Explorations in the Dynamics of Cognition by Port and van Gelder, which had a memorable description of the problem:
…many paradigmatically cognitive capacities seem to have nothing at all to do with being in a tightly coupled relationship with the environment. I can think about the St. Louis Arch while I’m sitting in a hot tub in southern California or while flying over the Atlantic Ocean.
Even this basic kind of decoupling from a situation – thinking about something that’s not happening to you right now – needs some capacities that are missing from the olfactory bulb model. Grush even uses the word ‘decoupling’ to describe this:
…what is needed, in slightly more refined terms, is an executive part, C (for Controller), of an agent, A, which is in an environment E, decoupling from E, and coupling instead to some other system E’ that stands in for E, in order for the agent to ‘think about’ E (see Figure 2). Cognitive agents are exactly those which can selectively couple to either the ‘real’ environment, or to an environment model, or emulator, perhaps internally supported, in order to reason about what would happen if certain actions were undertaken with the real environment.
This actually sounds like a plausible alternate history for Stanovich’s idea, with its intellectual roots in dynamical systems rather than the representational theory of mind. So maybe my hallucinations were not too silly after all.
I still think that the idea of cognitive decoupling is getting at something genuinely interesting – otherwise I wouldn’t have spent all this time rambling on about it! I don’t think the current representational story for how it works is much good. But the ability to isolate ‘abstract structure’ (whatever that means, exactly) from its surrounding context does seem to be a real skill that people vary in. In practice I expect that much of this context will be more of a diffuse associational soup than the sharp propositional statements of Leslie’s pretence model.
It’s interesting to me that the banana phone model and the olfactory bulb model both run into problems, but in opposite directions. Leslie’s banana phone relies on a bunch of free-floating propositions (‘this is a banana’), with no story for how they refer to actual bananas and phones out in the world. Freeman’s rabbit olfactory bulb has no problem with this – relevance is guaranteed through direct causal coupling to the outside world – but it’s so directly coupled that there’s no space for decoupling. We need something between these two extremes.
David Chapman pointed out to me that Brian Cantwell Smith already has a term for this in The Origin of Objects – he calls it ‘the middle distance’ between direct coupling and causal irrelevance. I’ve been reading the book and have already found his examples to be hugely useful in thinking about this more clearly. These are worth a post in their own right, so I’ll describe them in a followup to this one.
I’ve been looking back at some of the mess I produced while trying to get an initial grip on the ideas I wrote up in my lasttwo posts on negative probability. One of my main interests in this blog is the gulf between maths as it is formally written up and the weird informal processes that produce mathematical ideas in the first place, so I thought this might make a kind of mini case study. Apologies in advance for my handwriting.
I do all my work in cheap school-style exercise books. The main thread of what I’m thinking about goes front-to-back: that’s where anything reasonably well-defined that I’m trying to do will go. Working through lecture notes, doing exercises, any calculations where I’m reasonably clear on what I’m actually calculating. But if I have no idea what I’m even trying to do, it goes in the back, instead. The back has all kinds of scribbles and disorganised crap:
Most of it is no good, but new ideas also tend to come from the back. The Wigner function decomposition was definitely a back-of-the-book kind of thing. I’ve mostly forgotten what I was thinking when I made all these scribblings, and I wouldn’t trust the remembered version even if I had one, so I’ll try to refrain from too much analysis.
The idea seems to originate here:
This has the key idea already: start with equal probability for all squares, and then add on correction terms until the bottom left corner goes negative. But the numbers are complete bullshit! Looking back, I can’t make sense of them at all. For instance, I was trying to add to things, instead of . Why? No idea! It’s not like is a number that had come up in any of my previous calculations, so I have no idea what I was thinking.
Even with the bullshit numbers, I must have had some inkling that this line of thought was worth pursuing, so I wrote it out again. This time I realised the numbers were wrong and crossed them out, writing the correct ones to the right:
The little squares above the main squares are presumably to tell me what to do: add to the filled in squares and subtract it from the blank ones.
I then did a sanity check on an example with no negative probabilities, and it worked:
At that point, I think I was convinced it worked in general, even though I’d only checked two cases. So I moved to the front of the book. After that, the rest of it looks like actual legit maths that a sane person would do, so it’s not so interesting. But I had to produce this mess to get there.
OK, so this is where I go back through everything from the last post, but this time show how all the fiddling around with boxes relates back to quantum physics, and also go into some technical details like explaining what I meant by ‘half the information’ in the discussion at the end. This is unavoidably going to need more maths than the last post, and enough quantum physics knowledge to be OK with qubits and density matrices. I’ll start by translating everything into a standard physics problem.
Qubit phase space
So, first off, instead of the ‘strange machine’ of the last post we will have a qubit state – as a first example I’ll take the state. The three questions then become measurements on it. Specifically, these measurements are expectation values of the operators , where the are the three Pauli matrices.
For we get the following:
This can be represented on the same sort of 2×2 grid I used in the previous post:
The state has a definite value of 0 for the measurement, so the probabilities in the cells where must sum to 1. For the state there is an equal chance of either or . The third measurement, , can be shown to be associated with the diagonals of the grid, in the same way as in Piponi’s example in the previous post, and again there is an equal chance of either value. Imposing all these conditions gives the probability assignment above.
The 2×2 grid is called the phase space of the qubit, and the function that assigns probabilities to each cell is called the Wigner function . To save on drawing diagrams, I’ll represent this as a square-bracketed matrix from now on:
For much more detail on how this all works, the best option is probably to read Wootters, who developed a lot of the ideas in the first place. There’s his original paper, which has all the technical details, and a nice follow-up paper on Picturing Qubits in Phase Space which gives a bit more intuition for what’s going on.
In the previous post I gave the following formula for the Wigner function:
which simplifies to
This is a somewhat different form to the standard formula for the Wigner function, but I’ve checked that they’re equivalent. I’ve put the details on a separate notes page here, in a sort of blog post version of the really boring technical appendix you get at the back of papers.
As with the example in the last blog post, it’s possible to get qubit states where some of the values of the Wigner function are negative. The numbers don’t work out so nicely this time, but as one example we can take the qubit state . (This is the +1 eigenvector of the density matrix .)
The Wigner function for is
with one negative entry. I learned while writing this that the states with negative values are called magic states by quantum computing people! These are the states that provide the ‘magic’ for quantum computing, in terms of giving a speed-up over classical computing. I’d like be able to say more about this link, but I’ll never finish the post if I have to get my head around all of that too, so instead I’ll link to this post by Earl Campbell that goes into more detail and points to some references. A quick note on the geometry, though:
The six eigenvectors of the Pauli matrices form the corners of an octahedron on the Bloch sphere, as in my dubious sketch above. We’ve already seen that the state has no magic – all the values are nonnegative. This also holds for the other five, which have the following Wigner functions:
The other states on the surface of the octahedron or inside it also have no magic. The magic states are the ones outside the octahedron, and the further they are from the octahedron the more magic they are. So the most magic states are on the surface of the sphere opposite the middle of the triangular faces.
Half the information
Why can’t we have a probability of as before? Well, I briefly mentioned the reason in the previous blog post, but I can go into more detail now. There are constraints on the values of that forbids values that are this negative. First off, the values of have to sum to 1 – this makes sense, as they are supposed to be something like probabilities.
The second constraint is more interesting. Taking the state as an example again, this state has a definite answer to one of the questions and no information at all about the other two. There’s redundancy in the questions, so exact answers to two of them would be enough to pin down the state precisely. So we have half of the possible information.
This turns out to be the most information you can get from any qubit state, in some sense. I say ‘in some sense’ because it’s a pretty odd definition of information.
I learned about this from a fascinating paper by van Enk, A toy model for quantum mechanics, which was actually my starting point for thinking about this whole topic. He starts with the Spekkens toy model, a very influential idea that reproduces a number of the features of quantum mechanics using a very simple model. Again, this is too big a topic to get into all the details, but the most basic system in this model maps to the six ‘non-magic’ qubit states listed above, in the corners of the octahedron. These all share the half-the-knowledge property of the state, where we know the answer to one question exactly and have no idea about the others.
Now van Enk’s aim is to extend this idea of ‘half the knowledge’ to more general probability distributions over the four boxes. But this requires having some kind of measure of what half the knowledge means. He stipulates that this measure should have for the six half-the-knowledge states we already have, which seems reasonable. Also, it should have for states where we know all the information (impossible in quantum physics), and for the state of total ignorance about all questions. Or to put it a bit differently,
where is an entropy measure – it decreases from 2 to 1 to 0 as we learn more information about the system. There’s a parametrised family of entropies known as the Rényi entropies, which reproduce this behaviour for the cases above, and differ for other distributions over the boxes. (I have some rough notes about these here, which may or may not be helpful.) By far the most well-known one is the Shannon entropy , used widely in information theory, but it turns out that this one doesn’t reproduce the states found in quantum physics. Instead, van Enk picks , the collision entropy. This has quite a simple form:
where the are the four components of – we’re just summing the squares of them. So then our information measure is just , and the second constraint on is this can have value at most :
Why this particular entropy measure? That’s something I don’t really understand. Van Enk describes it as ‘the measure of information advocated by Brukner and Zeilinger’, and links to their paper, but so far I haven’t managed to follow the argument there, either. If anyone reads this and has any insight, I’d like to know!
In some ways, I know a lot more about negative probabilities than I did when I started getting interested in this. But conceptually I’m almost as confused as I was at the start! I think the main improvement is that I have some more focussed questions to be confused about:
Is the way of decomposing the Wigner function that I described in these posts any use for making sense of the negative probabilities? I found it quite helpful for Piponi’s example, in giving some more insight into how the negative value connects to that particular answer being ‘especially inconsistent’. Is it also useful for thinking about qubits?
Any link to the idea of negative probabilities representing events ‘unhappening’? As I said at the beginning of the first post, I love this idea but have never seen it fully developed anywhere in a satisfying way.
What’s going on with this collision entropy measure anyway?
I’m not a quantum foundations researcher – I’m just an interested outsider trying to understand how all these ideas fit together. So I’m likely to be missing a lot of context that people in the field would have. If you read this and have pointers to things that I’m missing, please let me know in the comments!
I’ve been thinking about the idea of negative probabilities a lot recently, and whether it’s possible to make any sense of them. (For some very muddled and meandering background on how I got interested in this, you could wade through my ramblings here, here, here and here, but thankfully none of that is required to understand this post.)
To save impatient readers the hassle of reading this whole thing: I’m not going to come up with any brilliant way of interpreting negative probabilities in this blog post! But recently I did notice a few things that are interesting and that I haven’t seen collected together anywhere else, so I thought it would be worth writing them up.
Now, why would you even bother trying to make sense of negative probabilities? I’m not going to go into this in any depth – John Baez has an great introductory post on negative probability that motivates the idea, and links to a good chunk of the (not very large) literature. This is well worth reading if you want to know more. But there are a couple of main routes that lead people to get interested in this thing.
The first route is pretty much pure curiosity: what happens if we try extending the normal idea of probabilities to negative numbers? This is often introduced in analogy with the way we often use negative numbers in applications to simplify calculations. For example, there’s a fascinating discussion of negative probability by Feynman which starts with the following simple situation:
A man starting a day with five apples who gives away ten and is given eight during the day has three left. I can calculate this in two steps: 5 – 10 = -5 and -5 + 8 = 3.
The final answer is satisfactorily positive and correct although in the intermediate steps of calculation negative numbers appear. In the real situation there must be special limitations of the time in which the various apples are received and given since he never really has a negative number, yet the use of negative numbers as an abstract calculation permits us freedom to do our mathematical calculations in any order, simplifying the analysis enormously, and permitting us to disregard inessential details.
So, although we never actually have a negative number of apples, allowing them to appear in intermediate calculations makes the maths simpler.
The second route is that negative probabilities actually crop up in exactly this way in quantum physics! This isn’t particularly obvious in the standard formulation learned in most undergrad courses, but the theory can also be written in a different way that closely resembles classical statistical mechanics. However, unlike the classical case, the resulting ‘distribution’ is not a normal probability distribution, but a quasiprobability distribution that can also take negative values.
As with Feynman’s apples, these negative values don’t map to anything we observe directly: all measurements we could make give results that occur with zero or positive probabilities, as you would expect. The negative probabilities instead come in as intermediate steps in the calculation.
This should become clearer when I work through a toy example. The particular example I’ll use (which I got from an excellent blog post by Dan Piponi) doesn’t come up in quantum physics, but it’s very close: its main advantage is that the numbers are a bit simpler, so it’s easier to concentrate on the ideas. I’ll do this in two pieces: one that requires no particular physics or maths background and just walks through the example using basic arithmetic, and one that makes connections back to the quantum mechanics literature and might drop in a Pauli matrix or two. This is the no-maths one.
Neither of these routes really get to the point of fully making sense of negative probabilities. In the apple example, we have a tool for making calculations easier, but we also have an interpretation of ‘a negative apple’, in terms of taking away one of the apples you have already. For negative probabilities, we mostly just have the calculational tool. It’s tempting to try and follow the apple analogy and interpret negative probabilities as being to do with something like ‘events unhappening’ – many people have suggested this (see e.g. Michael Nielsen here), and I certainly share the intuition that something like this ought to be possible, but I’ve never seen anything fully worked out along those lines that I’ve found really satisfying.
In the absence of a compelling intuitive explanation, I find it helpful to work through examples and get an idea of how they work. Even if we don’t end up with a good explanation for what negative probabilities are, we can see what they do, and start to build up a better understanding of them that way.
A strange machine
OK, so let’s go through Piponi’s example (here’s the link again). He describes it very clearly and concisely in the post, so it might be a good idea to just switch to reading that first, but for completeness I’ll also reproduce it here.
Piponi asks us to consider a case where:
a machine produces boxes with (ordered) pairs of bits in them, each bit viewable through its own door.
So you could have 0 in both boxes, 0 in the first and 1 in the second, and so on. Now suppose we ask the following three questions about the boxes:
Is the first box in state 0?
Is the second box in state 0?
Are the boxes both in the same state?
I’ll work through two possible sets of answers to these questions: one consistent and unobjectionable set, and one inconsistent and stupid one.
Example 1: consistent answers
Let’s say that we find that the answer to the first question is ‘yes’ , the answer to the second is ‘no’, and the answer to the third is ‘no’. This makes sense, and we can interpret this easily in terms of an underlying state of the two boxes. The first box is in state 0, the second box is in state 1, and so of course the two are in different states and the answer to the third question is also satisfied.
We can represent this situation with the grid below:
The system is in state ‘first box 0, second box 1’, with probability 1, and the other states have probability 0. This is all very obvious – I’m just labouring the point so I can compare it to the case of inconsistent answers, where things get weird.
Example 2: inconsistent answers
Now suppose we find a inconsistent set of answers when we measure the box: ‘no’ to all three questions. This doesn’t make much intuitive sense: both boxes are in state 0, but also they are in different states. Still, Piponi demonstrates that you can still assign something like ‘probabilities’ to the squares on the grid, as long as you’re OK with one of them being negative:
Let’s go through how this matches up with the answers to the questions. For the first question, we have
so the answer is ‘no’ as required. Similarly, for the other two questions we have
so we get ‘no’ to all three, at the expense of having introduced this weird negative probability in one cell of the grid.
It’s not obvious at all what the negative probability means, though! Piponi doesn’t explain how he came up with this solution, but I’m guessing it’s one of either ‘solve the equations and get the answer’ or ‘notice that these numbers happen to work’.
I wanted to think a bit more about interpretation, and although I haven’t fully succeeded, I did notice a more enlightening calculation method, which maybe points in a useful direction. I’ll describe it below.
A calculation method
Some motivating intuition: all four possible assignments of bits to boxes are inconsistent with the answers in Example 2, but ‘both bits are zero’ is particularly inconsistent. It’s inconsistent with the answers to all three questions, whereas the other assignments are inconsistent with only one question each (for example, ‘both bits are 1’ matches the answer to the first two questions, but is inconsistent with the two states being different).
So you can maybe think in terms of consecutively answering the three questions and penalising assignments that are inconsistent. ‘Both bits are zero’ is an especially bad answer, so it gets clobbered three times instead of just once, pushing the probability negative.
The method I’ll describe is a more formal version of this. I’ll go through it first for Example 1, with consistent answers, to show it works there.
Back to Example 1
Imagine that we start in a state of complete ignorance. We have no idea what the underlying state is, so we just assign probability ¼ to each cell of the grid, like this:
(I’ll stop drawing the axes every time from this point on.) We then ask the three questions in succession and make corrections. For the first question, ‘is the first box in state 0’, we have the answer ‘yes’, so after we learn this we know that the left two cells of the grid now have probability ½ each, and the right two have probability 0. We can think of this as adding a correction term to our previous state of ignorance:
Notice that the correction term has some negative probabilities in it! But these seem relatively benign from an interpretational point of view – they are just removing probability from some cells so that it can be reassigned to others, and the final answer is still positive. It’s kind of similar to saying , where we subtract some probability to get to the answer.
Next, we add on two more correction terms, one for each of the remaining two questions. The correction term for the second question needs to remove probability from the bottom row and add it to the top row, and the one for the third question corrects the diagonals:
Adding ‘em all up gives
So the system is definitely in the top left state, which is what we found before. It’s good to verify that the method works on a conventional example like this, where the final probabilities are positive.
Example 2 again
I’ll follow the same method again for Piponi’s example, starting from complete uncertainty and then adding on a correction for each question (this time the answer is ‘no’ each time). This time I’ll do it all in one go:
which adds up to
So we’ve got the same probabilities as Piponi, with the weird negative -½ probability for ‘both in state 0’. This time we get a little bit more insight into where it comes from: it’s picking up a negative correction term from all three questions.
This ‘strange machine’ looks pretty bizarre. But it’s extremely similar to a situation that actually comes up in quantum physics. I’ll go into the details in the follow-up post (‘now with added equations!’), but this example almost replicates the quasiprobability distribution for a qubit, one of the simplest systems in quantum physics. The main difference is that Piponi’s machine is slightly ‘worse’ than quantum physics, in that the -½ value is more negative than anything you get there.
The two examples I did were ones where all three questions have definite yes/no answers, but my method of starting from a state of ignorance and adding on corrections carries over in the obvious way when you have a probability distribution over ‘yes’ and ‘no’. As an example, say you have a 0.8 probability of ‘no’ for the first question. Then you add 0.8 times the correction matrix for ‘no’, with the negative probabilities on the left hand side, and 0.2 times the correction matrix for ‘no’, with the negative probabilities on the right hand side. That’s all there is to it. Just to spell it out I’ll add the general formula: if the three questions have answer ‘no’ with probabilities , , respectively, then we assign probabilities to the cells as follows:
(If you’re wondering where the comes from, it’s just the usual letter used to label this thing – it stands for ‘Wigner’, and is a discrete version of his Wigner function.)
It turns out that all examples in quantum physics are of the type where you don’t have certain knowledge of the answers to all three questions. It’s possible to know the answer to one of them for certain, but then you have to be completely ignorant about the other two, and assign probability ½ to both answers. More usually, you will have partial information about all three questions, with a constraint that the total information you get about the system is at most half the total possible information, in a specific technical sense. To go into this in detail will require some more maths, which I’ll get to in the next post.
In the references to The World Beyond Your Head I found an intriguing paper by Mizuko Ito, Mobilizing Fun in the Production and Consumption of Children’s Software, following the interactions between children and adults at an after-school computer club. It’s written in a fairly heavy dialect of academicese, but the dialogue samples are fascinating. Here a kid, Jimmy, is playing SimCity 2000, with an undergrad, Holly, watching:
J: (Budget window comes up and Jimmy dismisses it.) Yeah. I’m going to bulldoze a skyrise here. (Selects bulldozer tool and destroys building.) OK. (Looks at H.) Ummm! OK, wait, OK. Should I do it right here?
H: Sure, that might work… that way. You can have it …
J: (Builds highway around city.) I wonder if you can make them turn. (Builds highway curving around one corner) Yeah, okay.
H: You remember, you want the highway to be … faster than just getting on regular streets. So maybe you should have it go through some parts.
J: (Dismisses budget pop-up window. Points to screen.) That’s cool! (Inaudible.) I can make it above?
H: Above some places, I think. I don’t know if they’d let you, maybe not.
J: (Moves cursor over large skyscraper.) That’s so cool!
H: Is that a high rise?
J: Yeah. I love them.
H: Is it constantly changing, the city? Is it like …
J: (Builds complicated highway intersection. Looks at H.)
J: So cool. (Builds more highway grids in area, creating a complex overlap of four intersections.)
H: My gosh, you’re going to have those poor drivers going around in circles.
J: I’m going to erase that all. I don’t like that, OK. (Bulldozes highway system and blows up a building in process.) Ohhh …
H: Did you just blow up something else?
J: Yeah. (Laughs.)
J: I’m going to start a new city. I don’t understand this one. I’m going to start with highways. (Quits without saving city.)
As Ito puts it, “by the end Jimmy has wasted thousands of dollars on a highway to nowhere, blown up a building, and trashed his city.” So what’s the point of playing the game in this way?
Well, for a start, it lets him make cool stuff and then blow it up. That might be all the explanation we need! But I think he’s also doing something genuinely useful for understanding the game itself.
Ito mainly seems to be interested in the social dynamics of the situation – the conflict between Jimmy finding ‘fun’, ‘spectacular’ effects in the game, and Holly trying to drag him back to more ‘educational’ behaviours. I can see that too, but I’m interested in a slightly different reading.
To my mind, Jimmy is ‘sketching’: he’s finding out what the highway tool can do as a tool, rather than immediately subsuming it to the overall logic of the game. The highway he’s building is in a pointless location and doesn’t function very well as a highway, but that doesn’t matter. He’s investigating how to make it turn, how to make it intersect with other roads, how to raise it above ground level. While focussed on this, he ignores any more abstract considerations that would pull him out of engagement with the tool. For example, he dismisses the budget popup as fast as he can, so that he can get back to bulldozing buildings.
Now he knows what the tool does, he may as well just trash the current city and start a new one where he can use his knowledge in a more productive way. His explorations are useless in the context of the current game, but will give him raw material to work with later in a different city, where he might need a fancy junction or an overhead highway.
I first wrote a version of this for the newsletter last year. Reading it back this time, I noticed something else: Jimmy’s explorations are a great example of bricolage. I first learned this term from Sherry Turkle and Seymour Papert’s Epistemological Pluralism and the Revaluation of the Concrete, which I talked about here once before. In Turkle and Papert’s sense of the word, adapted from Lévi-Strauss, bricolage is a particular style of programming computers:
Bricoleurs construct theories by arranging and rearranging, by negotiating and renegotiating with a set of well-known materials.
… They are not drawn to structured programming; their work at the computer is marked by a desire to play with the elements of the program, to move them around almost as though they were material elements — the words in a sentence, the notes on a keyboard, the elements of a collage.
… bricoleur programmers, like Levi-Strauss’s bricoleur scientists, prefer negotiation and rearrangement of their materials. The bricoleur resembles the painter who stands back between brushstrokes, looks at the canvas, and only after this contemplation, decides what to do next. Bricoleurs use a mastery of associations and interactions. For planners, mistakes are missteps; bricoleurs use a navigation of midcourse corrections. For planners, a program is an instrument for premeditated control; bricoleurs have goals but set out to realize them in the spirit of a collaborative venture with the machine. For planners, getting a program to work is like ”saying one’s piece”; for bricoleurs, it is more like a conversation than a monologue.
One example in the paper is ‘Alex, 9 years old, a classic bricoleur’, who comes up with a clever repurposing of a Lego motor:
When working with Lego materials and motors, most children make a robot walk by attaching wheels to a motor that makes them turn. They are seeing the wheels and the motor through abstract concepts of the way they work: the wheels roll, the motor turns. Alex goes a different route. He looks at the objects more concretely; that is, without the filter of abstractions. He turns the Lego wheels on their sides to make flat ”shoes” for his robot and harnesses one of the motor’s most concrete features: the fact that it vibrates. As anyone who has worked with machinery knows, when a machine vibrates it tends to ”travel,” something normally to be avoided. When Alex ran into this phenomenon, his response was ingenious. He doesn’t use the motor to make anything ”turn,” but to make his robot (greatly stabilized by its flat ”wheel shoes”) vibrate and thus ”travel.” When Alex programs, he likes to keep things similarly concrete.
This is a similar mode of investigation to Jimmy’s. He’s seeing what kinds of things the motor and wheels can do, as part of an ongoing conversation with his materials, without immediately subsuming them to the normal logic of motors and wheels. In the process, he’s discovered something he wouldn’t have done if he’d just made a normal car. Similarly, Jimmy will have more freedom with the highway tool in the future than if he followed all the rules about budgets and city planning before he understood everything that it can do.
Alternatively, maybe I’m massively overanalysing this short contextless stretch of dialogue, and Jimmy just likes making stuff explode. Maybe he just keeps making and trashing a series of similarly broken cities for the sheer fun of it. Either way, mashing these two papers together has been a fun piece of bricolage of my own.
I wrote a version of this for the newsletter last year and decided to expand it out into a post. I’ve also added in a few thoughts based on an email conversation about the book with David MacIver a while back, and a few more thoughts inspired by a more recent post.
This wasn’t a book I’d been planning to read. In fact, I’d never even heard of it. I was just working in the library one day, and the cover caught my attention. It’s been given the subtitle ‘How To Flourish in an Age of Distraction’, and it looks like the publisher has tried to sell it as a sort of book-length version of one of those hand-wringers in The Atlantic about how we all gawp at our phones too much. I’m a sucker for those. This is a bit pathetic, I know, but there are certain repetitive journalist topics that I like simply because they’re repetitive, and the repetition has given them a comfortingly familiar texture, and ‘we all gawp at our phones too much’ is one of them. So I had a flick through.
The actual contents turned out to be less comfortingly familiar, but a lot more interesting. Actually, I recognised a lot of it! Merleau-Ponty on perception… Polanyi on tacit knowledge… lots of references to embodied cognition. This looks like my part of the internet! I hadn’t seen this set of ideas collected together in a pop book before, so I thought I’d better read it.
The author, Matthew Crawford, previously wrote a book called Shop Class As Soul Craft, on the advantages of working in the skilled trades. In this one he zooms out further to take a more philosophical look at why working with your hands with real objects is so satisfying. There’s a lot of good stuff in the book, which I’ll get to in a minute. I still struggled to warm to it, though, despite it being full of topics I’m really interested in. Some of this was just a tone thing. He writes in a style I’ve seen before and don’t get on with – I’m not American and can’t place it very exactly, but I think it’s something like ‘mild social conservatism repackaged for the educated coastal elite’. According to Wikipedia he writes for something called The New Atlantis, which may be of the places this style comes from. I don’t know. There’s also a more generic ‘get off my lawn’ thing going on, where we are treated to lots of anecdotes about how the airport is too loud and there’s too much advertising and children’s TV is terrible and he can’t change the music in the gym.
The oddest thing for me was his choice of pronouns for the various example characters he makes up throughout the book to illustrate his points. This is always a pain because every option seems to annoy someone, but using ‘he’ consistently would at least have fitted the grumpy old man image quite well. Maybe his editor told him not to do that, though, or maybe he has some kind of point to make, because what he actually decided to do was use a mix of ‘he’ and ‘she’, but only ever pick the pronoun that fits traditional expectations of what gender the character would be. Because he mostly talks about traditionally masculine occupations, this means that maybe 80% of the characters, and almost all of the sympathetic ones, are male – all the hockey players, carpenters, short-order cooks and motorcycle mechanics he’s using to demonstrate skilled interaction with the environment. The only female characters I remember are a gambling addict, a New Age self-help bore, a disapproving old lady, and one musician who actually gets to embody the positive qualities he’s interested in. It’s just weird, and I found it very distracting.
OK, that’s all my whining about tone done. I have some more substantive criticisms later, but first I want to talk about some of the things I actually liked. Underneath all the owning-the-libs surface posturing he’s making a subtle and compelling argument. Unpacking this argument is quite a delicate business, and I kind of understand why the publishers just rounded it off to the gawping at phones thing.
Violins and slot machines
Earlier, I said that Crawford is out to explain ‘why working with your hands with real objects is so satisfying’, but actually he’s going for something a little more nuanced and specific than that. Not all real objects are satisfying to work with. Here’s his discussion of one that isn’t, at least for an adult:
When my oldest daughter was a toddler, we had a Leap Frog Learning Table in the house. Each side of the square table presents some sort of electromechanical enticement. There are four bulbous piano keys; a violin-looking thing that is played by moving a slide rigidly located on a track; a transparent cylinder full of beads mounted on an axle such that any attempt, no matter how oblique, makes it rotate; and a booklike thing with two thick plastic pages in it.
… Turning off the Leap Frog Learning Table would produce rage and hysterics in my daughter… the device seemed to provide not just stimulation but the experience of agency (of a sort). By hitting buttons, the toddler can reliably make something happen.
The Leap Frog Learning Table is designed to take very complicated data from the environment – toddlers bashing the thing any old how, at any old speed or angle – and funnel this mess into a very small number of possible outcomes. The ‘violin-looking thing’ has only one translational degree of freedom, along a single track. Similarly, the cylinder can only be rotated around one axis. So the toddler’s glancing swipe at the cylinder is not dissipated into uselessness, but instead produces a satisfying rolling motion – they get to ‘make something happen’.
This is extremely satisfying for a toddler, who struggles to manipulate the more resistant objects of the adult world. But there is very little opportunity for growth or mastery there. The toddler has already mastered the toy to almost its full extent. Hitting the cylinder more accurately might make it spin for a bit longer, but it’s still pretty much the same motion.
At the opposite end of the spectrum would be a real violin. I play the violin, and you could describe it quite well as a machine for amplifying tiny changes in arm and hand movements into very different sounds (mostly horrible ones, which is why beginners sound so awful). There are a large number of degrees of freedom – the movements of the each jointed finger in three dimensional space, including those on the bow hand, contribute to the final sound. Also, almost all of them are continuous degrees of freedom. There are no keys or frets to accommodate small mistakes in positioning.
Crawford argues that although tools and instruments that transmit this kind of rich information about the world can be very frustrating in the short term, they also have enormous potential for satisfaction in the long term as you come to master them. Whereas objects like the Leap Frog Learning Table have comparatively little to offer if you’re not two years old:
Variations in how you hit the button on a Leap Frog Learning Table or a slot machine do not similarly produce variations in the effect you produce. There is a closed loop between your action and the effect that you perceive, but the bandwidth of variability has been collapsed… You are neither learning something about the world, as the blind man does with his cane, nor acquiring something that could properly be called a skill. Rather, you are acting within the perception-action circuits encoded in the narrow affordances of the game, learned in a few trials. This is a kind of autistic pseudo-action, based on exact repetition, and the feeling of efficacy that it offers evidently holds great appeal.
(As a warning, Crawford consistently uses ‘autistic’ in this derogatory sense throughout the book; if that sounds unpleasant, steer clear.)
Objects can also be actively deceptive, rather than just tediously simple. In the same chapter there’s some interesting material on gambling machines, and the tricks used to make them addictive. Apparently one of the big innovations here was the idea of ‘virtual reel mapping’. Old-style mechanical fruit machines would have three actual reels with images on them that you needed to match up, and just looking at the machine would give you a rough indication of the total number of images on the reel, and therefore the rough odds of matching them up and winning.
Early computerised machines followed this pattern, but soon the machine designers realised that there no longer needed to be this close coupling between the machine’s internal representation and what the gambler sees. So the newer machines would have a much larger number of virtual reel positions that are mostly mapped to losing combinations, with a large percentage of these combinations being ‘near misses’ to make the machine more addictive. The machine still looks simple, like the toddler’s toy, but the intuitive sense of odds you get from watching the machine becomes completely useless, because the internal logic of the machine is now doing something very complicated that the screen actively hides from you. A machine like this is actively ‘out to get you’, alienating you from the evidence of your own eyes.
Apples and sheep
Before reading the book I’d never really thought carefully about any of these properties of objects. For a while after reading it, I noticed them everywhere. Here’s one (kind of silly) example.
Shortly after reading the book I was visiting my family, and came across this wooden puzzle my aunt made:
I had a phase when I was ten or so where I was completely obsessed with this puzzle. Looking back, it’s not obvious why. It’s pretty simple and looks like the kind of thing that would briefly entertain much younger children. I was a weird kid and also didn’t have a PlayStation – maybe that’s explanation enough? But I didn’t have some kind of Victorian childhood where I was happy with an orange and a wooden toy at Christmas. I had access to plenty of plastic and electronic nineties tat that was more obviously fun.
I sat down for half an hour to play with this thing and try and remember what the appeal was. The main thing is that it turns out to be way more controllable than you might expect. The basic aim of the puzzle is just to get the ball bearings in the holes in any old order. This is the game that stops being particularly rewarding once you’re over the age of seven. But it’s actually possible to learn to isolate individual ball bearings by bashing them against the sides until one separates off, and then tilt the thing very precisely to steer one individually into a specific hole. That gives you a lot more options for variants on the basic game. For example, you can fill in the holes in a spiral pattern starting from the middle. Or construct a ‘fence’ of outer apples with a single missing ‘gate’ apple, steer two apples into the central pen (these apples are now sheep), and then close the gate with the last one.
The other interesting feature is that because this is a homemade game, the holes are not uniformly deep. The one in the top right is noticeably shallower than the others, and the ball bearing in this slot can be dislodged fairly easily while keeping the other nine in their place. This gives the potential for quite complicated dynamics of knocking down specific apples, and then steering other ones back in.
Still an odd way to have spent my time! But I can at least roughly understand why. The apple puzzle is less like the Leap Frog Learning Table than you might expect, and so the game can reward a surprisingly high level of skill. Part of this is from the continuous degrees of freedom you have in tilting the board, but the cool thing is that a lot of it comes from unintentional parts of the physical design. My aunt made the basic puzzle for small children, and the more complicated puzzles happened to be hidden within it.
The ability to dislodge the top right apple is not ‘supposed’ to be part of the game at all – an abstract version you might code up would have identical holes. But the world is going about its usual business of being incorrigibly plural, and there is just way more of it than any one abstract ruleset needs. The variation in the holes allows some of that complexity to accidentally leak in, breaking the simple game out into a much richer one.
Pebbles and birdsong
Now for the criticism part. I think there’s a real deficiency in the book that goes deeper than the tone issues I pointed out at the start. Crawford is insightful in his discussions of the kind of complexity that many handcrafted objects exhibit, that’s often standardised away in the modern world. But in his enthusiasm for people doing real things with real tools he’s forgotten the advantages of the systematised, ‘autistic’ world he dislikes. Funnelling messy reality into repeatable categories is how we get shit done at scale. It’s not just some unpleasant feature of modernity, either. Even something as simple as counting with pebbles relies on this:
To make the method work, you must choose bits-of-rock of roughly even sizes, so you can distinguish them from littler bits—stray grains of sand or dust in the bucket—that don’t count. How even? Even enough that you can make a reliable-enough judgement.
The counting procedure abstracts away the vivid specificity of the individual pebbles, and reduces them to simplistic interchangeable tokens. But there’s not much point in complaining about this. You need to do this to get the job done! And you can always break them back out into individuality later on if you want to do something else, like paint a still life of them.
I’m finding myself going back yet again to Christopher Norris’s talk on Derrida, which I discussed in my braindump here. (I’m going to repeat myself a bit in the next section. This was the most thought-provoking single thing I read last year, and I’m still working through the implications, so everything seems to lead back there at the moment.) Derrida picks apart some similar arguments by Rousseau, who was concerned with the bad side of systematisation in music:
One way of looking at Rousseau’s ideas about the melody/harmony dualism is to view them as the working-out of a tiff he was having with Rameau. Thus he says that the French music of his day is much too elaborate, ingenious, complex, ‘civilized’ in the bad (artificial) sense — it’s all clogged up with complicated contrapuntal lines, whereas the Italian music of the time is heartfelt, passionate, authentic, spontaneous, full of intense vocal gestures. It still has a singing line, it’s still intensely melodious, and it’s not yet encumbered with all those elaborate harmonies.
Crawford is advocating for something close to Rousseau’s pure romanticism. He brings along more recent and sophisticated arguments from phenomenology and embodied cognition, but he’s still very much on the side of spontaneity over structure. And I think he’s still vulnerable to the same arguments that Derrida was able to use against Rousseau. Norris explains it as follows:
… Rousseau gets into a real argumentative pickle when he say – lays it down as a matter of self-evident truth – that all music is human music. Bird-song just doesn’t count, he says, since it is merely an expression of animal need – of instinctual need entirely devoid of expressive or passional desire – and is hence not to be considered ‘musical’ in the proper sense of that term. Yet you would think that, given his preference for nature above culture, melody above harmony, authentic (spontaneous) above artificial (‘civilized’) modes of expression, and so forth, Rousseau should be compelled – by the logic of his own argument – to accord bird-song a privileged place vis-à-vis the decadent productions of human musical culture. However Rousseau just lays it down in a stipulative way that bird-song is not music and that only human beings are capable of producing music. And so it turns out, contrary to Rousseau’s express argumentative intent, that the supplement has somehow to be thought of as always already there at the origin, just as harmony is always already implicit in melody, and writing – or the possibility of writing – always already implicit in the nature of spoken language.
Derrida is pointing out that human music always has a structured component. We don’t just pour out a unmarked torrent of frequencies. We define repeatable lengths of notes, and precise intervals between pitches. (The evolution of these is a complicated engineering story in itself.) This doesn’t make music ‘inauthentic’ or ‘artificial’ in itself. It’s a necessary feature of anything we’d define as music.
I’d have been much happier with the book if it had some understanding of this interaction – ‘yes, structure is important, but I think we have too much of it, and here’s why’. But all we get is the romantic side. As with Rousseau’s romanticism, this tips over all too easily into pure reactionary nostalgia for an imagined golden age, and then we have to listen to yet another anecdote about how everything in the modern world is terrible. It’s not the eighteenth century any more, and we can do better now. And for all its genuine insight, this book mostly just doesn’t.
Last month I finally got round to reading The Eureka Factor by John Kounios and Mark Beeman, a popular book summarising research on ‘insightful’ thinking. I first mentioned it a couple of years ago after I’d read a short summary article, when I realised it was directly relevant to my recurring ‘two types of mathematician’ obsession:
The book is not focussed on maths – it’s a general interest book about problem solving and creativity in any domain. But it looks like it has a very similar way of splitting problem solvers into two groups, ‘insightfuls’ and ‘analysts’. ‘Analysts’ follow a linear, methodical approach to work through a problem step by step. Importantly, they also have cognitive access to those steps – if they’re asked what they did to solve the problem, they can reconstruct the argument.
‘Insightfuls’ have no such access to the way they solved the problem. Instead, a solution just ‘pops into their heads’.
Of course, nobody is really a pure ‘insightful’ or ‘analyst’. And most significant problems demand a mixed strategy. But it does seem like many people have a tendency towards one or the other.
I wasn’t too sure what I was getting into. The replication crisis has made me hyperaware of the dangers of uncritically accepting any results in psychology, and I’m way too ignorant of the field to have a good sense for which results still look plausible. However, the book turned out to be so extraordinarily Relevant To My Interests that I couldn’t resist writing up a review anyway.
The final chapters had a few examples along the lines of ‘[weak environmental effect] primes people to be more/less insightful’, and I know enough to stay away from those, but the earlier parts look somewhat more solid to me. I haven’t made much effort to trace back references, though, and I could easily still be being too credulous.
(I didn’t worry so much about replication with my previous post on the Cognitive Reflection Test. Getting the bat and ball question wrong is hardly the kind of weak effect that you need a sensitive statistical instrument to detect. It’s almost impossible to stop people getting it wrong! I did steer clear of any more dubious priming-style results, though, like the claim that people do better on the CRT when reading it ‘in a disfluent font’.)
Insight and intuition
First, it’s worth getting clear on exactly what Kounious and Beeman mean by ‘insight’. As they use it, insight is a specific type of creative thinking, which they define more generally as ‘the ability to reinterpret something by breaking it down into its elements and recombining these elements in a surprising way to achieve some goal.’ Insight is distinguished by its suddenness and lack of conscious control:
When this kind of creative recombination takes place in an instant, it’s an insight. But recombination can also result from the more gradual, conscious process that cognitive psychologists call “analytic” thought. This involves methodically and deliberately considering many possibilities until you find the solution. For example, when you’re playing a game of Scrabble, you must construct words from sets of letters. When you look at the set of letters “A-E-H-I-P-N-Y-P” and suddenly realize that they can form the word “EPIPHANY,” then that would be an insight. When you systematically try out different combinations of the letters until you find the word, that’s analysis.
Insights tend to have a few other features in common. Solving a problem by insight is normally very satisfying: the insight comes into consciousness along with a small jolt of positive affect. The insight itself is usually preceded by a longer period of more effortful thought about the problem. Sometimes this takes place just before the moment of insight, while at other times there is an ‘incubation’ phase, where the solution pops into your head while you’ve taken a break from deliberately thinking about it.
I’m not really going to get into this part in my review, but the related word ‘intuition’ is also used in an interestingly specific sense in the book, to describe the sense that a new idea is lurking beneath the surface, but is not consciously accessible yet. Intuitions often precede an insight, but have a different feel to the insight itself:
This puzzling phenomenon has a strange subjective quality. It feels like an idea is about to burst into your consciousness, almost as though you’re about to sneeze. Cognitive psychologists call this experience “intuition,” meaning an awareness of the presence of information in the unconscious mind — a new idea, solution, or perspective — without awareness of the information itself, at least until it pops into consciousness.
To study insight, psychologists need to come up with problems that reliably trigger an insight solution. One classic example discussed in The Eureka Factor is the Nine Dot Problem, where you are asked to connect the following set of black dots using only four lines, without taking your pen off the page:
If you’ve somehow avoided seeing this puzzle before, think about it for a while first. In the absence of any kind of built-in spoiler blocks for wordpress.com sites, I’ll insert a bunch of blank space here so that you hopefully have to scroll down off your current screen to see my discussion of the solution:
If you didn’t figure it out, a solution can be found in the Wikipedia article on insight problems here. It’ll probably look irritatingly obvious once you see it. The key feature of the solution is that the lines you draw have to extend outside the confines of the square of dots you start with (thus spawning a whole subgenre of annoying business literature on ‘thinking outside the box’). Nothing in the rules forbids this, but the setup focusses most people’s attention on the grid itself, and breaking out of this mindset requires a kind of reframing, a throwing away of artificially imposed constraints. This is a common characteristic of insight problems.
This characteristic also makes insight hard to test. For testing purposes, it’s useful to have a large stock of similar puzzles in hand. But a good reframing like the one in the Nine Dot Problem tends to be a bit of a one-off: once you’ve had the idea of extending the lines outside the box, it applies trivially to all similar puzzles, and not at all to other types of puzzle.
(I talked about something similar in my last post, on the Cognitive Reflection Test. The test was inspired by one good puzzle, the ‘bat and ball problem’, and adds two other questions that were apparently picked to be similar. Five thousand words and many comments later, it’s not obvious to me or most of the other commenters that these three problems form any kind of natural set at all.)
Kounios and Beeman discuss several of these eyecatching ‘one-off’ problems in the book, but their own research that they discuss is focussed on a more standardisable kind of puzzle, the Remote Associates Test. This test gives you three words, such as
and asks you to find the common word that links them. The authors claim that these can be solved either with or without insight, and asked participants to self-categorise their responses as either fitting in the ‘insightful’ or ‘analytic’ categories:
The analytic approach is to consciously search through the possibilities and try out potential answers. For example, start with “pine.” Imagine yourself thinking: What goes with “pine”? Perhaps “tree”? “Pine tree” works. “Crab tree”? Hmmm … maybe. “Tree sauce”? No. Have to try something else. How about “cake”? “Crab cake” works. “Cake sauce” is a bit of a reach but might be acceptable. However, “pine cake” and “cake pine” definitely don’t work. What else? How about “crabgrass”? That works. But “pine grass”? Not sure. Perhaps there is such a thing. But “sauce grass” and “grass sauce” are definitely out. What else goes with “sauce”? How about “applesauce”? That’s good. “Pineapple” and “crab apple” also work. The answer is “apple”!
This is analytical thought: a deliberate, methodical, conscious search through the possible combinations. But this isn’t the only way to come up with the solution. Perhaps you’re trying out possibilities and get stuck or even draw a blank. And then, “Aha! Apple” suddenly pops into your awareness. That’s what would happen if you solved the problem by insight. The solution just occurs to you and doesn’t seem to be a direct product of your ongoing stream of thought.
This categorisation seems suspiciously neat, and if I rely on my own introspection for solving one of these (which is obviously dubious itself) it feels like more of a mix. I’ll often generate some verbal noise about cakes and trees that sounds vaguely like I’m doing something systematic, but the main business of solving the thing seems to be going on nonverbally elsewhere. But I do think there’s something there – the answer can be very immediate and ‘poppy’, or it can surface after a longer and more accessible process of trying plausible words. This was tested in a more objective way by seeing what people do when they don’t come up with the answer:
Insightfuls made more “errors of omission.” When waiting for an insight that hadn’t yet arrived, they had nothing to offer in its place. So when the insight didn’t arrive in time, they let the clock run out without having made a guess. In contrast, Analysts made more “errors of commission.” They rarely timed out, but instead guessed – sometimes correctly – by offering the potential solution they had been consciously thinking about when their time was almost up.
Kounios and Beeman’s research focussed on finding neural correlates of the ‘aha’ moment of insight, using a combination of an EEG test to pinpoint the time of the insight, and fMRI scanning to locate the brain region:
We found that at the moment a solution pops into someone’s awareness as an insight, a sudden burst of high-frequency EEG activity known as “gamma waves” can be picked up by electrodes just above the right ear. (Gamma waves represent cognitive processing in the brain, such as paying attention to something or linking together different pieces of information.) We were amazed at the abruptness of this burst of activity—just what one would expect from a sudden insight. Functional magnetic resonance imaging showed a corresponding increase in blood flow under these electrodes in a part of the brain’s right temporal lobe called the “anterior superior temporal gyrus” (see figure 5.2), an area that is involved in making connections between distantly related ideas, as in jokes and metaphors. This activity was absent for analytic solutions.
So we had found a neural signature of the aha moment: a burst of activity in the brain’s right hemisphere.
I’m not sure how settled this is, though. I haven’t tried to do a proper search of the literature, but certainly a review from 2010 describes the situation as very much in flux:
A recent surge of interest into the neural underpinnings of creative behavior has produced a banquet of data that is tantalizing but, considered as a whole, deeply self-contradictory.
(The book was published somewhat later, in 2015, but mostly cites research from prior to this review, such as this paper.)
As an outsider it’s going to be pretty hard for me to judge this without spending a lot more time than I really want to right now. However, regardless of how this holds up, I was really interested in the authors’ discussion of why a right-hemisphere neural correlate of insight would make sense.
Insight and context
One of the authors, Mark Beeman, had previously studied language deficits in people who had suffered brain damage to the right hemisphere. One such patient was the trial attorney D.B.:
What made D.B. “lucky” was that the stroke had damaged his right hemisphere rather than his left. Had the stroke occurred in the mirror-image left-hemisphere region, he would have experienced Wernicke’s aphasia, a profound deficit of language comprehension. In the worst cases, people with Wernicke’s aphasia may be completely unable to understand written or spoken language.
Nevertheless, D.B. didn’t feel lucky. He may have been better off than if he’d had a left-hemisphere stroke, but he felt that his language ability was far from normal. He said that he “couldn’t keep up” with conversations or stories the way he used to. He felt impaired enough that he had stopped litigating trials—he thought that it would have been a disservice to his clients to continue to represent them in court.
D.B. and the other patients were able to understand the straightforward meanings of words and the literal meanings of sentences. Even so, they complained about vague difficulties with language. They failed to grasp the gist of stories or were unable to follow multiple-character or multiple-plot stories, movies, or television shows. Many didn’t get jokes. Sarcasm and irony left them blank with incomprehension. They could sometimes muddle along without these abilities, but whenever things became subtle or implicit, they were lost.
An example of the kind of problem D.B. struggled with is the following:
Saturday, Joan went to the park by the lake. She was walking barefoot in the shallow water, not knowing that there was glass nearby. Suddenly, she grabbed her foot in pain and called for help, and the lifeguard came running.
If D.B. was given a statement about something that occurred explicitly in the text, such as ‘Joan went to the park on Saturday’, he could say whether it was true or false with no problems at all. In fact, he did better than all of the control subjects on these sorts of explicit questions. But if he was instead presented with a statement like ‘Joan cut her foot’, where some of the facts are left implicit, he was unable to answer.
This was interesting to me, because it seems so directly relevant to the discussionlast year on ‘cognitive decoupling’. This is a term I’d picked up from Sarah Constantin, who herself got it from Keith Stanovich:
Stanovich talks about “cognitive decoupling”, the ability to block out context and experiential knowledge and just follow formal rules, as a main component of both performance on intelligence tests and performance on the cognitive bias tests that correlate with intelligence. Cognitive decoupling is the opposite of holistic thinking. It’s the ability to separate, to view things in the abstract, to play devil’s advocate.
The patients in Beeman’s study have so much difficulty with contextualisation that they struggle with anything at all that is left implicit, even straightforward inferences like ‘Joan cut her foot’. This appears to match with other evidence from visual half-field studies, where subjects are presented with words on either the right or left half of the visual field. Those on the left half will go first to the right hemisphere, so that the right hemisphere gets a head start on interpreting the stimulus. This shows a similar difference between hemispheres:
The left hemisphere is sharp, focused, and discriminating. When a word is presented to the left hemisphere, the meaning of that word is activated along with the meanings of a few closely related words. For example, when the word “table” is presented to the left hemisphere, this might strongly energize the concepts “chair” and “kitchen,” the usual suspects, so to speak. In contrast, the right hemisphere is broad, fuzzy, and promiscuously inclusive. When “table” is presented to the right hemisphere, a larger number of remotely related words are weakly invoked. For example, “table” might activate distant associations such as “water” (for underground water table), “payment” (for paying under the table), “number” (for a table of numbers), and so forth.
Why would picking up on these weak associations be relevant to insight? The story seems to be that this tangle of secondary meanings – the ‘Lovecraftian penumbra of monstrous shadow phalanges’ – works to pull your attention away from the obvious interpretation you’re stuck with, helping you to find a clever new reframing of the problem.
This makes a lot of sense to me as a rough outline. In my own experience at least, the kind of thinking that is likely to lead to an insight experience feels softer and more diffuse than the more ‘analytic’ kind, more a process of sort of rolling the ideas around gently in your head and seeing if something clicks than a really focussed investigation of the problem. ‘Thinking too hard’ tends to break the spell. This fits well with the idea that insights are triggered by activation of weak associations.
There’s a lot of other interesting material in the book about the rest of the insight process, including the incubation period leading up to an insight flash, and the phenomenon of ‘intuitions’, where you feel that an insight is on its way but you don’t know what it is yet. I’ll never get through this review if I try to cover all of that, so instead I’m going to finish up with a couple of weak associations of my own that got activated while reading the book.
I’ve been getting increasingly dissatisfied with the way dual process theories split cognition into a fast/automatic/intuitive ‘System 1’ and a slow/effortful/systematic ‘System 2’. System 1 in particular has started to look to me like an amorphous grab bag of all kinds of things that would be better separated out.
The Eureka Factor has pushed this a little further, by bringing out a distinction between two things that normally get lumped under System 1 but are actually very different. One obvious type of System 1-ish behaviour is routine action, the way you go about tasks you have done many times before, like making a sandwich or walking to work. These kinds of activities require very little explicit thought and generally ‘just happen’ in response to cues in the environment.
The kind of ‘insightful’ thinking discussed in The Eureka Factor would also normally get classed under System 1: it’s not very systematic and involves a fast, opaque process where the answer just pops into your head without much explanation. But it’s also very different to routine action. It involves deliberately choosing to think about a new situation, rather than one you have seen many times before, and a successful insight gives you a qualitatively new kind of understanding. The insight flash itself is a very noticeable, enjoyable feature of your conscious attention, rather than the effortless, unexamined state of absorbed action.
You seem to be lumping “flashes of insight” in with “effortless flow-state”. I don’t think they’re the same. For one thing, inspiration generally comes in bursts, while flow-states can persist for a while (driving on a highway, playing the piano, etc.) Definitely, “flashes of insight” aren’t the same type of thought as “effortful attention” — insight feels easy, instant, and unforced. But they might be their own, unique category of thought. Still working out my ontology here.
I’d sort of had this at the back of my head since then, but the book has really brought out the distinction clearly. I’m sure these aren’t the only types of thinking getting shoved into the System 1 category, and I get the sense that there’s a lot more splitting out that I need to do.
I also thought about how the results in the book fit in with my perennial ‘two types of mathematician’ question. (This is a weird phenomenon I’ve noticed where a lot of mathematicians have written essays about how mathematicians can be divided into two groups; I’ve assembled a list of examples here.) ‘Analytic’ versus ‘insightful’ seems to be one of the distinctions between groups, at least. It seems relevant to Poincaré’s version, for instance:
The one sort are above all preoccupied with logic; to read their works, one is tempted to believe they have advanced only step by step, after the manner of a Vauban who pushes on his trenches against the place besieged, leaving nothing to chance.
The other sort are guided by intuition and at the first stroke make quick but sometimes precarious conquests, like bold cavalrymen of the advance guard.
Just at this time, I left Caen, where I was living, to go on a geologic excursion under the auspices of the School of Mines. The incidents of the travel made me forget my mathematical work. Having reached Coutances, we entered an omnibus to go some place or other. At the moment when I put my foot on the step, the idea came to me, without anything in my former thoughts seeming to have paved the way for it, that the transformations I had used to define the Fuchsian functions were identical with those of non-Euclidean geometry. I did not verify the idea; I should not have had time, as, upon taking my seat in the omnibus, I went on with a conversation already commenced, but I felt a perfect certainty. On my return to Caen, for conscience’ sake, I verified the result at my leisure.
If the insight/analysis split is going to be relevant here, it would require that people favour either ‘analytic’ or ‘insight’ solutions as a general cognitive style, rather than switching between them freely depending on the problem. The authors do indeed claim that this is the case:
Most people can, and to some extent do, use both of these approaches. A pure type probably doesn’t exist; each person falls somewhere on an analytic-insightful continuum. Yet many—perhaps most—people tend to gravitate toward one of these styles, finding their particular approach to be more comfortable or natural.
This is based on their own research where they recorded participant’s self-report of whether they were using a ‘insight’ or ‘analytic’ approach to solve anagrams, and compared it with EEG recordings of their resting state. They found a number of differences, including more right-hemisphere activity in the ‘insight’ group, and lower levels of communication between the frontal lobe and other parts of the brain, indicating a more disorderly thinking style with less top-down control. This may suggest more freedom to allow weak associations between thoughts to have a crack at the problem, without being overruled by the dominant interpretation.
Again, and you’re probably got very bored of this disclaimer, I have no idea how well the details of this will hold up. That’s true for pretty much every specific detail in the book that I’ve discussed here. Still, the link between insight and weak associations makes a lot of sense to me, and the overall picture certainly triggered some useful reframings. That seems very appropriate for a book about insight.
Almost everyone we ask reports an initial tendency to answer “10 cents” because the sum $1.10 separates naturally into $1 and 10 cents, and 10 cents is about the right magnitude. Many people yield to this immediate impulse. The surprisingly high rate of errors in this easy problem illustrates how lightly System 2 monitors the output of System 1: people are not accustomed to thinking hard, and are often content to trust a plausible judgment that quickly comes to mind.
In Thinking, Fast and Slow, the bat and ball problem is used as an introduction to the major theme of the book: the distinction between fluent, spontaneous, fast ‘System 1’ mental processes, and effortful, reflective and slow ‘System 2’ ones. The explicit moral is that we are too willing to lean on System 1, and this gets us into trouble:
The bat-and-ball problem is our first encounter with an observation that will be a recurrent theme of this book: many people are overconfident, prone to place too much faith in their intuitions. They apparently find cognitive effort at least mildly unpleasant and avoid it as much as possible.
This story is very compelling in the case of the bat and ball problem. I got this problem wrong myself when I first saw it, and still find the intuitive-but-wrong answer very plausible looking. I have to consciously remind myself to apply some extra effort and get the correct answer.
However, this becomes more complicated when you start considering other tests of this fast-vs-slow distinction. Frederick later combined the bat and ball problem with two other questions to create the Cognitive Reflection Test:
(2) If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets? _____ minutes
(3) In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake? _____ days
These are designed to also have an ‘intuitive-but-wrong’ answer (100 minutes, 24 days), and an ‘effortful-but-right’ answer (5 minutes, 47 days). But this time I seem to be immune to the wrong answers, in a way that just doesn’t happen with the bat and ball:
I always have the same reaction, and I don’t know if it’s common or I’m just the lone idiot with this problem. The ‘obvious wrong answers’ for 2. and 3. are completely unappealing to me (I had to look up 3. to check what the obvious answer was supposed to be). Obviously the machine-widget ratio hasn’t changed, and obviously exponential growth works like exponential growth.
When I see 1., however, I always think ‘oh it’s that bastard bat and ball question again, I know the correct answer but cannot see it’. And I have to stare at it for a minute or so to work it out, slowed down dramatically by the fact that Obvious Wrong Answer is jumping up and down trying to distract me.
If this test was really testing my propensity for effortful thought over spontaneous intuition, I ought to score zero. I hate effortful thought! As it is, I score two out of three, because I’ve trained my intuitions nicely for ratios and exponential growth. The ‘intuitive’, ‘System 1’ answer that pops into my head is, in fact, the correct answer, and the supposedly ‘intuitive-but-wrong’ answers feel bad on a visceral level. (Why the hell would the lily pads take the same amount of time to cover the second half of the lake as the first half, when the rate of growth is increasing?)
The bat and ball still gets me, though. My gut hasn’t internalised anything useful, and it’s super keen on shouting out the wrong answer in a distracting way. My dislike for effortful thought is definitely a problem here.
I wanted to see if others had raised the same objection, so I started doing some research into the CRT. In the process I discovered a lot of follow-up work that makes the story much more complex and interesting.
I’ve come nowhere near to doing a proper literature review. Frederick’s original paper has been cited nearly 3000 times, and dredging through that for the good bits is a lot more work than I’m willing to put in. This is just a summary of the interesting stuff I found on my limited, partial dig through the literature.
Thinking, inherently fast and inherently slow
Frederick’s original Cognitive Reflection Test paper describes the System 1/System 2 divide in the following way:
Recognizing that the face of the person entering the classroom belongs to your math teacher involves System 1 processes — it occurs instantly and effortlessly and is unaffected by intellect, alertness, motivation or the difficulty of the math problem being attempted at the time. Conversely, finding to two decimal places without a calculator involves System 2 processes — mental operations requiring effort, motivation, concentration, and the execution of learned rules.
I find it interesting that he frames mental processes as being inherently effortless or effortful, independent of the person doing the thinking. This is not quite true even for the examples he gives — faceblind people and calculating prodigies exist.
This framing is important for interpreting the CRT. If the problem inherently has a wrong ‘System 1 solution’ and a correct ‘System 2 solution’, the CRT can work as intended, as an efficient tool to split people by their propensity to use one strategy or the other. If there are ‘System 1’ ways to get the correct answer, the whole thing gets much more muddled, and it’s hard to disentangle natural propensity to reflection from prior exposure to the right mathematical concepts.
My tentative guess is that the bat and ball problem is close to being this kind of efficient tool. Although in some ways it’s the simplest of the three problems, solving it in a ‘fast’, ‘intuitive’ way relies on seeing the problem in a way that most people’s education won’t have provided. (I think this is true, anyway – I’ll go into more detail later.) I suspect that this is less true the other two problems – ratios and exponential growth are topics that a mathematical or scientific education is more likely to build intuition for.
(Aside: I’d like to know how these other two problems were chosen. The paper just states the following:
Motivated by this result [the answers to the bat and ball question], two other problems found to yield impulsive erroneous responses were included with the “bat and ball” problem to form a simple, three-item “Cognitive Reflection Test” (CRT), shown in Figure 1.
I have a vague suspicion that Frederick trawled through something like ‘The Bumper Book of Annoying Riddles’ to find some brainteasers that don’t require too much in the way of mathematical prerequisites. The lilypads one has a family resemblance to the classic grains-of-wheat-on-a-chessboard puzzle, for instance.)
However, I haven’t found any great evidence either way for this guess. The original paper doesn’t break down participants’ scores by question – it just gives mean scores on the test as a whole. I did however find this meta-analysis of 118 CRT studies, which shows that the bat and ball question is the most difficult on average – only 32% of all participants get it right, compared with 40% for the widgets and 48% for the lilypads. It also has the biggest jump in success rate when comparing university students with non-students. That looks like better mathematical education does help on the bat and ball, but it doesn’t clear up how it helps. It could improve participants’ ability to intuitively see the answer. Or it could improve ability to come up with an ‘unintuitive’ solution, like solving the corresponding simultaneous equations by a rote method.
What I’d really like is some insight into what individual people actually do when they try to solve the problems, rather than just this aggregate statistical information. I haven’t found exactly what I wanted, but I did turn up a few interesting studies on the way.
No, seriously, the answer isn’t ten cents
My favourite thing I found was this (apparently unpublished) ‘extremely rough draft’ by Meyer, Spunt and Frederick from 2013, revisiting the bat and ball problem. The intuitive-but-wrong answer turns out to be extremely sticky, and the paper is basically a series of increasingly desperate attempts to get people to actually think about the question.
One conjecture for what people are doing when they get this question wrong is the attribute substitution hypothesis. This was suggested early on by Kahneman and Frederick, and is a fancy way of saying that they are instead solving the following simpler problem:
(1) A bat and a ball cost $1.10 in total. The bat costs $1.00.
How much does the ball cost? _____ cents
Notice that this is missing the ‘more than the ball’ clause at the end, turning the question into a much simpler arithmetic problem. This simple problem does have ‘ten cents’ as the answer, so it’s very plausible that people are getting confused by it.
Meyer, Spunt and Frederick tested this hypothesis by getting respondents to recall the problem from memory. This showed a clear difference: 94% of ‘five cent’ respondents could recall the correct question, but only 61% of ‘ten cent’ respondents. It’s possible that there is a different common cause of both the ‘ten cent’ response and misremembering the question, but it at least gives some support for the substitution hypothesis.
However, getting people to actually answer the question correctly was a much more difficult problem. First they tried bolding the words more than the ball to make this clause more salient. This made surprisingly little impact: 29% of respondents solved it, compared with 24% for the original problem. Printing both versions was slightly more successful, bumping up the correct response to 35%, but it was still a small effect.
After this, they ditched subtlety and resorted to pasting these huge warnings above the question:
These were still only mildly effective, with a correct solution jumping to 50% from 45%. People just really like the answer ‘ten cents’, it seems.
At this point they completely gave up and just flat out added “HINT: 10 cents is not the answer.” This worked reasonably well, though there was still a hard core of 13% who persisted in writing down ‘ten cents’.
That’s where they left it. At this point there’s not really any room to escalate beyond confiscating the respondents’ pens and prefilling in the answer ‘five cents’, and I worry that somebody would still try and scratch in ‘ten cents’ in their own blood. The wrong answer is just incredibly compelling.
So, what are people doing when they solve this problem?
Unfortunately, it’s hard to tell from the published literature (or at least what I found of it). What I’d really like is lots of transcripts of individuals talking through their problem solving process. The closest I found was this paper by Szaszi et al, who did carry out these sort of interview, but it doesn’t include any examples of individual responses. Instead, it gives a aggregated overview of types of responses, which doesn’t go into the kind of detail I’d like.
Still, the examples given for their response categories give a few clues. The categories are:
Correct answer, correct start. Example given: ‘I see. This is an equation. Thus if the ball equals to x, the bat equals to x plus 1… ‘
Correct answer, incorrect start. Example: ‘I would say 10 cents… But this cannot be true as it does not sum up to €1.10…’
Incorrect answer, reflective, i.e. some effort was made to reconsider the answer given, even if it was ultimately incorrect. Example: ‘… but I’m not sure… If together they cost €1.10, and the bat costs €1 more than the ball… the solution should be 10 cents. I’m done.’
No reflection. Example: ‘Ok. I’m done.’
These demonstrate one way to reason your way to the correct answer (solve the simultaneous equations) and one way to be wrong (just blurt out the answer). They also demonstrate one way to recover from an incorrect solution (think about the answer you blurted out and see if it actually works). Still, it’s all rather abstract and high level.
How To Solve It
However, I did manage to stumble onto another source of insight. While researching the problem I came across this article from the online magazine of the Association for Psychological Science, which discusses a variant ‘Ford and Ferrari problem’. This is quite interesting in itself, but I was most excited by the comments section. Finally some examples of how the problem is solved in the wild!
The simplest ‘analytical’, ‘System 2’ solution is to rewrite the problem as two simultaneous linear equations and plug-and-chug your way to the correct answer. For example, writing for the bat and for the ball, we get the two equations
which we could then solve in various standard ways, e.g.
which then gives
There are a couple of variants of this explained in the comments. It’s a very reliable way to tackle the problem: if you already know how to do this sort of rote method, there are no surprises. This sort of method would work for any similar problem involving linear equations.
However, it’s pretty obvious that a lot of people won’t have access to this method. Plenty of people noped out of mathematics long before they got to simultaneous equations, so they won’t be able to solve it this way. What might be less obvious, at least if you mostly live in a high-maths-ability bubble, is that these people may also be missing the sort of tacit mathematical background that would even allow them to frame the problem in a useful form in the first place.
That sounds a bit abstract, so let’s look at some responses (I’ll paste all these straight in, so any typos are in the original). First, we have these two confused commenters:
The thing is, why does the ball have to be $.05? It could have been .04 0r.03 and the bat would still cost more than $1.
This is exactly what bothers me and resulted in me wanting to look up the question online. On the quiz the other 2 questions were definitive. This one technically could have more than one answer so this is where phycologists actually mess up when trying to give us a trick question. The ball at .4 and the bat at 1.06 doesn’t break the rule either.
These commenters don’t automatically see two equations in two variables that together are enough to constrain the problem. Instead they seem to focus mainly on the first condition (adding up to $1.10) and just use the second one as a vague check at best (‘the bat would still cost more than $1’). This means that they are unable to immediately tell that the problem has a unique solution.
In response, another commenter, Tony, suggests a correct solution which is an interesting mix of writing the problem out formally and then figuring out the answer by trial and error:
I hear your pain. I feel as though psychologists and psychiatrists get together every now and then to prove how stoopid I am. However, after more than a little head scratching I’ve gained an understanding of this puzzle. It can be expressed as two facts and a question A=100+B and A+B=110, so B=? If B=2 then the solution would be 100+2+2 and A+B would be 104. If B=6 then the solution would be 100+6+6 and A+B would be 112. But as be KNOW A+B=110 the only number for B on it’s own is 5.
This suggests enough half-remembered mathematical knowledge to find a sensible abstract framing, but not enough to solve it the standard way.
Finally, commenter Marlo Eugene provides an ingenious way of solving the problem without writing all the algebraic steps out:
Linguistics makes all the difference. The conceptual emphasis seems to lie within the word MORE.
X + Y = $1.10. If X = $1 MORE then that leaves $0.10 TO WORK WITH rather than automatically assign to Y
So you divide the remainder equally (assuming negative values are disqualified) and get 0.05.
So even this small sample of comments suggests a wide diversity of problem-solving methods leading to the two common answers. Further, these solutions don’t all split neatly into ‘System 1’ ‘intuitive’ and ‘System 2’ ‘analytic’. Marlo Eugene’s solution, for instance, is a mixed solution of writing the equations down in a formal way, but then finding a clever way of just seeing the answer rather than solving them by rote.
I’d still appreciate more detailed transcripts, including the time taken to solve the problem. My suspicion is still that very few people solve this problem with a fast intuitive response, in the way that I rapidly see the correct answer to the lilypad question. Even the more ‘intuitive’ responses, like Marlo Eugene’s, seem to rely on some initial careful reflection and a good initial framing of the problem.
If I’m correct about this lack of fast responses, my tentative guess for the reason is that it has something to do with the way most of us learn simultaneous equations in school. We generally learn arithmetic as young children in a fairly concrete way, with the formal numerical problems supplemented with lots of specific examples of adding up apples and bananas and so forth.
But then, for some reason, this goes completely out of the window once the unknown quantity isn’t sitting on its own on one side of the equals sign. This is instead hived off into its own separate subject, called ‘algebra’, and the rules are taught much later in a much more formalised style, without much attempt to build up intuition first.
(One exception is the sort of puzzle sheets that are often given to young kids, where the unknowns are just empty boxes to be filled in. Sometimes you get 2+3=□, sometimes it’s 2+□=5, but either way you go about the same process of using your wits to figure out the answer. Then, for some reason I’ll never understand, the worksheets get put away and the poor kids don’t see the subject again until years later, when the box is now called for some reason and you have to find the answer by defined rules. Anyway, this is a separate rant.)
This lack of a rich background in puzzling out the answer to specific concrete problems means most of us lean hard on formal rules in this domain, even if we’re relatively mathematically sophisticated. Only a few build up the necessary repertoire of tricks to solve the problem quickly by insight. I’m reminded of a story in Feynman’s The Pleasure of Finding Things Out:
Around that time my cousin, who was three years older, was in high school. He was having considerable difficulty with his algebra, so a tutor would come. I was allowed to sit in a corner while the tutor would try to teach my cousin algebra. I’d hear him talking about x.
I said to my cousin, “What are you trying to do?”
“I’m trying to find out what x is, like in 2x + 7 = 15.”
I say, “You mean 4.”
“Yeah, but you did it by arithmetic. You have to do it by algebra.”
I learned algebra, fortunately, not by going to school, but by finding my aunt’s old schoolbook in the attic, and understanding that the whole idea was to find out what x is – it doesn’t make any difference how you do it.
I think this reliance on formal methods might be somewhat less true for exponential growth and ratios, the subjects underpinning the lilypad and widget questions. Certainly I seem to have better intuition there, without having to resort to rote calculation. But I’m not sure how general this is.
How To Visualise It
If you wanted to solve the bat and ball problem without having to ‘do it by algebra’, how would you go about it?
My original post on the problem was a pretty quick, throwaway job, but over time it picked up some truly excellent comments by anders and Kyzentun, which really start to dig into the structure of the problem and suggest ways to ‘just see’ the answer. The thread with anders in particular goes into lots of other examples of how we think through solving various problems, and is well worth reading in full. I’ll only summarise the bat-and-ball-related parts of the comments here.
We all used some variant of the method suggested by Marlo Eugene in the comments above. Writing out the basic problem again, we have:
Now, instead of immediately jumping to the standard method of eliminating one of the variables, we can just look at what these two equations are saying and solve it directly ‘by thinking’. We have a bat, . If you add the price of the ball, , you get 110 cents. If you instead remove the same quantity you get 100 cents. So the bat’s price must be exactly halfway between these two numbers, at 105 cents. That leaves five for the ball.
Now that I’m thinking of the problem in this way, I directly see the equations as being ‘about a bat that’s halfway between 100 and 110 cents’, and the answer is incredibly obvious.
Kyzentun suggests a variant on the problem that is much less counterintuitive than the original:
A centered piece of text and its margins are 110 columns wide. The text is 100 columns wide. How wide is one margin?
Same numbers, same mathematical formula to reach the solution. But less misleading because you know there are two margins, and thus know to divide by two after subtracting.
In the original problem, the 110 units and 100 units both refer to something abstract, the sum and difference of the bat and ball. In Kyzentun’s version these become much more concrete objects, the width of the text and the total width of the margins. The work of seeing the equations as relating to something concrete has mostly been done for you.
Similarly, anders works the problem by ‘getting rid of the 100 cents’, and splitting the remainder in half to get at the price of the ball:
I just had an easy time with #1 which I haven’t before. What I did was take away the difference so that all the items are the same (subtract 100), evenly divide the remainder among the items (divide 10 by 2) and then add the residuals back on to get 105 and 5.
The heuristic I seem to be using is to treat objects as made up of a value plus a residual. So when they gave me the residual my next thought was “now all the objects are the same, so whatever I do to one I do to all of them”.
I think that after reasoning my way through all these perspectives, I’m finally at the point where I have a quick, ‘intuitive’ understanding of the problem. But it’s surprising how much work it was for such a simple bit of algebra.
Rather than making any big conclusions, the main thing I wanted to demonstrate in this post is how complicated the story gets when you look at one problem in detail. I’ve written about close reading recently, and this has been something like a close reading of the bat and ball problem.
Frederick’s original paper on the Cognitive Reflection Test is in that generic social science style where you define a new metric and then see how it correlates with a bunch of other macroscale factors (either big social categories like gender or education level, or the results of other statistical tests that try to measure factors like time preference or risk preference). There’s a strange indifference to the details of the test itself – at no point does he discuss why he picked those specific three questions, and there’s no attempt to model what was making the intuitive-but-wrong answer appealing.
The later paper by Meyer, Spunt and Frederick is much more interesting to me, because it really starts to pick apart the specifics of the bat and ball problem. Is an easier question getting substituted? Can participants reproduce the correct question from memory?
I learned the most from the individual responses, though, and seeing the variety of ways people go about solving the problem. It’s very strange to me that I had an easier time digging this out from an internet comment thread than the published literature! I would love to see a lot more research into what people actually do when they do mathematics, and the bat and ball problem would be a great place to start.
I’m interested in any comments on the post, but here are a few specific things I’d like to get your answers to:
My rapid, intuitive answer for the bat and ball question is wrong (at least until I retrained it by thinking about the problem way too much). However, for the other two I ‘just see’ the correct answer. Is this common for other people, or do you have a different split?
If you’re able to rapidly ‘just see’ the answer to the bat and ball question, how do you do it?
How do people go about designing tests like these? This isn’t at all my field and I’d be interested in any good sources. I’d kind of assumed that there’d be some kind of serious-business Test Creation Methodology, but for the CRT at least it looks like people just noticed they got surprising answers for the bat and ball question and looked around for similar questions. Is that unusual compared to other psychological tests?
[I’ve cross-posted this at LessWrong, because I thought the topic fits quite nicely – comments at either place are welcome.]