The Image Fulgurator

Many of you will have probably seen this (e.g. on Wired), but just in case you haven't, Rod at Reasonable Deviations has a discussion of Julius von Bismarck's device, which looks a bit like a camera with a speedgun mounted in the back of it. Camera flashes when other people take pictures set off its own flash unit, which fires through the image on film in the camera, projecting it onto whatever the camera is focused on... like a kind of instantaneous graffiti - it's too brief to be seen by eye, but it shows up in the shot.

I'm not sure what to think of it (aside from, "Damn, that's clever"), but I can see the potential for a lot of shenannigans of various kinds.

Examples of the genre

In my previous post I said:

Unfortunately, based only on the information that the third central moment was zero, many people would in fact describe it as symmetric.

I've seen many examples over the years, so I figured it shouldn't be hard to find one or two. A quick google search reveals some examples:

• This page on using a TI83 calculator has:
  • "Skewness measures the departure from symmetry" (it defines skewness as the third moment measure I mentioned, so it is saying that 0 third moment implies symmetry)
  • it goes on to suggest a test statistic for symmetry based on this, and concludes that discussion with "If that fraction is between −2 and 2, you can’t say whether the population is symmetric (skewness = 0) or skewed."
• This paper on brain evolution has:
  • "SK, subclade skewness (- negative skew; 0, symmetric distribution; + positive skew)"
  • "the system is probably passive if average subclade skew is neutral (symmetric distribution) or negative"
    
• This set of notes for a university* subject called "Introduction to Statistics" has the following complete howlers:
  
  • "If the skewness is approximately zero, the histogram (distribution) for the data is symmetric and usually normal"
  • "'varB' has a skewness close to zero so that its distribution should be normal and mean and median should be similar."
    

*I won't name the institution here. Let's just say you've heard of it.
That last site approaches farce, and it's trying to teach statistics!. This is often what happens when people whose own area is not statistics get put in charge of teaching it. (For some reason mathematicians are among the worst offenders.)

There were more examples. The above ones are pretty standard.

Not fooling ourselves (I) - the unmeasuring of asymmetry

In order to impose some kind of structure on our understanding of complex phenomena, we tend to use simple terms to describe what may be surprisingly nonsimple.

In order to make progress, we often attempt to quantify those descriptives. This is not just useful, it's often unavoidable, but the act carries with it a special danger, because we then (almost universally) invest the nonunique quantification of the concept with a "reality" that it doesn't merit - and that can lead to nonsense.

[It may be this is another version of the common phenomenon of believing mathematical models when at the beginning we knew them to be at best a rough approximation. Models tend to take on a life of their own, and their conclusions are often treated with a respect we did not accord the original model when it was first tentatively adopted. We need to step back and remember the model was never exactly the thing it was used to describe.]

I think this may be the flip side of what Blake Stacey was talking about when he was discussing the confusions that come in when an inherently mathematical concept is translated into a nonmathematical description.

Let me take a concrete example with which I am familiar. It relates to descriptions of probability distributions.

We begin with something that is inherently mathematical - symmetry. Symmetry is an extraordinarily useful, almost universal concept in mathematics. For what I'll be talking about you can just use the more common senses of reflection symmetry and rotational symmetry.

In the case of symmetry of distributions, there are several ways to define it (if you have a continuous or a discrete random variable, you can define it in the usual "mirror symmetry" sense), but the more general definition would have something like "m is the centre of symmetry if Prob(X≤ m-a) = Prob(X≥ m+a) for all a" (which effectively corresponds to rotational symmetry of the distribution function about the points (m, 1/2), if you tidy up details the right way).

Anyway, "symmetry" is both easily understood and fairly easy to pin down. Of course, almost all distributions are not symmetric (but symmetry arises in a natural way in particular circumstances, so it's far from a useless concept).

Aside from "unimodal" (one "hump"), or the overused and badly abused "bell-shaped", one of the most common descriptions of a distributional shape that is applied would probably be "skewed". An elementary book might have a diagram like the top one below (see for example, the diagram at Wikipedia's entry on skewness). The corresponding distribution function is underneath.

Diagram of a right (positive) skewed density and distribution function.

If you flipped the above density left-to-right (or rotated the distribution about the median), it would have left (negative) skewness.

Notice that "right skewed" means the long tail is to the right (this is often the opposite of a beginner's intuition about what the term should mean).

The problem comes when this seemingly clear but actually vague notion is quantified. There are numerous quantities that have been called "skewness". By far the most popular is the standardized third moment (or, equivalently, the standardized third cumulant) - so much so that it is frequently called "the" skewness. Equivalent sample statistics are used for samples.

Now for a distribution like the one above, this quantity is positive (when it exists). If you flip that density shape left to right, the skewness measure is negative. Importantly, when the density is symmetric and the first three moments exist, the skewness is 0. So far so good - left and right skewness and symmetry in pictures generally correspond to negative, positive and zero quantities on the measurement.

However, the problem comes when interpreting a standardized third moment back in terms of the density.

A positive number will lead people to call the distribution "right skew" without looking at it. A number near zero will cause people to call the distribution "symmetric". In the first case, the distribution will be asymmetric but it may not appear to be skewed with a tail to the right. And the value can be exactly zero without the distribution being symmetric. While symmetry implies zero third moment (if it exists), the implication does not go back the other way.

Consider a fair six-sided die with the following labels on its faces: 0, 0, 5, 5, 5, 9. The distribution of outcomes is not symmetric, yet its "skewness" measure is zero. Unfortunately, based only on the information that the third central moment was zero, many people would in fact describe it as symmetric.[Added later: here's one that most people would say on inspection was "right skew" - an ordinary die labelled 0, 0, 5, 5, 7, 10. But the third central moment is zero.]

Other examples, both continuous and discrete, abound. Continuous asymmetric distributions exist for which all odd central moments are zero.

Other measures of skewness (and there have been many) also have their problems, though some are very useful.
The sort of problem described here is essentially unavoidable - there are so many ways a distribution may be asymmetric that a single measure of asymmetry cannot possibly suffice, except perhaps within the framework of a particular family of distributions.

This cautionary tale is not an argument against using measures like the third central moment to attempt to capture something of deviations from symmetry - it's an argument against investing them with more than the limited meaning than they posses.

Tyson

Saw Neil deGrasse Tyson on Colbert last night.

He was very quick with his response to Colbert's bit about "Goddiddit" ("You can say that, but then you can't find out anything," if memory is working okay).

Then again, I guess he's been on enough to know it was coming.

Is the man hot, or is it just me?

(Link now goes to correct segment, not just the right show, so you don't have to find it in the list.)

Ecstathy in Wordle

click for larger version

Am I A Positive Atheist?

I wrote a longer version of this post, but it has been eaten by a grue. This one is different.

I was about to start writing the first version of this when I noticed that Dana asks "What kind of atheist are you?". An interesting coincidence, because in some way this post is related.

I must admit a certain amount of confusion with the term "positive atheism". I don't see atheism as inherently positive or negative. (I see the step of throwing off irrational faith in favour of reality as a positive one, but that's a benefit of becoming an atheist, not of being one in the sense that it's not an active process.)

Being an atheist, living as an atheist, is neither positive nor negative. You can live a positive life, or a negative one, or anything in between, and be an atheist. Atheism is orthogonal to positivity.

This might seem incongruous, given I just had a post in the 21st Humanist Symposium. Humanism is the sort of philosphy people tend to associate with "positive atheism" and the Humanist Symposium's brief is definitely about living positively, but humanism and atheism are quite different - many atheists are humanists, and many others are not. Many humanists are also not atheists. But, nevertheless, many atheists are attracted to humanism. I agree with many of its tenets, but I don't see humanism as connected to atheism, so to me it's not really "positive atheism", it's just humanism, and neither is necessary to the other. Similarly, my post was about using probability to figure out that some apparently worrisome things may be nothing to worry about, and that an understanding of probability could help lift burdens of worry from our lives. But again, it's not atheism that's doing that. Atheism's job came earlier, in removing the attempt to explain such events as the acts of an incomrehensible and all-powerful god (the positive step of becoming an atheist). With that non-answer out of the way, there is room for the rational explanations of probability to make their positive contribution. So it is with humanism and other positive worldviews that many atheists have.

I'm generally a very positive and content person. I happen to be positive, and I happen to be an atheist.

But the phrase "positive atheism" (and for that matter, all the negative epithets theists throw at atheism) just doesn't have any more resonance for me than "yellow music" does.

The fate of the planet may depend on it!

Scott Aaronson of Shtetl Optimized explains why safety considerations demand that we must turn on the LHC right away!

The tale of "The square root of a rectangle"

Wow. Just wow.

I was just flicking through channels and caught a little of "Engineering an Empire" on the History Channel. The episode was on the Maya.

The narrator said something about the Maya knowing how to "compute the square root of a rectangle".

I was so boggled, I concluded I must have misheard - but no, it was repeated a minute or so later. It was emphasized; apparently we were expected to find this an impressive feat.

The Maya, apparently, knew how to compute the square root of a rectangle. That's pretty clever of them, because that, as far as I can see, is utter nonsense.

As Inigo Montoya might have put it: I do not think it means what you think it means.

I wondered if they meant "compute the hypotenuse of a right angled triangle" (which is equivalent to the diagonal of a rectangle). I also wondered if perhaps they meant "the geometric mean of two numbers" (since that would be the square root of the area of a rectangle). I also wondered if they possibly intended something else.

I concluded they probably meant the first thing (since that's a very valuable thing for a civilization to be able to do, crucial to surveying land - and then they had a shot of a Mayan guy squinting across the top of a stick as if he was indeed surveying). So that would kind of make sense.

But what the heck is someone who isn't comfortable with mathematics supposed to make of it?

The show was no cheap-and-nasty affair. They made some pretty fancy graphics. They had actors dress up in costumes, and they had some pretty fancy sets and location shots. This was a pretty involved documentary. Why the hell couldn't they have had the script looked over by anyone with even a modest bit of mathematics? Say, a mathematics undergrad? (Heck, I could have told them that was obvious nonsense before I was 16.)

That bit of (repeated, emphasized) mathematical nonsense is enough to make the entire program suspect - because if they were that lax with the mathematics, who knows what care was taken with the rest of it?

A misplaced sense of self-importance (II - you ain't so special)

(Part I is here.)

If deflating misplaced self-importance in hypocrites is a good thing, we should be prepared to deflate our own as well.

Fortunately this is easy. The universe is happy to disabuse us of any sense of importance, if we're prepared to listen to it. Here's a few examples from the last few days:

Other apes can plan for their future needs just as we humans can – by using self-control and imagining future events.

At the same time, our human sense of empathy is nothing special either. Chimps calm each other after an attack with hugs and kisses, in the same way that humans do.

Our memory of what happens to us is not as good as we think

And we may (just possibly) be in for a biggie - since there's ice on Mars, the chances improve substantially that one day we'll find evidence of life there in the past. There will be good reason to think that the universe is filled with life. We would know, then, that life just tends to pop up whenever the conditions are suitable.

Which I find is kind of a cosy feeling.

A misplaced sense of self-importance (I: "elitism"-ism)

A month ago, I made the point that accusations of elitism tend to come from people who are in fact from an elite group themselves.

Whether it's to deflect attention from their own situation, or an attempt to "get in" with a group to which they do not belong, or simply an attempt to shut up a message they don't like, or for some other reason, it's a common occurence.

This was recently brought home by the ironic situation of David Brooks of the New York Times claiming "Obama had a problem" because he wouldn't be comfortable with the people at an Applebee's. Except Brooks fucked up, showing he's completely ignorant of what can be found at an Applebee's himself.

This is not an occasional happenstance - the "charge" of elitism always seems to come from a member of some elite. It's hypocrisy writ large - writ large because the stakes are usually big.

They're making another person or group scapegoats for a "crime" that they themselves are guilty of, in order to incite support among a group of people to which they simply do not belong.

[And really, it's not such a surprise that this is almost always the case. An ordinary Joe doesn't use a word like elite. It's insufficiently exoteric (as are words like "insufficiently" and "exoteric" - the irony isn't lost on me, but at least I've linked to a definition - I may be an elitist bastard, but anyone's welcome to join me!)]

I'm almost tempted to call this a law:
(1) If a person calls another an elitist, they are almost guaranteed to be a member of a social, economic or political elite themselves.

If that law is true, it carries with it something like Godwin's law as a corollary -

(2) the first person to level a charge of "elitism" is a loser

The appropriate response is to simply indicate that we know the speaker is hypocrite - a loser of the worst kind. Let them know you're onto them. You know you want to.

I think this hypocrisy comes from a misplaced sense of self-importance - that the rules they play by don't apply to them. Helping deflate that sense of self-importance is a public service.



(Image from Demotivate Us)

(Part II is here.)

Cectic's take on the atheist billboard

Regular readers will have seen my take on the atheist billboard hullaballoo:

(click image to embiggen)


Cectic has a terrific comic on the same issue.

(Displayed here with permission.)

Be careful what you wish for

One thing the Republicans have done over the past couple of decades is increasingly polarize the electorate, and really put effort into the politics of hate. That's a popular strategy with totalitarian governments, because it gets people looking the direction you want them to (anywhere but at what you're actually doing).

It worked, as it usually does. Among other things, they pandered to what they call "values voters". The thing is, people on both "sides" of politics (an oversimplification if ever there was one) vote their values. The Republican fear-mongering grabbed a bit of the middle as well, and so it worked well - a die-hard core plus a good chunk of the ordinary everyday person, at the expense of the people who are less likely to vote for you will get you power - for a time.

But the problem is that the people they have upset the most are growing, while the people that they're relying on are both shrinking and abandoning(1) them(2) in revulsion(3). Young voters, in particular, are increasingly secular, and that's a problem for Republicans, because they have attacked the secular and their values at every turn.

The Republicans are beginning to reap what they have sown.

Blogger Nate at FiveThirtyEight talks about the problem McCain has with nonreligious voters.

The religious only slightly favour McCain over Obama. The nonreligious overwhelmingly favour Obama. As time goes on, unless the Republicans move toward the centre, they're going to increasingly lose the now-secularizing middle, and their history of vile attacks on them won't be quickly forgotten. If they do move toward the centre, they'll lose what remains of the right-wing evangelicals. Things are not looking good for the Republicans in the next 3 or 4 elections, unless they can completely reinvent themselves. I don't think they'll be able to do it.

They could stew for a generation in the cesspit they've dug for themselves - and they'll still stink less than they do now.

Weekend news

A couple of items from the local news over the weekend:

Obama has the confidence of people in Europe, Asia, Africa and Australia and is strongly preferred as president over McCain

"In Australia, 80 per cent of participants said they had confidence in Senator Obama, against 40 per cent for Senator McCain. Similar results were reported from Britain, France, Germany, Spain, Japan and Tanzania."


The secretive religious group at the heart of US politics

"The Family organises Washington politicians into intimate "prayer cells", influences foreign policy, inspired the creation of the president's annual prayer breakfast in 1953 and sponsored George Bush's faith-based 2001 policy of transferring social welfare responsibilities to religious groups."

Read about them, and be scared. Or angry. Or both. But know they they're there, and they may well be behind a lot of what you really hate about what goes on in US politics. They're the elitists we should be fighting against.

Technorati blog claim

Taking vjack's advice and trying to add my blog to Technorati - here I'm claiming my blog there ... so here goes. There's my Technorati Profile.

Edit: Currently not showing my blog under blog tags nor my posts under post tags. A lot of effort for zip so far. Now seems to work thanks to admin assistance

A possible place to look for an explanation for religious belief?

Dana has a review of Julie Sweeney's movie Letting Go of God.

Now, on a completely unrelated note, there are various potential explanations that have been suggested for the fact that religious belief is very common - generally either taking the form that it in some sense carries a social benefit, or that it is a side effect of something else (or an interaction of several somethings) that carry a benefit. Some of these explanations have some evidence for them, though I don't think there's really conclusive evidence that any of them forms a complete explanation.

Anyway, something Dana said brought up a thought:

She doesn't ever state this directly, but the end of the movie talks about her daughter reaching for magical explanations when Julia's trying to explain things like death in rational, material terms. And that struck me: religion is never growing up. What her daughter invented to make herself feel better about things she didn't understand sounded exactly like the answers most human religions invent.

We as a species have never seemed to mature past the age of four.

And then I thought... if you add in the "invisible friends" bit, you've got a couple of indicators of development typical of maybe a three year old.

That reminded me of neoteny - the retention of juvenile characteristics into adulthood. This is a common evolutionary trick - rather than change an underlying gene, alter the timing of its development - it's generally going to be a much smaller change (this sort of idea should be standard fare for the evo-devo crowd, but I first encountered the concept in one of Dawkins' books, many years ago), but the effects can be dramatic.

Humans are "masters" of neoteny. We retain dozens of juvenile-ape characterstics. It's why our faces tend to be almost hairless. It's why we play games and retain curiosity into adulthood. It is one of the major drivers of our success; it is, not to put too fine a point on it, a prime cause of why we are who we are: I gather chimp brain development continues until about 1 year of age. In humans it lasts until we're about 23!

Is it so surprising then, that certain mental development characteristics of toddlers might hang on? If our evolution has caused us to so dramatically retain certain juvenile aspects of brain development, might associated mental milestones tend to come along with them?

Is religious belief a side effect of neoteny?


If speculation is not to your taste, have some (tangentially related) science - on genes in humans that have shown a much higher rate of evolution since our last common ancestor with chimps - what potentially makes us what we are.

Any day now...

(click comic for a larger version)

Adding percentiles means there's probably more oil than we thought

Isabel Lugo has an interesting post up that points out that because "proven reserves" figures for individual oil fields are given as the tenth percentile (that is, the amount of oil in a field is estimated to have a 90% chance of being larger than the stated amount), you can't just add up the individual "proven reserve" figures and have a figure that means the same thing. But apparently that's exactly what many bodies do with the figures!

(She refers to this New Scientist story.)


Isabel gives an example where adding two tenth percentiles gives much lower than the tenth percentile of the estimate of the total oil for both (with the implication that if you add enough of these things together, the true amount of oil is probably far, far larger).

[I pointed out in comments that the conservatism in the sum is not necessarily the case (but for readers here - it very likely is the case; the counterexample to conservatism that I give is not a likely situation).]

Changing patterns of belief

A very interesting article on falling rates of belief (in percentage of population terms).

I'm not sure the evidence is completely clear for a few things they say, but overall gist looks pretty solid.

Sign of the tomes

(click image to embiggen)

Scary atheists

Ebonmuse has an interesting post up at Daylight Atheism, about what's going on with the sense of distrust believers feel for atheists.

I disagree with some of it, but it's a great post, and brings up some great points.

Contingency in evolution vs the journalists

Answers in Genesis BUSTED! points to New Scientist's reporting of the interesting story of contingent evolution in e. coli.

Fortunately, nowadays we have the resource of actual scientist-bloggers rather than being restricted to what passes for science reporting these days, because the New Scientist story drives me nuts. As much as I love being able to hold a magazine in my hands to read, this reminds me why I decided a few years back that New Scientist was too expensive for what I was getting. (I have boxes and boxes of the things. Is there a good charity that needs a whole lot of old science magazines that I won't go broke getting them to?)

Frankly, it's entirely typical of science reporting, and another example of why science reporting sucks, and why blogging by scientists beats it hands down.

Aside from the fact that the New Scientist report has the emphasis wrong, it's just a generally typical sucky example of science reporting, with distortions of fact and silliness. (To be fair, here and there it does an okay job of explaining some bits, but really, I'm no expert, and I could tell it sucked before I'd read about it anywhere else.)

[Actually, what used to really drive me to screaming fits of rage was whenever they let Ian Stewart start talking about statistics. The guy may be a reasonable mathematician, but his explanations of statistics sucked dogs balls.]

I highly recommend reading the Pharyngula
discussion.

For goodness sake, even the abstract of the paper he quotes (jargon aside) is in many places better written than the New Scientist article.

Let me quote a paragraph from the New Scientist report:
"In the meantime, the experiment stands as proof that evolution does not always lead to the best possible outcome. Instead, a chance event can sometimes open evolutionary doors for one population that remain forever closed to other populations with different histories."

Gah!

"doors ... that remain forever closed to other populations" is one of the more ridiculous statements I've seen from New Scientist, and they've had some doozys.

forever closed? forever??

Bullshit. If the "tricky part" - the enabling mutation - evolved once, in only 20000 generations (which then further evolved several times to produce the ability to metabolize citrate), it's obviously possible for it to happen again. The probability of it evolving is not zero.

The fact that Blount, Borland & Lenski couldn't get it to evolve independently a second time just means that probability is likely very low. Maybe extremely low.

But forever is actually a pretty long while. You might think it's a long time 'til lunch, but forever is such a really long time that any mathematically nonzero probability of it evolving in a finite amount of time (as was observed to occur) means that by forever, it will have evolved an infinite number of times. And, if you're not real good with the counting thing, take it from me that infinity times is just a tad bit bigger than zero.

And what's with that bit about "evolution does not always lead to the best possible outcome"? Which cryptotheist retarded version of evolution do they think they are arguing with there? All you have to do is take a look at a human eye (heck, just get a small light and grab a handy animal with a spine) to know that. The blood vessels of the eye lie on top of the rods and cones! They're in the way! Anyone can see them! And the nerves, too - that's why we have a blindspot. And, funnily enough, the light sensitive bits point backwards, away from the light. Our eyes are arranged ass-backwards!
(That it needn't have happened this way is obvious from squid and related creatures, whose eyes are not arranged "backwards" like ours. And there are hundreds of little examples of this sort of thing.)

Evolution is completely cobbled-together (i.e. contingent), and it's already completely obvious that it's so - to anyone with eyes. Well, and maybe a brain.

[based on a comment I made at AIGBusted.]

The problem of suffering and free will

There's a standard apologetic response to what's usually called "the problem of evil". That response is, basically, that evil is a consequence of free will.

However, that utterly fails to deal with suffering that has nothing to do with free will. Disasters, painful disease and so on have nothing whatever to do with free will, and so the apologetic response does not apply.

There's a second response I've seen (usually from the more fundamentally minded), which is that these things are either "caused by the devil" or "caused by the fall".

I could easily deal with both of these point by point, but they're directly covered by Epicurus' 33AD argument (slightly recast):

"Is God willing to prevent suffering, but not able?
Then he is not omnipotent.
Is he able, but not willing?
Then he is malevolent.
Is he both able and willing?
Then whence cometh suffering?
Is he neither able nor willing?
Then why call him God?"

Either God is able to stop the devil or not. If not, "caused by the devil" is answered by the first pair of lines, but if God could stop the devil, it's answered by the second pair of lines. If it's caused by the fall, then (second pair) God is punishing the innocent because "hundreds of generations ago" someone didn't believe him when he lied (it's right there in the bible) about the fruit on the tree.

(Yep, the snake told the truth and God lied, and he condemns humans to floods and earthquakes and painful deaths forever - nice guy. Oh, and he pulls the legs off talking snakes that point out he's a big shifty fibber. Niice. And then yells at everyone to get the hell off his lawn.)

So we get stuck - the problem of suffering is not answered by free will. If it's the devil, God is either not omnipotent or malevolent, and if it's the fall, God is plainly malevolent. As well as a big fat liar.

Government in SF - a failure of imagination?

(Within this post, I'm going to take the stance that SF stands for "speculative fiction" rather than science fiction. My main reason is that SF these days rarely contains much actual "sciency" stuff, and the category has become so broad that the term "science fiction" is not even vaguely true of most of it. It is also possible that it serves my argument. )

I've long had a problem with how government is represented in SF, particularly film and television, but it also happens in written SF - that the governments almost always seem to be of only two very narrow types:
(i) near-absolute monarchy (though possibly called an emperor or something, often also with a noble class);
(ii) US-style presidential democracy.

I have no doubt that in many stories, these are plausible models. However, I simply cannot fathom why so many writers assume that if it's not the first, it must be the second. Sometimes the US political system is reproduced to an astonishing level of detail, including a near duplicate constitution and everything.

According to wikipedia:
- there are 123 democratic countries
- there are 72 parliamentary democracies
- in addition, there are a large number of "hybrid" models (particularly in Eastern Europe)

That is, a US-style democracy is by far in the minority. On earth, parliamentary-style democracies are the rule. Yet in SF, they're almost unheard of.

So why in visual-media SF (the majority of whose writers are from the US), if you don't have some form of monarchy or monarchy analogue, will it be (apart from a very few exceptions), a US-clone democracy, often down to its boot-buckles? Is this a failure of imagination? Are US film and TV writers largely unable to imagine anything else? Are they unaware of anything else? Or do they imagine that a US audience won't accept anything else (and everyone else will, presumably)? If it's the last, that, too, is a failure of imagination - I certainly don't believe US fans of SF are that unimaginative and unwilling to think about different ideas.

Obviously, people tend to write about what they're familiar with, but if you only ever write about only what's in your back yard, your fiction hardly deserves the name "speculative".

Come on, people, have a look at the world around you. Try a different model for government once in a while!

The Consolations of Probability

There's little doubt that many people find comfort in the presence of an all-powerful, all-knowing being watching over them, in knowing that everything happens for a reason, even if they don't know what those reasons might be. People who don't believe in such things cannot have such consolation. After all, what consolation is there if things just happen?

I'm not just talking about the fact that there's a lot of random stuff that happens that we aren't responsible for, or that we're incredibly lucky to be here. Those pieces of understanding are major consolations, no doubt (or at least they are to me). Greta Christina has touched on these, and similar issues several times, and written about them better than I could.

Probability and statistics, or at least an understanding of them, are incredibly useful things to have. I make a living out of them, so they're obviously useful in that sense, but I am talking here about ordinary everyday getting-through-life stuff.

For example, they're essential tools when responding to the news, because they help you identify the difference between a real issue and a likely manufacturoversy.


The tale of the traffic blackspot

Some time ago I saw a headline about how a particular busy intersection had become a pedestrian "black-spot" because there had been a "50 percent increase" in pedestrian deaths and serious injuries last year; it was suggested that it may have been caused by increased traffic from changes several intersections back along the same road. The relevant state minister was grilled about his "lack of action" and induced to promise some very expensive changes at the intersection.


My initial, emotional response was the one the journalist was doubtless trying to induce - horror at an apparently huge increase in deaths and injuries. But that nice round "50 percent" figure worried me. Was that number so rounded off from say 48 or 52 percent, or was it exactly 50, suggesting the numbers were very small? So I looked through the story for the figures. The number of deaths and serious inuries had gone from 8 the previous year to 12 in the last year. A quick calculation in my head showed that, even if the underlying probability of incidents had not changed, such a change in actual numbers could easily have happened. The fuss over the change from 8 to 12 may well be a big fuss over nothing but randomness. It could well be that the figure of 8 was itself too high - maybe something should have been done years ago - but the controversy highlighted by the article was misplaced; there was no particular reason to believe anything had changed. The journalist had managed to induce the minister to reallocate funds that may well have been much better left where they had been before.

[Since the details of the mathematics is not the point of this post, I haven't included them - in another post at a later time, I may explore a little of the specific calculations in these (real) examples. ]

If you're watching for it, this kind of thing comes up a lot. H.G. Wells once wrote "Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write". Is he overselling the case? When government policy is increasingly driven by media attention to unimportant detail rather than substantive policy, I do think it's become increasingly important. We need to be able to understand the flood of information, and if necessary, debunk the nonsense it contains. Can the broader awareness suggested by Wells' comment come to pass?

It seems that educators are responding.


In the part of the world where I live, statistics and probability are now taught throughout school (K-12). My six-year-old has already learned to record data and produce simple graphs of it on the computer. My nine-year-old has learned some basic probability. By the end of year 12, they will have a pretty solid grounding in the basic ideas. They might not end up with all the tools to do the calculations I used, but they will at least be able to ask the right questions.

While important, this sort of understanding of what's going on in the world - certainly a form of comfort - is perhaps a relatively abstract one. However, the comforts can also be much more personal.


The tale of the overtested driver

Some years ago, a fellow was asking a group of statistically-minded people (of which I was one) whether he was being victimized at work. His employer, who owned a business that had many employees driving trucks, had instituted random drug testing. Every week a small number of employees were selected and tested.


This particular driver felt he was being targetted, because he had been tested seven times in 18 months, while he knew of several others who had not been tested at all. He was worried that someone must have it in for him; he thought that maybe another employee he'd had a falling out with may have said that he was taking drugs in order to make trouble for him. He was wondering if he should confront that person.

I pointed out that for it to be an effective deterrent, drug testing had to be done at random - you can't test someone once and then not again until everyone else had been done, because if someone were inclined to take drugs while driving trucks, once they'd been tested, they'd know they were "safe" for a while. There always had to be a risk you would be tested next month. Random testing in turn meant that there would always be some people tested substantially more than others. Since he was asking, rather than someone else, it was likely that he was one of the most heavily tested people in that company, probably the most heavily tested - but someone had to be the most tested person.

The question was - if testing really was being done randomly, would seven tests in 18 months be an unusually high number for the most-tested person? Of course, you need to look at the number of people in the company and the number being tested each week. You'd also want to ask whether you'd really expect to find some people still untested at the same time.

It turned out that after 18 months, it was reasonably likely someone at the company was going to be tested seven times (around a 20% chance - a higher chance than picking up a die and rolling a six), and at the same time there would be a fair number of still-untested people (enough so that he'd likely know several). So there was no particular reason to think he was being targetted. No reason, based on the numbers so far, to expect the person he was worried about was making trouble for him.

Such knowledge was indeed, a consolation. Sure, the drug testing was a nuisance, but there wasn't any reason to think it was personal, or that anybody had it in for him. A significant source of stress in his life was lifted.

And understanding not just that things can happen at random, but of the specific consequences of how randomness works can show us that nobody may be to blame. It can show us that we are not a victim.

Probability can be a consolation, sometimes quite a personal one.

What I think of when sports stars thank God

16000 children died of hunger today, but God made your football team win.

God must really like football.

Right change

NEW ORLEANS, Louisiana -- Senator John McCain portrayed himself as the candidate of "right change". "The choice is between the right change and the wrong change," he said.

"No matter what you're buying, you should always count your change. Senator McCain thinks it's very important that you get the right amount of money back," an aide explained. "Also, check down the back of your sofa. I found a dollar fifty three last week. When elected, Senator McCain plans to check every sofa in the White House. Do you have any idea how much the last Hundred Years War cost? And wars are apparently more expensive nowadays. If everyone in America found a dollar fifty three every week, umm... how many people live in America? Let's say a trillion? Okay. Oh wow, is that a penny on the ground? Senator McCain! Senator McCain! I found another one!"

To know the mind of God

(Click the comic for a larger version)

Time again for the triple quote

Hmm. I've almost hit the 8 month mark with this blog. This will be my 71st post, so that's...say (1/5)/(2/3)... 0.3 posts per day on average. (Which is better than I originally thought I'd manage.)

I haven't done the "Three quotes" thing in a while (and sometimes it's not three, anyway - it's not like it's a rule). So I guess it's time to dust it off.

In honour of the just-posted Carnival of Elitist Bastards #1, here's three quotes relating to knowledge and truth ... and their lack.

"I have observed that the world has suffered far less from ignorance than from pretensions to knowledge." - Daniel Boorstin

"The truth may be puzzling. It may take some work to grapple with. It may be counterintuitive. It may contradict deeply held prejudices. It may not be consonant with what we desperately want to be true. But our preferences do not determine what’s true." –Carl Sagan

"If I look over my life, every single step of maturing for me, every single one, has had the exact same common denominator, and that was accepting what was true over what I wished were true." –Julia Sweeney