Tuesday, November 17, 2015

Probability and Statistics in Science Fiction

My previous post on mathematics in science fiction had a serious omission -- I ignored probability and statistics. These two subjects are often lumped together, but they are, in a way, opposites. In probability, you know the rules of the game ahead of time (I roll a die, and each number from one to six is equally likely) and you have to calculate the odds of various outcomes (how likely is it that I will roll three sixes in a row?)  Statistics is just the opposite:  you are given the outcomes and are trying to figure out the rules of the game. If you have a bunch of "data", what is the likelihood that they were produced from a particular model of the universe? In his book Numerical Recipes, Bill Press and his collaborators describe statistics as "that gray area which is surely not a branch of mathematics as it is neither a branch of science."  I couldn't agree more.

Probability is clean and precise, and I really enjoy it.  I've written several papers about the pattern of galaxy clustering in the universe (technically, this is called the "large-scale structure" of the universe) based largely on different aspects of probability theory.  Probability is my friend.

Statistics, on the other hand, is not my friend.  It's a necessary evil, like a decennial colonoscopy. Statistics is the Norse trickster god Loki -- it's slippery and untrustworthy.  There are even different sects within statistics whose members base their analyses on completely different fundamental assumptions.  For example, you can be a Bayesian or a frequentist -- these two groups fought an inconclusive war that devastated Germany in the 17th century.  (By the way, don't believe everything you read on the internet). For a less jaundiced and undoubtedly more accurate view of statistics, read the various posts on Michael Flynn's blog, such as this one.

How have these subjects entered science fiction? A very clever treatment of statistics is "The Law," by Robert M. Coates, which appeared in 1947 in The New Yorker of all places (in the unlikely event that you subscribe to the The New Yorker, you can read the full story here).  In Coates's story, the law of averages breaks down, so that, for instance, everybody tries to get on the same bridge at the same time. A similar idea is presented in "A Very Good Year," by Jack C. Haldemann (not to be confused with his brother Joe Haldemann).  In Haldemann's story, nobody dies for a whole year, but then the law of averages takes over the following year with a vengeance.

It's harder to find treatments of probability in science fiction, unless you count the many times it appears in stories of quantum mechanics (some of which I've discussed earlier).  Greg Egan's Quarantine makes particularly interesting use of probability theory as it applies to quantum mechanical measurements.


TheOFloinn said...

I have sometimes described the difference between probability, statistics, and process control by using a box containing white and red beads.

1. Probability: knowing what is in the box, what sort of samples are we likely to get?
2. Statistics: knowing what is in the sample, what is the likely make-up in the box?
3. Process control: Is there a box?

The first two are as you say. But the third is also interesting. Both 1 and 2 implicitly assume the existence of a common cause system. But it may be more crucial to learn whether the system is constant before doing classical statistics on it. You cannot calculate a meaningful average from heterogeneous data [pun intended]. The average human being has only one testicle.

Anonymous said...

Where do y'all (i.e. both Our Host and TheOFloinn) draw the Line between "Science Fiction" and "Technothriller"?

On which side of the Line would each of you put Eaten?

Robert Scherrer said...

If it gets made into a movie with the location of each scene appearing as if from a typewriter, then it's definitely a technothriller.... Otherwise I have no deep insight.

Kathy said...

I travel once a year to Vegas. I play in a casual, recreational, smart(ish) way. The past two years I got ahead of the curve with a nickel loose deuces (a deuces wild variant) video poker machine.

By its paytable it pays back 101.75% when using a full strategy perfectly (ie without mistakes). I use the simple strategy because it's easier to memorize.

My first royal flush in that machine (it pays only $200, it's a nickel game) happened when I misremembered strategy and held suited Jack, King and Ace. the strategy says to hold only the Jack and King (better chance for a paying hand that way). Holding the Ace was a mistake. This time it paid.

I figure all that week I made that error several other times, but turned it into a Royal only once.

Sometimes a probability is only a probability :)

I'm sorry to say this machine was taken out of service last summer :( I was way up on it. Besides, it was a really cheap way to gamble. $20 could last me literally hours.

Kathy said...

I had a notion about a team of physicists using time travel to cheat at roulette.

The problem would be they're rewinding time, rather than traveling back in time. Therefore while most events happen again as they did previously, random events like roulette don't. For example, Daryl might spend an hour recording results, give them to Larry after time-traveling one hour back, but Larry would see different numbers come up.

I just couldn't come up with a story.

Robert Scherrer said...

A collection of physicists did once try to cheat at roulette. Read "The Eudaemonic Pie".

Kathy said...

I know a bit about the Eudomonics group. As I understand, their hardware fell short in actual operations.

Their system could predict the octant where the ball would likely land, given known or approximate variables like ball and wheel speed. In an early season of "Mission: Impossible!" the good guys have a similar system, though much exaggerated. Theirs could predict to a certainty where the ball would land (given the limitations of a TV show, I'm willing to let it go).

The most common form of cheating at roulette is past posting. This involves increasing one's bet after it wins. Of course without the dealer noticing and before the dealer pays off. That's one reason casinos tape the whole floor at all times.

Robert Scherrer said...

There's a story (possibly apocryphal) that when the American Physical Society held their annual meeting in Las Vegas, the physicists (who understood the laws of probability) gambled so little that the hotel asked them to never come back.

TheOFloinn said...

I dunno, but I was at an American Mathematical Society annual conference in Las Vegas, back in the day, with a similar behavioral pattern. It was worse, because Dr. Thron did not even want to stay for the girlie show.

I once wrote a short story, "Probably Murder," in which a miscreant (an actuary) killed his wife by ensuring she was subject to as many risk factors as possible (e.g., he discouraged her from quitting smoking) in the serene knowledge that the probability of surviving any one of them was high, the probability of surviving all of them was vanishingly small.

Kathy said...

I'd like to know a physicist's take on dice control for craps.

The notion is that if the dice are set a certain way and thrown in a controlled, consistent manner, the odds of hitting a particular number are higher.

This is a very controversial issue in the gaming boards and blogs.

Indirect evidence against it is that 1) casinos don't commonly ban dice setting and 2) the people making money off it do so by conducting seminars and selling books and videos about it, not at the craps tables.

Robert Scherrer said...

I'm not an expert on this. But as you noted, empirical evidence suggests that it doesn't work, or it would already have been banned. (My experience is that if it's possible to extract money from a system in an efficient way, then it's already been extracted!)

Kathy said...

My own take is that it sounds plausible, but the lack of firm proof after several years means that either 1) it's not really possible, or 2) it's so hard even the best at it can't do it regularly.

And there remains three: luck.

I've seen long, long rolls without a seven-out (not without sevens). The one I was playing (others I was staring a few paces off a full table), hit the point four times (three different numbers), and a bunch of come bets I kept placing. I made out ok, considering I was playing the table minimum and the max odds were set at 2X (that's chintzy; odds bets pay real odds, and therefore have no house edge).

Craps is awfully complex, too. I don't think any other game comes even close to the number of possible bets available, nor to the number of bad bets (ie with a high house edge) available for tourists and locals to bet.

LJ@Sheldon said...

As a PhD student in physics as well as a MS student in BioStatistics, I used to share the same preference of probability over statistics. But my opinion changed now.

The statistics nowadays is divided into two parts: the inference part (aka, old school one) and the learning part (e.g. machine learning). While I am mastering well in the first part thanks to my mathematical foundation, I am really amazed by the second part. I have seen so many concepts in theoretical physics being applied to machine learning again and again...

After all, statistics is largely based on data just as physics is a subject based on experiments. Statistics Models are trained to be more robust as more and more data show up, just as the Standard Model is improved as the colliders are telling us more truth.

I truly love and appreciate physics even after I make a decision to quit with no luck in postdoc job hunting,,,

Evan Browns said...

Your previous post and also this one, they are awesome!