Edwards on Fisher (2005)

In his discussion of Fisher's 1925 book Statistical Methods for Research Workers, Edwards writes:

It was not until 1950 that the word ‘Bayesian’ was coined, by Fisher himself, to refer to inverse probability (p. 862)

Can this really be true? To be sure, "inverse probability" was the more common term before 1950, but was "Bayesian" really never used, at all?

I thought this claim would not stand up to 30 seconds of googling, but I was wrong. The only reference I've found so far is from the bibliography in Jimmy Savage's Lecture Notes on Mathematical Statistics, which includes the following item:

Jeffreys, Harold. Theory of Probability. New York: Oxford University Press, 1939. 380 pp. A highly controversial book on the philosophical foundations of statistics by the most foremost modern exponent of the Bayesian heresy. Examples largely from geophysics.

I don't have access to Savage's book here, so I can't vouch for the year of publication, but Amazon, WorldCat, Google Books, and a few other resources all give the date as 1947.

If that's true, then the word "Bayesian" was used at least once before 1950. But one is certainly not a crowd.

Chrystal: "On Some Fundamental Principles in the Theory of Probability" (1891)

I've been trying to get my hands on the following paper:

George Chrystal (University of Edinburgh): "On Some Fundamental Principles in the Theory of Probability." Transactions of the Actuarial Society of Edinburgh, Volume 2, January 1891, pages 420–439.

So far, no luck. The Cambridge Journals database has a copy, but it's behind a paywall, and my university doesn't have a subscription.

However, a number of other sources quote extensively from the paper, so I've been able to piece together an understanding of what it looks like.

Posterior Frequencies

It seems that Chrystal's main beef is with the use of Bayes' rule to update the probability of a certain set of hypotheses whose long-term frequencies are already given in advance. His reasoning seems to be that this conflates our subjective degree of belief (which may indeed change) with the objective frequency (which, by assumption, cannot).

This philosophical distinction is nicely presented in the following quote. It comes from an 1894 review (also available in book form on the Internet Archive) by somebody called G. F. Hardy, not to be confused with G. H. Hardy.

"There is," says Professor Chrystal, "in Laplace's view, a confusion between two senses of the word 'Probability', which although distinct are often more or less associated in point of fact. In common speech we say that a single event is more or less 'probable', and by this word we indicate our own mental attitude towards the event, an attitude that may be well or ill justified by facts. When an actuary says that the probability that a man of 20 will live to be 60 is $\frac{59}{97}$, he is not, strictly speaking, referring to any one event at all, but merely making an assertion to the effect that out of any considerable number of men of 20 years of age about $\frac{59}{97}$ will reach the age of 60. No one knows better than an actuary that this statement is a fact, established (under certain circumstances, and with certain limitations), by experience, and that it has nothing whatever to do with the mental attitude of anyone. Everyone will admit that we could never arrive at this result by analyzing the event of a man of 20 reaching or not reaching the age of 60 into cases regarding each of which we should be equally undecided,—mentally suspended, as it were, like Buridan's ass between the equal bundles of hay." (Hardy, p. 316)

It's not clear whether the emphases are in the original, but I'm guessing not.

Hardy does not give a page reference, but proper reference seems to be page 423. I got that figure from the manuscript of a 1893 presentation by a certain John Govan, F.F.A. (whose name, location, and date fit the proselytizing businessman John George Govan).

Govan's rendition of the quote occurs on his page 212. He doesn't use the emphases.

The Burial of Bayes

Several sources also report that Chrystal summarizes his discussion with the following tirade:

… both from the point of view of practical common-sense, and from the point of view of logic, the so-called laws of Inverse Probability are a useless appendage to the first principles of probability, if indeed they be not a flat contradiction of those very principles.

This is cited by a number of authors, including E. T. Whittaker, F.R.S. (in a footnote to a 1920 presentation, p. 165), Andrew I. Dale (1999, p. 485) and Sharon McGrayne (2011, p. 37).

Dale reports that this quote is found on page 438. Whittaker apparently reports it as page 421 (but that would be at odds with Dale's description of the quote as a occurring near the conclusion of the essay). McGrayne doesn't give a page number.

According to Whitaker, the quote continues as follows:

The laws of Inverse Probability being dead, should be decently buried out of sight, and not embalmed in text-books and examination papers.

McGrayne further reports the following conclusion:

The indiscretions of great men should be quietly allowed to be forgotten.

Checking the relevant footnote of McGrayne's book (note 9 of ch. 3, p. 260), this turns out to be a recycled quote from Anders Hald's A History of Mathematical Statistics, page 275. That book doesn't have a Google Books preview, and it's not in my library.

Sort-of-Long-Term Frequencies

Hardy's review continues to quote Chrystal's discussion of how probability is to be defined:

"The notion of probability is always attached to a class or series of events, which usually have more or less of other attributes in common, but are always distinguished by this mark, that certain phases of them, although not predicable with the smallest certainty in any individual case, are predicable with more or less uniformity in a certain proportion of cases in the long run. The fundamental features of this series are statistical uniformity combined with irregularity of every conceivable kind in the individual instance. The number of the events in the series must be large. Its extension both as to space and time is arbitrary, and in certain ideal cases infinite. It is in this last respect alone that probability has anything to do with our mental attitude; we may choose our standpoint, and this determines the probability to which our knowledge may make a better or worse approximation. As the series is varied the probability alters. . . .  We are thus led to the following abstract definition of the probability or chance or an event. If, on taking any very large number, N out of a series of cases in which an event A is in question, A happens on pN occasions, the probability of the event A is said to be p." (Hardy, p. 317)

Again, no page number is given.

Posterior Priors

The examples in Chrystal's paper seem all to be of the same kind: He describes a set-up in which certain a priori frequencies are given, and he then tells us that no amount of evidence should be able to change those frequencies; the only mental operation we can perform is to exclude logically impossible cases, not to compute posterior probabilities.

Govan thus quotes him as discussing a situation in which you draw two white balls from a bag of black and white balls. Then:

"Any one," says Professor Chrystal, "who knows the definition of mathematical probability, and who considers this question apart from the Inverse Rule, will not hesitate for a moment to say that the chance is $\frac{1}{2}$; that is to say, that the third ball is just as likely to be white as black. For there are four possible constitutions of the bag . . . each of which we are told occurs equally often in the long run, and among those cases there are two . . . in which there are two white balls, and among these the case in which there are three white occurs in the long run, just as often as the case in which there are only two." (Govan, p. 208)

According to the text of Govan's discussion, this quote must be on or around page 435 of Chrystal's text.

Another very similar example is attributed to Chrystal's page 437:

"A bag contains five balls which are known to be either all black or all white—and both these are equally probable. A white ball is dropped into the bag, and then a ball is drawn out at random and found to be white. What is now the chance that the original balls were all white?" Professor Chrystal asserts that the chance is precisely what it was before, viz. $\frac{1}{2}$. (Govan, p. 208–209)

"The ball drawn out," says Professor Chrystal, "may have been the one we put in, it may not; and this is all that any one can say." (Govan, p. 209)

Note that this is quite upside-down compared to how we usually think about frequentism: Here, Chrystal tells us to ignore the likelihoods and put all our confidence in the priors. We are used to thinking about frequentists as doing the exact opposite.

The Essential Tension

Govan objects to Chrystal's principle of rejecting the evidence:

… let us say we have two bags before us, one containing six white balls, the other five black balls and one white. There is nothing to indicate which is which. We draw from one of the bags chosen at random a ball which proves to be white. It is difficult to believe that any man in the possession of his faculties, say if his life depended on his guessing aright from which bag the ball had come, would hesitate to guess the former. Even according to Professor Chrystal he would be right 6 times out of 7 in the long run. Yet, again according to Professor Chrystal he would would be just as likely to be wrong as to be right. (Govan, p. 209)

Although they are talking past each other, this is certainly the core of the issue: The distinction between optimal, adaptive gambling behavior and fixed, objective frequencies.

Cardano: The Book on Games of Chance (1961/1663)

Cardano; from Wikimedia.

My library has refused to buy the new translation of Jacob Bernoulli's Ars Conjectandi (1713), possibly because the price tag is on the order of $3000. Consequently, I don't have an English translation of the complete book, arguably the most influential one in the history of probability (perhaps with the exception of Kolmogorov's little pamphlet).

Meanwhile, they hold two copies of Cardano's weird, chatty, and uneven book on gambling. I checked out both of them, only two find out that one (the 1961 version) was a reprint of the other, the translation by Øystein Ore appended to his own 1953 commentary on the book (which, incidentally, also included an interpretation of Cardano's erroneous calculus of probabilities).

I'll deal with Ore's book later. Now a few comments on quotes on Cardano.

Contents

Cardano's book can roughly be divided up into sections as follows. If it looks jumbled up, it's because it is.

• Chapters 1–10: Preliminaries, survey of games, moralistic preaching, etc.
• Chapters 9–11: Combinatorics of dice games.
• Chapters 16–19: On card games (mostly non-mathematical discussions of cheating, moral issues, and the like).
• Chapters 20–21: On luck.
• Chapters 22–25: More game taxonomies and definitions.
• Chapter 26: On theoretical and practical knowledge.
• Chapter 27: More on luck
• Chapter 28: Recommendations on playing styles in backgammon.
• Chapter 29: On why you shouldn't play against hotheads.
• Chapters 30–31: Games in ancient Greece.
• Chapter 32: Computing expectations for a die.

From the perspective of the history of mathematics proper, the most directly relevant parts are those on combinatorics and those on luck.

"If Fortune Be Equal"

One of the more remarkable features of Cardano's book is the prominent place it gives to "luck." As Lorraine Daston has noted, this concept seemed to fill out any gap his calculus didn't account for, including the gap between frequencies and probabilities, or between expectations and actual outcomes.

Mean playing dice; illustration from a 1531 print of Cicero's On Duties.

The first time the word occurs is during Cardano's dicussion of fair bets. Apparently, the most common dice game at the time was a bet on whether a specific throw (e.g., double six) would show up or not show up within n throws of a number of dice.

Cardano was therefore interested in computing the number n for which this bet would be fair — or in his words, for which "there is equality." He explains in Chapter 11:

This gives eighteen casts for equality for a throw (1,1); for in that number of casts this throw can appear and not appear with equal probability; and similarly for the throws (2,2) and (3,3).

But the throw (1,2) can turn up in two ways, so that for it there is equality in nine casts; and if it turns out up more frequently or more rarely, that is a matter of luck. (p. 11; my emphasis)

Later, in Chapter 14:

If, therefore, someone should say, "I want an ace, a deuce, or a trey," you know that there are 27 favorable throws, and since the [size of the] circuit [= sample space] is 36, the rest of the throws in which these points will not turn up will be 9; the odds will therefore 3 to 1.

• Therefore in 4 throws, if fortune be equal, an ace, deuce, or trey will turn up 3 times and only one throw will be without any of them;
• if therefore, the player who wants an ace, deuce, or trey were to wager three ducats and the other player one, then the former would win three times and would gain three ducats; and the other once and would win three ducats;
• therefore in the circuit [= observation] of 4 throws they would always be equal. (p. 16; my epmhasis; my itemization)

His language use here suggests that, on some level, he believes that the expected values must somehow be realized — they are what the game should pay off if it weren't for all these kinks and imperfections in the universe.

"The Length of Time … shows Forth All Forms"

Apparently, he is completely serious about this. In a later chapter on skill (ch. 27), he lists two "methods" by which fortunes can change, the second being the more occult one:

But of the other method there is also some secret principle. To these matters belong amulets, witchcraft, and the like, and just as in each case (as they say) the sword fits its own sheath and the foot its own show, so the hour, the day, the year, and the place must fit; so also in this question, what will make one man happy will make another wretched. (p. 44)

Austrian 16th century woodcut of two soldiers playing dice.

Cardano does seem to think, however, that time tends to cancel out luck. In the chapter on sequential successes, he computes the probability of observing an unbroken string of 20 repetitions of a probability 1/2 event, wrongly getting the answer to be 1/8000. He comments:

Yet it can scarcely be believed that in 7,999 throws a player can cast an even number twenty times in a row. But we have demonstrated this; because, although in 8,000 performed only once, this reasoning can deceive us, still in an infinite number of throws, it is almost necessary for it to happen; for the magnitude of the circuit is the length of time which shows forth all forms. (pp. 19–20).

Here, "luck" almost seems to be synonymous with "noise."

Squares and Cubes

Since I'm talking about this error anyway, and since this is essentially the sole remaining mathematical component of the book, let me just quickly summarize what Cardano seems to be doing in the rest of Chapters 14 and 15.

At the end of Chapter 14, he claims that if the odds in favor of a single success is p : 1 – p, then odds in favor of k successes in a row are p^k : (1 – p)^k. This is not correct, since (1 – p)^k is not in general equal to 1 – p^k. He recognizes this in the opening of Chapter 15, noting that if it were really true, any run of consecutive probability 1/2 events would also have probability 1/2:

But this reasoning seems to be false, even in the case of equality, as, for example, the chance of getting one of any three chosen faces in one cast of one die is equal to the chance of getting one of the other three, but according to this reasoning there would be an even chance of getting a chosen face each time in two casts, and thus in three, and four, which is most absurd. For if a player with two dice can with equal chances throw an even and an odd number, it does not follow that he can with equal fortune throw an even number in each of three successive casts. (p. 19)

 Cardano 1 

 Cardano 2  

Correct 

1 : 1

1 : 1

1 : 1

1 : 1

3 : 1

3 : 1

1 : 1

8 : 1

7 : 1

1 : 1

15 : 1

15 : 1

1 : 1

24 : 1

31 : 1

1 : 1

35 : 1

63 : 1

1 : 1

48 : 1

123 : 1

So now Cardano owes us a different argument. He therefore goes on to claim that the correct answer for p = 1/2 in fact is k² – 1 : 1. This coincides with the correct answer for a couple of small values, but then diverges exponentially from it. This leads him to make the "infinity" remark quoted above.

Parenthetically, I'm not sure why he would cube rather than square the number 20 in that example. Perhaps Ore has something intelligent to say about this.

Late 15th century book illustration showing a dice game.

How To Gamble If You Must

In Chapter 20, Cardano tells a little autobiographical anecdote as an illustration of his points about fortune, luck, etc. This story is not strictly relevant to my topic here, but it is simply to bizarre not to quote. Hence, Ladies and Gentlemen, Cardano without filter:

Yet I have decided to submit to the judgment of my readers what happened to me in the year 1526 in the company of Thomas Lezius, the patrician of Venice, leaving it to each reader to form his own opinion. I had just duly resigned from the office of rector of the scholars in the University of Padua on the third of August, and now I was journeying with Hieronymus Rivola, a scholar from Bergamo, on a certain night of the same month toward Venice. We were playing a game (called Bassette) and I won all the money he had. Then he asked me to play with him on credit, if I am not mistaken up to two or three aurei, and I won again. Then, finally, he wanted to carry it on endlessly, but I refused. He promised to pay what he owed me within three ways; he did not come.

Then he chanced to meet me and said that he would come to pay the money on Saturday (which was the day of the Nativity of the Virgin) and promised to take me to a beautiful prostitute. At that time I was just completing my twenty-fifth year, but I was impotent. Nevertheless, I accepted the condition; there was not a word about the game. He came on the day agreed; and in that year the festival of the Blessed Virgin was on Saturday. He took me to the home of Thomas Lezius; there was no Thais there, but a bearded man with a young servant. No money was paid but we played with marked cards. I lost to him all the money which he owed me, and he reckoned it as part of his debts just as though he had given it to me. I list about twenty-five aurei or even a few more which I had, and played on, giving my clothes and my rings as security.

I returned home in sadness (as was natural), especially since there was no hope of getting money from home because uprisings and plots were raging at Milan. And so (and now I tell the truth, there being no reason why I should lie), I contrived for myself a certain art; I do not now remember what it was, since thirty-eight years have passed, but I think it took its rise in geomancy, by which I kept in mind on up to twenty-four plays all the numbers whereby I should win and all those whereby I should lose; by chance the former were far more numerous than the latter, even in the proportion (if I am not mistaken) of seven to one; and I do not recall now in what order these were against me.

But when I saw that I could not safely hold more numbers in my memory, I admonished my young servant, whose name was Jacob, that when he saw I had won back my clothes and my money he was to call me. I threatened that if he did not do it I would beat him severely. He promised and we went. As the game went on I won and lost in all the plays just as I had foreseen and after the third play I realized that there was no trickery or deceit about it. They laid down money freely and I accepted the wagers, but he was delighted be the example of the previous day and also on account of the marked cards (as I have said).

Thus his thoughts were inflamed by his youthful ardor; but the result was otherwise, for, on those plays in which I saw (as it were, without any real foreknowledge) that I would win, I did not rehect any amount of money and made large bets of my own, and in the other cases, where I knew he would win, I refused if he was the first to wager, and wagered very meagerly myself: thus the result was that within twenty plays I regained my clothes, my rings, and money and also what he had added besides. As for the clothes, the rings, and a collar for the boy, I sent them home piecemeal. Out of the total number there remained four deals; I played and won, and also came out victor in a few deals which were not contained in the number.

He was already perturbed and full of admiration, since he saw that in all the plays in which we played for high stakes I came out the victor, and in those in which he won I myself wagered little and when he wished to wager a great deal I refused. So (he said) I believe some demon is advising you, or that you know by some enchantment what is going to happen. What happened after that I remember that I have narrated elsewhere. (pp. 32–34)

He also gives a bit of extra detail in Chapter 17, the chapter on fraud in card games:

There are also some who smear the cards with soap so that they may slide easily and slip past one another. This was the trick practiced upon me by the well-known Thomas Lezius of Venice, patrician, when in my youth I was addicted to gambling. (p. 27)

Extraordinary, isn't it?

Mieroop: A History of the Ancient Near East (2004)

Two pieces of information from this textbook surprised and fascinated me:

The first is that the Babylonian scribes during the Uruk period apparently used different number systems for different things (ch. 2.2). They would thus have one system for discrete objects (including human beings) based on symbols for the numbers

1/2 (or 1/10), 1, 10, 60, 600, 3600, 36000.

Another system would be used for things that would come in volume, like grain. This system was based on the unit symbols for the numbers

1/2, 1, 10, 60, 120, 1200, 7200.

A third system would be used to express areas.

What's particularly whacky about this practice is that dried fish would count as discrete objects, while fresh fish would not. Once you dried a fish, you would thus count it using a different number system.

20th century BCE list of stones, plants, fish, birds, and clothing (from the Cuneiform Digital Library Initiative).

A second this is the role of lists in the ancient educational system. There are hundreds and hundreds of so-called lexical lists remaining from the period, recording the names of, for instance, types of swine, often in several languages or with pronunciation guides.

It would seem that copying lists like this would be a core exercise for scribes during this period, perhaps also because they might not always have spoken the official language of the administration as mother tongue.

But there seems to be a particular philosophy expressed by this practice as well, a kind of "ordered universe" conception, with every object occupying its place in the great chain of being. This would might also explain certain aspects of the story of Noah, which seems to have roots in ancient Babylonia, too.