Skip to main content

Lies, damned lies, and frequentist statistics
Lies, damned lies, and frequentist statistics

Earlier this year I rekindled an interest in probability theory. In my classes, Bayes' theorem was little more than a footnote, and we drilled frequentist techniques. Browsing a few books led me to question this. In particular, though parts of Jaynes' "Probability Theory: The Logic of Science" sounded like a conspiracy theory at first, I was soon convinced that the author’s militant condemnation of frequentism was justified.

Today, I had the pleasure of reading a Nature article about a paper by Valen E. Johnson directly comparing Bayesian and frequentist methods in scientific publications, who suggests the latter is responsible for a plague of irreproducible findings. I felt vindicated; or rather, I felt I had several more decibels of evidence for the hypothesis that Bayesian methods produce far better results than frequentist methods when compared against the hypothesis that the two methods produce equivalent results!

This post explains it well. In short, frequentist methods have led to bad science.

An apologist might retort that it’s actually the fault of bad scientists, who are misusing the methods due to insufficient understanding of the theory. There may be some truth here, but I still argue that Bayesian probability should be taught instead. I need only look at my undergraduate probability and statistics textbook. On page 78, I see the 0.05 P-value convention castigated by Johnson, right after recipe-like instructions for computing a P-value. If other textbooks are similar, no wonder scientists are robotically misapplying frequentist procedures and generating garbage.

Johnson’s recommended fix of using 0.005 instead 0.05 is curious. I doubt it has firm theoretical grounding, but perhaps the nature of data that most scientists collect mean that this rule of thumb will usually work well enough. Though perhaps striving for the arbitrary 0.005 standard may require excessive data: a Bayesian method might yield similar results with less input. I guess it’s an expedient compromise. Those with poor understanding of statistical inference can still obtain decent results, at the cost of gathering more data than necessary.

The above post also mentions a paper describing how even a correctly applied frequentist technique leads to radically different inferences from a Bayesian one. The intriguing discussion within is beyond me, but I’m betting Bayesian is better; or rather, the prior I’d assign to the probability that Bayesian inference will one day shown to be better is extremly close to one!



Popular posts from this blog

"The West Does Not Understand the Extreme Danger from Its Arming of Ukraine The Idiot Biden Regime and Insane Neoconservatives Have the World on the Path to Armegeddon"

Paul Craig Roberts wrote: How Many More Red Lines Can Be Crossed Before Armageddon Arrives? Paul Craig Roberts The Organization for Security and Co-operation in Europe published on April 15, 2016, more than six years ago, a report on the torture of Donbass Russians by the Ukrainian military and police forces.   The report documents horrendous torture and it was done out of racial hatred of Russians. You can read the report here:     Few Westerners, being so poorly educated, are aware that Western Ukraine fought for Nazi Germany during World War II.   When Washington overthrew the Ukrainian government in 2014, Washington installed a Nazi government. The Nazi government in Ukraine shelled the Russian residents of the two break-away Donbass republics for 8 years while Washington and NATO trained and equipped a Ukrainian army to retake the breakaway republics. .. ... These statements from Russian leaders indicate that the I

Vox Cantoris vs. Aqua

The Catholic Monitor commenter Aqua had this to say to the Vox Cantoris website: Aqua said… Fred, your topic here reminds me of a dust-up, a few days ago, on Vox Cantoris. He asserted that it is our duty as Christians to wear masks to the Holy Sacrifice of the Mass if the government tells us we must, or they will close our Churches. My response to him was that I find it inconceivable that an orthodox Catholic, such as himself, would ever submit to unjust dictates from secular government over how we approach Our Lord in Holy Mass. My response to him was that the Mass belongs to Catholics and we decide, within the bounds of Tradition, and in accord with the Word of Jesus, how we conduct ourselves in Holy Mass. Only one authority prevails over Mass and that is our God and the Sacred Tradition given by Him to guide us in all times and places. Understand, there is nothing inherently wrong with wearing a mask to Mass. But there is EVERYTHING wrong with wearing a symbol

Mainstream Media: "Is America the Real Victim of Anti-Russia Sanctions?" & "Biden's arrogant anti-Russian sanctions have amounted to a price hike on working class Americans that have so far failed to weaken the Russian economy"

  Mainstream Media Acknowledges Biden’s “Arrogant” Sanctions On Russia Are Damning Americans:     @MaxBlumenthal Biden's arrogant anti-Russian sanctions have amounted to a price hike on working class Americans that have so far failed to weaken the Russian economy. His neocon policy accelerates the process of de-dollarization, diplomatic isolation & imperial decline. The mainstream new outlet asked "Is America the Real Victim of Anti-Russia Sanctions?": Remember the claims that Russia’s economy was more or less irrelevant, merely the equivalent of a small, not very impressive European country? “Putin, who has an economy the size of Italy,” Sen. Lindsey Graham, R-S.C., said in 2014 after the invasion of Crimea, “[is] playing a poker game with a pair of twos and winning.” Of increasing Russian diplomatic and geopolitical influence in Europe, the Middle East, and East Asia, The Economist asked in 2019, “How did a country with an economy the size of Spain … ach