Skip to main content

Lies, damned lies, and frequentist statistics

https://benlynn.blogspot.com/2013/11/lies-damned-lies-and-frequentist_12.html
Lies, damned lies, and frequentist statistics

Earlier this year I rekindled an interest in probability theory. In my classes, Bayes' theorem was little more than a footnote, and we drilled frequentist techniques. Browsing a few books led me to question this. In particular, though parts of Jaynes' "Probability Theory: The Logic of Science" sounded like a conspiracy theory at first, I was soon convinced that the author’s militant condemnation of frequentism was justified.

Today, I had the pleasure of reading a Nature article about a paper by Valen E. Johnson directly comparing Bayesian and frequentist methods in scientific publications, who suggests the latter is responsible for a plague of irreproducible findings. I felt vindicated; or rather, I felt I had several more decibels of evidence for the hypothesis that Bayesian methods produce far better results than frequentist methods when compared against the hypothesis that the two methods produce equivalent results!

This post explains it well. In short, frequentist methods have led to bad science.

An apologist might retort that it’s actually the fault of bad scientists, who are misusing the methods due to insufficient understanding of the theory. There may be some truth here, but I still argue that Bayesian probability should be taught instead. I need only look at my undergraduate probability and statistics textbook. On page 78, I see the 0.05 P-value convention castigated by Johnson, right after recipe-like instructions for computing a P-value. If other textbooks are similar, no wonder scientists are robotically misapplying frequentist procedures and generating garbage.

Johnson’s recommended fix of using 0.005 instead 0.05 is curious. I doubt it has firm theoretical grounding, but perhaps the nature of data that most scientists collect mean that this rule of thumb will usually work well enough. Though perhaps striving for the arbitrary 0.005 standard may require excessive data: a Bayesian method might yield similar results with less input. I guess it’s an expedient compromise. Those with poor understanding of statistical inference can still obtain decent results, at the cost of gathering more data than necessary.

The above post also mentions a paper describing how even a correctly applied frequentist technique leads to radically different inferences from a Bayesian one. The intriguing discussion within is beyond me, but I’m betting Bayesian is better; or rather, the prior I’d assign to the probability that Bayesian inference will one day shown to be better is extremly close to one!

 

Comments

Popular posts from this blog

Bioweapons Expert Dr. Francis Boyle On Coronavirus

March 13, 2020              https://greatgameindia.com/transcript-bioweapons-expert-dr-francis-boyle-on-coronavirus/ : A recent interview with Bioweapons expert Dr. Francis Boyle published by GreatGameIndia and conducted by Geopolitics & Empire , has been exploding across the world the past few days as the truth is emerging on the origins of the Coronavirus Bioweapon . Francis Boyle is a professor of international law at the University of Illinois College of Law. He drafted the U.S. domestic implementing legislation for the Biological Weapons Convention, known as the Biological Weapons Anti-Terrorism Act of 1989, that was approved unanimously by both Houses of the U.S. Congress and signed into law by President George H.W. Bush... ... Dr. Francis Boyle:   All these BSL-4 labs are by United States, Europe, Russia, China, Israel are all there to research, develop, test biological warfare agents.  There’s really no legitimate scientific reas

Former advisor to Sec. of Defense, Col. Douglas MacGregor: "the next phase of the Ukrainian War will not only destroy the Ukrainian state. It will also demolish the last vestiges of the postwar liberal order"

Macgregor appears to cite intel sources for his numbers and timetable—and I assume he really does have good sources. He says the offensive will happen “not sooner than 12/10 and not later than 12/19.” It will involve something like 540,000 troops with 1,500 tanks, thousands of other armored vehicles, huge numbers of helicopters, fixed wing aircraft, and bombers. These are numbers he says that have been confirmed by US intel. - Mark Wauck [https://meaninginhistory.substack.com/p/doug-macgregors-timetable-and-scale?utm_source=post-email-title&publication_id=473679&post_id=89073832&isFreemail=true&utm_medium=email] Modern Diplomacy reports that  "Douglas Macgregor , Col. (ret.), who was the former advisor to the Secretary of Defense in the Trump administration" says that  "the next phase of the Ukrainian War will not only destroy the Ukrainian state. It will also demolish the last vestiges of the postwar liberal order": The Biden administration repeated

Mainstream Media: "Is America the Real Victim of Anti-Russia Sanctions?" & "Biden's arrogant anti-Russian sanctions have amounted to a price hike on working class Americans that have so far failed to weaken the Russian economy"

  Mainstream Media Acknowledges Biden’s “Arrogant” Sanctions On Russia Are Damning Americans:     @MaxBlumenthal Biden's arrogant anti-Russian sanctions have amounted to a price hike on working class Americans that have so far failed to weaken the Russian economy. His neocon policy accelerates the process of de-dollarization, diplomatic isolation & imperial decline. The mainstream new outlet asked "Is America the Real Victim of Anti-Russia Sanctions?": Remember the claims that Russia’s economy was more or less irrelevant, merely the equivalent of a small, not very impressive European country? “Putin, who has an economy the size of Italy,” Sen. Lindsey Graham, R-S.C., said in 2014 after the invasion of Crimea, “[is] playing a poker game with a pair of twos and winning.” Of increasing Russian diplomatic and geopolitical influence in Europe, the Middle East, and East Asia, The Economist asked in 2019, “How did a country with an economy the size of Spain … ach