Skip to main content

The “frequentist” will counter they only have to test the X they believe are relevant. But then they have to say how the winnowed down the infinite list to choose the “relevants”. How did they do this? It can’t be by testing. So there must exist procedures that are not testing that allow picking Xs and excluding Xs. Read that over and over until it sinks in.

https://www.wmbriggs.com/post/40373/

Spurious Correlations Proves Hypothesis Testing Should Be Abandoned

This week traditionally is a slow week on the blog, so let me have a go at explaining something I’ve explained a few hundred times before, a thing which has not yet stuck. Maybe the pace of the day will help us.

I enjoy collecting statistics (I use that word in its old-fashioned sense) like this:

Well, you tell me: do sunspots cause war? Would a “null hypothesis” significance test produce a wee p, thus “proving” the cause? Or not proving the cause, because we all know correlation isn’t causation, so proving the correlation instead. Which didn’t need proving, because the correlation is there, and is proof of itself. A large p doesn’t make what is there disappear.

So the wee p doesn’t prove causation and it doesn’t prove correlation. What does it prove?

Nothing.

As I have quoted more times than I can recall, De Finetti in 1974 shouted “PROBABILITY DOES NOT EXIST.” Most of his audience didn’t understand him, and most still don’t.

If probability doesn’t exist, any procedure or mechanism or measure that purports to tell you about probability as if it is a thing that exists must therefore be wrong. Right?

P-values are measures, hypothesis tests are procedures, and some obscure machines (used to measure statistics) all purport to tell you about probability as if it is a thing that exists. Since probability does not exist, they are all therefore wrong.

Right?

That’s the true, valid, and sound proof against p-values. But it doesn’t stick. Many still have the vague belief that, yes, probability does exist, therefore there are “right ways” or “good ways” to use p-values. This is false. So let’s use a deeper proof. Here is where you have to pay attention.

We’ve seen the site Spurious Correlations many times. Here’s one of his plots, “US spending on science, space, and technology” with “Suicides by hanging, strangulation and suffocation.”

It’s clear that this correlation would give a wee p, or that it’s easily extended so that it will. But we all recognize it’s a silly correlation.

Why?

Because we bring in outside information that says there is no way one of these things would cause the other. Simple!

Problem is, there is no way to fit in “outside” information in frequentist theory. In old-fashioned, real old-school, non-frequentist, non-subjective probability, probability as logic, we gather all information we believe is relevant—and even more importantly, we ignore all evidence we believe is irrelevant—and then use it to judge the probability of some proposition. Here the proposition is that spending causes suicides, or suicides cause spending.

In notation: Pr(Y|X), where Y is the proposition and X all information we deem relevant—and the absence of all information we deem irrelevant.

What should surprise you is that there is no mechanism to do this in frequentism, which is the probability theory that treats probability as if it is a real thing.

In frequentism, the criterion for including or excluding evidence is the “significance” test. Stick with me here, because here’s the subtle part. There are always (as in always) an infinite number of premises or evidence that we can add to the “X”, in frequentism or in logical probability.

There is no guarantee, of course, that in any logical probability assessment, we have chosen the “right” X—and indeed the right X is that one that gives the full cause of Y (so that Pr(Y|right X) = 1). But we are free (and this is the semi-subjective nature of probability and decision) to pick what we like. When we like. Before taking data, after, whenever. And then asking what is Pr(Y|X)?

But there is no way to pick the “right” X for frequentism without subjecting every potential element of X to a significance test. Since there are an infinite number of X for any Y, and since hypothesis testing acknowledges spurious correlations can appear as legitimate (if the p is wee), then either every problem if carried out in strict accordance with frequentist theory will have an infinite number of spurious correlations, or the test will “saturate” at some point and the analysis has to end incomplete.

By “saturate” I mean the significance testing math breaks down with finite n: after adding too many premises, the testing can no longer be computed. I won’t here explain the math of this, but all statisticians will understand this point.

There thus can be no genuine frequentist result, not one entirely consistent with the theory, unless the theory is abandoned at some point. And that point—which always comes—is when the analyst acts like a logical probabilist and excludes certain X.

As it is, the “frequentist” (the scare quotes indicate there is no true or pure such creature, but only approximations) will conduct “tests” on such X he picks, and ignore the tests for X he excludes. All “frequentists” are only partial frequentists. And even more partial than you think.

Here, with spending and suicides, even with a wee p, most “frequentists” would again reject the rejection of the “null”—that is, toss out the wee p—because other X are brought in from which it deduced the correlation is silly.

The “frequentist” will counter they only have to test the X they believe are relevant. But then they have to say how the winnowed down the infinite list to choose the “relevants”. How did they do this? It can’t be by testing. So there must exist procedures that are not testing that allow picking Xs and excluding Xs.

Read that over and over until it sinks in.

So why not use these other procedures all the time? Or you have to show us exactly precisely what is this strange inherent measure or procedure that says when to use testing, when to not to.

These questions are never answered. At most, we will hear “Well, there are still some good uses for p-values.” To which I answer: no there aren’t, and we just proved it.

Buy my new book and learn to argue against the regime: Everything You Believe Is Wrong.

 

Comments

Popular posts from this blog

Vox Cantoris vs. Aqua

The Catholic Monitor commenter Aqua had this to say to the Vox Cantoris website: Aqua said… Fred, your topic here reminds me of a dust-up, a few days ago, on Vox Cantoris. He asserted that it is our duty as Christians to wear masks to the Holy Sacrifice of the Mass if the government tells us we must, or they will close our Churches. My response to him was that I find it inconceivable that an orthodox Catholic, such as himself, would ever submit to unjust dictates from secular government over how we approach Our Lord in Holy Mass. My response to him was that the Mass belongs to Catholics and we decide, within the bounds of Tradition, and in accord with the Word of Jesus, how we conduct ourselves in Holy Mass. Only one authority prevails over Mass and that is our God and the Sacred Tradition given by Him to guide us in all times and places. Understand, there is nothing inherently wrong with wearing a mask to Mass. But there is EVERYTHING wrong with wearing a symbol

Inventor of mRNA Vaccines, Robert W. Malone MD: "'Archbishop Carlo Maria Viganò is calling for the creation of an Anti-Globalist Alliance 'under the Cross of Our Lord Jesus Christ, the only King and Savior,' to fight back against the establishment of the New World Order'"

T his is what the inventor of mRNA vaccines, Robert W. Malone MD himself, reported about the vaccines: Robert W Malone, MD @RWMaloneMD "We conclude that the mRNA vacs dramatically increase inflammation on the endothelium and T cell infiltration of cardiac muscle and may account for the observations of increased thrombosis, cardiomyopathy, and other vascular events following vaccination." [https://mobile.twitter.com/RWMaloneMD/status/1462194727407730693]   Moreover, he goes even further and is quoting Archbishop Carlo Maria Viganò's call for the creation of an Anti-Globalist Alliance:   Robert W Malone, MD @RWMaloneMD "Archbishop Carlo Maria Viganò is calling for the creation of an Anti-Globalist Alliance “under the Cross of Our Lord Jesus Christ, the only King and Savior,” to fight back against the establishment of the New World Order." youtube.com Abp. Viganò calls for Anti-Globalist Alliance to stop global enslav... Archbishop Carlo Maria Viganò is calli

"The West Does Not Understand the Extreme Danger from Its Arming of Ukraine The Idiot Biden Regime and Insane Neoconservatives Have the World on the Path to Armegeddon"

Paul Craig Roberts wrote: How Many More Red Lines Can Be Crossed Before Armageddon Arrives? Paul Craig Roberts The Organization for Security and Co-operation in Europe published on April 15, 2016, more than six years ago, a report on the torture of Donbass Russians by the Ukrainian military and police forces.   The report documents horrendous torture and it was done out of racial hatred of Russians. You can read the report here:   https://www.osce.org/files/f/documents/e/7/233896.pdf     Few Westerners, being so poorly educated, are aware that Western Ukraine fought for Nazi Germany during World War II.   When Washington overthrew the Ukrainian government in 2014, Washington installed a Nazi government. The Nazi government in Ukraine shelled the Russian residents of the two break-away Donbass republics for 8 years while Washington and NATO trained and equipped a Ukrainian army to retake the breakaway republics. .. ... These statements from Russian leaders indicate that the I