Bias in Research
Someone on Twitter once said in a reply comment, "Your entire train of thought here is unscientific, because as you clearly point out you came with a prexisting assumption and tried to justify it with data. It's good that you found an argument, but it isn't enough to accept it, you need to control for you biases."
That is wrong-headed, or at least must be taken with caveats. It's the kind of thinking the logical positivists had, and Milton Friedman with his prediction and testable hypotheses methodology too, I think, though I'd have to check. It's what taught in high schools too, as the scientific method: form a hypothesis, test it with an experiment or observation, and then form your conclusion. And it's the pure form of classical statistics: form a null hypothesis, see how closely a random sample of data fits it, if a sample that fits that badly would only be expected 5% of the time you draw samples if the null were indeed true, reject the null (but never "accept" anything).
In reality, this is unworkable and undesirable. Instead, what we do is this. I conjecture that X is true, or maybe I just want to persuade people it is true, whether I believe it or not. That's fine; I can be biased, so long as the rest of the process works out. I start trying to prove X, with theoretical equations or with real data. I find X does not quite fit the equations or data, so I change it to X', which does. Then I see if X' is interesting or useful, which it often is not because I had to get rid of the most intersetnig parts to fit reality. I present X' at seminars and conferences and journal referees tear into it. Again, I may decide X' is boring or wrong, and I modify it to X. The idea dies, or it gets published in an obscure place and disappears, or it gets published in a good place and maybe is noticed.
The virtue of this is that my bias doesn't matter. If I can't connect my idea to reality convincingly, it dies. Nobody in the scholarly world believes it just because I said it (this is different in "the real world", where people argue from authority and assertion rather than logic). I originally wrote "academic world," but that is misleading. There are many nonacademic scholarly people too, e.g. on intelligent Twitter threads and blogs, and there, too, credentials don't impress them and you have to match up your idea with reality. "Link or it's a lie."
