How most people are unable to control their biases in social science
This will be a short blog post about how scientists get to the wrong conclusions and what causes it. Unfortunately, I’d need to write 20 pages to explain all the details, but I’ll link to sources you should check out.
The replication crisis is still a fresh topic and a simple way to understand the faults in modern science. It wasn’t totally unexpected as we already knew about the big problem of p-hacking and overblown conclusions. But it is a topic that brought the faults to the surface. You may want to watch this short video on p-hacking if you haven’t heard about it.
By using p-hacking and other methods many scientists deceive and tell white-lies in their experiments to get positive results. The very interesting book, The Seven Deadly Sins of Psychology, illustrates just some of the scope of the problem on page 19:
Norbert Kerr’s survey of 156 psychologists in 1998 suggested that about 40 percent of respondents had observed HARKing by other researchers; strikingly, the surveyed psychologists also suspected that HARKing was about 20 percent more prevalent than the classic H- D method. A more recent survey of 2,155 psychologists by Leslie John and colleagues estimated the true prevalence rate to be as high as 90 percent despite a self- admission rate of just 35 percent.
Then in a meta-analysis of questionnaire studies, 14% of scientists said that they have seen a colleague falsifying results. Up to 72% said they had observed questionable research practices. Only 2% admitted to fraud themselves, but I assume a few didn’t find the courage, to tell the truth.
The Seven Deadly Sins of Psychology also explains why this happens. A researcher can only get hired and do research if she is largely successful in finding funding to conduct her research. Fund money is largely given to researchers who publish in big journals. To publish in a big journal your paper needs to be novel research with important positive results. Hence finding positive novel results is what keeps scientists in their job. Good science in itself is not something that is directly measurable.
Goodhart’s law explains what goes wrong in most labs, companies, and parliaments:
“When a measure becomes a target, it ceases to be a good measure.”
People strive to get positive numbers on the scale that is measured. This is part of the reason why we see fields like psychology produce over 90% positive results in articles even though it’s one of the fields most guided by biases and therefore should see a lot of assumptions be proven wrong. Furthermore, the test subjects are just as biased and unpredictable too so they are hard to study consistently.
The chart above shows how soft sciences work as clay and is so moldable that often results turn out positive by sheer pressure from the scientist. The abstract explains the hypothesis:
If the hierarchy hypothesis is correct, then researchers in “softer” sciences should have fewer constraints to their conscious and unconscious biases, and therefore report more positive outcomes.
You won’t get any scientists to openly admit that this is what is happening in her own lab. And even though most of this is unsaid most scientists do know about the problem and have seen it themselves. Which implies that everyone is covering over everyone to not shake the boat. They don’t want to drown so the problem is made much smaller than it actually is and famous social scientists even call people out for retesting their results without asking them first.
The “p-hacking to publish and profit” bias is found in all scientific fields as people compete for wealth and status. Unfortunately, this competition may be even greater in social science. A soft science degree does not give you a clear road to make a profit for companies. It’s hard to find a job with such loose degrees and it may make many students seek out the only jobs that are aimed at their degree: educating other people about the degree. This may cause these scientists to use dirty tricks to get ahead in this extremely competitive field. Of course, all scientists may be equally desperate for power, but just not have the same moldable field to get positive results in.
All of this does not break science fully. The boat still slowly drifts forward and we will know more in 10 years' time than we do now. The perfect sail would be the pure want for progress by discovering facts. But what we have now is better than nothing and a few people are even trying to make all scientists publish their raw data online which will revolutionize the field as p-hacking and fake data will be much harder to hide.
Social science
Here comes the scary thing. Social science is levels worse than any other academic field. While most fields could replicate something like 50% of their big studies in social psychology only 25% of studies replicated. I suggest you read the Wikipedia article on it. I also suggest you watch this Youtube series on it though it may be a bit progressive political at times.
How is such low replicability even possible? Is it even science at that point? Well, much of social science is not a science as such but functions more like a cultural doctrine. And many soft science professors even admit to that and claim that the scientific method is just one way to do science. There is a social constructivism thinking in social science fighting against the scientific method. But how did it even come about? I will present part of the thinking that leads to terrible results and makes whole scientific fields unscientific.
The main issue is the inborn need to belong
The main issue with the scientific method is that it’s not fully natural for us. We are created to solve problems in our natural environment on the spot and that may complicate how we view stats and data and especially how we view topics about humanity itself as these topics make us emotionally engaged. This feeling of needing a certain result is great to solve those social problems in a split moment. But as these emotions activate while we discuss social science research they will kickstart our emotional thinking and lead us astray.
I will go over 3 of the main issues that make social science produce worse results than hard science. There are many more issues to explore, but I don’t want to make this too long.
- Fighting your instincts which is extremely hard and nearly impossible for most people. Especially for emotional topics.
- Learning logical ways to get to solutions. This demands great brainpower and a great narrow focus that STEM people have in a higher degree. It’s not that widely applied in social science as it’s too hard to learn for most people.
- Accepting hate from people when you fight your instincts and therefore come to objective conclusions instead of following the subjective moral opinions of the group.
1. Fighting your instincts
First step in getting anywhere in understanding yourself and humans overall is to look at this emotional stuff from the outside in. There are several huge things holding most people back from looking into the brutal and “unfair” truth of things even if we try to. To understand our biased thinking we need to understand what problems our thinking is created to actually solve.
Just like chimpanzees groom each other to create a stronger bond we have the same need. We want to help people out because it will make us seem as good and productive people in the group and therefore gain us status and goodwill. People with an extra strong need to groom their ingroup and gain favors this way also have a stronger need to work in fields requiring these emotional tools. They have a need to understand themselves and their social circle a bit better to then live a more fulfilling life. It’s not a want to learn about facts of life. It’s a holistic inner need to belong that drives this curiosity and it may as well lead people to study astrology, pseudo personality theories or shallow therapy tools. It’s a want to make the world a better place by learning secrets about human beings and fixing some of the “wars” in the world. Just like chimpanzees keep the group calm and friendly by practicing grooming. So these human-focused sciences are peculiar as they don’t always have a direct practical and measurable motivation behind them. An engineer, on the other hand, may just want to develop a better flashlight to help people out in the dark and then make a profit that way.
The grooming need also carries with it a moral leadership need to sustain the helpful group so that we can keep helping it (put resources into it) and then seek help in it (extract owed resources). These people have great sympathy for suffering people and spot them easily. This also gives them an extra keen sense for spotting anyone who makes other people suffer mentally by breaking any moral code. They are very eager to make everyone around them follow the grooming rules so that their system of help will grow as big as possible. Then on top of that we all have a bunch of proactivity biases that make us see ourselves as better and more worthy of praise and resources. In the extremes these 2 things combined can turn into a complex that creates a cult leader or a politician who applies forceful extreme political ideologies to a mass of people causing mass suffering. These extreme people will use extreme mass grooming to entice the mass by great vague emotional promises and then end up needing to destroy and kill to remain in power. But even at the less extreme level of teachers or writers, it may create great harm when the natural need to help out becomes an extreme need to fix a large group of people you have never even met by sheer force.
In conclusion, in our day to day life we use complicated automatic instincts to solve group problems. We are not able to clearly see these instincts, but we feel them holistically. We feel that we just need to develop these instincts and see social science through this point of view to also solve scientific problems. But these instincts hurt much more than they help as many social science results are counterintuitive and go against our very basic moral belief about life. A belief where we see ourselves as the ever so helpful Messiah in the center of the universe.
In some ways it’s easier to get a physics P.hD. than to be a critical social scientist. Because to get a P.hD. in physics you just need to be born extremely smart and hardworking. To understand social science you need to… not to be. You’ll need to not to try to solve problems, but just let yourself explore without judging first. That’s not really something humans are born to do for social issues. We don’t have an off switch for social instincts. You may be lucky and be born on the autistic scale with weaker social instincts. It will leave you misguided and lonely in life, but having weaker social tools also makes you able to see social theories more clearly and directly. The foggy view is usually there to help and guide us and calm us down. As we see things clearly we suffer in the lonely meaningless of life. But it’s a state good and stubborn explorers seek out.
2. Discovering logical tools to come to correct conclusions
So how does your social science logic currently work? You use your social instincts to solve issues in your family, friend group and maybe help out in small quarrels around you. It’s automatic behavior guided by feelings. If something feels right and good you assume it’s the right action. A friend needs a shoulder to cry on? You lend them a shoulder continuing a natural grooming behavior in your social group unconsciously making sure that you can profit from it later on.
Meanwhile to understand theories in science you need a logical mindset. It’s easier said than done. People in STEM usually have high IQ’s and learn to think logically and critically by learning advanced math and programming. In social science, the average student has a much lower IQ and a stronger social thinking creating stronger biases.
The logic of math is essential for all scientific fields. You learn proper social science by studying other scientific fields and understand how they look into problems and find correlations and causations. Because this way you will find topics that don’t make your emotions cloud logic. This is why everyone who wants to study social science should first and foremost study logical thinking and then subjects such as economics and personal psychology based on quantitative studies. Later on when you are a logical thinker in some other field you can dabble in social science and fail a bit less than most other people.
In conclusion, don’t just assume great things about emotional topics. Read and learn. Then slowly develop a scientific point of view that will go on top of your moral point of view. Both will coexist, but you will learn to lock out the moral point of view when you talk about science or want to be logical and actually solve big complicated problems long-term. If you just talk with friends and family you may want to just use that instinctual moral thinking that will put you in the center of this small universe as a grooming savior.
3. Accepting conclusions that make you worse off
In hard science you can come to a conclusion that goes against the norm and people may just see you are semi-crazy, but not hate you for it unless you challenge cultural social norms. In social science most research will challenge some cultural social norms in some groups. And as I pointed out before there is an extreme need to groom in social science. This need creates a very strong coercive cultural norm that people try to uphold and defend to be able to consistently gain resources from the group. It’s a norm of not wanting to hurt any feelings. A need to constantly be helpful and especially help out suffering people. The need to help out creates a very narrow frame in which all social science must take place as anything outside the frame may hurt feelings and cause the harm that these people are desperate to avoid. And followers will also learn to be obeying soldiers and guard these barriers. So there is no social science without ideology. A mass campaign is created to keep social science chained down as the free market of ideas is stopped and progress is slowed down to a halt. Why progress something that functions based on million years old instincts? You can’t. You can just describe what you feel holistically and keep doing what you feel you want to do.
All this means that if an outsider learns to understand hard to get social science concepts that go against the grooming people in the field they will dislike him and his thinking for breaking their moral doctrines. They would also counter his conclusions by stating that the scientific way is just one way to look into things and that the holistic moral intuition they personally have is more important in their field.
Even so, there is a considerable amount of people who do achieve to fight or ignore their instincts with the help of the few good objective books on these topics. And there are very few of them still. There are also a lot of people who take it a step further and actually learn about advanced social science concepts by applying more advanced logic. All of this creates some great inner conflict but keeps the person largely in the same situation in life. But after achieving all of this most of these clever and critical people are still not be able to take the very last step. They can’t handle the great hate/distrust they may receive for it. Being hated is one of the worst fears we have as we depended on our band to survive in our natural environment. It could mean certain death to get thrown out from the band. It feels horrible to lose friends and make enemies. And then there may be the loss of a potential job and loss of other opportunities to influence the world and earn resources.
If you have a close family that respects you no matter what, you may have the emotional foundation to do what is right and fight bad thinking while receiving hate. But you still need to get out into the world and earn a profit for your family. And besides that, you need to protect them from the outside hate. It’s just easier to not tell the full truth even if you have done all the stuff you needed to do to uncover it. There are plenty of people who try to speak out, but then get attacked and learn to keep quiet. They may even backtrack and apologize to not lose out on opportunities in life.
Conclusion
Obviously I would suggest just studying facts by looking into numbers and avoiding all that social tam-tam. But I don’t think most people will even get close to achieving this. Why pay the price when you can remain rich but blissfully ignorant?
The goal is not to get all people to take a physics degree or learn basic social science. That’s not possible as most people just don’t have the inborn tools needed for this. Learning more social science will just make them more ignorant as they will apply their ever so helpful social instincts to solve scientific problems and just create even bigger biases. But if we try to talk sense we may get just a few more people to learn correct things about the wold and then speak about them convincingly so that politics and leadership may get better for all of us.
So how do you notice this problem and fix it? Firstly, what is your goal? If your motivation is to be seen as honorable and sympathetic then that will be what drives your thinking about theories. You’ll just dream about being a great politician or superhero and then try to find solutions to this want.
Your goal is to help suffering people = you will be a bad social scientist
Let me give you an example. I once saw this counter-argument to a study on gender differences: “I don’t believe this. I think women should be able to do anything men do.” It’s a bias that makes you unable to see results that you consider “evil”. The only way to conquer this bias and become more knowledgeable about life is to not let the biases guide you. Think about what you want in life and how your ideal group and society should look like. Then try to see if that biases your thinking. If it does you may be a horrible logical thinker. But then how should you think about goals for society?
What you need to do is to let all people and groups suffer and die in your head if the theory leads you there. You don’t need to practice this in real life. But it needs to be how you think about people, they are just objects you can study and count. If you support a cause you will search for evidence to prove your points. And that evidence will ultimately not really make your group better off as it will be p-hacking for results that work mentally, but won’t work practically. If you really want to help you need to have a need to uncover the true problem no matter how cruel and ruthless it may appear. Good scientists learn to avoid anecdotes, personal experience, personal emotions, and bad studies based on these things. Then when they need to convey their objective information to laymen they use anecdotes as that is the way we naturally communicate knowledge. Laymen will not readily believe this stuff, but this is why scientists need to be great storytellers too.
Your goal is to let nature sort things out = you will be a good social scientist
The goal should not be to help suffering groups out but to get to the truth. The problem with having an extreme need for grooming is that it becomes the hammer and everything is then seen as a nail. All issues and solutions are seen as social issues. Inner suffering is then blamed for lack of grooming and some powerful group with lots of resources will be blamed for not grooming enough people. Bad social scientists will tell themselves: “Surely my great inner instincts for grooming tell the truth about the world! All problems in the world are solved by putting me in power and applying my theories.” This, of course, is faulty illogical thinking. Life is not a zero-sum game. Powerful people have often created extreme wealth for millions of people. Of course, people still suffer in this rich society if we are not all fully equal. This is because our suffering is largely relative to other people and having a small family, a small house and a fine job may still feel like hell if our neighbor runs a big charity. We may also feel hopeless about helping this mass of people. Grooming works great 1-to-1. And that’s as good as it gets. While hard science cures sickness and alleviates suffering every year on a mass scale. Wanting to help a mass group of people with our small grooming instincts will lead to mass suffering as these instincts are only made to solve minuscule problems. Having this need will just create more pain in yourself as you will never really understand why your great motivation is never fulfilled. And worse of all it will create suffering around you as you will keep hurting people by trying to help them. And this makes me sad.
