[0:01]I've had a lot of conversations recently about what's going wrong with scientific research. And that's a good thing. It's good we're talking about it, though I'm a little surprised it turned out to be so controversial. But this made me realize that I've confused myself and potentially everybody else by mixing together a bunch of different things. The most visible problems in scientific research are in some sense the least important ones. I mean misconduct and fraud. These cases make a lot of headlines but they're rare. Like we had this honesty researcher at Harvard who was accused of having faked survey responses. We have Ranga Dias, who was accused of having faked super conductor measurements and several other prominent examples. And yes, this is terrible, but you'll find some rotten eggs in any profession. There are circumstances that may make this more likely and I wonder whether science has such circumstances, but ultimately I think there's no way to avoid it. The second problem with scientific research is far less visible: it's organized scams. These are becoming an increasing problem. These scams include so-called paper mills that are networks of people who sell off paper authorships or citations for money, but also just networks of pseudo scientists who crank out fake papers with fake data and fake images. The methods these people are using are becoming more and more sophisticated and include things like planting fake papers online to generate profiles of imaginary researchers, letting AI write their papers and generating images to try and attract citations by quoting themselves on Wikipedia. This used to be a thing which happened predominantly in some Eastern countries, notably China and India, but it's been spreading West in the past years with cases showing up in Europe and the Americas. We'll likely see more of this with AI becoming better. And this is a growing problem, but it's not the major problem I worry about, at least not yet. It's however indicative of the much bigger problem, because you can ask, and I think you should ask, why do people do this? Why do people buy authorships and citations to pretend being scientists? What's the point? The answer is quite simple: because it's a good investment. They spend money on fake papers with fake citations and then they can make more money by using this fake research to get grants or find a well-paid job. The much bigger problem that I'm worried about is that even if the research isn't outright fake, the incentive still exists and they're still pushing researchers into the wrong direction. That is economic pressure encourages researchers to publish papers that get cited. And so they develop strategies to make that as simple as possible, which is not by doing good science. The easiest way to publish papers that get cited is to create useless garbage that the public doesn't understand or doesn't care about and that their colleagues approve of, but that ultimately benefits no one besides scientists themselves. This works particularly well if no one besides the people in the field feels competent to judge what they're even doing. When I say they develop strategies, I don't mean they do this deliberately. It's not like they sit down and decide to do useless research for the rest of their life just to continue getting paid. It's that the way that the system is organized. This is the winning strategy. The winning strategy in science is to be useless, isn't the winning strategy to, you know, make a revolutionary breakthrough? Yes, but to do that you have to do research in the first place and where do you get the funding for that? Right. And so scientists come to think of useless paper production as a necessary evil on the way to a breakthrough that never happens because in the end all they do is produce useless papers, which of course, they'd never admit. Scientists will defend their useless research as allegedly normal and to be fair, this has been going on for so long that in some sense, it has indeed become normal. This is why the return on investment in science has gone down. They think it's fine that way, they don't want to change it, they want you to believe it's fine and most people believe it. With all this complacency, it'd be surprising if progress had not slowed down. In my interactions with scientists, I've gotten the impression that almost everyone is aware of this problem. It's possible, of course, that my own experience is biased for one reason or another, biased towards wine drinking cheese eaters and away from beer drinking pizza people. But that doesn't make my experience go away. Scientists constantly have pressure to produce more papers, to publish them in high impact journals and to get their paper cited. And they know what is the sort of paper they'll get published and cited, and that's what they work on. I've seen tenured professors advising younger researchers to work in areas where many other people work because that's the only way to get cited. That's what causes scientific bubbles. I've seen department heads reprimanding staff for not publishing enough multiple times. I've been criticized for not publishing enough myself. I know that researchers are now being pressured to publish more at earlier and earlier stages in their career because everyone else is doing it. This is a race to the bottom and it's happening as we speak. I've heard faculty members admitting that the relevant section of their grant proposals is made up nonsense more than once. They justify it to themselves and others by arguing that one can never predict what'll be useful anyway, same old argument. I have myself received offers to work on theories that were throwing together two or more already speculative ideas because that'd be a fast paper and get published, not because it made any sense. Sometimes I've done it. I've talked to many researchers who knew that the topic they work on has no scientific relevance, but who tell themselves and their colleagues that they do it because it pays the bills. And on the side, they have a project that really matters to them. And I've heard senior researchers advising younger ones to do exactly that, to swim with the mainstream to finance something else. That isn't per se bad advice, it's just that these side projects basically never go anywhere just by lack of time. And I know many researchers who made up research topics just because there was funding in a particular area. This is why I say it's a planned economy. Most of them would never admit any of this in public. You sometimes hear Nobel Prize winners speak out about it, because once you have a Nobel Prize, you're basically untouchable and no one will question you. If a German YouTuber talks about these problems, then that's highly controversial. In the literature it's been called the natural selection of bad science. Basically, the problem is that bad science is easier than real science. So if no one checks the use of the research, the bad science will take over and that's what we're seeing. I want to stress this again, I think that most of the people who work in academia don't see anything wrong with what they're doing. Because they've been taught that whatever they're doing is standard procedure, everyone does it and they can't change it anyway. It's just the system. Most of them also buy into the idea that it doesn't really matter what they work on because you never know if not maybe one day it'll turn out to be useful. The problem presents itself differently in different fields and naturally, I know most about the foundations of physics. Here the problem has taken the form of what I call mathematical fiction. Jim Baggot has called fairytail science and what Arvin Vihle has called mathematical gymnastics. It's why Ellis and Silk have argued that one should designate an area of mathematical cosmology to defend the integrity of physics and that was 10 years ago. You know what has happened since is that we now have more mathematical cosmology than ever before. In other research areas the problem presents itself differently. For example, in psychology and some parts of sociology, they had a big problem with using flawed measures of statistical significance. This has literally been going on for decades despite the fact that the problem was widely known. There's an interesting statement from 2016 from Jessica Utz, who was then the President of the American Statistical Association, who wrote statisticians and other scientists have been writing on the topic for decades.
[10:03]You must ask them if the problem was so widely known for so long, why didn't psychologists and sociologists do anything about it? Because that would have been effort and that would have made it more difficult for them to publish papers. So of course they didn't do it until they really, really had to because it attracted so much public attention. Now, in psychology an interesting thing has happened, which is that psychologists went and tried to understand what caused the problem. And they correctly identified the cause of the problem in a lack of self-correction in the community. This self-correction didn't happen because they all thought it was okay, they were just doing what they'd learned. And psychologists have since tried to clean up the mess by trying to enforce certain standards. Physicists lack this ability for self-reflection. There are similar problems in many other areas of science like biomedicine, there are some wild stories about they've been using mislabeled cell lines for decades or antibodies that aren't what they're supposed to be or the widely known shortcomings of mouse models that is tests on mice that are known to rarely carry over to humans. So why do they keep using them? Because mice are cheap, easy to use and everyone else is doing it. I learned all this from Richard Harris's book, Rigor Mortis. Why do these things keep happening? He quotes social scientist Brian Martinson. Most people who work in science are working as hard as they can. They are working as long as they can in terms of the hours they're putting in. They're often going beyond their own physical limits and they're working as smart as they can. And so if you're doing all those things, what else can you do to get an edge, to get ahead, to be the person who crosses the finish line first? All you can do is cut corners, that's the only option left to you. I think this broken incentive structure in academia is why scientific progress has slowed down so much as I discussed in an earlier video. It's not that it stopped, but we've seen a declining return on investment. The reason for this, I think, is that we're paying a lot of researchers who no longer care about being useful to the society that pays them. They think it's okay to be useless and they'll even defend their uselessness. Let me be clear that when I talk about useful research, I don't necessarily mean technological applications. Knowledge per se also has a use. That the current organization of scientific research is extremely inefficient has also been studied by economists. The best book I know about this comes from Paula Stephan. She points towards a variety of problems like that the current system exploits PhD students and postdocs as cheap and expandible labor to produce more papers and bring in money for universities, but the key problem she points out is that the system strongly discourages risk-taking and this is why there's so little progress. She writes:
[13:33]The system that has evolved discourages faculty from pursuing research with uncertain outcomes. Lack of success can mean that one's next grant will not be funded. Proposals that do not look like a sure bet may be hard to get funded in the first place. To quote the Nobel Laureate Roger Kornberg, if the work that you propose to do isn't virtually certain of success, then it won't be funded. Risk avoidance is particularly acute for faculty on soft money. As Stephen Quake, a Stanford Professor of bioengineering says, the rubric for today's faculty has gone from publish or perish to funding or famine. The book is from 2012. That the incentive structure in academia is fundamentally broken, isn't a new insight, of course. People have discussed this already when I was a student. That was 30 years ago. This tells you how hard the problem is to solve. But despite the enormous relevance of the topic, it's admittedly somewhat of a niche interest of mine. I think that most scientists are not aware the problem has been studied and documented so much. The reason you don't hear much about it is that the scientists you're most likely to hear of are the top researchers at top institutions who don't feel most of the pressure in academia. Those are the 0.1%, the creme de la creme of science. Whereas what I'm talking about is the other 99.9%. And this is why I don't trust scientists. I know that this upsets some people, it's something I shouldn't be saying because you know, it could damage trust in science. But if I said otherwise, I'd be lying to you. The truth is, I don't trust scientists because I know how the system skews their interests. And I think that you need to know how scientific research works so you can properly evaluate scientific claims. This is why I talk about it, I want you to be well informed, I want you to know the truth. If the truth reduces trust in science, then so be it. I talk about this problem because we need to fix it. I've seen in my own discipline how scientists get stuck on pseudo scientific arguments, and this story isn't over, it's still going on. I've good reason to think that the same thing can happen and almost certainly does happen in other disciplines. The only thing that would make me trust scientists more would be if they introduced deliberate measures to prevent sociological and economic pressure from affecting research. Like for example, that if a community ends up making wrong predictions for decades, they should see consequences. Don't you think that this seems reasonable? I think so. Paula Stefan has a list of recommendations in her book, I have a list in my own book. Lots of people have come up with recommendations for how to fix the problem, but nothing has changed. So, these are the different problems. There's outright fraud on occasion, possibly made worse by systematic pressure on researchers. There are organized scams that are demonstrably getting more widespread, and then there's the generally broken incentive structure that affects pretty much everyone. Finally, a personal comment, I am aware that some people have attacked me for pointing out that scientific research has a big and systematic problem that wastes an enormous amount of money and that slows down progress. These are for the most part, I think people who mean well. They want to believe that science is doing well, they think they're doing the right thing when they're defending science. Their way to defend science is to deny that a problem exists and in some cases to attack the people who draw attention to the problem. They may be well meaning but they are incredibly ill informed. This is a real problem and denying its existence doesn't help science. Scientific research is broken. This has been going on for decades and the situation is not improving, it's getting worse. We need to finally do something about this. That's it, that's the video. No really.
