Do’s and Don’ts Regarding How To Assess Scientific Studies in the Age of the Internet and Social MediaRead Now
A positive aspect of the advent of the internet is that scientific studies can be made public as soon as they are ready to be published. However, these studies are highly technical publications that are intended for scientists to study and analyze. Thus, one negative effect of greater accessibility to the scientific literature is that individuals without the education and technical knowledge necessary to evaluate the studies can now gain access to them. As a result, these individuals may disseminate in their blogs, podcasts, social media, and other outlets erroneous claims about these studies either because they misunderstood them, or because they may have an agenda directed at favoring certain interpretations of the studies, even if these interpretations are not supported by the data.
I have lost track of the number of times I have seen someone on Twitter making claims about some issue by citing the latest published scientific article. Invariably the purpose of the individuals making these claims is not to discover or debate the truth, but rather to support their political or social agendas. I have tried to explain that truth in science is not established by one or even a few studies, even if they are published in peer-reviewed journals. Scientists have to debate the merits and flaws of each other’s studies, and this is a process that will take time. During this process scientists may make claims that they may later recant when more evidence becomes available, or a study that was heralded as a good study may fall in disfavor if it is realized that certain variables that turn out to be important were not controlled. But when scientists do these normal things that are part of the scientific process, they are accused of flip-flopping or selling out to special interests.
The above process is amplified by various types of media which reach millions of people and contribute to create confusion and suspicion when people see narratives change. I saw this happen with hydroxychloroquine. A study would come out indicating hydroxychloroquine (HCQ) had an effect against COVID-19, and all the HCQ proponents would brag about how the issue was settled and HCQ worked. Then another study would come out showing that HCQ did not have an effect, and all the HCQ critics would claim HCQ did not work. In the middle of the storm, certain responsible scientists or organizations would comment about the studies pointing out flaws or strengths, and they would be denounced by the pro or con side. Eventually enough studies accumulated, and they showed not only that HCQ does not work against COVID-19, but also why it does not work. However, by then HCQ had lost its appeal as a political issue.
I have seen this happening again with the drug ivermectin. A study came out of Brazil using a population of 88,012 subjects where ivermectin brought about a reduction of 92% in COVID-19 mortality rate. The pro-ivermectin crowd declared victory, bragged about how they had been right all along, and pointed out that the withholding of ivermectin had led to many preventable deaths from COVID-19. The truth, however, was very different. This was an observational study where the allocation of patients to treatments was not randomized, which can lead to serious biases in the data. And while a sample size of 88,012 subjects sounds impressive, the actual comparisons were performed on much smaller subsets. For example, the 92% result came from comparing 283 ivermectin users to 283 non-users. Additional problems involved the exclusion of a large number of subjects and the non-control of ivermectin use. Finally, there is no way that an effect of such a large magnitude (92%) would not have been detected when performing better designed and controlled trials, but that has not been the case.
As I have pointed out before, the politization of science creates a caustic environment where the work of scientists is mischaracterized or attacked by unscrupulous individuals, and this makes the process of science much harder than it already is.
To avoid all the problems mentioned above, I have put together a list of do’s and don’ts regarding how to assess scientific studies in the age of the internet and social media.
1) Do listen to what scientists have to say about the studies. They are experts in their field, and an expert is called that for a reason. They have studied many years and trained to do what they are doing. Do not assume you know more than the experts. Do not merely quote a study in your blog or social media to defend a position. Rather, do report on the debate among scientists regarding the strengths and weaknesses of the study and identify unanswered questions.
2) Do give scientists the time to evaluate and debate the studies and to reassess the studies as more information becomes available. Do not attack scientists for changing their minds.
3) To make up your mind, do wait for several studies to accumulate and for the majority of scientists to reach a consensus regarding the studies. However, this consensus will not be arrived at based on the total number of studies, but rather on their quality. One study of good quality can be more meaningful than dozens of low-quality studies, and the community of scientists (not a single scientist) is the ultimate arbiter regarding the quality of the studies.
4) Do not judge scientists or the results from their studies by their affiliations to companies or other organizations. The studies have to be judged on their merits. Do not make offhand claims that conflicts of interest have corrupted the science if you don’t have any evidence for it. Hearsay, innuendo, and ignorance are not proof of anything.
5) Success in science is measured by the ability of scientists to convince their peers. Scientists who have been unable to convince their peers and who bypass the normal scientific process to take their case to “the people” are a huge red flag. Do not blindly trust the renegade scientists who claim they are ignored by their peers. These scientists are often ignored because their studies are deficient and their ideas are unconvincing.
6) Do not defend and promote a scientific claim just because a celebrity or politician whom you trust or like has endorsed it. Endorsement by a non-scientist of a scientific claim without any hard evidence is irrelevant to the scientific debate. Science is not politics.
If everyone follows these guidelines, we can hopefully restore a measure of rationality to the scientific discourse among the public.
Image by Tumisu from pixabay is free for commercial use and was modified.
Dr. Fauci has been getting a bad rap lately. His critics claim that he funded the creation of the COVID-19 virus and profited from it. That he conspired with the pharmaceutical industry to discourage the acceptance of cheap effective drugs such as hydroxychloroquine. That he promoted the COVID-19 vaccines which have harmed and killed people. That he pushed for masks, social distancing, lockdowns, and other ineffective measures which caused unnecessary pain. And that while he did this he lied about his true motivations. His critics also point out that Fauci’s support of certain ideas and treatments and his cozying up to the pharma industry also led to the deaths of hundreds of thousands during the AIDS epidemic. To sum it up, they claim he is an evil man who lies constantly and is responsible for untold deaths and suffering from which he profited, and that he belongs in jail.
Considering the rapidly evolving nature of the COVID-19 pandemic and the science (and nonscience) behind it, I consider that Dr. Fauci in general did a good job of informing us about the latest developments and measures to stop the spread of COVID-19. And while I also think he made some mistakes (some of which he has acknowledged), I have often defended him in my blog against all the nonsense that people write about him out of either malice, ignorance, or design, including unfounded conspiracy theories and baseless claims. For example, the COVID-19 vaccines and the mitigation measures against the virus saved millions of lives, and hydroxychloroquine does not work against COVID-19. These are solid facts. Fauci’s support of vaccines and mitigation measures as well as his opposition to hydroxychloroquine were grounded in science, evidence, and reason.
But I realize that quite a number of people do not know who Anthony Fauci is. In fact, many think he is an uncaring bureaucrat with no specific accomplishments under his belt. In this post I will endeavor to set the record straight.
As a researcher, Dr. Fauci developed successful therapies against the fatal diseases polyarteritis nodosa, granulomatosis with polyangiitis (formerly Wegener's granulomatosis), and lymphomatoid granulomatosis. And in the field of AIDS he made seminal contributions to the understanding of how the disease works and the developing of treatments. By 2022 Fauci was the 44th most cited researcher in the world. In the field of immunology, he was ranked 9th out of 3.3 million authors, in the field of research and experimental medicine he was 22nd out of 3.3 million authors, and in the field of general and internal medicine he was ranked 715th out of 1.4 million authors. Dr. Fauci’s work has clearly captured the attention of his peers who often cite his work. In science this is one of the most common measures of success. Dr. Fauci also has won many prestigious awards such as the National Medal of Science, the Lasker Award for Public Service, and the Robert Koch Gold Medal, and he has more than 50 honorary doctoral degrees.
Dr. Fauci has been the head of the National Institute of Allergy and Infectious Diseases (NIAID) since 1984, and he has advised seven presidents beginning with Ronald Reagan on matters of public health including working on the federal response to AIDS, Ebola, the Zika virus, Anthrax, and COVID-19. And while Dr. Fauci’s research contribution are important, some of his greatest contributions have taken place at the managerial level. For example, Dr. Fauci was the chief architect of the President’s Emergency Plan for AIDS Relief (PEPFAR) launched in 2003. This program, which provided treatment for people with HIV, prevented new infections, and made possible epidemic control, saved more than 20 million lives. For this accomplishment, President George W. Bush presented Dr. Fauci with the nation’s highest civilian honor, the Presidential Medal of Freedom.
But the accomplishment that best describes the type of person Dr. Fauci is, is that which involved the notorious firebrand AIDS activist Larry Kramer. Kramer criticized Dr. Fauci for moving too slowly in finding a treatment for AIDS, and said he was evil and represented a callous government. Kramer called Fauci a pill-pushing tool of the medical establishment, an incompetent idiot, a disgrace, and a murderer who should be put in front of a firing squad. Kramer compared him to a Nazi and even insulted Fauci’s wife. So what did Dr. Fauci do? He talked to Kramer and other AIDS activists, he listened to their concerns and ideas, he realized they had a point, and he pushed for changes in the way clinical trials were conducted, thus expanding access to experimental medicines, speeding up the process, making it more flexible, and giving patients a greater voice. He reached out to those who insulted him and worked with them to change medicine for the better and make history. Eventually, Dr. Fauci and Kramer became good friends. Fauci helped Kramer get medical treatment for his health problems, and Kramer made Fauci a character in one of his award winning plays.
This is the man Dr. Fauci is. He is not some cold-hearted bureaucrat. He genuinely cares about people and patients, and those who know him can testify to his empathy for others. On top of this, as I’ve mentioned above, Dr, Fauci is among the best scientists in the world in terms of his research. And finally, Dr. Fauci has achieved what the majority of scientists only dream of. His research and managerial skills have had a real-life impact on the world in terms of saving or improving the lives of tens of millions of people.
All of the above is why I will not put up with anyone slandering Dr. Fauci. Sure, we can discuss all you want specific mistakes he has made, and what he could have done better, but he must be treated with the respect he deserves.
The image of Dr. Fauci by NIAID is used here under an Attribution 2.0 Generic (CC BY 2.0) license.