Ratio Scientiae
RATIO SCIENTIAE BLOG
  • Home
  • Meet the Author
  • Ratio Scientiae Blog
  • Random Science
  • Writing & News
  • Nonfiction Books
    • Science Can Be Right Because It Can Be Wrong
    • The Gift of Science
    • Random Science
  • Fiction Books
    • The Sun Zebra
    • Spirit Women
  • Science Cat (Mascot)
  • Contact

9/27/2022

Do’s and Don’ts Regarding How To Assess Scientific Studies in the Age of the Internet and Social Media

2 Comments

Read Now
 
Picture
A positive aspect of the advent of the internet is that scientific studies can be made public as soon as they are ready to be published. However, these studies are highly technical publications that are intended for scientists to study and analyze. Thus, one negative effect of greater accessibility to the scientific literature is that individuals without the education and technical knowledge necessary to evaluate the studies can now gain access to them. As a result, these individuals may disseminate in their blogs, podcasts, social media, and other outlets erroneous claims about these studies either because they misunderstood them, or because they may have an agenda directed at favoring certain interpretations of the studies, even if these interpretations are not supported by the data.
 
I have lost track of the number of times I have seen someone on Twitter making claims about some issue by citing the latest published scientific article. Invariably the purpose of the individuals making these claims is not to discover or debate the truth, but rather to support their political or social agendas. I have tried to explain that truth in science is not established by one or even a few studies, even if they are published in peer-reviewed journals. Scientists have to debate the merits and flaws of each other’s studies, and this is a process that will take time. During this process scientists may make claims that they may later recant when more evidence becomes available, or a study that was heralded as a good study may fall in disfavor if it is realized that certain variables that turn out to be important were not controlled. But when scientists do these normal things that are part of the scientific process, they are accused of flip-flopping or selling out to special interests.
 
The above process is amplified by various types of media which reach millions of people and contribute to create confusion and suspicion when people see narratives change. I saw this happen with hydroxychloroquine. A study would come out indicating hydroxychloroquine (HCQ) had an effect against COVID-19, and all the HCQ proponents would brag about how the issue was settled and HCQ worked. Then another study would come out showing that HCQ did not have an effect, and all the HCQ critics would claim HCQ did not work. In the middle of the storm, certain responsible scientists or organizations would comment about the studies pointing out flaws or strengths, and they would be denounced by the pro or con side. Eventually enough studies accumulated, and they showed not only that HCQ does not work against COVID-19, but also why it does not work. However, by then HCQ had lost its appeal as a political issue.
 
I have seen this happening again with the drug ivermectin. A study came out of Brazil using a population of 88,012 subjects where ivermectin brought about a reduction of 92% in COVID-19 mortality rate. The pro-ivermectin crowd declared victory, bragged about how they had been right all along, and pointed out that the withholding of ivermectin had led to many preventable deaths from COVID-19. The truth, however, was very different. This was an observational study where the allocation of patients to treatments was not randomized, which can lead to serious biases in the data. And while a sample size of 88,012 subjects sounds impressive, the actual comparisons were performed on much smaller subsets. For example, the 92% result came from comparing 283 ivermectin users to 283 non-users. Additional problems involved the exclusion of a large number of subjects and the non-control of ivermectin use. Finally, there is no way that an effect of such a large magnitude (92%) would not have been detected when performing better designed and controlled trials, but that has not been the case.
 
As I have pointed out before, the politization of science creates a caustic environment where the work of scientists is mischaracterized or attacked by unscrupulous individuals, and this makes the process of science much harder than it already is.
 
To avoid all the problems mentioned above, I have put together a list of do’s and don’ts regarding how to assess scientific studies in the age of the internet and social media.
 
1) Do listen to what scientists have to say about the studies. They are experts in their field, and an expert is called that for a reason. They have studied many years and trained to do what they are doing. Do not assume you know more than the experts. Do not merely quote a study in your blog or social media to defend a position. Rather, do report on the debate among scientists regarding the strengths and weaknesses of the study and identify unanswered questions.
 
2) Do give scientists the time to evaluate and debate the studies and to reassess the studies as more information becomes available. Do not attack scientists for changing their minds.
 
3) To make up your mind, do wait for several studies to accumulate and for the majority of scientists to reach a consensus regarding the studies. However, this consensus will not be arrived at based on the total number of studies, but rather on their quality. One study of good quality can be more meaningful than dozens of low-quality studies, and the community of scientists (not a single scientist) is the ultimate arbiter regarding the quality of the studies.
 
4) Do not judge scientists or the results from their studies by their affiliations to companies or other organizations. The studies have to be judged on their merits. Do not make offhand claims that conflicts of interest have corrupted the science if you don’t have any evidence for it. Hearsay, innuendo, and ignorance are not proof of anything.
 
5) Success in science is measured by the ability of scientists to convince their peers. Scientists who have been unable to convince their peers and who bypass the normal scientific process to take their case to “the people” are a huge red flag. Do not blindly trust the renegade scientists who claim they are ignored by their peers. These scientists are often ignored because their studies are deficient and their ideas are unconvincing.
 
6) Do not defend and promote a scientific claim just because a celebrity or politician whom you trust or like has endorsed it. Endorsement by a non-scientist of a scientific claim without any hard evidence is irrelevant to the scientific debate. Science is not politics.
 
If everyone follows these guidelines, we can hopefully restore a measure of rationality to the scientific discourse among the public.

 
Image by Tumisu from pixabay is free for commercial use and was modified.


Share

2 Comments

7/14/2018

Predatory Journals: The Problem is Much Worse than You Think

0 Comments

Read Now
 
I recently published an article in a scientific journal. Within a few days of publication of the article, the invitations started coming. Scientists I had never heard of before wrote to tell me about how they read my article, thought it was very good, and wanted me to publish in the journals for which they serve as editors. One of these persons who claimed to be an “assitant editor” stated he was “impressed deeply by the novelty, advance, and potential extensive use” of my research and was “deeply honored” to also extend me an invitation to join their team as an editor or a reviewer. I did not recognize the names of any of these journals, and when I checked the links, I found they were all exclusively open access (online) journals. What had just happened to me is something that most scientists who publish nowadays face. I had been targeted by predatory journals.
​

Picture
Predatory journals are journals that masquerade as legitimate scientific journals but charge authors for publication without providing any editorial and publishing services. One of the cornerstones of scientific publishing is peer-review. This means that articles containing research results are submitted to scientific journals where other scientists with comparable knowledge of the field (a peer) review them. These reviewers make recommendations to the authors and/or the journal’s editor regarding revisions or publication of the research. A so called “predatory journal” makes a mockery of the peer review system by indiscriminately publishing any submission it receives with minimal review.
​


In addition, unlike mainstream journals, predatory journals often hide their publishing fees. The editors of predatory journals target scientist by sending flattery-laden invitations to publish with them without spelling out that there will be a fee involved. The scientist goes through all the work of putting together an article and sending it for review. Once the article has been accepted, an e-mail is sent to the scientist informing them that there is a publication fee involved and the scientist is not allowed to withdraw their article until the fee is payed.
 
Predatory journals have been made possible by the advent of the internet. Most predatory journals are open access journals (online journals) that have no print version. There are legitimate open access journals such as PlosOne that have a rigorous editorial review process, and their editorial team includes scientists of renown. Predatory journals, on the other hand, have editorial teams either made up of low caliber scientists that often review articles outside their area of expertise, or bona fide scientists that the journal has duped into joining as editors, or even scientists that have been included as editors without their consent! Many predatory journals sport names and websites that are similar to those of mainstream journals, and they report fraudulent metrics with regards to how often the articles published in the journal are cited to make that journal look good.
 
To illustrate the problem that predatory publishing can cause, consider the sting operation carried out by the mainstream scientific journal Science. Several members of its team put together a spoof article with glaring scientific errors that would be picked by an average competent reviewer, and they sent them to a few hundred open access journals that had been identified as predatory. They found that 62% of the journals accepted the spoof article for publication. Of those journals that conducted any discernible kind of review of the article (most often limited to details not involving the science), 70% accepted it. Here consider not only that the acceptance rate for legitimate articles sent to bona fide scientific journals is between 20 and 30%, but that this particular spoof article described research designed to be easily identifiable as bad science. What this indicates is that predatory journals are highly likely to contain shoddy science that can mislead scientists searching for clues to solve their research problems, leading to wasted time and resources.
 
By 2014, about 400,000 scientific articles had been published in 8,000 journals regarded by some metrics as predatory. Today the number of such journals has increased to more than 10,000. If predatory journals were readily identifiable, this would not be as much of a problem, but for the average researcher with limited time on their hands, the process of weeding out the good journals from the bad can prove daunting. The scientist Jeffrey Beal elaborated and maintained a public list of predatory journals for a few years, but due to harassment from the publishers of the journals he was forced to take his list down.
 
Many people believe that scientists that publish in predatory journals are usually inexperienced young scientists who are deceived into publishing in these journals. After all, what possible value can be obtained from accumulating publications in unknown, low-quality journals? One would expect that at the time the researcher’s credentials are evaluated, this would be considered a big negative, right? As it turns out, the problem is much worse than previously thought. I have published a post about several ways by which scientists game the system to advance their career. Well, add publishing in predatory journals to the list! In what is turning out to be not quite predation but a twisted interdependence, many scientists from developing countries, and from institutions with few resources where the metric for academic promotion relies more on the total number of publications, are flocking to predatory journals to beef up their publication numbers.
 
So what is there to be done? The issues concerning predatory journals as they relate to the criteria for faculty promotions will have to be addressed at the institutional level. The practices of predatory journals of misrepresenting themselves to scientists can be addressed at the judicial level. However, at the individual level there are several guidelines that researchers can follow to avoid not only publishing in predatory journals, but also taking seriously the science contained in them. I myself, for example, view with suspicion anything published in a journal not included in reputable bibliographic databases such as MEDLINE. And, of course, if you get a message in your e-mail describing what a wonderful first class researcher you are and inviting you to publish in a journal you’ve never heard of before and to join its editorial board, leave your ego aside and ignore it!

 
Inage by Sarahmirk is used under an Attribution-Share Alike 4.0 International license.

Share

0 Comments

7/5/2018

Gaming the System in Science

0 Comments

Read Now
 
A long time ago I had a conversation with a colleague regarding another scientist. This other scientist was a seemingly successful individual who had published more than a hundred articles in peer-reviewed journals. My colleague stated that he thought that this individual was not a real researcher because he had merely “gamed the system”.
 
Somewhat puzzled I inquired as to why he thought this individual, whom everyone regarded as a successful scientist, was not a researcher. He responded that this individual’s research was devoid of any guiding set of questions. It was disjointed and chaotic. Many of this individual’s publications resembled a mass production conveyor belt set up in collaboration with other labs with the aim of churning out articles that addressed low risk questions. Additionally, my colleague argued that in many publications this individual had been included as an author solely because of access granted to other researchers to technology or tissue samples that they would not have otherwise had. He concluded by stating that this individual had made a career by drilling where drilling is easy and mastering the art of serial scientific publishing. My colleague again added, “This person is not a real researcher, he has just figured out how to game the system.”
 
Now, many scientists are opinionated individuals with strong personalities, and more often than not they don’t have the nicest things to say about other scientists with whom they have butted heads. I don’t know if my colleague was right, but most scientists will tell you that they know someone who fits the unflattering description that my colleague made of this other scientist. There are indeed scientists that use several methods to game the system, and these methods range from those that don’t quite follow the “spirit” of what science should be to those which are flagrantly criminal.
 
Here I list some of these methods:
​
Picture
1) In science there is a huge pressure to publish. The axiom “publish or perish” embodies this conception of results-oriented science. The upside of this approach is that you can gauge the productivity of researchers by their number of publications. This approach allows accountability and rational planning in the allocation of resources based on performance. The downside of this approach is that it encourages researchers to think about publications rather than science. Thus there will invariably be individuals who will excel at publishing rather than at answering meaningful scientific questions. These individuals have mastered the art of breaking up scientific problems into many little parts each of which will generate sufficient data to produce at least one publication (what has been dubbed the LPU or least publishable unit), and they team up with other like-minded labs to produce a steady stream of publications where they are coauthors in each other’s articles. 
​

2) Another metric employed to evaluate scientists is citations. The concept is very straightforward. If what you publish is of interest to other scientists, they will cite your published articles in their publications. This metric allows evaluators to go beyond the mere volume of published articles in evaluating scientists. Thus scientists who publish a lot of inconsequential articles can be singled out using this metric and separated from those whose publications generate a lot of excitement within the scientific community. However, a way around this approach has been found in the form of the citation tit for tat (i.e. I will cite your articles if you cite mine). The extent to which this practice occurs is difficult to gauge, but it ranges from something that may happen among a few labs in an uncoordinated way to full-fledged “citations cartels” whose individuals blatantly engage in boosting each other’s citations.
 
3) In each field of science there are a series of top journals in which all scientists in the field wish to publish. Publication in a top journal means more exposure, more prestige, and more citations. In fact the quality of the journals in which scientists publish is also a metric that is taken into account when evaluating them. But what is a scientist to do if the work they are doing is not good enough to be published in a top journal? As it turns out, you can buy the authorship! In certain areas of the world there are black markets where scientists can purchase a slot in a publication as a coauthor for a certain amount of money.
 
4) In today’s fragmented scientific landscape where practically a lifetime of study and research is required to become an expert even in relatively small scientific fields, it is virtually impossible for a journal to have enough reviewers to cover the full breadth of topics represented by the articles that are submitted for publication. To remedy this, many journals allow authors to recommend reviewers for their articles. This practice has led to abuses ranging from authors recommending their friends to review their papers, to outright fake positive reviews.
 
And last, but definitely not least, we come to the most infamous practice of them all to game the system.
 
5) In the idealized notion of science, scientists formulate hypotheses, perform experiments, and learn from the outcome of these experiments whether it supports their hypotheses or not. However, the cold hard truth is that if none of your hypotheses prove to be true and this goes on for too long, your career may be in serious trouble. Look at it from the point of view of the agencies that fund scientific research. Why support someone who keeps barking up the wrong tree? It is then that some individuals in this bind are tempted to engage in fraud by faking their results. This fakery can range from selective publishing, where positive data is reported and negative data is ignored, to massive and systematic forging of data on dozens of publications.
 
To be fair, the use of some of the above practices by scientists (with the exception of the most extreme forms of gaming the system) is not necessarily negative. Competent scientists may wish to tackle worthy scientific questions that may take years to solve with potentially little to show for it during the process.  However, these individuals realize that if they try to answer these questions head-on they will not be favored by the current evaluation system. Thus they divide their research into low risk “bread and butter” projects designed to meet pesky publication requirements, and those projects where they address the meaningful but risky questions they really want to tackle. These scientists may also figure out that if they collaborate with and cite the right people or recommend friendly reviewers, this will provide them with the stability they need to devote themselves to the important issues.
 
Many scientists have been known to engage in the more benign forms of gaming the system, but whereas most use these procedures to fulfil evaluation requirements while they address important scientific questions, some use these practices merely to survive and further their careers. Of course, the ultimate evaluation of the achievements of a researcher will come not from citation metrics or number of publications, but rather from the actual real world impact of their research.
 
This is a metric that can’t be gamed.

 
Figure by Selena N. B. H. used here under an Attribution 2.0 Generic (CC BY 2.0) license.

Share

0 Comments
Details

    Categories

    All
    911
    Absence Of Evidence
    Adrenaline
    Adrenochrome
    Advances In War Medicine
    Affidavits
    Airplane On Conveyor Belt
    Alder's Razor
    Alfred Wegener
    Alzheimer's Disease
    Amyloid Theory
    Ancient Astronauts
    Animal Rights
    Animals
    Anthony Fauci
    Anthropomorphism
    Antibodies
    Anti Vaccination
    Astronauts
    Authority Figure
    Autumn
    Bambi
    Believers
    Bias
    Big Bang Theory Sitcom
    Bigfoot
    Bill Ney
    Black Death
    Black Holes
    Blind Experimental Design
    Brain
    Brains
    Building 7
    Buoyancy
    Cancer
    Cannonballs
    Carnivores
    Catoctin Mountains
    Cell Culture
    Center Of Mass
    Challenger
    Chance
    Chemical Names
    Chemical Reactions
    Chemtrails
    Chesapeake Bay
    Child Abuse
    Child Testimonies
    Chimborazo
    Citations
    Citogenesis
    Civil War
    Climate Change
    Climategate
    Climate Skeptics
    Clinical Trial
    Coelacanth
    Coincidence
    Cold-Stress
    Communicating Science
    Completeness Of Scientific Theories
    Consciousness
    Conspiracy
    Contrails
    Controls
    Coquina Rock
    Coronavirus
    Counterintuitive Facts
    COVID19
    COVID 19 Vaccine
    COVID-19 Vaccine
    Creationism
    Cross-Sectional Method
    Cryptozoology
    Dan Shechtman
    Demonic Possession
    Density
    Devotion To Science
    Dihydrogen Monoxide
    Dinosaurs
    Discrimination
    Disgust
    Dishonesty In Science
    Diversity In Methodologies
    Diversity In Scientists
    DNA
    Doctors
    Dog Experiments
    Donald Trump
    Double Slit Experiment
    Do Your Own Research
    Dreams
    Dr. Gloom's Crypt Of Curiosities
    Drugs
    Dust Bunnies
    Eclipse
    Efrain Racker
    Einstein
    Elections 2020
    End Of The World
    Erosion
    Established Science
    Everest
    Evidence
    Evolution
    Exorcism
    Experimenter Bias
    Experiments You Can Do In Your Home
    Experts
    Extinction
    Face Masks
    Facts
    Faith
    Faith Healing
    Fall
    Falsifiability
    Fantastical Claims
    Fauna
    Fear
    Feynman
    Finger Snapping
    Fosbury Flop
    Francis Crick
    Frankenstein
    Fraud
    Frederick's Municipal Forest
    Free Will
    Friction
    Fruit Fly
    Funerals
    Funny
    Galaxies
    Genes
    Genius
    Geology
    Ghost
    Global Warming
    Global Warming Denial
    God
    Gravitational Lens
    Gravitational Waves
    Gravity
    Gun Violence
    Hanlon’s Razor
    Hearing
    Hearing Aids
    Hearing Loss
    HeLa Cells
    Henrietta Lacks
    Heroic Science
    Hitchens's Razor
    Hoax
    Homosexuality
    HPV Vaccine
    Human Experimentation
    Human Folly
    Hunting
    Hurricane
    Hydroxychloroquine
    Hypothesis
    Ignaz Semmelweis
    Ignorance As Evidence
    Immunotherapy
    Infamy Or Glory
    Influenza
    Insults
    Intelligence
    Intelligence Tests
    Intelligent Design
    Intersex
    Iraq War
    Irish Washerwoman
    Isaac Asimov
    James T. Kirk
    James Watson
    Katherine Hayhoe
    Language
    Lazarus Effect
    Left Fork Rocks
    Level Of Detail
    Libet Experiment
    Lightning
    Limericks
    Limitations Of Science
    Loch Ness Monster
    Lord Howe Island Stick Insect
    Loud Music
    Luck
    Mad Scientist
    Magnus Effect
    Malaria Vaccine
    Max Planck
    Medical Risks
    Medical Terms
    Mediums
    Men
    Mental Illness
    Mice
    Microbiome
    Milgram Experiment
    Mind
    Mind In The Gutter
    Misuse Of Science
    MMR Vaccine
    Mnemonic Devices
    Moments Of Discovery
    Monty Hall Puzzle
    Moon
    Movies
    NASA
    Natural
    Nature
    Nature Of Science
    Negative Evidence
    Nerds
    Newton
    Nobel Prize
    NOMA
    N-Rays
    Obesity
    Occam's Razor
    Open Mind
    Ouija Board
    Outreau
    Oxbow Lakes
    Para-Dimethylaminobenzaldehyde
    Paranormal
    Pasteur Louis
    Peak Of Illusion
    Peers
    Perception
    Philosophy
    Phobias
    Phrenology
    Physics
    Pink Lady's Slippers
    Plague
    Politicization Of Science
    Polywater
    Popper’s Falsifiability Principle
    Possible/Impossible
    Power To The People!
    Predatory Journals
    Pregnancy
    Premonitions
    Probability
    Propaganda
    Prophesy
    Pseudoscience
    Psychic
    Publication
    Puerperal Fever
    QAnon
    Quack
    Quackery
    Quantum Mechanics
    Quartzite
    Racism
    Radical New Ideas
    Radioactivity
    Radium
    Randomness
    Ratio Sapientiae
    Ratio Scientiae
    Reality
    Reason
    Religion
    Rock Climbing
    Rosalyn Franklin
    RSV Vaccine
    Sagan's Standard
    Salem Witch Trials
    Sample Size
    Science Fiction
    Science Jokes
    Science Pranks
    Scientific Consensus
    Scientific Establishment
    Scientific Guidelines
    Scientific Journals
    Scientific Method
    Scientific Names
    Scientific Question
    Scientific Terms
    Scientific Theories
    Seashells
    Self-Experimentation
    Shape Of The Earth
    Skepticism
    Skeptics
    Skin Color
    Slavery
    Snake Oil
    Soccer
    Sonic Hedgehog
    Space
    Spanish Flu
    Spontaneous Generation
    Stanley Prusiner
    Stars
    Star Trek
    Statistical Significance
    Statistics
    Sublime/Ridiculous
    Suicide
    Sun
    Superstition
    Surface To Volume Ratio
    Swanson Conversion
    Swine Flu
    Tangier Island
    Taxonomy
    Technical Details
    The Support Of God
    Thomas Young
    Timeline Method
    Tooth Worm
    Transposable Elements
    Trusted Messenger
    Trust In Scientists
    Truth
    Tyranny Of Fantasy
    Understanding
    Universe
    Vaccine Hesitancy
    Vaccines
    Vacuum
    VAERS
    Valley Of Despair
    Values
    Video Games
    West Side Story
    West Virginia Penitentiary
    Wikipedia
    William Shatner
    Wolf Rock
    Women
    World Trade Center
    Xenophobia
    Y Chromosome
    Zinc
    Zombies

    Archives

    October 2024
    September 2024
    August 2024
    July 2024
    June 2024
    May 2024
    April 2024
    December 2023
    November 2023
    October 2023
    September 2023
    May 2023
    April 2023
    March 2023
    February 2023
    January 2023
    November 2022
    October 2022
    September 2022
    August 2022
    July 2022
    June 2022
    May 2022
    April 2022
    August 2021
    July 2021
    June 2021
    May 2021
    April 2021
    March 2021
    February 2021
    January 2021
    November 2020
    October 2020
    September 2020
    August 2020
    July 2020
    June 2020
    May 2020
    April 2020
    March 2020
    February 2020
    January 2020
    November 2019
    October 2019
    September 2019
    August 2019
    July 2019
    June 2019
    May 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    May 2017
    April 2017

    RSS Feed

Proudly powered by Weebly
  • Home
  • Meet the Author
  • Ratio Scientiae Blog
  • Random Science
  • Writing & News
  • Nonfiction Books
    • Science Can Be Right Because It Can Be Wrong
    • The Gift of Science
    • Random Science
  • Fiction Books
    • The Sun Zebra
    • Spirit Women
  • Science Cat (Mascot)
  • Contact