The Illusion of Knowledge and Expertise in the Field of Evidence Synthesis and Systematic Reviewing
The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge. Stephen Hawking
Yesterday, a colleague approached me with a few questions. He also asked if I knew how to run meta-analyses. I said, yes, I know; however, I will speak to our statistician to see her availability and fees. He found my suggestion very strange, considering that I had the knowledge and availability and he was paying handsomely, but I refused to run the analysis. After two decades of working in the field, I know that staying ethical and professional is extremely hard for greedy ones, especially when there is good money waiting right at the corner. I mean, entering/importing numbers into SPSS, Stata, Review Manager, or R and running analysis by watching YouTube training is not complicated; think about the money :D
Ethically, it is right to refer the work to its professional where and when possible. As an information scientist, I have seen so many people who have claimed to know how to search — a claim that I dream to make after two decades of practice in information retrieval. Such claimers usually even don’t know the difference between two buttons of Google Search, and I’m Feeling Lucky on Google’s homepage, leave alone the search in bibliographic databases with proper controlled vocabularies and many features:
Interestingly, about 30% of the time I get involved in a systematic review is not to design the search strategies for a systematic review protocol:
- It is to clean the review team’s mess after rejection from a journal;
- or to reply to a peer-review comment that asks the reason for missing half of the studies;
- or because the editor cannot reproduce the search results;
- or the search strategy is not there and the librarian’s retired, and we cannot find the searches!
I don’t mind giving my students the knowledge of search in a session; however, when it comes to a search to update a WHO guideline that the world will follow, I wouldn’t put global health in my search-trained students’ hands.
I am sure you are familiar with Golden Rule regardless of your background:
- Treat others as you would like others to treat you
- Do not treat others in ways that you would not like to be treated
- What you wish upon others, you wish upon yourself
While it would bore you to death if I write about the typology of knowledge and its classes, I summarise that knowledge in books and theory is very different from the knowledge earned in practice (expertise).
You can learn open heart surgery from a 5-min YouTube video; I don’t even dare you try doing it at home! I know you know, but you are not an expert. I have seen systematic reviews in the cardiovascular field without any clinical experts among the authors or acknowledgements!
Would you like to see people without knowledge and experience claiming or taking over your role as a professional? If not, don’t do the same to the others. Do your open heart surgery, and let me do my searches with input from you.
Nowadays, systematic reviewing is part of any primary or secondary research. It’s almost impossible to avoid being involved in at least one systematic review during your study or career. Would that make everyone a systematic reviewer? We live in a world where people want to reach their goals in the fastest possible way rather than the right way. These are the times that you receive the following queries from the people who claim to know how to do a systematic review:
- I want to do a systematic review in one week (no team, no question, no protocol).
- I want to publish a systematic review with X number of studies (yeah, you wish! Tune your exclusion criteria and kill more people or add to research waste).
- I want a search strategy that gives me only these 11 papers (put each title between two double quotations and combine their titles with OR; you won’t need me).
- I need to publish this review that has no protocol in 3 weeks (and I need to retire ASAP).
- Would that be enough if I search EBSCO and Ovid? (of course, give it a try and show me how to do it)
- I need a publication for my CV, and my supervisor said the systematic review is the easiest option. Can I have one, please (of course; where’s your protocol and team?)
- I want to publish a systematic review on my own and only want to search PubMed for my systematic review (As long as you are OK with receiving rejection letters from the journals).
What is more interesting is that there are sub-specialities even in the field of expert searching and information retrieval. Last year, I declined to conduct two realist searches because I didn’t have time to read the papers on realist searching and practice it. When the client asked for the list of those who can conduct realist searches, I barely could list six information scientists globally! It is the same with other specialities such as health economics. One of my colleagues confessed that we would pass on cancer modelling to another more expert colleague.
On the other hand, it is almost impossible to design search strategies or conduct economic or statistical analysis without involving topic experts! I wonder how some people design and run searches in a day! I have seen professors claiming that they can do the searches in an hour! Well, I also can be a professor in half an hour!
I wish it were ending just there; many have no idea why they are conducting a review, how to interpret the results, draw narratives from GRADE tables, or translate all these numbers and analyses into the language that patients and the public could understand. It seems to me that many think that just because there is literature, they should publish a systematic review even though there is no question to answer. Even if there are questions, they may not be the most important (priority) questions to answer.
The people with illusion of knowledge have degraded the systematic review into a structurally-formatted literature review, adding to waste in research. They follow the steps mechanically but forget about synthesising the evidence for their evidence synthesis. The purpose is to publish!
Bottomline
- Respect the professionalism and ethics in research practice.
- Little knowledge is more dangerous than lack of knowledge.
- Depending on the complexity, a proper systematic review requires a team of clinicians, systematic reviewers, information specialists, methodologists, statisticians, epidemiologists, and health economists.
- Finding out the difference between Google Search and I’m Feeling Lucky buttons on Google’s homepage will NOT make you a search expert, but it is fun!
If you liked this blog post, please support me by pressing the green Follow button and signing up so I can write more. Email Subscription Dysfunctions. Thank you :D
