Why EndNote cannot find some of the duplicate titles? Can Automation or EndNote help?
Currently, there is no way to identify 100% of duplicate records in EndNote during a systematic review. Do we need to identify and remove 100% of the duplicates? If yes, does it worth it? If No, why bother reading this post :D
Anyway, if you have ever been questioned by fellow review team members about missing a few duplicates among the search results, this post is for you.
Missing a few duplicates is not a big deal and will not invalidate your systematic review.
But why is it that EndNote cannot find all the duplicates no matter how you tweak the duplication filter and combine the fields?
1. To Dash or not to Dash or to double-Dash!
In typography, there is more than one type of dash or hyphen. These hyphens can get as long as it gets. We have en dash (n or –) and em dash (m or — ). Changing encoding (ANSI vs Unicode or UTF-8) can make such characters unreadable, and the system may replace them with black rhombus with a white question mark.
The databases had no choice but to avoid surprising conversions popular with ANSI encoding. So, they started converting the en and em dashes into simple dashes. MEDLINE decided to replace them with double…
