When Metamap Meets Social Media In Healthcare: Are The Word Labels Correct?

Health forums have gained attention from researchers for studying various topics on healthcare. In many of these studies, identifying biomedical words by using the MetaMap is often a pre-processing step. MetaMap is a popular tool for recognizing Unified Medical Language System (UMLS) concepts in free text. However, MetaMap favors identifying terminologies used by professionals rather than laymen terms by the common users. The word labels given by MetaMap on social media may not be accurate, and may adversely affect the next level studies. In this study, we manually annotate the correctness of medical words extracted by MetaMap from 100 posts in HealthBoards and get a precision of 43.75 %. We argue that directly applying MetaMap on social media data in healthcare may not be a good choice for identifying the medical words.