Designing an Efficient Way to Create New Veterinary Clinical Codes

In this report, various deduplication methods are described in order to assist Vet-AI with the removal of redundant clinical codes from their database system. Their system currently operates whereby clinicians enter codes for diagnoses, leaving open the possibility that multiple clinicians assign the same code to disparate diagnoses. It is also possible that two new entries in the database may be the same diagnosis, with synonymous terminology used. By formulating this as a graph problem, we sought to reduce redundancies by identifying the most probable duplicated codes. A probabilistic model was used, where the probability that two codes are duplicates is a function of a suitable similarity measure (e.g. the Hamming distance). A heuristic method for graph edge pruning is also outlined, based on the application of principles of logical consistency.