Inter- and Intra-Rater Level of Agreement in Ultrasonic Video Grading of Venous Gas Emboli

INTRODUCTION: This study aimed to evaluate whether a short familiarization session is sufficient for individuals with no prior experience of sonography to both reliably and consistently evaluate the prevalence of venous gas emboli (VGE) from precordial ultrasonic videos. METHODS: A total of 10 adults with no prior experience of sonography were introduced to the Eftedal-Brubakk 6-grade scale and were shown 6 video sequences, each of a maximum of 10 heartbeats, representing each grading level. Thereafter, they independently evaluated the prevalence of VGE in 70 ultrasonic videos before and after a 14-d interval (test-retest; intrarater), with these being compared to an experienced sonographer's grading (inter-rater). RESULTS: A significant inter-rater level of agreement was found between the naive and experienced sonographers'bubble grading both during the first (W = 0.945) and second (W = 0.952) round of bubble evaluation. The naive observers' evaluations were on average 79% (range: 61-95%) and 75% (range: 48-95%) in complete agreement with the experienced sonographer's gradings, while the level of agreement was 99% and 98% within 1 grade unit. There was a significant intra-rater level of agreement (Kappa = 0.845) during the test-retest series, with a mean percentage level of agreement of 87% (range: 72-93%). CONCLUSION: This study demonstrates that a short familiarization session enables individuals with no prior sonography experience to consistently evaluate VGE prevalence from precordial ultrasonic videos.