Fusing Echocardiography Images and Medical Records for Continuous Patient Stratification
CoRR(2024)
摘要
Deep learning now enables automatic and robust extraction of cardiac function
descriptors from echocardiographic sequences, such as ejection fraction or
strain. These descriptors provide fine-grained information that physicians
consider, in conjunction with more global variables from the clinical record,
to assess patients' condition. Drawing on novel transformer models applied to
tabular data (e.g., variables from electronic health records), we propose a
method that considers all descriptors extracted from medical records and
echocardiograms to learn the representation of a difficult-to-characterize
cardiovascular pathology, namely hypertension. Our method first projects each
variable into its own representation space using modality-specific approaches.
These standardized representations of multimodal data are then fed to a
transformer encoder, which learns to merge them into a comprehensive
representation of the patient through a pretext task of predicting a clinical
rating. This pretext task is formulated as an ordinal classification to enforce
a pathological continuum in the representation space. We observe the major
trends along this continuum for a cohort of 239 hypertensive patients to
describe, with unprecedented gradation, the effect of hypertension on a number
of cardiac function descriptors. Our analysis shows that i) pretrained weights
from a foundation model allow to reach good performance (83
with limited data (less than 200 training samples), ii) trends across the
population are reproducible between trainings, and iii) for descriptors whose
interactions with hypertension are well documented, patterns are consistent
with prior physiological knowledge.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要