DictLLM: Harnessing Key-Value Data Structures with Large Language Models for Enhanced Medical Diagnostics
CoRR(2024)
摘要
Structured data offers a sophisticated mechanism for the organization of
information. Existing methodologies for the text-serialization of structured
data in the context of large language models fail to adequately address the
heterogeneity inherent in key-value structured data. These methods are not
ideal and frequently result in larger input sizes and poor adaptability to
input changes. In this paper, we introduce DictLLM, an innovative framework
designed to improve the modeling of key-value structured data, like medical
laboratory reports, for generating medical diagnoses. DictLLM integrates three
key components: (1) group positional encoding to maintain permutation
invariance, (2) hierarchical attention bias to capture the inherent bias in
structured data, and (3) an optimal transport alignment layer that aligns the
embedding generated by the dictionary encoder with the LLM, thereby producing a
sequence of fixed-length virtual tokens. We carry out experiments using various
LLM models on a comprehensive real-world medical laboratory report dataset for
automatic diagnosis generation, our findings illustrate that DictLLM
significantly outperforms established baseline methods and few-shot GPT-4
implementations in terms of both Rouge-L and Knowledge F1 scores. Furthermore,
our evaluation of the framework's scalability and robustness, through a series
of experiments, underscores its exceptional capability in accurately modeling
the complex key-value data structure of medical dictionary data.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要