VOICE: Visual Oracle for Interaction, Conversation, and Explanation
CoRR(2023)
摘要
We present VOICE, a novel approach to science communication that connects
large language models' (LLM) conversational capabilities with interactive
exploratory visualization. VOICE introduces several innovative technical
contributions that drive our conversational visualization framework. Our
foundation is a pack-of-bots that can perform specific tasks, such as assigning
tasks, extracting instructions, and generating coherent content. We employ
fine-tuning and prompt engineering techniques to tailor bots' performance to
their specific roles and accurately respond to user queries. Our interactive
text-to-visualization method generates a flythrough sequence matching the
content explanation. Besides, natural language interaction provides
capabilities to navigate and manipulate the 3D models in real-time. The VOICE
framework can receive arbitrary voice commands from the user and respond
verbally, tightly coupled with corresponding visual representation with low
latency and high accuracy. We demonstrate the effectiveness of our approach by
applying it to the molecular visualization domain: analyzing three 3D molecular
models with multi-scale and multi-instance attributes. We finally evaluate
VOICE with the identified educational experts to show the potential of our
approach. All supplemental materials are available at https://osf.io/g7fbr.
更多查看译文
关键词
visual oracle,interaction,conversation,voice
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要