LLM-SQL-Solver: Can LLMs Determine SQL Equivalence?
CoRR(2023)
摘要
Judging the equivalence between two SQL queries is a fundamental problem with
many practical applications in data management and SQL generation (i.e.,
evaluating the quality of generated SQL queries in text-to-SQL task). While the
research community has reasoned about SQL equivalence for decades, it poses
considerable difficulties and no complete solutions exist. Recently, Large
Language Models (LLMs) have shown strong reasoning capability in conversation,
question answering and solving mathematics challenges. In this paper, we study
if LLMs can be used to determine the equivalence between SQL queries under two
notions of SQL equivalence (semantic equivalence and relaxed equivalence). To
assist LLMs in generating high quality responses, we present two prompting
techniques: Miniature & Mull and Explain & Compare. The former technique is
used to evaluate the semantic equivalence in which it asks LLMs to execute a
query on a simple database instance and then explore if a counterexample exists
by modifying the database. The latter technique is used to evaluate the relaxed
equivalence in which it asks LLMs to explain the queries and then compare if
they contain significant logical differences. Our experiments demonstrate using
our techniques, LLMs is a promising tool to help data engineers in writing
semantically equivalent SQL queries, however challenges still persist, and is a
better metric for evaluating SQL generation than the popular execution
accuracy.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要