API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs
CoRR(2024)
Abstract
There is a growing need for Large Language Models (LLMs) to effectively use
tools and external Application Programming Interfaces (APIs) to plan and
complete tasks. As such, there is tremendous interest in methods that can
acquire sufficient quantities of train and test data that involve calls to
tools / APIs. Two lines of research have emerged as the predominant strategies
for addressing this challenge. The first has focused on synthetic data
generation techniques, while the second has involved curating task-adjacent
datasets which can be transformed into API / Tool-based tasks. In this paper,
we focus on the task of identifying, curating, and transforming existing
datasets and, in turn, introduce API-BLEND, a large corpora for training and
systematic testing of tool-augmented LLMs. The datasets mimic real-world
scenarios involving API-tasks such as API / tool detection, slot filling, and
sequencing of the detected APIs. We demonstrate the utility of the API-BLEND
dataset for both training and benchmarking purposes.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined