PeaTMOSS: A Dataset and Initial Analysis of Pre-Trained Models in Open-Source Software
CoRR(2024)
摘要
The development and training of deep learning models have become increasingly
costly and complex. Consequently, software engineers are adopting pre-trained
models (PTMs) for their downstream applications. The dynamics of the PTM supply
chain remain largely unexplored, signaling a clear need for structured datasets
that document not only the metadata but also the subsequent applications of
these models. Without such data, the MSR community cannot comprehensively
understand the impact of PTM adoption and reuse. This paper presents the
PeaTMOSS dataset, which comprises metadata for 281,638 PTMs and detailed
snapshots for all PTMs with over 50 monthly downloads (14,296 PTMs), along with
28,575 open-source software repositories from GitHub that utilize these models.
Additionally, the dataset includes 44,337 mappings from 15,129 downstream
GitHub repositories to the 2,530 PTMs they use. To enhance the dataset's
comprehensiveness, we developed prompts for a large language model to
automatically extract model metadata, including the model's training datasets,
parameters, and evaluation metrics. Our analysis of this dataset provides the
first summary statistics for the PTM supply chain, showing the trend of PTM
development and common shortcomings of PTM package documentation. Our example
application reveals inconsistencies in software licenses across PTMs and their
dependent projects. PeaTMOSS lays the foundation for future research, offering
rich opportunities to investigate the PTM supply chain. We outline mining
opportunities on PTMs, their downstream usage, and cross-cutting questions.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要