A Testing and Evaluation Framework for the Quality of DNN Models

Guang Yang, Zhiwen Li,Pengqi Wang,Yuan Shi,Lingzhong Meng

ISSSR（2024）

Institute of Software Chinese Academy of Sciences

Cited 0|Views7

Abstract

Recently, the application of advanced AI algorithms represented by deep neural network (DNN) in various fields of human society has shown explosive growth. These powerful AI tools have not only revolutionized the traditional working mode of the technology industry, but also deeply penetrated into people’s daily life. However, since AI algorithms generally have problems such as uncertain capability boundaries and difficult interpretability, failing to adequately and accurately measure their quality will bring about great hidden dangers. Traditional software testing and evaluation methods can hardly meet the need of comprehensively measuring the quality of DNN models. To solve this problem, we propose a testing and evaluation framework for the quality of DNN models. Experiments demonstrate that our proposed framework can comprehensively test and evaluate the quality of DNN models, as well as improve the adequacy and accuracy of testing and evaluation through dataset quality analysis and test adequacy analysis.

Translated text

Key words

deep neural network,quality of models,testing and evaluation framework,dataset quality analysis,test adequacy analysis

求助PDF

上传PDF

Bibtex

AI Read Science

AI Summary

AI Summary is the key point extracted automatically understanding the full text of the paper, including the background, methods, results, conclusions, icons and other key content, so that you can get the outline of the paper at a glance.

Example

Background

Key content

Introduction

Methods

Results

Related work

Fund

Key content

Pretraining has recently greatly promoted the development of natural language processing (NLP)
We show that M6 outperforms the baselines in multimodal downstream tasks, and the large M6 with 10 parameters can reach a better performance
We propose a method called M6 that is able to process information of multiple modalities and perform both single-modal and cross-modal understanding and generation
The model is scaled to large model with 10 billion parameters with sophisticated deployment, and the 10 -parameter M6-large is the largest pretrained model in Chinese
Experimental results show that our proposed M6 outperforms the baseline in a number of downstream tasks concerning both single modality and multiple modalities We will continue the pretraining of extremely large models by increasing data to explore the limit of its performance

Upload PDF to Generate Summary

Must-Reading Tree

Example

Generate MRT to find the research sequence of this paper

Data Disclaimer

The page data are from open Internet sources, cooperative publishers and automatic analysis results through AI technology. We do not make any commitments and guarantees for the validity, accuracy, correctness, reliability, completeness and timeliness of the page data. If you have any questions, please contact us by email: report@aminer.cn

Chat Paper

【要点】：本文提出了一种深度神经网络（DNN）模型质量的测试与评估框架，以解决传统软件测试方法无法满足DNN模型质量全面测量的需求，创新点在于结合数据集质量分析和测试充分性分析，提高了测试与评估的充分性和准确性。

【方法】：作者构建了一个涵盖数据集质量分析和测试充分性分析的DNN模型质量测试与评估框架。

【实验】：通过实验使用不同数据集验证了该框架的综合测试与评估能力，实验结果表明框架有效提高了DNN模型测试与评估的充分性和准确性，但论文中未具体提及所使用的数据集名称。