HomeResearch-feedChannelRankingsGCTTHU AI TROpen DataMust Reading
HomeMachine Learning Operations
Machine Learning OperationsMachine learning operations (MLOps) is the use of machine learning models by development/operations (DevOps) teams. MLOps seeks to add discipline to the development and deployment of machine learning models by defining processes to make ML development more reliable and productive.
Year#CitationLike
Building Continuous Integration Services for Machine Learning
Bojan Karla_,Matteo Interlandi,Cedric Renggli,Wentao Wu,Ce Zhang,Deepak Mukunthu Iyappan Babu,Jordan Edwards,Chris Lauren, Andy Xu,Markus Weimer
KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining Virtual Event..., pp.2407-2415, (2020)
Methods
Conclusion
We have presented our efforts and experiences with building an industrial-strength “continuous integration for machine learning” service
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by0BibtexViews72Links
0
0
MLModelCI: An Automatic Cloud Platform for Efficient MLaaS
Zhang Huaizheng, Li Yuanming, Huang Yizheng,Wen Yonggang,Yin Jianxiong, Guan Kyle
MM '20: The 28th ACM International Conference on Multimedia Seattle WA USA ..., pp.4453-4456, (2020)
Conclusion
We described MLModelCI, an automated and easy-touse platform for building efficient and robust MLaaS in cloud
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by0BibtexViews52Links
0
0
Beyond Accuracy: Behavioral Testing of NLP models with CheckList
Ribeiro Marco Tulio,Wu Tongshuang,Guestrin Carlos,Singh Sameer
ACL, pp.4902-4912, (2020)
Conclusion
Adopting principles from behavioral testing in software engineering, we propose CheckList, a model-agnostic and task-agnostic testing methodology that tests individual capabilities of the model using three different test types
Must Reading
Comments
Weibo
Cited by0BibtexViews1302Links
0
0
Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology
Studer Stefan, Bui Thanh Binh, Drescher Christian, Hanuschkin Alexander, Winkler Ludwig, Peters Steven, Mueller Klaus-Robert
Methods
Conclusion
Due to the lack of a process model for machine learning applications, many project organizations rely on alternative models that are closely related to ML, such as, the Cross-Industry Standard Process for Data Mining
Must ReadingMust Reading
Comments
Weibo
Cited by0BibtexViews11Links
0
0
Towards ML Engineering: A Brief History Of TensorFlow Extended (TFX)
Konstantinos, Katsiapis, Abhijit Karmarkar, Ahmet Altay, Aleksandr Zaks, Neoklis Polyzotis, Anusha Ramesh, Ben Mathes, Gautam Vasudevan, Irene Giannoumis, Jarek Wilkiewicz, Jiri Simsa
Conclusion
We argue that in order to unlock the gains Machine Learning can bring, organizations should advance the maturity of their ML teams by investing in robust ML infrastructure and promoting ML Engineering education
default
Comments
Weibo
Cited by0BibtexViews7Links
0
0
Principles and Practice of Explainable Machine Learning
Vaishak Belle, Ioannis Papantonis
Conclusion
Visual explanation aim at generating visualizations that facilitate the understanding of a model
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by0BibtexViews10Links
0
0
Adoption and Effects of Software Engineering Best Practices in Machine Learning
Alex Serban, Koen van der Blom,Holger Hoos, Joost Visser
ESEM, (2020)
Methods
Conclusion
We studied how teams develop, deploy and maintain software solutions that involve machine learning components
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by0BibtexViews10Links
0
0
Monitoring and explainability of models in production
Janis Klaise, Arnaud Van Looveren, Clive Cox, Giovanni Vacanti, Alexandru Coca
Conclusion
Best practices are still being established in the MLOps community and we feel that in order to succeed, the open source tools need to be general enough to cover a majority of use cases whilst being flexible enough to allow for use case specific customization
Must ReadingMust Reading
Comments
Weibo
Cited by0BibtexViews7Links
0
0
Developments in MLflow: A System to Accelerate the Machine Learning Lifecycle
Andrew Chen, Andy Chow,Aaron Davidson, Arjun DCunha,Ali Ghodsi, Sue Ann Hong,Andy Konwinski, Clemens Mewald, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe
SIGMOD/PODS '20: International Conference on Management of Data Portland OR ..., pp.1-4, (2020)
MLflow is a popular open source platform for managing ML development, including experiment tracking, reproducibility, and deployment. In this paper, we discuss user feedback collected since MLflow was launched in 2018, as well as three major features we have introduced in respons...
default
Comments
Weibo
Cited by0BibtexViews92Links
0
0
Data Engineering for Data Analytics: A Classification of the Issues, and Case Studies
Nazabal Alfredo, Williams Christopher K. I., Colavizza Giovanni, Smith Camila Rangel, Williams Angus
(2020)
Conclusion
In this paper we provide a description and classification of such tasks into high-levels groups, namely data organization, data quality and feature engineering
Must Reading
Comments
Weibo
Cited by0BibtexViews7Links
0
0
Towards Automating the AI Operations Lifecycle
Arnold Matthew, Boston Jeffrey, Desmond Michael, Duesterwald Evelyn, Elder Benjamin, Murthi Anupama, Navratil Jiri, Reimer Darrell
Conclusion
This paper described a set of enabling technologies that help increase the level of automation during AI operations, reducing the human effort and cost required
Must Reading
Comments
Weibo
Cited by0BibtexViews10Links
0
0
Addressing the Memory Bottleneck in AI Model Training
Ojika David, Patel Bhavesh, Reina G. Anthony, Boyer Trent, Martin Chad, Shah Prashant
Conclusion
To the best of our knowledge, the results presented in this paper represent the first milestone in training a deep neural network having large memory footprint on a single-node server without hardware accelerators like GPUs
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by0BibtexViews9Links
0
0
Simulating Performance of ML Systems with Offline Profiling
Huang Hongming, Cheng Peng, Xu Hong,Xiong Yongqiang
Methods
Conclusion
To fully demonstrate its feasibility and potential, we are conducting extensive offline profiling to cover all operations in popular frameworks and common hardware, and investigating the Machine learning approach for op-level performance estimation
Must ReadingMust Reading
Comments
Weibo
Cited by0BibtexViews12Links
0
0
CodeReef: an open platform for portable MLOps, reusable automation actions and reproducible benchmarking
Fursin Grigori, Guillou Herve, Essayan Nicolas
Conclusion
We present CodeReef - an open platform to share all the components necessary to enable cross-platform MLOps, i.e. automating the deployment of ML models across diverse systems in the most efficient way
Must Reading
Comments
Weibo
Cited by0BibtexViews7Links
0
0
Engineering AI Systems: A Research Agenda
Bosch Jan,Crnkovic Ivica,Olsson Helena Holmström
Conclusion
We provide an overview of the engineering challenges surrounding ML/DL solutions and, through this, present a research agenda and overview of open items that need to be addressed by the research community at large
default
Comments
Weibo
Cited by0BibtexViews11Links
0
0
Teaching Software Engineering for AI-Enabled Systems
Kästner Christian,Kang Eunsuk
ICSE, pp.45-48, (2020)
Conclusion
This paper reports on design considerations for a course that teaches software engineering techniques for building systems with Artificial intelligence components and experience from teaching the course for the first time
Must Reading
Comments
Weibo
Cited by0BibtexViews10Links
0
0
Machine Learning Testing: Survey, Landscapes and Horizons
Jie M. Zhang,Mark Harman, Lei Ma,Yang Liu
CoRR, (2019)
Conclusion
This paper provides a comprehensive survey of Machine Learning Testing research
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by46BibtexViews154Links
0
0
Studying Software Engineering Patterns for Designing Machine Learning Systems
Hironori Washizaki, Hiromu Uchida,Foutse Khomh,Yann-Gaël Guéhéneuc
IWESEP, pp.49-54, (2019)
Conclusion
We answered four research questions through a survey of software developers of ML systems and a systematic-literature review of both academic and gray literatures pertaining to ML systems and their software development
Must ReadingMust Reading
Comments
Weibo
Cited by4BibtexViews18Links
0
0
Overton: A Data System for Monitoring and Improving Machine-Learned Products
Ré Christopher,Niu Feng, Gudipati Pallavi, Srisuwananukorn Charles
conference on innovative data systems research, (2019)
Methods
Conclusion
This paper presented Overton, a system to help engineers manage the lifecycle of production machine learning systems
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by2BibtexViews17Links
0
0
Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
Rob Ashmore,Radu Calinescu, Colin Paterson
arXiv: Learning, (2019)
Methods
Conclusion
machine learning technology is increasingly at the core of sophisticated functionality provided by smart devices, household appliances and online services, often unbeknownst to their users
Must Reading
Comments
Weibo
Cited by0BibtexViews14Links
0
0
Keywords
Adaptive AlgorithmAdaptive LearningApplied EthicsArtificial IntelligenceCategorize Existing StrategyChange DetectionComprehensive IntroductionConcept DriftConcept Drift AdaptationData Governance
Authors
Markus Weimer
Markus Weimer
Paper 1
Indrė Žliobaitė
Indrė Žliobaitė
Paper 1
Yonggang Wen
Yonggang Wen
Paper 1
Jianxiong Yin
Jianxiong Yin
Paper 1
Radu Calinescu
Radu Calinescu
Paper 1
Matteo Interlandi
Matteo Interlandi
Paper 1
Todd Phillips
Todd Phillips
Paper 1
Niu Feng
Niu Feng
Paper 1
Adrian Weller
Adrian Weller
Paper 1
Cedric Renggli
Cedric Renggli
Paper 1
IJCAI 2020cvpr2020Papers, Authors, Comments