Towards Interactively Improving ML Data Preparation Code via "Shadow Pipelines"
arxiv(2024)
摘要
Data scientists develop ML pipelines in an iterative manner: they repeatedly
screen a pipeline for potential issues, debug it, and then revise and improve
its code according to their findings. However, this manual process is tedious
and error-prone. Therefore, we propose to support data scientists during this
development cycle with automatically derived interactive suggestions for
pipeline improvements. We discuss our vision to generate these suggestions with
so-called shadow pipelines, hidden variants of the original pipeline that
modify it to auto-detect potential issues, try out modifications for
improvements, and suggest and explain these modifications to the user. We
envision to apply incremental view maintenance-based optimisations to ensure
low-latency computation and maintenance of the shadow pipelines. We conduct
preliminary experiments to showcase the feasibility of our envisioned approach
and the potential benefits of our proposed optimisations.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要