Point and Instruct: Enabling Precise Image Editing by Unifying Direct Manipulation and Text Instructions
CoRR(2024)
摘要
Machine learning has enabled the development of powerful systems capable of
editing images from natural language instructions. However, in many common
scenarios it is difficult for users to specify precise image transformations
with text alone. For example, in an image with several dogs, it is difficult to
select a particular dog and move it to a precise location. Doing this with text
alone would require a complex prompt that disambiguates the target dog and
describes the destination. However, direct manipulation is well suited to
visual tasks like selecting objects and specifying locations. We introduce
Point and Instruct, a system for seamlessly combining familiar direct
manipulation and textual instructions to enable precise image manipulation.
With our system, a user can visually mark objects and locations, and reference
them in textual instructions. This allows users to benefit from both the visual
descriptiveness of natural language and the spatial precision of direct
manipulation.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要