Referential communication in heterogeneous communities of pre-trained visual deep networks
arxiv(2023)
摘要
As large pre-trained image-processing neural networks are being embedded in
autonomous agents such as self-driving cars or robots, the question arises of
how such systems can communicate with each other about the surrounding world,
despite their different architectures and training regimes. As a first step in
this direction, we systematically explore the task of referential
communication in a community of heterogeneous state-of-the-art pre-trained
visual networks, showing that they can develop, in a self-supervised way, a
shared protocol to refer to a target object among a set of candidates. This
shared protocol can also be used, to some extent, to communicate about
previously unseen object categories of different granularity. Moreover, a
visual network that was not initially part of an existing community can learn
the community's protocol with remarkable ease. Finally, we study, both
qualitatively and quantitatively, the properties of the emergent protocol,
providing some evidence that it is capturing high-level semantic features of
objects.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要