Dynamic Backtracking in GFlowNets: Enhancing Decision Steps with Reward-Dependent Adjustment Mechanisms
arxiv(2024)
摘要
Generative Flow Networks (GFlowNets or GFNs) are probabilistic models
predicated on Markov flows, and they employ specific amortization algorithms to
learn stochastic policies that generate compositional substances including
biomolecules, chemical materials, etc. With a strong ability to generate
high-performance biochemical molecules, GFNs accelerate the discovery of
scientific substances, effectively overcoming the time-consuming,
labor-intensive, and costly shortcomings of conventional material discovery
methods. However, previous studies rarely focus on accumulating exploratory
experience by adjusting generative structures, which leads to disorientation in
complex sampling spaces. Efforts to address this issue, such as LS-GFN, are
limited to local greedy searches and lack broader global adjustments. This
paper introduces a novel variant of GFNs, the Dynamic Backtracking GFN
(DB-GFN), which improves the adaptability of decision-making steps through a
reward-based dynamic backtracking mechanism. DB-GFN allows backtracking during
the network construction process according to the current state's reward value,
thereby correcting disadvantageous decisions and exploring alternative pathways
during the exploration process. When applied to generative tasks involving
biochemical molecules and genetic material sequences, DB-GFN outperforms GFN
models such as LS-GFN and GTB, as well as traditional reinforcement learning
methods, in sample quality, sample exploration quantity, and training
convergence speed. Additionally, owing to its orthogonal nature, DB-GFN shows
great potential in future improvements of GFNs, and it can be integrated with
other strategies to achieve higher search performance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要