基本信息
浏览量:11
职业迁徙
个人简介
My research interests lie in AI reasoning and decision-making and my goal is to create AI systems that effectively scale with compute to process the world’s information and fast-forward scientific progress.
Research highlights from the last year include:
Automatic prompt engineering, which shows for the first time that when given access to trial-and-error, LLMs can control themselves through prompting often better than a human can (resulting in the discovery of a prompting trick that gives insight into the psychology of GPT).
STEVE-1, an instruction-following agent in Minecraft created using a novel methodology where we finetune a model pretrained on years of Minecraft videos on goal-relabeled data. STEVE-1 acts directly using keyboard and mouse input and follows open-ended text and visual instructions.
OpenWebMath, 14.7B tokens of mathematical documents gathered from Common Crawl for use in LLM pretraining and midtraining. OpenWebMath improves mathematical reasoning performance over 20x more effectively per-token than general-domain data and has already been used to train several open and closed models.
Llemma, the strongest open 7B and 34B base models for mathematical reasoning. These models are trained for up to 200B tokens primarily of OpenWebMath and show GPT-3.5-level performance with few-shot prompting even on held-out math evaluations.
Research highlights from the last year include:
Automatic prompt engineering, which shows for the first time that when given access to trial-and-error, LLMs can control themselves through prompting often better than a human can (resulting in the discovery of a prompting trick that gives insight into the psychology of GPT).
STEVE-1, an instruction-following agent in Minecraft created using a novel methodology where we finetune a model pretrained on years of Minecraft videos on goal-relabeled data. STEVE-1 acts directly using keyboard and mouse input and follows open-ended text and visual instructions.
OpenWebMath, 14.7B tokens of mathematical documents gathered from Common Crawl for use in LLM pretraining and midtraining. OpenWebMath improves mathematical reasoning performance over 20x more effectively per-token than general-domain data and has already been used to train several open and closed models.
Llemma, the strongest open 7B and 34B base models for mathematical reasoning. These models are trained for up to 200B tokens primarily of OpenWebMath and show GPT-3.5-level performance with few-shot prompting even on held-out math evaluations.
研究兴趣
论文共 10 篇作者统计合作学者相似作者
按年份排序按引用量排序主题筛选期刊级别筛选合作者筛选合作机构筛选
时间
引用量
主题
期刊级别
合作者
合作机构
arXiv (Cornell University) (2023)
ICLR 2023 (2022)
引用447浏览0EI引用
447
0
user-5aceb7ef530c7001b97ba534(2021)
引用1浏览0引用
1
0
user-5d54d98b530c705f51c2fe5a(2018)
引用3浏览0引用
3
0
作者统计
合作学者
合作机构
D-Core
- 合作者
- 学生
- 导师
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn