研究兴趣

论文共 18 篇作者统计合作学者相似作者

按年份排序按引用量排序主题筛选期刊级别筛选合作者筛选合作机构筛选
时间
引用量
主题
期刊级别
合作者
合作机构
引用0浏览0EI引用
0
0
加载更多
作者统计
  • 合作者
  • 学生
  • 导师
暂无相似学者,你可以通过学者研究领域进行搜索筛选
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
' that allows the reversal of operators. We demonstrate the elegance of the operators approach in a suitable programming language consisting of generic linear algebra operators such as Julia \\cite{bezanson2017julia}, and that it is possible to realize this abstraction in code. Our implementation shows how generic linear algebra can allow operators as elements of matrices, and without rewriting any code, the software carries through to completion giving the correct answer. ","add_method":true,"authors":[{"id":"560286ad45cedb3395fc699c","name":"Alan Edelman"},{"id":"64352fa0f2699869fc1e19f1","name":"Ekin Akyurek"},{"id":"542d3fe4dabfae12b980055a","name":"Yuyang Wang"}],"create_time":"2021-01-19T03:12:41.479Z","flags":[{"flag":"affirm_author","person_id":"5630fa0545cedb3399c28fe6"}],"hashs":{"h1":"-backpropagation backsubstitution backslash"},"id":"60064e299e795e124e208c1e","lang":"en","num_citation":0,"pdf":"https:\u002F\u002Fcz5waila03cyo0tux1owpyofgoryroob.aminer.cn\u002FFE\u002F34\u002F9B\u002FFE349B268D3212D71089D926325B4C03.pdf","title":"BACKpropagation through BACK substitution with a BACKslash","urls":["https:\u002F\u002Fopenalex.org\u002FW4361230439","db\u002Fjournals\u002Fcorr\u002Fcorr2303.html#abs-2303-15449","https:\u002F\u002Fdoi.org\u002F10.48550\u002FarXiv.2303.15449","https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.15449","https:\u002F\u002Fscholar.google.com.hk\u002Fcitations?view_op=view_citation&hl=zh-CN&user=QVBIKh4AAAAJ&pagesize=100&sortby=pubdate&citation_for_view=QVBIKh4AAAAJ:SjuI4pbJlxcC"],"versions":[{"id":"60064e299e795e124e208c1e","sid":"60064e299e795e124e208c1e","src":"user-5f8cf7e04c775ec6fa691c92","year":2020},{"id":"6423ac6c90e50fcafd55c06f","sid":"2303.15449","src":"arxiv","year":2023},{"id":"6456477dd68f896efae21b17","sid":"journals\u002Fcorr\u002Fabs-2303-15449","src":"dblp","year":2023},{"id":"656fe2d6939a5f4082a74508","sid":"W4361230439","src":"openalex","year":2023}],"year":2023},{"abstract":" Despite their unprecedented success, even the largest language models make mistakes. Similar to how humans learn and improve using feedback, previous work proposed providing language models with natural language feedback to guide them in repairing their outputs. Because human-generated critiques are expensive to obtain, researchers have devised learned critique generators in lieu of human critics while assuming one can train downstream models to utilize generated feedback. However, this approach does not apply to black-box or limited access models such as ChatGPT, as they cannot be fine-tuned. Moreover, in the era of large general-purpose language agents, fine-tuning is neither computationally nor spatially efficient as it results in multiple copies of the network. In this work, we introduce RL4F (Reinforcement Learning for Feedback), a multi-agent collaborative framework where the critique generator is trained to maximize end-task performance of GPT-3, a fixed model more than 200 times its size. RL4F produces critiques that help GPT-3 revise its outputs. We study three datasets for action planning, summarization and alphabetization and show improvements (~5% on average) in multiple text similarity metrics over strong baselines across all three tasks. ","authors":[{"id":"64b916a016b3d9192137baa3","name":"Afra Feyza Akyürek"},{"id":"64352fa0f2699869fc1e19f1","name":"Ekin Akyürek"},{"id":"53f42957dabfaeb22f3d42cd","name":"Aman Madaan"},{"id":"63724c37ec88d95668cc9755","name":"Ashwin Kalyan"},{"id":"540561dddabfae8faa5ca7ee","name":"Peter Clark"},{"id":"5601bd5c45cedb3395eab6a6","name":"Derry Wijaya"},{"id":"53f48030dabfaec09f29ea31","name":"Niket Tandon"}],"create_time":"2023-05-16T04:58:21.706Z","hashs":{"h1":"rgnlf","h3":"rlrmo"},"id":"6462f13cd68f896efa911ee9","num_citation":0,"pages":{"end":"7733","start":"7716"},"pdf":"https:\u002F\u002Fcz5waila03cyo0tux1owpyofgoryroob.aminer.cn\u002FEC\u002F59\u002F48\u002FEC5948C9B9DC115DA06411A4603494FB.pdf","title":"RL4F: Generating Natural Language Feedback with Reinforcement Learning\n for Repairing Model Outputs","update_times":{"u_c_t":"2024-01-02T12:12:47.916Z"},"urls":["https:\u002F\u002Fopenalex.org\u002FW4376654477","db\u002Fconf\u002Facl\u002Facl2023-1.html#AkyurekAKCWT23","https:\u002F\u002Faclanthology.org\u002F2023.acl-long.427","https:\u002F\u002Faclanthology.org\u002F2023.acl-long.427\u002F","db\u002Fjournals\u002Fcorr\u002Fcorr2305.html#abs-2305-08844","https:\u002F\u002Fdoi.org\u002F10.48550\u002FarXiv.2305.08844","https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.08844"],"venue":{"info":{"name":"conf_acl"},"volume":"abs\u002F2305.08844"},"venue_hhb_id":"5ea1afddedb6e7d53c00c104","versions":[{"id":"6462f13cd68f896efa911ee9","sid":"2305.08844","src":"arxiv","year":2023},{"id":"6479e3add68f896efa4e705b","sid":"journals\u002Fcorr\u002Fabs-2305-08844","src":"dblp","year":2023},{"id":"64ae66e53fda6d7f06849331","sid":"2023.acl-long.427","src":"conf_acl","year":2023},{"id":"64c78b9f3fda6d7f06db9a3d","sid":"conf\u002Facl\u002FAkyurekAKCWT23","src":"dblp","year":2023},{"id":"6578d88f939a5f40826f2899","sid":"W4376654477","src":"openalex","year":2023}],"year":2023},{"abstract":" Few-shot class incremental learning -- the problem of updating a trained classifier to discriminate among an expanded set of classes with limited labeled data -- is a key challenge for machine learning systems deployed in non-stationary environments. Existing approaches to the problem rely on complex model architectures and training procedures that are difficult to tune and re-use. In this paper, we present an extremely simple approach that enables the use of ordinary logistic regression classifiers for few-shot incremental learning. The key to this approach is a new family of subspace regularization schemes that encourage weight vectors for new classes to lie close to the subspace spanned by the weights of existing classes. When combined with pretrained convolutional feature extractors, logistic regression models trained with subspace regularization outperform specialized, state-of-the-art approaches to few-shot incremental image classification by up to 22% on the miniImageNet dataset. Because of its simplicity, subspace regularization can be straightforwardly extended to incorporate additional background information about the new classes (including class names and descriptions specified in natural language); these further improve accuracy by up to 2%. Our results show that simple geometric regularization of class representations offers an effective tool for continual learning. ","authors":[{"id":"64b916a016b3d9192137baa3","name":"Afra Feyza Akyürek"},{"id":"64352fa0f2699869fc1e19f1","name":"Ekin Akyürek"},{"id":"5601bd5c45cedb3395eab6a6","name":"Derry Wijaya"},{"id":"64fad981bb3ef99d3528b551","name":"Jacob Andreas"}],"citations":{"google_citation":6,"last_citation":5},"create_time":"2021-10-15T13:44:50.479Z","hashs":{"h1":"srfci","h3":"l"},"id":"6168f19c5244ab9dcbe2f90d","keywords":["few-shot class incremental learning","incremental learning","incremental classification","subspace regularization","manifold regularization","few-shot learning"],"lang":"en","num_citation":1,"pdf":"https:\u002F\u002Fcz5waila03cyo0tux1owpyofgoryroob.aminer.cn\u002F5A\u002F0D\u002F2F\u002F5A0D2FAED9EA1C13B25DB833F3481F95.pdf","pdf_src":["https:\u002F\u002Farxiv.org\u002Fpdf\u002F2110.07059","https:\u002F\u002Fopenreview.net\u002Fpdf\u002F8e50ad238e1d64a51be6a65f6da2e3606da02bef.pdf"],"title":"Subspace Regularizers for Few-Shot Class Incremental Learning","update_times":{"u_a_t":"2022-10-19T08:53:21.21Z","u_c_t":"2024-01-02T10:29:14.03Z","u_v_t":"2022-10-18T17:25:38.756Z"},"urls":["https:\u002F\u002Fdblp.org\u002Frec\u002Fconf\u002Ficlr\u002FAkyurekAWA22","https:\u002F\u002Fsemanticscholar.org\u002Fpaper\u002Ff8d77d2d33a86a02c7d2ddd3dcbb1dc48ccf265c","https:\u002F\u002Fopenreview.net\u002Fforum?id=boJy41J-tnQ","https:\u002F\u002Fsemanticscholar.org\u002Fpaper\u002F3a43859a9feecc53c0104d27c07577973c582f75","db\u002Fjournals\u002Fcorr\u002Fcorr2110.html#abs-2110-07059","https:\u002F\u002Farxiv.org\u002Fabs\u002F2110.07059"],"venue":{"info":{"name":"International Conference on Learning Representations (ICLR)","name_s":"ICLR"},"volume":"abs\u002F2110.07059"},"venue_hhb_id":"5ea1d518edb6e7d53c0100cb","versions":[{"id":"6168f19c5244ab9dcbe2f90d","sid":"2110.07059","src":"arxiv","year":2022},{"id":"6257c5a75aee126c0f467854","sid":"boJy41J-tnQ","src":"conf_iclr","vsid":"ICLR.cc\u002F2022\u002FConference","year":2022},{"id":"634d805190e50fcafd4dfa25","sid":"conf\u002Ficlr\u002FAkyurekAWA22","src":"dblp","vsid":"conf\u002Ficlr","year":2022},{"id":"61c8f9c05244ab9dcbd5f0c0","sid":"3a43859a9feecc53c0104d27c07577973c582f75","src":"semanticscholar"},{"id":"627ce0085aee126c0f5fe069","sid":"f8d77d2d33a86a02c7d2ddd3dcbb1dc48ccf265c","src":"semanticscholar"},{"id":"645647b8d68f896efae3cfda","sid":"journals\u002Fcorr\u002Fabs-2110-07059","src":"dblp","year":2021},{"id":"65778d4d939a5f4082dd61ca","sid":"W4286905055","src":"openalex","vsid":"conf\u002Ficlr","year":2022},{"id":"65781fa0939a5f40824db60f","sid":"W3207580420","src":"openalex","vsid":"conf\u002Ficlr","year":2022}],"year":2022},{"abstract":" Language models (LMs) have been shown to memorize a great deal of factual knowledge contained in their training data. But when an LM generates an assertion, it is often difficult to determine where it learned this information and whether it is true. In this paper, we propose the problem of fact tracing: identifying which training examples taught an LM to generate a particular factual assertion. Prior work on training data attribution (TDA) may offer effective tools for identifying such examples, known as \"proponents\". We present the first quantitative benchmark to evaluate this. We compare two popular families of TDA methods -- gradient-based and embedding-based -- and find that much headroom remains. For example, both methods have lower proponent-retrieval precision than an information retrieval baseline (BM25) that does not have access to the LM at all. We identify key challenges that may be necessary for further improvement such as overcoming the problem of gradient saturation, and also show how several nuanced implementation details of existing neural TDA methods can significantly improve overall fact tracing performance. ","authors":[{"id":"64352fa0f2699869fc1e19f1","name":"Ekin Akyürek"},{"id":"562d24f045cedb3398d667b5","name":"Tolga Bolukbasi"},{"id":"6565b3affbb0f21d747255d2","name":"Frederick Liu"},{"id":"652e6e7250dee4c4226dfeab","name":"Binbin Xiong"},{"name":"Ian Tenney"},{"id":"64fad981bb3ef99d3528b551","name":"Jacob Andreas"},{"name":"Kelvin Guu"}],"create_time":"2022-05-24T13:48:36.958Z","doi":"10.48550\u002Farxiv.2205.11482","hashs":{"h1":"tklmb","h3":"td"},"id":"628c4ce65aee126c0ff59f82","keywords":["factual knowledge","language models","training data"],"lang":"en","num_citation":0,"pdf":"https:\u002F\u002Fcz5waila03cyo0tux1owpyofgoryroob.aminer.cn\u002F98\u002F6B\u002F1D\u002F986B1D45873BF69E8B32A937B65FE616.pdf","pdf_src":["https:\u002F\u002Farxiv.org\u002Fpdf\u002F2205.11482"],"title":"Towards Tracing Factual Knowledge in Language Models Back to the\n Training Data","urls":["https:\u002F\u002Fopenalex.org\u002FW4307411724","https:\u002F\u002Fdoi.org\u002F10.48550\u002Farxiv.2205.11482","https:\u002F\u002Farxiv.org\u002Fabs\u002F2205.11482"],"versions":[{"id":"628c4ce65aee126c0ff59f82","sid":"2205.11482","src":"arxiv","year":2022},{"id":"65783d40939a5f408281e9df","sid":"W4307411724","src":"openalex","year":2022},{"id":"63d7ae8590e50fcafdad4d5c","sid":"journals\u002Fcorr\u002Fabs-2205-11482","src":"dblp","vsid":"journals\u002Fcorr","year":2022}],"year":2022},{"abstract":" Neural sequence models, especially transformers, exhibit a remarkable capacity for in-context learning. They can construct new predictors from sequences of labeled examples $(x, f(x))$ presented in the input without further parameter updates. We investigate the hypothesis that transformer-based in-context learners implement standard learning algorithms implicitly, by encoding smaller models in their activations, and updating these implicit models as new examples appear in the context. Using linear regression as a prototypical problem, we offer three sources of evidence for this hypothesis. First, we prove by construction that transformers can implement learning algorithms for linear models based on gradient descent and closed-form ridge regression. Second, we show that trained in-context learners closely match the predictors computed by gradient descent, ridge regression, and exact least-squares regression, transitioning between different predictors as transformer depth and dataset noise vary, and converging to Bayesian estimators for large widths and depths. Third, we present preliminary evidence that in-context learners share algorithmic features with these predictors: learners' late layers non-linearly encode weight vectors and moment matrices. These results suggest that in-context learning is understandable in algorithmic terms, and that (at least in the linear case) learners may rediscover standard estimation algorithms. Code and reference implementations are released at https:\u002F\u002Fgithub.com\u002Fekinakyurek\u002Fgoogle-research\u002Fblob\u002Fmaster\u002Fincontext. ","authors":[{"id":"64352fa0f2699869fc1e19f1","name":"Ekin Akyürek","org":"Massachusetts Institute of Technology","orgid":"62331e350a6eb147dca8a7ec","orgs":["Massachusetts Institute of Technology"]},{"id":"53f4694ddabfaeb1a7c989d2","name":"Dale Schuurmans","org":"University of Alberta","orgid":"5f71b2941c455f439fe3cd7c","orgs":["University of Alberta"]},{"id":"64fad981bb3ef99d3528b551","name":"Jacob Andreas","org":"Massachusetts Institute of Technology","orgid":"62331e350a6eb147dca8a7ec","orgs":["Massachusetts Institute of Technology"]},{"id":"53f46708dabfaec09f242380","name":"Tengyu Ma","org":"Stanford University","orgid":"62331e330a6eb147dca8a6e8","orgs":["Stanford University"]},{"id":"53f4334fdabfaedce550f474","name":"Denny Zhou","org":"Google Brain","orgid":"5f71b2d21c455f439fe3e823","orgs":["Google Brain"]}],"citations":{"google_citation":0,"last_citation":0},"create_time":"2022-11-29T05:04:43.293Z","hashs":{"h1":"laili","h3":"lm"},"id":"6385789190e50fcafdf4c6be","keywords":["in-context learning","transformers","sequence models","deep learning","meta learning"],"lang":"en","num_citation":157,"pdf":"https:\u002F\u002Fcz5waila03cyo0tux1owpyofgoryroob.aminer.cn\u002FA7\u002FC8\u002F22\u002FA7C822733C120428745609F5B04D5E5F.pdf","pdf_src":["https:\u002F\u002Farxiv.org\u002Fpdf\u002F2211.15661"],"title":"What learning algorithm is in-context learning? Investigations with\n linear models","update_times":{"u_a_t":"2022-11-30T14:54:51.759Z","u_c_t":"2024-01-02T12:42:32.647Z"},"urls":["https:\u002F\u002Fopenreview.net\u002Fforum?id=0g0X4H8yN4I","db\u002Fconf\u002Ficlr\u002Ficlr2023.html#AkyurekSA0Z23","https:\u002F\u002Fopenreview.net\u002Fpdf?id=0g0X4H8yN4I","https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.15661"],"versions":[{"id":"6385789190e50fcafdf4c6be","sid":"2211.15661","src":"arxiv","year":2022},{"id":"63dcdb422c26941cf00b6052","sid":"0g0X4H8yN4I","src":"conf_iclr","vsid":"ICLR.cc\u002F2023\u002FConference","year":2023},{"id":"6433f61f90e50fcafd6c035a","sid":"2023#0g0X4H8yN4I","src":"conf_iclr","vsid":"ICLR.cc\u002F2023\u002FConference","year":2023},{"id":"64a407ddd68f896efaf1cbb4","sid":"conf\u002Ficlr\u002FAkyurekSA0Z23","src":"dblp","year":2023},{"id":"63d7ae8490e50fcafdad2984","sid":"journals\u002Fcorr\u002Fabs-2211-15661","src":"dblp","vsid":"journals\u002Fcorr","year":2022}],"year":2022},{"abstract":"Neural sequence models, especially transformers, exhibit a remarkable capacity for in-context learning. They can construct new predictors from sequences of labeled examples $(x, f(x))$ presented in the input without further parameter updates. We investigate the hypothesis that transformer-based in-context learners implement standard learning algorithms implicitly, by encoding smaller models in their activations, and updating these implicit models as new examples appear in the context. Using linear regression as a prototypical problem, we offer three sources of evidence for this hypothesis. First, we prove by construction that transformers can implement learning algorithms for linear models based on gradient descent and closed-form ridge regression. Second, we show that trained in-context learners closely match the predictors computed by gradient descent, ridge regression, and exact least-squares regression, transitioning between different predictors as transformer depth and dataset noise vary, and converging to Bayesian estimators for large widths and depths. Third, we present preliminary evidence that in-context learners share algorithmic features with these predictors: learners' late layers non-linearly encode weight vectors and moment matrices. These results suggest that in-context learning is understandable in algorithmic terms, and that (at least in the linear case) learners may rediscover standard estimation algorithms. Code and reference implementations are released at https:\u002F\u002Fgithub.com\u002Fekinakyurek\u002Fgoogle-research\u002Fblob\u002Fmaster\u002Fincontext.","authors":[{"id":"64352fa0f2699869fc1e19f1","name":"Ekin Akyürek"},{"id":"53f4694ddabfaeb1a7c989d2","name":"Dale Schuurmans"},{"name":"Jacob Andreas"},{"id":"53f46708dabfaec09f242380","name":"Tengyu Ma"},{"id":"53f4334fdabfaedce550f474","name":"Denny Zhou"}],"create_time":"2024-02-09T15:08:24.66Z","hashs":{"h1":"laili","h3":"lm"},"id":"6578b241939a5f408236efd1","keywords":["learning","models","algorithm","in-context"],"num_citation":0,"title":"What learning algorithm is in-context learning? Investigations with\n linear models","urls":["https:\u002F\u002Fopenalex.org\u002FW4310509152"],"venue":{"info":{"name":"arXiv (Cornell University)"}},"versions":[{"id":"6578b241939a5f408236efd1","sid":"W4310509152","src":"openalex"}],"year":2022},{"authors":[{"id":"64352fa0f2699869fc1e19f1","name":"Ekin Akyürek"},{"id":"562d24f045cedb3398d667b5","name":"Tolga Bolukbasi"},{"id":"6565b3affbb0f21d747255d2","name":"Frederick Liu"},{"id":"652e6e7250dee4c4226dfeab","name":"Binbin Xiong"},{"id":"6176c0fb60a96543aa816ee1","name":"Ian Tenney"},{"id":"64fad981bb3ef99d3528b551","name":"Jacob Andreas"},{"id":"61770f8960a96543aa816f7b","name":"Kelvin Guu"}],"create_time":"2023-04-07T15:35:21.079Z","hashs":{"h1":"tklmb","h3":"td"},"id":"6426ed4d90e50fcafd44a50d","num_citation":4,"pages":{"end":"2446","start":"2429"},"title":"Towards Tracing Knowledge in Language Models Back to the Training Data.","update_times":{"u_c_t":"2023-07-18T07:00:22.419Z","u_v_t":"2023-04-15T03:19:50.733Z"},"urls":["db\u002Fconf\u002Femnlp\u002Femnlp2022f.html#AkyurekBLXTAG22","https:\u002F\u002Faclanthology.org\u002F2022.findings-emnlp.180"],"venue":{"info":{"name":"EMNLP (Findings)"}},"venue_hhb_id":"5eba7087edb6e7d53c1009a5","versions":[{"id":"6426ed4d90e50fcafd44a50d","sid":"conf\u002Femnlp\u002FAkyurekBLXTAG22","src":"dblp","vsid":"conf\u002Femnlp","year":2022},{"id":"6578479f939a5f408292397a","sid":"W4281557775","src":"openalex","vsid":"conf\u002Femnlp","year":2022}],"year":2022}],"profilePubsTotal":18,"profilePatentsPage":0,"profilePatents":null,"profilePatentsTotal":null,"profilePatentsEnd":false,"profileProjectsPage":1,"profileProjects":{"success":true,"msg":"","data":null,"log_id":"2clvZRxsaTvDbDAi1C1AJELhp7s"},"profileProjectsTotal":0,"newInfo":null,"checkDelPubs":[]}};