九九热视频精品在线_国产成人mv视频在线观看_国产精品三级一区二区_亚洲精品国产av成拍色拍婷婷_久久伊人一区二区_国产高清精品网站_九九99热久久精品离线6_国产在线一卡2卡三卡4卡免费_日韩美女乱淫aaa高清视频_亚洲大胆_啪啪一级视频_午夜av免费观看_国产高清精品福利私拍国产写真_亚洲婷婷五月综合狠狠_日韩欧美一区二区三区视频_岛国av免费看_精品国产乱码久久久久久久软件_亚洲国产成人精品无码区99_www日本色_免费黄色看片

基于上表所的統(tǒng)的實(shí)驗(yàn)結(jié)果，結(jié)合所提出的探索中的主要挑戰(zhàn)，綜述中詳細(xì)分析了各類探索策略在這些任務(wù)上的優(yōu)劣。

關(guān)于探索策略的開放問題和未來方向

盡管探索策略的研究取得了常前沿的進(jìn)展，但是仍然存在些問題沒有被完全解決。綜述主要從以下五個(gè)度討論了尚未解決的問題。

在規(guī)模動(dòng)作空間的探索。在規(guī)模動(dòng)作空間上，融合表征學(xué)習(xí)、動(dòng)作語義等法，降低探索算法的計(jì)算復(fù)雜度仍然是個(gè)急需解決的問題。

在復(fù)雜任務(wù)（時(shí)間步較長、極度稀疏、延遲的獎(jiǎng)勵(lì)設(shè)置）上的探索，雖然取得了一定的進(jìn)展，如蒙特祖瑪?shù)膹?fù)仇，但這些解決辦法代價(jià)通常較，甚要借助量類先驗(yàn)知識。這其中還存在較多普遍性的問題值得探索。

噪聲問題。現(xiàn)有的些解決案都需要額外估計(jì)動(dòng)態(tài)模型或狀態(tài)表征，這疑增加了計(jì)算消耗。除此之外，針對噪聲問題，利對抗訓(xùn)練等式增加探索的魯棒性也是值得研究的問題。

收斂性。在向不確定性的探索中，線性MDP下認(rèn)知不確定性是可以收斂到0的，但在深度神經(jīng)絡(luò)下維度爆炸使得收斂困難。對于向內(nèi)在激勵(lì)的探索，內(nèi)在激勵(lì)往往是啟發(fā)式設(shè)計(jì)的，缺乏理論上合理性論證。

多智能體探索。多智能體探索的研究還處于起步階段，尚未很好地解決上述問題，如局部觀測、不穩(wěn)定、協(xié)同探索等。

主要作者介紹

楊天培博，現(xiàn)任University of Alberta博后研究員。楊博在2021年從天津?qū)W取得博學(xué)位，她的研究興趣主要包括遷移強(qiáng)化學(xué)習(xí)和多智能體強(qiáng)化學(xué)習(xí)。楊博致于利遷移學(xué)習(xí)、層次強(qiáng)化學(xué)習(xí)、對建模等技術(shù)提升強(qiáng)化學(xué)習(xí)和多智能體強(qiáng)化學(xué)習(xí)的學(xué)習(xí)效率和性能。前已在IJCAI、AAAI、ICLR、NeurIPS等頂級會議發(fā)表論余篇，擔(dān)任多個(gè)會議期刊的審稿。

湯宏垚博，天津?qū)W博在讀。湯博的研究興趣主要包括強(qiáng)化學(xué)習(xí)、表征學(xué)習(xí)，其學(xué)術(shù)成果發(fā)表在AAAI、IJCAI、NeurIPS、ICML等頂級會議期刊上。

甲博，哈爾濱業(yè)學(xué)博在讀，研究興趣包括探索與利、離線強(qiáng)化學(xué)習(xí)，學(xué)術(shù)成果發(fā)表在ICML、NeurIPS等。

劉毅，天津?qū)W智能與計(jì)算學(xué)部碩在讀，研究興趣主要包括強(qiáng)化學(xué)習(xí)、離線強(qiáng)化學(xué)習(xí)等。

郝建業(yè)博，天津?qū)W智能與計(jì)算學(xué)部副教授。主要研究向?yàn)樯疃葟?qiáng)化學(xué)習(xí)、多智能體系統(tǒng)。發(fā)表智能領(lǐng)域國際會議和期刊論100余篇，專著2部。主持參與國家基委、科技部、天津市智能重等科研項(xiàng)10余項(xiàng)，研究成果榮獲ASE2019、DAI2019、CoRL2020最佳論獎(jiǎng)等，同時(shí)在游戲AI、告及推薦、動(dòng)駕駛、絡(luò)優(yōu)化等領(lǐng)域落地應(yīng)。

Reference

[1]P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of the multiarmed bandit problem,” Machinelearning, vol. 47, no. 2-3, pp. 235–256, 2002.

[2]I. Osband, B. V. Roy, and Z. Wen, “Generalization and exploration via randomized value functions,” inInternational Conference on Machine Learning, 2016, pp. 2377–2386.

[3]I. Osband, C. Blundell, A. Pritzel, and B. V. Roy, “Deep exploration via bootstrapped DQN,” in Advances inNeural Information Processing Systems 29, 2016, pp. 4026–4034.

[4]K. Ciosek, Q. Vuong, R. Loftin, and K. Hofmann, “Better exploration with optimistic actor critic,” inAdvances in Neural Information Processing Systems, 2019, pp. 1785–1796.

[5]C. Bai, L. Wang, L. Han, J. Hao, A. Garg, P. Liu, and Z. Wang, “Principled exploration via optimisticbootstrapping and backward induction,” in International Conference on Machine Learning, 2021.

[6]J. Kirschner and A. Krause, “Information directed sampling and bandits with heteroscedastic noise,” inConference On Learning Theory, 2018, pp. 358–384.

[7]B. Mavrin, H. Yao, L. Kong, K. Wu, and Y. Yu, “Distributional reinforcement learning for efficientexploration,” in International Conference on Machine Learning, 2019, pp. 4424–4434.

[8]D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, “Curiosity-driven exploration by self-supervisedprediction,” in International Conference on Machine Learning, 2017, pp. 2778–2787.

[9]H. Kim, J. Kim, Y. Jeong, S. Levine, and H. O. Song, “EMI: exploration with mutual information,” inInternational Conference on Machine Learning, 2019, pp. 3360–3369.

[10]Y. Burda, H. Edwards, A. J. Storkey, and O. Klimov, “Exploration by random network distillation,” inInternational Conference on Learning Representations, 2019.

[11]R. Y. Tao, V. Franois-Lavet, and J. Pineau, “Novelty search in representational space for sample efficientexploration,” in Advances in Neural Information Processing Systems, 2020.

[12]Y. Du, L. Han, M. Fang, J. Liu, T. Dai, and D. Tao, “LIIR: learning individual intrinsic reward in multi-agentreinforcement learning,” in Advances in Neural Information Processing Systems, 2019, pp. 4405– 4416

[13]R. Houthooft, X. Chen, Y. Duan, J. Schulman, F. D. Turck, and P. Abbeel, “VIME: variational information maximizing exploration,” in Advances in Neural Information Processing Systems, 2016, pp. 1109–1117.

[14]T. Wang, J. Wang, Y. Wu, and C. Zhang, “Influence-based multi-agent exploration,” in International Conference on Learning Representations, 2020

[15]D. Horgan, J. Quan, D. Budden, G. Barth-Maron, M. Hessel, H. van Hasselt, and D. Silver, “Distributed prioritized experience replay,” in International Conference on Learning Representations, 2018.

[16]S. Kapturowski, G. Ostrovski, J. Quan, R. Munos, and W. Dabney, “Recurrent experience replay in distributed reinforcement learning,” in International Conference on Learning Representations, 2019.

[17]M. Fortunato, M. G. Azar, B. Piot, J. Menick, M. Hessel, I. Osband, A. Graves, V. Mnih, R. Munos, D. Hassabis, O. Pietquin, C. Blundell, and S. Legg, “Noisy networks for exploration,” in International Conference on Learning Representations, 2018.

[18]E. Adrien, H. Joost, L. Joel, S. K. O, and C. Jeff, “First return, then explore,” Nature, vol. 590, no. 7847, pp.580–586, 2021.

[19]A. Mahajan, T. Rashid, M. Samvelyan, and S. Whiteson, “MAVEN: multi-agent variational exploration,” inAdvances in Neural Information Processing Systems, 2019, pp. 7611–7622.