تقرير
Gradient Informed Proximal Policy Optimization
| العنوان: | Gradient Informed Proximal Policy Optimization |
|---|---|
| المؤلفون: | Son, Sanghyun, Zheng, Laura Yu, Sullivan, Ryan, Qiao, Yi-Ling, Lin, Ming C. |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Machine Learning, Computer Science - Artificial Intelligence |
| الوصف: | We introduce a novel policy learning method that integrates analytical gradients from differentiable environments with the Proximal Policy Optimization (PPO) algorithm. To incorporate analytical gradients into the PPO framework, we introduce the concept of an {\alpha}-policy that stands as a locally superior policy. By adaptively modifying the {\alpha} value, we can effectively manage the influence of analytical policy gradients during learning. To this end, we suggest metrics for assessing the variance and bias of analytical gradients, reducing dependence on these gradients when high variance or bias is detected. Our proposed approach outperforms baseline algorithms in various scenarios, such as function optimization, physics simulations, and traffic control environments. Our code can be found online: https://github.com/SonSang/gippoTest. Comment: 27 pages, NeurIPS 2023 Conference |
| نوع الوثيقة: | Working Paper |
| الوصول الحر: | http://arxiv.org/abs/2312.08710Test |
| رقم الانضمام: | edsarx.2312.08710 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |