强化学习中基于马尔可夫决策过程的标准Q-Learning算法可以取得较优路径, 但是方法存在收敛速度慢及规划效率低等问题, 无法直接应用于真实环境。针对此问题, 提出一种基于势能场知识的Q-Learning移动机器人路径规划算法。通过引入环境的势能值作为搜索启发信息对Q值进行初始化, 从而在学习初期便能引导移动机器人快速收敛, 改变了传统强化学习过程的盲目性, 适用于真实环境中直接学习。仿真实验表明, 与现有的算法相比, 所提算法不仅提高了收敛速度, 而且还缩短了学习时间, 使得移动机器人能够迅速找到一条较优的无碰撞路径。

The standard Q-Learning algorithm based on Markov decision process in reinforcement learning can obtain an optimum path, but the method has the shortcomings of slow convergence rate and low planning efficiency, and thus can not be directly applied to the real environment. This paper proposes a Q-Learning path planning algorithm for mobile robots based on the potential energy field knowledge.By introducing the potential energy value into the environment as the search heuristic information to initialize the Q value, the rapid convergence of the mobile robot can be guided in the early stage of learning, and the blindness of the traditional reinforcement learning process is avoided, which makes it suitable for direct learning in a real environment. The simulation result shows that: Compared with existing algorithms, the proposed algorithm not only improves the convergence speed, but also shortens the learning time, which can make the mobile robot find a better collision-free path quickly.