earticle

영어

Reinforcement Learning is an area of machine learning that studies the problem of solving sequential decision making problems. The agent must learn behavior through trial-and-error interaction with a dynamic environment. Learning efficiently in large scale problems and complex tasks demands a decomposition of the original complex task into simple and smaller subtasks. In this paper, we present a subgoal-based method for automatically creating useful skills in reinforcement Learning. Our method identifies subgoals using a local graph clustering algorithm. The main advantage of the proposed algorithm is that only the local information of the graph is considered to cluster the agent state space. Clustering of the transition graphs corresponding to MDPs can be performed in linear time using the proposed method. Subgoals discovered by the algorithm are then used to generate skills using the option framework. Experimental results show that the proposed subgoal discovery algorithm has a dramatic effect on the learning performance.