The synergy of the two paradigms, neural network and fuzzy inference system, has given rise to rapidly emerging filed, neuro-fuzzy systems. Evolving neuro-fuzzy systems are intended to use online learning to extract knowledge from data and perform a high-level adaptation of the network structure. We explore the potential of evolving neuro-fuzzy systems in reinforcement learning (RL) applications. In this paper, a novel on-line sequential learning evolving neuro-fuzzy model design for RL is proposed. We develop a dynamic evolving fuzzy neural network (DENFIS) function approximation approach to RL systems. Potential of this approach is demonstrated through a case study-two-link robot manipulator. Simulation results have demonstrated that the proposed approach performs well in reinforcement learning problems. © 2014 IFAC.