<p>Shiv Nadar University is a comprehensive, multidisciplinary, research-focused, and student-centric University offering a full range of academic programs at the undergraduate, postgraduate and doctoral level. With state-of-the-art infrastructure, the University comprises of academic wings, sophisticated labs, international standard sports facilities, amphitheaters, auditorium, conference rooms and smart classrooms. The University&rsquo;s goal is to become internationally recognized for the quality of its research and creative endeavors and their applicability to improving quality of life, generating new insights and expanding the boundaries of human knowledge creativity. Committed to excellence in teaching, research and service, the University aims to serve the higher education needs of India and the world beyond.</p>

Shiv Nadar University

In this study, an optimal control algorithm based on Hamilton-Jacobi- Bellman (HJB) equation, for the bounded robust controller design for finite-time-horizon nonlinear systems, is proposed. The HJB equation formulated using a suitable nonquadratic term in the performance functional to take care of magnitude constraints on the control input. Utilising the direct method of Lyapunov stability, we have proved the optimality of the controller with respect to a cost functional, that includes penalty on the control effort and the maximum bound on system uncertainty. The bounded controller requires the knowledge of the upper bound of system uncertainty. In the proposed algorithm, neural network is used to approximate the time-varying solution of HJB equation using least squares method. Proposed algorithm has been applied on the nonlinear system with matched and unmatched system uncertainties. Necessary theoretical and simulation results are presented to validate proposed algorithm. © 2009 The Institution of Engineering and Technology.

World University root node

Electrical Engineering

(SoE) School of Engineering

IET Control Theory and Applications

Fixed final time optimal control approach for bounded robust controller design using Hamilton-Jacobi-Bellman solution

In this paper we formulate a least squares version of the recently proposed twin support vector machine (TSVM) for binary classification. This formulation leads to extremely simple and fast algorithm for generating binary classifiers based on two non-parallel hyperplanes. Here we attempt to solve two modified primal problems of TSVM, instead of two dual problems usually solved. We show that the solution of the two modified primal problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems along with two systems of linear equations in TSVM. Classification using nonlinear kernel also leads to systems of linear equations. Our experiments on publicly available datasets indicate that the proposed least squares TSVM has comparable classification accuracy to that of TSVM but with considerably lesser computational time. Since linear least squares TSVM can easily handle large datasets, we further went on to investigate its efficiency for text categorization applications. Computational results demonstrate the effectiveness of the proposed method over linear proximal SVM on all the text corpuses considered. © 2008 Elsevier Ltd. All rights reserved.

Expert Systems with Applications

Least squares twin support vector machines for pattern classification

In this paper we present a novel shape descriptor based on shape context, which in combination with hierarchical distance based hashing is used for word and graphical pattern based document image indexing and retrieval. The shape descriptor represents the relative arrangement of points sampled on the boundary of the shape of object. We also demonstrate the applicability of the novel shape descriptor for classification of characters and symbols. For indexing, we provide a new formulation for distance based hierarchical locality sensitive hashing. Experiments have yielded promising results. © 2009 IEEE.

Proceedings of the International Conference on Document Analysis and Recognition, ICDAR

Shape descriptor based document image indexing and symbol recognition

In this paper, a Hamilton-Jacobi-Bellman (HJB) equation based optimal control algorithm for robust controller design, is proposed for a nonlinear system. Utilizing the Lyapunov direct method, controller is shown to be optimal with respect to a cost functional that includes maximum bound on system uncertainty. Controller is continuous and requires the knowledge of the upper bound of system uncertainty. In the proposed algorithm, Neural Network (NN) is used to find approximate solution of HJB equation. Proposed algorithm has been applied on a nonlinear uncertain system. © 2009 IEEE.

Proceedings of the 7th International Conference on Advances in Pattern Recognition, ICAPR 2009

Optimal control approach to robust control of nonlinear systems using neural network based HJB solution

In this paper, a Hamilton-Jacobi-Bellman (HJB) equation based optimal control algorithm for robust controller design is proposed for a nonlinear system. Utilising the Lyapunov direct method, the controller is shown to be optimal with respect to a cost functional, which includes penalty on the control effort, the maximum bound on system uncertainty and cross-coupling between system state and control. The controllers are continuous and require the knowledge of the upper bound of system uncertainty. In the present algorithm, neural network is used to approximate value function to find approximate solution of HJB equation using least squares method. Proposed algorithm has been applied on a nonlinear system with matched uncertainties. It is also applied to the system having uncertainties in input matrix. Results are validated through simulation studies. Copyright © 2009, Inderscience Publishers.

International Journal of Automation and Control

Robust control of nonlinear systems using neural network based HJB solution

Text Categorization (TC) remains as a potential application area for linear support vector machines (SVMs). Among the numerous linear SVM formulations, we bring forward linear PSVM together with recently proposed distributional clustering (DC) of words to realize its potential in TC realm. DC has been presented as an efficient alternative to conventionally used feature selection in TC. It has been shown that, DC together with linear SVM drastically brings down the dimensionality of text documents without any compromise in classification performance. In this paper we use linear PSVM and its extension Fuzzy PSVM (FPSVM) together with DC for TC. We present experimental results comparing PSVM/FPSVM with linear SVMlight and SVMlin on popular WebKB text corpus. Through numerous experiments on subsets of WebKB, we reveal the merits of PSVM and FPSVM over other linear SVMs. © Springer-Verlag Berlin Heidelberg 2009.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Text categorization using fuzzy proximal svm and distributional clustering of words

In this paper, constrained controller design is proposed for a class of nonlinear discrete-time uncertain system having matched type system uncertainties, using the solution of HJB (Hamilton-Jaccobi-Bellman) equation. The discrete-time HJB equation is formulated using a suitable nonquadratic term in the performance functional to tackle constraints on the control input. Based on the non-quadratic functional, a greedy HDP algorithm is used to obtain the constrained robust-optimal controller. The constrained robust controller requires knowledge of the upper bound of system uncertainties. For facilitating the implementation of the iterative algorithm, two neural networks are used to approximate the value function and to compute the optimal control policy, respectively. Their weights have been tuned using least squares method. Proposed algorithm has been applied on a nonlinear discrete-time system with matched uncertainties. © 2009 AACC.

Proceedings of the American Control Conference

Constrained controller design for a class of nonlinear discrete-time uncertain systems

Reinforcement learning (RL) is a popular learning paradigm to adaptive learning control of nonlinear systems, and is able to work without an explicit model. However, learning from scratch, i.e., without any a priori knowledge, is a daunting undertaking, which results in long training time and instability of learning process with large continuous state space. For physical systems, one must consider that the design of controller is very rarely a tabula rasa: some approximate mathematical model of the system is always available. In this paper, our focus is on control applications wherein the system to be controlled is a physical system. We can always obtain at least an approximate mathematical model of the plant to be controlled. We propose a method for hybridization of model-based approach with RL, which is the right solution for such control problems. The superiority of proposed hybrid approach has been established through simulation experiments on a cart-pole balance bench mark problem, comparing it with model-free RL system. We have used fuzzy inference system for function approximation; it can deal with continuous action space in Q-Iearning. Comparison with other function approximators has shown its superiority in terms of robustness of the controller to parameter variations in the plant. ©2009 IEEE.

IEEE International Symposium on Industrial Electronics

Hybridization of model-based approach with reinforcement fuzzy system design

Considerable attention has been given to the design of stable controllers for robot manipulators, in the presence of uncertainties. We investigate here the robust tracking performance of reinforcement learning control of manipulators, subjected to parameter variations and extraneous disturbances. Robustness properties in terms of average error, absolute maximum errors and absolute maximum control efforts, have been compared for reinforcement learning systems using various parameterized function approximators, such as fuzzy, neural network, decision tree, and support vector machine. Simulation results show the importance of fuzzy Q-learning control. Further improvements in this control approach through dynamic fuzzy Q-learning have also been highlighted.

Proceedings of the IEEE International Conference on Industrial Technology

Reinforcement learning control of robot manipulators in uncertain environments

In this paper, a new algorithm is proposed for the constrained control of weakly coupled nonlinear systems. The controller design problem is solved by solving Hamilton-Jacobi-Bellman(HJB) equation with modified cost to tackle constraints on the control input and unknown coupling. In the proposed controller design framework, coupling terms have been formulated as model uncertainties. The bounded controller requires the knowledge of the upper bound of the uncertainty. In the proposed algorithm, Neural Network (NN) is used to approximate the solution of HJB equation using least squares method. Necessary theoretical and simulation results are presented to validate proposed algorithm. © 2009 Springer-Verlag Berlin Heidelberg.

Constrained control of weakly coupled nonlinear systems using neural network

This paper develops an adaptive fuzzy controller for robot manipulators using a Markov game formulation. The Markov game framework offers a promising platform for robust control of robot manipulators in the presence of bounded external disturbances and unknown parameter variations. We propose fuzzy Markov games as an adaptation of fuzzy Q-learning (FQL) to a continuous-action variation of Markov games, wherein the reinforcement signal is used to tune online the conclusion part of a fuzzy Markov game controller. The proposed Markov game-adaptive fuzzy controller uses a simple fuzzy inference system (FIS), is computationally efficient, generates a swift control, and requires no exact dynamics of the robot system. To illustrate the superiority of Markov game-adaptive fuzzy control, we compare the performance of the controller against a) the Markov game-based robust neural controller, b) the reinforcement learning (RL)-adaptive fuzzy controller, c) the FQL controller, d) the H∞ theory-based robust neural game controller, and e) a standard RL-based robust neural controller, on two highly nonlinear robot arm control problems of i) a standard two-link rigid robot arm and ii) a 2-DOF SCARA robot manipulator. The proposed Markov game-adaptive fuzzy controller outperformed other controllers in terms of tracking errors and control torque requirements, over different desired trajectories. The results also demonstrate the viability of FISs for accelerating learning in Markov games and extending Markov game-based control to continuous state-action space problems. © 2007 IEEE.

IEEE Transactions on Fuzzy Systems

A markov game-adaptive fuzzy controller for robot manipulators

Materials Science and Technology: A Comprehensive Treatment

Effect of Multiphase Flow on Corrosion

Using fuzzy-rough hybrids, we have proposed a measure to quantify the functional dependency of decision attribute(s) on condition attribute(s) within fuzzy data. We have shown that the proposed measure of dependency degree is a generalization of the measure proposed by Pawlak for crisp data. In this paper, this new measure of dependency degree has been encapsulated into the decision tree generation mechanism to produce fuzzy-rough classification trees (FRCT); efficient, top-down, multi-class decision tree structures geared to solving classification problems from feature-based learning examples. The developed FRCT generation algorithm has been applied to 16 real-world benchmark datasets. It is experimentally compared with the five fuzzy decision tree generation algorithms reported so far, and the rough decomposition tree algorithm. Comparison has been made in terms of number of rules, average training time, and classification accuracy. Experimental results show that the proposed algorithm to generate FRCT outperforms existing fuzzy decision tree generation techniques and rough decomposition tree induction algorithm. © 2007 Springer-Verlag London Limited.

Pattern Analysis and Applications

FRCT: Fuzzy-rough classification trees

This paper proposes a novel control approach that incorporates a hybrid game strategy in Markov-game-based fuzzy control. Specifically, we aim at designing a "safe and universally consistent"controller that exhibits an ability to maintain performance against large disturbance and environment variations. The proposed hybrid control is a convex combination (based on experiential information) of "a variation of cautious fictitious play" approach and the "minimax" control approach implemented on a fuzzy Markov game platform. We show analytical convergence of Markov-game-based control in the presence of bounded external disturbances, and extend the analysis to show convergence of the proposed Markov-game-based hybrid control approach. Controller simulation and comparison against baseline Markov game fuzzy control and fuzzy Q-learning control on a highly nonlinear two-link robot brings out the superiority of the approach in handling severe environment and disturbance variations over different desired trajectories. This paper illustrates the possibility of obtaining "universal consistency," i.e., reasonable performance against severe environment and disturbance variations, by hybridizing "cautious fictitious play" with "minimax" approaches in Markov-game-based control. © 2008 IEEE.

Hybrid game strategy in fuzzy Markov-game-based control

In this paper, a Hamilton-Jacobi-Bellman (HJB) equation based optimal control algorithm is proposed for a bilinear system. Utilizing the Lyapunov direct method, the controller is shown to be optimal with respect to a cost functional, which includes penalty on the control effort and the system states. In the proposed algorithm, Neural Network (NN) is used to And approximate solution of HJB equation using least squares method. Proposed algorithm has been applied on bilinear systems. Necessary theoretical and simulation results are presented to validate proposed algorithm. © 2008 IEEE.

Proceedings of the International Joint Conference on Neural Networks

Constrained optimal control of bilinear systems using neural network based HJB solution

This paper enhances the recently proposed twin SVM Jayadeva et al. [Jayadeva, Khemchandani, R., Chandra, S., 2007. Twin support vector machines for pattern classification. IEEE Trans. Pattern Anal. Machine Intell. 29 (5), 905-910] using smoothing techniques to smooth twin SVM for binary classification. We attempt to solve the primal quadratic programming problems of twin SVM by converting them into smooth unconstrained minimization problems. The smooth reformulations are solved using the well-known Newton-Armijo algorithm. The effectiveness of the enhanced method is demonstrated by experimental results on available benchmark datasets. © 2008 Elsevier B.V. All rights reserved.

Pattern Recognition Letters

Application of smoothing technique on twin support vector machines

In this paper, we use the approach of adaptive critic design (ACD) for control, specifically, the action-dependent heuristic dynamic programming (ADHDP) method. A least squares support vector machine (SVM) regressor has been used for generating the control actions, while an SVM-based tree-type neural network (NN) is used as the critic. After a failure occurs, the critic and action are retrained in tandem using the failure data. Failure data is binary classification data, where the number of failure states are very few as compared to the number of no-failure states. The difficulty of conventional multilayer feedforward NNs in learning this type of classification data has been overcome by using the SVM-based tree-type NN, which due to its feature to add neurons to learn misclassified data, has the capability to learn any binary classification data without a priori choice of the number of neurons or the structure of the network. The capability of the trained controller to handle unforeseen situations is demonstrated. © 2007 IEEE.

IEEE Transactions on Neural Networks

SVM-based tree-type neural networks as a critic in adaptive critic designs for control

This paper proposes a reinforcement learning (RL)-based game-theoretic formulation for designing robust controllers for nonlinear systems affected by bounded external disturbances and parametric uncertainties. Based on the theory of Markov games, we consider a differential game in which a 'disturbing' agent tries to make worst possible disturbance while a 'control' agent tries to make best control input. The problem is formulated as finding a min-max solution of a value function. We propose an online procedure for learning optimal value function and for calculating a robust control policy. Proposed game-theoretic paradigm has been tested on the control task of a highly nonlinear two-link robot system. We compare the performance of proposed Markov game controller with a standard RL-based robust controller, and an H∞ theory-based robust game controller. For the robot control task, the proposed controller achieved superior robustness to changes in payload mass and external disturbances, over other control schemes. Results also validate the effectiveness of neural networks in extending the Markov game framework to problems with continuous state-action spaces. © 2006 Elsevier B.V. All rights reserved.

Applied Soft Computing Journal

A robust Markov game controller for nonlinear systems

When performing classification of large set of samples, Neural Trees (NTs) are preferably used. To circumvent the problem of poor generalization of Neural Trees, hybrid Neural Trees have been proposed. Recently hybrid SVM based Neural Tree has been shown to be an effective binary classifier. In this paper, we examine the performance of SVM based Neural Trees relative to the nonlinear SVMs. We observe that nonlinear SVMs are more effective, though at higher computational cost. Our conclusions will provide important guidelines in data mining applications on real world datasets. © 2007 IEEE.