Acta Medica Okayama 3 2004 A double layered state space construction method for reinforcement learning agents 2698 2703 EN Hisashi Handa In this paper, we propose a new double-layered state space construction method, which consists of Fritzke's Growing Neural Gas algorithm and a class management mechanism of GNG units. The classification algorithm yields a new class by referring to anticipation error, anticipation vectors of an originated class, and anticipation vectors GNG units belonging in the originated class. No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 1 2005 Dynamic salting route optimisation using evolutionary computation 158 165 EN Hisashi Handa Lee Chapman Xin Yao On marginal winter nights, highway authorities face a difficult decision as to whether or not to salt the road network. The consequences of making a wrong decision are serious, as an untreated network is a major hazard. However, if salt is spread when it is not actually required, there are unnecessary financial and environmental consequences. In this paper, a new salting route optimisation system is proposed which combines evolutionary computation (EC) with the next generation road weather information systems (XRWIS). XRWIS is a new high resolution forecast system which predicts road surface temperature and condition across the road network over a 24 hour period. ECs are used to optimise a series of salting routes for winter gritting by considering XRWIS temperature data along with treatment vehicle and road network constraints. This synergy realises daily dynamic routing and it will yield considerable benefits for areas with a marginal ice problem. No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 2 2004 State space construction of reinforcement learning agents based upon anticipated sensory changes 1115 1120 EN Hisashi Handa We propose herein a new incremental state construction method which consists of Fritzke's growing neural gas algorithm and a class management mechanism of GNG units. The GNG algorithm condenses sensory inputs and learns which areas are frequently sensed. The CMM yields a new state based upon the anticipated behaviors of the agent, i.e., a couple of actions by an agent and the resultant change in sensory inputs. Computational simulations on the mountain-car task confirm the effectiveness of the proposed method. No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 2 2001 Coevolutionary GA with schema extraction by machine learning techniques and its application to knapsack problems 1213 1219 EN Hisashi Handa Tadashi Horiuchi Osamu Katai Takeshi Kaneko Tadataka Konishi Mitsuru Baba The authors introduce a novel coevolutionary genetic algorithm with schema extraction by machine learning techniques. Our CGA consists of two GA populations: the first GA (H-GA) searches for the solutions in the given problems and the second GA (P-GA) searches for effective schemata of the H-GA. We aim to improve the search ability of our CGA by extracting more efficiently useful schemata from the H-GA population, and then incorporating those extracted schemata in a natural manner into the P-GA. Several computational simulations on multidimensional knapsack problems confirm the effectiveness of the proposed method No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 1 1 2006 Robust route optimization for gritting/salting trucks: a CERCIA experience 6 9 EN Hisashi Handa Lee Chapman Xin Yao Highway authorities in marginal winter climates are responsible for the precautionary gritting/salting of the road network in order to prevent frozen roads. For efficient and effective road maintenance, accurate road surface temperature prediction is required. However, this information is useless if an effective means of utilizing this information is unavailable. This is where gritting route optimization plays a crucial role. The decision whether to grit the road network at marginal nights is a difficult problem. The consequences of making a wrong decision are serious, as untreated roads are a major hazard. However, if grit/salt is spread when it is not actually required, there are unnecessary financial and environmental costs. The goal here is to minimize the financial and environmental costs while ensuring roads that need treatment will. In this article, a salting route optimization (SRO) system that combines evolutionary algorithms with the neXt generation Road Weather Information System (XRWIS) is introduced. The synergy of these methodologies means that salting route optimization can be done at a level previously not possible. No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 4 2000 An incremental state-segmentation method for reinforcement learning using ART neural network 2732 2737 EN Hisashi Handa Akira Ninomiya Tadashi Horiuchi Tadataka Konishi Mitsuru Baba In this paper, we propose a new incremental state segmentation method by utilizing information of the agents' state transition table which consists of a tuple of (state; action, state) in order to reduce the effort of designers and which is generated using the ART neural network. In the proposed method, if an inconsistent situation in the state transition table is observed, agents refine their map from perceptual inputs to states such that inconsistency is resolved. We introduce two kinds of inconsistency, i.e., different results caused by the same states and the same actions, and contradiction due to ambiguous states. Several computational simulations on cart-pole problems confirm the effectiveness of the proposed method No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 4 2001 Perception-action rule acquisition by coevolutionary fuzzy classifier system 2405 2410 EN Hisashi Handa Takashi Noda Tadataka Konishi Mitsuru Baba Osamu Katai Recently, many researchers have studied the techniques in applying a fuzzy classifier system (FCS) to control mobile robots, since the FCS can easily treat continuous inputs, such as sensors and images by using a fuzzy number. By using the FCS, however, only reflective rules are acquired. Thus, in the proposed approach, an additional genetic algorithm is incorporated in order to search for strategic knowledge, i.e., the sequence of effective activated rules in the FCS. Therefore, the proposed method consists of two modules: an ordinal FCS and the genetic algorithm. Computational experiments based on WEBOTS, one of the Khepera robot simulators, confirm the effectiveness of the proposed method No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 3 2002 Evolutionary constitution of game player agents 1609 1612 EN Hisashi Handa Tadashi Horiuchi In this paper, we propose a constitution method of game player agent that adopts a neural network as a state evaluation function for the game player, and evolves its weights and structure by evolutionary strategy. In this method, we attempt to acquire a better state evaluation function by evolving weights and structure simultaneously. No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 2006 Robust Solution of Salting Route Optimisation Using Evolutionary Algorithms 3098 3105 EN Hisashi Handa Dan Lin Lee Chapman Xin Yao The precautionary salting of the road network is an important maintenance issue for countries with a marginal winter climate. On many nights, not all the road network will require treatment as the local geography will mean some road sections are warmer than others. Hence, there is a logic to optimising salting routes based on known road surface temperature distributions. In this paper, a robust solution of Salting Route Optimisation using a training dataset of daily predicted temperature distributions is proposed. Evolutionary Algorithms are used to produce salting routes which group together the colder sections of the road network. Financial savings can then be made by not treating the warmer routes on the more marginal of nights. Experimental results on real data also reveal that the proposed methodology reduced total distance traveled on the new routes by around 10conventional salting routes. No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 2 2000 A new fitness function for discovering a lot of satisfiable solutions in constraint satisfaction problems 1184 1189 EN Hisashi Handa Osamu Katai Tadataka Konishi Mitsuru Baba In this paper, we discuss how many satisfiable solutions a genetic algorithm can find in a problem instance of a constraint satisfaction problems in a single execution. Hence, we propose a framework for a new fitness function which can be applied to traditional fitness functions. However, the mechanism of the proposed fitness function is quite simple, and several experimental results on a variety of instances of general constraint satisfaction problems demonstrate the effectiveness of the proposed fitness function No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 1 2003 Estimation of Bayesian network algorithm with GA searching for better network structure 436 439 EN Hisashi Handa Osamu Katai Estimation of Bayesian network algorithms, which adopt Bayesian networks as the probabilistic model were one of the most sophisticated algorithms in the estimation of distribution algorithms. However the estimation of Bayesian network is key topic of this algorithm, conventional EBNAs adopt greedy searches to search for better network structures. In this paper, we propose a new EBNA, which adopts genetic algorithm to search the structure of Bayesian network. In order to reduce the computational complexity of estimating better network structures, we elaborates the fitness function of the GA module, based upon the synchronicity of specific pattern in the selected individuals. Several computational simulations on multidimensional knapsack problems show us the effectiveness of the proposed method. No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 3 2001 Adaptive state construction for reinforcement learning and its application to robot navigation problems 1436 1441 EN Hisashi Handa Akira Ninomiya Tadashi Horiuchi Tadataka Konishi Mitsuru Baba This paper applies our state construction method by ART neural network to robot navigation problems. Agents in this paper consist of ART neural network and contradiction resolution mechanism. The ART neural network serves as a mean of state recognition which maps stimulus inputs to a certain state and state construction which creates a new state when a current stimulus input cannot be categorized into any known states. On the other hand, the contradiction resolution mechanism (CRM) uses agents' state transition table to detect inconsistency among constructed states. In the proposed method, two kinds of inconsistency for the CRM are introduced: "Different results caused by the same states and the same actions" and "Contradiction due to ambiguous states." The simulation results on the robot navigation problems confirm the effectiveness of the proposed method No potential conflict of interest relevant to this article was reported.

Acta Medica Okayama 4 2000 Constraint satisfaction on dynamic environments by the means of coevolutionary genetic algorithms 2935 2940 EN Hisashi Handa Osamu Katai Tadaaki Konishi Mitsuru Baba We discuss adaptability of evolutionary computations in dynamic environments. We introduce two classes of dynamic environments which are utilizing the notion of constraint satisfaction problems: changeover and gradation. The changeover environment is a problem class which consists of a sequence of the constraint networks with the same nature. On the other hand, the gradation environment is a problem class which consists of a sequence of the constraint networks such that the sequence is associated with two constraint networks, i. e., initial and target, and all constraint networks in the sequence metamorphosis from the initial constraint network to the target constraint network. We compare coevolutionary genetic algorithms with SGA in computational simulations. Experimental results on the above dynamic environments confirm us the effectiveness of our approach, i.e., coevolutionary genetic algorithm No potential conflict of interest relevant to this article was reported.

IEEE SMC Hiroshima Chapter Acta Medica Okayama 1883-3977 2009 1 2009 Evolving FPS Game Players by Using Continuous EDA-RL 143 146 EN Hisashi Handa This paper extends EDA-RL, Estimation of Distribution Algorithms for Reinforcement Learning Problems, to continuous domain. The extended EDA-RL is used to constitiute FPS game players. In order to cope with continuous input-output relations, Gaussian Network is employed as in EBNA. Simulation results on Unreal Tournament 2004, one of major FPS games, confirm the effectiveness of the proposed method. No potential conflict of interest relevant to this article was reported.

IEEE SMC Hiroshima Chapter Acta Medica Okayama 1883-3977 2009 1 2009 Rule Induction by EDA with Instance-Subpopulations 3 7 EN Hisashi Handa In this paper, a new rule induction method by using EDA with instance-subpopulations is proposed. The proposed method introduces a notion of instance-subpopulation, where a set of individuals matching a training instance. Then, EDA procedure is separately carried out for each instance-subpopulation. Individuals generated by each EDA procedure are merged to constitute the population at the next generation. We examined the proposed method on Breast-cancer in Wisconsin and Chess End-Game. The comparisons with other algorithms show the effectiveness of the proposed method. No potential conflict of interest relevant to this article was reported.

IEEE SMC Hiroshima Chapter Acta Medica Okayama 1883-3977 2008 1 2008 Rule Acquisition for Cognitive Agents by Using Estimation of Distribution Algorithms 185 190 EN Tokue Nishimura Hisashi Handa Cognitive Agents must be able to decide their actions based on their recognized states. In general, learning mechanisms are equipped for such agents in order to realize intellgent behaviors. In this paper, we propose a new Estimation of Distribution Algorithms (EDAs) which can acquire effective rules for cognitive agents. Basic calculation procedure of the EDAs is that 1) select better individuals, 2) estimate probabilistic models, and 3) sample new individuals. In the proposed method, instead of the use of individuals, input-output records in episodes are directory used for estimating the probabilistic model by Conditional Random Fields. Therefore, estimated probabilistic model can be regarded as policy so that new input-output records are generated by the interaction between the policy and environments. Computer simulations on Probabilistic Transition Problems show the effectiveness of the proposed method. No potential conflict of interest relevant to this article was reported.

IEEE SMC Hiroshima Chapter Acta Medica Okayama 2008 1 2008 Constitution of Ms.PacMan Player with Critical-Situation Learning Mechanism 48 53 EN Hisashi Handa We previously proposed evolutionary fuzzy systems of playing Ms.PacMan for the competitions. As a consequence of the evolution, reflective action rules such that PacMan tries to eat pills effectively until ghosts come close to PacMan are acquired. Such rules works well. However, sometimes it is too reflective so that PacMan go toward ghosts by herself in longer corridors. In this paper, a critical situation learning module is combined with the evolved fuzzy systems, i.e., reflective action module. The critical situation learning module is composed of Q-learning with CMAC. Location information of surrounding ghosts and the existence of power-pills are given to PacMan as state. This module punishes if PacMan is caught by ghosts. Therefore, this module learning which pairs of (state, action) cause her death. By using learnt Q-value, PacMan tries to survive much longer. Experimental results on Ms.PacMan elucidate the proposed method is promising since it can capture critical situations well. However, as a consequence of the large amount of memory required by CMAC, real time responses tend to be lost. No potential conflict of interest relevant to this article was reported.