Knowledge Free and Learning Based Methods in Intelligent Game Playing Studies in Computational Intelligence Volume 276 1st edition by Jacek Mandziuk 9783642116780, 3642116787

Part l AI Tools and State-of-the-Art Accomplishments in Mind Games

Foundations of AI and CI in Games. Claude Shannon’s Postulates

Basic AI Methods and Tools

Definitions and Notation

Game Tree Searching

Minimax Algorithm

Alpha-Beta Search

Heuristic Enhancements

SCOUT and Negascout (PVS)

MTD(f) and MTD-bi

Rollout Simulations and Selective Sampling

UCT Algorithm

State of the Art

…, Deep Blue, Fritz, Hydra, Rybka, …

Chinook – A Perfectly Playing Checkers Machine

Logistello – The Othello World Champion

Maven and Quackle – The Top Level Machine Scrabble Players

Poker and Bridge – Slightly behind Humans

Go – The Grand Challenge

Part ll CI Methods in Mind Games. Towards Human-Like Playing

An Overview of Computational Intelligence Methods

Artificial Neural Networks

Multilayer Perceptron

Neural Networks and Games

Evolutionary Methods

Evolution Strategies

Evolutionary Methods and Games

Reinforcement Learning

Temporal Difference Algorithm

Q-Learning Algorithm

Temporal Difference Learning in Games

Summary

CI in Games – Selected Approaches

TD-Gammon and Neurogammon

Anaconda vel Blondie24

TD-Learning Systems

KnightCap

TDL-Chinook

TD-GAC

Neuro-evolutionary Approaches

Blondie24-Like Approach to Othello

Blondie25

Moriarty and Miikkulainen’s Othello Player

Part lll An Overview of Challenges and Open Problems

Evaluation Function Learning

Game Representation

Local, Neighborhood-Based Representations

Invariants-Based Representations

Incremental Learning

Double Dummy Bridge Problem

Efficient TD Training

External Opponents vs. Self-playing

Self-playing

Training with External Opponents

Human vs. Artificial Trainers

Exploration-vs.-Exploitation Dilemma

TD-Learning Strategies

Learning on All Games

Learning Exclusively on Lost or Drawn Games

Learning on “No Blunders”

Learning with Focus on Strong Opponents

Successive Elimination of Weaker Opponents

Random Opponent

TD-GAC Experiment

Lessons from TD-GAC Experiment

The Choice of Training Strategy

Selection of Training Opponents

Training and Testing with Random Opponent

Oscillations in the Learning Process

Tuning and

TD-Learning vs. Evolution

Move Ranking and Search-Free Playing

Methods Relying on Historical Goodness of a Move

Pattern-Based Methods

Methods Using Predefined Game Features

Chess-Related Approaches

Modeling the Opponent and Handling the Uncertainty

Part IV Grand Challenges

Intuition

Perception in Chess

CHREST/CHUMP Model

SYLPH: A Pattern-Based Learning System

COPYCAT-Like Model

Other Approaches

Focused Minimax Search

Linguistic Geometry and Pioneer Project

Emergent Intuitive Playing

Perception in Go

fMRI Scanning

Summary

Creativity and Knowledge Discovery

Spontaneous Knowledge Discovery

TD-Gammon

DDBP Neural Network Solver

Blondie

Deliberate Knowledge Discovery

Morph

Zenith

Discovering Playing Strategies in Othello

Analysis of Archive Game Records

Summary

Multi-game Playing

SAL

Hoyle

METAGAMER

Morph II

General Game Playing Competition

Transfer of Knowledge

Summary

Summary and Perspectives

References