PDF] Monte-Carlo Graph Search for AlphaZero

A new, improved search algorithm for AlphaZero is introduced which generalizes the search tree to a directed acyclic graph, which enables information flow across different subtrees and greatly reduces memory consumption. The AlphaZero algorithm has been successfully applied in a range of discrete domains, most notably board games. It utilizes a neural network, that learns a value and policy function to guide the exploration in a Monte-Carlo Tree Search. Although many search improvements have been proposed for Monte-Carlo Tree Search in the past, most of them refer to an older variant of the Upper Confidence bounds for Trees algorithm that does not use a policy for planning. We introduce a new, improved search algorithm for AlphaZero which generalizes the search tree to a directed acyclic graph. This enables information flow across different subtrees and greatly reduces memory consumption. Along with Monte-Carlo Graph Search, we propose a number of further extensions, such as the inclusion of Epsilon-greedy exploration, a revised terminal solver and the integration of domain knowledge as constraints. In our evaluations, we use the CrazyAra engine on chess and crazyhouse as examples to show that these changes bring significant improvements to AlphaZero.

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

AlphaZero: A General Reinforcement Learning Algorithm that Masters Chess, Shogi and Go through Self-Play

PDF) Targeted Search Control in AlphaZero for Effective Policy Improvement

Monte Carlo Tree Search (MCTS) in AlphaGo Zero, by Jonathan Hui

Acquisition of chess knowledge in AlphaZero

Deep bidirectional intelligence: AlphaZero, deep IA-search, deep IA-infer, and TPC causal learning, Applied Informatics

Q* Some kind of Alpha Zero self-play applied to LLMs according to Musk : r/singularity

Monte-Carlo Graph Search for AlphaZero

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner [PeerJ]

Student of Games: A unified learning algorithm for both perfect and imperfect information games

From Alpha Go to Alpha Zero - Vaas Madrid 2018

Mastering Atari, Go, chess and shogi by planning with a learned model

Monte-Carlo Tree Search (MCTS) — Introduction to Reinforcement Learning

Deep bidirectional intelligence: AlphaZero, deep IA-search, deep IA-infer, and TPC causal learning, Applied Informatics