An AlphaZero-inspired reinforcement learning chess engine built from scratch. Uses a Transformer neural network as the evaluation backbone, Monte Carlo Tree Search (MCTS) for move selection, and a deterministic Alpha-Beta endgame solver that takes over in simplified positions to force checkmate.
π GitHub Repository: RLChess
Sample Game

White - 300000 episodes
Black - 700000 episodes
Architecture
βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Board State β
β (42 Γ 8 Γ 8 tensor) β
β turn + pieces + attacks + castling + en passant β
β + 8-move history β
βββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β 1Γ1 Conv Projection (42 β 512) β
β + Learnable Positional Encoding β
β (64 square tokens) β
βββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Transformer Encoder (Γ10 layers) β
β d_model=512, 8 heads, FFN=2048 β
β Pre-Norm, batch_first β
ββββββββββββ¬βββββββββββββββββββββββββββββ¬ββββββββββββββ
β β
βΌ βΌ
ββββββββββββββββββββββββ ββββββββββββββββββββββββββββ
β Policy Head β β Value Head β
β Linear(32768β4096) β β Linear(32768β512) β
β Linear(4096β4672) β β Linear(512β1) β
β β raw logits β β β tanh β [-1, 1] β
ββββββββββββββββββββββββ ββββββββββββββββββββββββββββ
- Policy Head: Outputs logits over 4672 possible actions (4096 normal moves + queen promotions, 144 underpromotion slots for Knight/Bishop/Rook)
- Value Head: Outputs a single scalar predicting the game outcome from the current playerβs perspective
- ~30M parameters β designed for an RTX 3080
MCTS (Opening + Middlegame)
Standard AlphaZero-style MCTS with PUCT selection, Dirichlet noise at root for exploration, and 400 simulations per move. The neural network provides both the prior probabilities for edge selection and the leaf evaluation.
Alpha-Beta Solver (Endgame)
When β€ 7 pieces remain on the board, the engine automatically switches from MCTS to a deterministic alpha-beta search with iterative deepening (up to depth 20). This solver uses MVV-LVA move ordering, king-cornering heuristics, and passed-pawn proximity bonuses to force checkmate reliably.