google-deepmind
Implement the board game Hive and its expansions
This is my implementation of Hive. While I plan on continually making improvements, what's here is in a good working state and has been tested thoroughly for correctness. I hope that the matter of copyright gets sorted out at some point, but for now, I figured I'd at least put up what I have for review!
I copied the README.md I wrote down below, as it goes into most of the implementation details in depth:
Hive
Implements the base game of Hive and its three expansion pieces: Mosquito, Ladybug, and Pillbug.
This implementation follows the rules outlined by the Universal Hive Protocol (UHP), which means states can be serialized to and deserialized from valid UHP game strings. With a bit of I/O handling, this can also be used as a UHP-compliant Hive Engine, making interactions with other engines straightforward.
State
Observation Tensor
First, the hexagonal grid needs to be represented as a rectangular one for 2D convolution:
The observation tensor then takes the form of multiple 2D feature planes describing the board and turn state, similar to what was done for AlphaZero chess.
However, since Hive's "board" is a repeating hexagonal tiling, the size is bounded only by the maximum number of tiles that can be laid in a straight line (28 total tiles for all expansions). Yet, a grid of size 28x28 is far too large to be computationally practical.
To help offset the complications this would bring for training in AlphaZero, the board can be paramaterized with board_size to reduce the tensor's overall sparsity. Using a board_size smaller than kMaxBoardSize means that some outlier games cannot be perfectly represented and are instead forced to a Draw. In practice, games that would approach that board length are extremely rare, so the trade-off feels acceptable.
The 2D feature planes are one-hot encodings that indicate:
- the presence of a particular bug type, for each player
- which bugs are pinned
- which bugs are covered
- the available positions that each player can place a new bug tile
- all 1's or all 0's to distinguish the current player's turn
Action Space
An action in Hive is described as:
- choosing which tile to move
- choosing which tile it moves next to (or on top of)
- the relative direction of the tile it moves next to
e.g. "wA2 bL/" - White moves their 2nd Ant to the top right edge of Black's Ladybug
With there being 28 unique tiles and 7 directions (the 6 hexagonal edges and "above"), the action space can be thought of as entries into a 3D matrix with dimensions 7 x 28 x 28 = 5488 total actions.
This is not a perfect action space representation as there are a handful of unused actions (e.g. moving a tile next to itself?), but it does capture every legal move. Unfortunately, with the introduction of the Pillbug, each player is able to move their own piece or the enemy's, meaning we can't implicitly expect the tile being moved to be the colour of the current player. This ends up doubling the action space size from 7x14x28 to 7x28x28
To-do
Below are some concrete features and fixes I intend to implement to either help speed up training or improve the interoperability between other Hive software (e.g. displaying games directly to MzingaViewer):
- [ ] Address the efficiency of code that uses the most compute time (
HiveState::GenerateValidSlides()andHiveState::IsGated()from recent perf tests) - [ ] Implement zobrist hashing to handle a "3-repeated moves" forced draw (unofficial community rule)
- [ ] Undo()
- [ ] Perft()
- [ ] Make it possible to load many UHP gamestrings from a file for training, or to collect interesting game statistics
- [ ] Create a separate binary that handles I/O and behaves as a proper UHP-compliant engine
- [ ] Provide a simplified action space for games that do not use expansion pieces
Future Improvements / Thoughts
While developing this engine, I came across many interesting ideas that have the potential for serious progress towards a viable expert-level AZ-bot for Hive. And as of this submission, no such Hive AI exists, making the prospect of any improvements much more appealing.
Below is a record of those miscellaneous thoughts, in approximate order of the potential I think it has:
-
Design a more exact action space. There are a handful of other suggested notations from the Hive community, each with their own advantages and drawbacks, that may be useful to look into for an alternative action space. One that looks very promising is Direction-Based Notation, as it implicitly covers all rotations and reflections by design.
-
Use a Hexagonal CNN model or filter. One problem that has been conveniently unaddressed is the fact that 2D convolution is performed on Hexagonal data that has be refitted onto a square. The typical 3x3 filter then doesn't accurately describe the 6 neighbours of a hex, as 2 extra values are contained in the filter. One option would be to use a custom 3x3 filter that zeroes-out the two values along the diagonal, or to attempt using a more advanced implementation like HexCNN or Rotational-Invariant CNN. The first option would be much easier to implement into the existing AlphaZero framework.
-
Attempt a graph/node-based representation. With how a game of Hive is structed like a graph itself, I think there is potential in using Graph Neural Networks (GNN) for learning. Some recent research has been done by applying GNNs to AlphaZero for board game AI, which indicates there is at least some proven success already.
@rhstephens This looks awesome!!
Just chiming in quickly to say sorry for the delay in response. I was on holiday and have been catching up a putting out a few fires since my return. I'll take a proper look soon.
Meanwhile, could I ask that you pull changes from master and push the merge commit? We needed to fix a few things to get Github Actions CI working, so you will need changes from https://github.com/google-deepmind/open_spiel/pull/1250.
Once that's done, I'll approve and run the tests and take a deeper look after that.
No worries at all, thanks for responding! I've rebased my changes on top of #1250, but let me know if anything else needs to be done on my end.
Also, if needed, I'm more than willing to go over any parts of the code that may need clarification. Just let me know
A few minor comments for now (I may add more), but on the first glance the code looks very nice. Well done!
This was a year-long effort but with the help from many people, we were able to get permissions to include this game into OpenSpiel! Thank you @rhstephens for your patience and to the folks at Gen42 for their cooperation, and of course our legal team.
With the success of this PR merging, this opens the possibility of having more board game implementations included in OpenSpiel! Having experienced what was involved for this PR, it may be even faster next time.
If you use this game for AI research or even just designing AI for the game as a hobby, we (and our contacts at Gen42) would love to hear about it. Please reach out!