README.md

musweeper

AI project using MuZero to play Minesweeper for IMT3104

todo

• Get muzero up and running (with tests)
• Merge in Gym-minesweeper
• Evaluate muzero vs mrgris
• Use mrgris for giving the model "dark Knowledge" (simulate states from mrgris and use his actions as optimal)
• Create a small test which trains the model and plays minesweeper, and calculates the average win rate.
• Create a simple script to run this in the colab notebook for offloading the training and eventual data analysis to Colab.
• Calculate average win rate on muzero

Testing

cd muzero
python -m unittest

Using pycharm: right-click test folder -> run 'unittest in test'

Environments

We evaluated the same agent (MuZero) on multiple different environments. We increased the board area from beginner (8*8 with 10 mines) to intermediate (16*16 with 40 mines) and expert (16*30 with 99 mines). The muzero model cannot use previously trained models on different board sizes as the model has to be retrained. To solve this we created a "gridworld" environment where the model plays as an agent moving inside the game. The agent can view an area of 5*5 cells around it, and to open a cell it has to stand over it and open. This reduces the observation space to always have the same size of 5*5 and reduces the observation space to 5 actions (up, down, left, right and open the cell it is standing on). The true board can then have unbounded size. Before we tested this environment, we hypothesize that the agent will have a harder time learning the rules of the game, as it has to additionally movement and exploration of the environment in addition to the traditional minesweeper rules, but this is required for the game to be of arbitrary size.

The classical algorithm cannot solve the minesweeper board by only looking at a small view of the board as it has not been designed for this case.

We hypothesize that the trained model will learn the rules of minesweeper to a good degree, but will not beat the classical algorithm in efficiency or accuracy. We hypothesize that by combining the two, (ie. give the learned model domain-specific knowledge generated by the classical model, and only apply the trained model in cases the classical algorithm can not handle) we will see an increased win-rate.

We also tried to train the model using intermediary rewards, and without. The gridworld environment uses only intermediary results, and ends when a mine is pressed.

We expect the trained model on the gridworld environment to prioritize opening cells that are connected to the outside, and have a lower priority on islands left within the opened area.