Analyzing the game with TwixtBot TWIXT PP

35 replies. Last post: 2021-05-06

Reply to this topic Return to forum

Analyzing the game with TwixtBot
  • Tomek at 2021-03-08

    I’ve played a little bit with TwixtBot source code to find out if I can use it for game analyze. It’s really well written and wasn’t hard to make some research on that. Here is a short description what is possible.

    For a given game and particular move we can get following information:

    • Move rank, ie. what is the position of the move from all possible moves ordered by strength (strongest -> weakest)
    • N best possible moves from the ranking
    • Player score + score for a draw, the number describing probability of winning or probability of draw
    • (optional, because it’s slow) Possible game play (can differ from proposed positions, since the algorithm is more non-deterministic as far as I understand)

    Here is the example taken from my first twixt game ever and what AI thinks what to do next.

    • my move was h11 (rank > 30)
    • ranking of first best moves from AI (strongest -> weakest) goes as follows: k13 (green) or f12 (red) or j14 (orange) or e12 (yellow) and the rest marked as greyish)
    • probability for winning by white is around 93%, by black (me): 57%, draw: 5%
    • actual game play from this point consist following moves (marked from 1 to 4): d15,i14,d11,e14

    This is only for one move, all above can be created for every step of the game. Of course, it’s not easy to make an app from that, these are just Python scripts which can be used after some setup. But I can imagine creating automatic report for given game.

    I’ve read all the concerns about having TwixtBot (it’s easier to cheat having such tool for example). But I think it’s better to have such tool than not having. What do you think about it?

  • Tomek at 2021-03-08

    oops, sorry, white was me (57%), black was actual winner (93%)

  • add3993 ★ at 2021-03-08

    Awesome work, I really hope it can be put into usable form for offline play against TwixtBot, with win% estimates and map of better/worse moves like the above!  Are you planning to make your scripts publicly available?

  • David J Bush ★ at 2021-03-08

    I think it would be more effective for you, at your stage in development, to ask humans about  what went wrong on the duckdns analysis page for your game. The game does not have to take place on Little Golem. Here is a list of “blank” LG games which contain zero moves. Please start with the first available game on the list.

    |     2067447 ||     2067454 ||     2067455 ||     2067458 ||     2073577 ||     2073583 ||     2073587 ||     2077853 ||     2081756 ||     2081758 ||     2081759 ||     2084053 |
    |     2084063 ||     2086121 ||     2088046 ||     2089997 ||     2090007 ||     2091199 ||     2092620 ||     2092622 ||     2092688 ||     2092690 ||     2092698 ||     2093161 |
    |     2093450 ||     2093460 ||     2093463 ||     2095620 ||     2096978 ||     2096979 ||     2098980 ||     2098993 ||     2098998 ||     2100510 ||     2100514 ||     2100515 |
    |     2100516 |

    Make the moves on the board, then copy the move list and paste it into a comment, along with whatever question you may have.


  • Tomek at 2021-03-09

    @add3993 – Yes, they are freely available. I’ve made a fork of Jordan’s scripts here: https://github.com/genmeblog/twixtbot

    @David – yes, I’m aware of this method but at my stage it’s really hard to ask specific question. I think that the only way to progress at the very beginning is just to play and play and play until some mental patterns emerge. After this I would be ready to talk. 

  • Hairspring at 2021-03-15

    I have said something like the following many times here: I think that the problem with using TwixtBot to try to learn to play TwixT is that the way the bot plays is very different from how a human plays. It has the ability to game out a long tree of moves to see what peg placement is best and a human can’t take the time to do this. If you are, in fact, using the bot to figure out what your next move should be against a human opponent than you are, indeed, cheating and, more importantly, destroying the integrity of the game. That will lead to all of us just using bots to decide our moves and that seems pretty pathetic ultimately. How much of an accomplishment is it to win by having your bot come up with a solution to moves my bot made for me? Sounds really sad. Just play lots of games and learn to play like a human.


  • HappyHippo ★ at 2021-03-15

    I believe the suggestion is that you learn from the bot by having it analyse games you’ve already played, not that you cheat and use the bot to select moves.

    I don’t play Twixt so I can’t comment on how the bot plays compared to humans. But I will say I’ve learned quite a bit from the Hex bots that are around, even if they’re so good I have no way of beating them myself.

  • Tomek at 2021-03-15

    Believe me or not, I’m not cheating and never be. It will be visible in my games I hope. So no fear is needed. I know that there are people against using software in training and there are people who use such. I’m in the second group. There are plenty of tools for chess used around from years and these are highly helpful. The same should be possible for more games. Also for Twixt.

    AlphaZero is state of the art and it’s constructed this way that it learns strategies. Not only best moves. That’s why it’s so good and valuable in training. Twixtbot IS AlphaZero. Please watch AlphaGo movie (available on YT) and comments of the best player after losing a tournament.

  • shyryan ★ at 2021-03-15

    Tomek, I think this is fabulous and I am so excited that you’re expanding the usefulness of TwixtBot. It’s so hard to extrapolate strategic nuggets by playing one move at a time, or sifting back through TwixtBot’s other games. Really excited to learn from this!

  • agtoever at 2021-03-23

    There is a GUI for twixtbot, called [twixtbot-ui on GitHub](https://github.com/stevens68/twixtbot-ui) which wraps around twistbot. What is nice is that the twistbot AlphaZero backend is part of the repository, so it’s a matter of checking out the code, installing python and the python libraries and go!

    I recently contributed a heatmap functionality to the application, which does exactly the type of analysis as mentioned by @Tomek; see image below.

    Any suggestions, feedback, comments, etc. is welcome on the GitHub page.

  • add3993 ★ at 2021-03-25

    Amazing.  It’s actually working.  I’m getting trounced by the bot, who takes no time at all to play.  I’m looking at heat maps and position evaluations.  Now I can get play experience and expert feedback 100x faster than on LG.  This is OUTSTANDING work!  Huge thanks to everyone involved!!

    It was not especially easy to set up (as a very-occasional programmer and slightly Terminal-shy person), but I want to share some quick notes on the hurdles I had to overcome (Win 10).  To get this to work,

    1. understand that you want python 3.6-3.8, NOT the latest 3.9

    2. so you may need a second python install on your machine.  possibly in a virtual environment, although I didn’t go that route.

    3. also, you need 64-bit python, because 32-bit python (although it will happily install and run on your 64-bit machine) is not a supported option of Tensorflow, an important machine-learning framework used by this project.

    so as I follow the install instructions on agtoever’s github link above, I am using not the plain command ‘python’ (which defaults to 3.9 on my PC) but  replacing it everywhere with ‘py -3.8-64’, after installing that version.  The first command-line there will install Tensorflow and other packages using pip (the python package installer, which connects to the internet to get what it needs). 

  • add3993 ★ at 2021-03-25

    It’s extremely entertaining to use and I predict it will help many players improve.  The heat map feels very illuminating.

    Striking how little difference the MCTS seems to make on top of the NN.  I’m curious why there aren’t more options for trial-depth, it seems like 20 or 100 would be welcome settings.

    If I could make just one change, it would be to make it easier to see whose move it is.  This would make it easier to follow while watching ‘bot TV’.  One stylish way would be to illuminate the two board sides belonging to that player (or just thicken/brighten the boundary color lines).  A simple square of color somewhere prominent near the board would also help.

    A next suggestion would be to add additional text on github or in-game on the difference between P-values and evaluation function.

  • add3993 ★ at 2021-03-26

    suggestions (but, I LOVE this software...)

    keyboard shortcuts for the main commands would be wonderful.

    -at times I wished the Bot Move and Accept were the same button, e.g. when watching a MCTS bot play against a no-trials one.

    ‘smart root’ MCTS option doesn’t seem to be explained

    -the 3-top-moves display seemingly always gives a hint, it would be nice to be able to hide it.  to a lesser extent this is true of the evaluation bar as well.

  • add3993 ★ at 2021-03-26

    didn’t mean to strikethru that text, keyboard shortcuts good.

    would be nice to highlight the last-played move.

    with the way the Undo function un-clicks the Auto-Move option, it’s harder to rewind while playing the bot.

  • Tomek at 2021-03-26

    Great comments. I also opt for keyboard shortcuts and lower trial depth search from 700 to 20-50.

    I’ve been using TB to analyze games for some time and I want to raise some concerns about how it behaves:

    * the most important: TB always wants to play a perfect game (even using pure P-values). When you play against it you’ll almost never face a bad move. That means, that there is no chance to simulate a play like with a weaker player. For a beginner like me, it’s impossible to train progressively. In games against human I can try to use what I know now and win if my opponent makes mistakes. Being able to recognize such mistakes is crucial for a progress. With TB is normally impossible.

    * quite common – when game is lost, useless moves are proposed constantly as the best one.

    * analysis – it’s not as easy as I thought at the beginning. It requires a lot of clicking to find which move was the crucial one for winning or losing.  NN very often thinks that everything is ok while it’s already not and after 2-3 moves the winning prediction suddenly changes to the opponent without recovery path. In such case you have to go back at earlier moment in the game and start analysis there.

    Good point is that you can try several variants to learn recovery paths in certain situations. It’s highly valuable.

    I’ve tested it against twixtpuzzles (first 10). Generally expected moves match but for example in puzzle 4, TB thinks that proposed correct move is a losing one. Also in other puzzle (need to find which) decision made by TB was wrong completely.

  • Tomek at 2021-03-26

    Great comments. I also opt for keyboard shortcuts and lower trial depth search from 700 to 20-50.

    I’ve been using TB to analyze games for some time and I want to raise some concerns about how it behaves:

    * the most important: TB always wants to play a perfect game (even using pure P-values). When you play against it you’ll almost never face a bad move. That means, that there is no chance to simulate a play like with a weaker player. For a beginner like me, it’s impossible to train progressively. In games against human I can try to use what I know now and win if my opponent makes mistakes. Being able to recognize such mistakes is crucial for a progress. With TB is normally impossible.

    * quite common – when game is lost, useless moves are proposed constantly as the best one.

    * analysis – it’s not as easy as I thought at the beginning. It requires a lot of clicking to find which move was the crucial one for winning or losing.  NN very often thinks that everything is ok while it’s already not and after 2-3 moves the winning prediction suddenly changes to the opponent without recovery path. In such case you have to go back at earlier moment in the game and start analysis there.

    Good point is that you can try several variants to learn recovery paths in certain situations. It’s highly valuable.

    I’ve tested it against twixtpuzzles (first 10). Generally expected moves match but for example in puzzle 4, TB thinks that proposed correct move is a losing one. Also in other puzzle (need to find which) decision made by TB was wrong completely.

  • Tomek at 2021-03-26

    (sorry for a duplicate, I don’t know what has happend)

  • David J Bush ★ at 2021-03-26

    Thanks for the info everyone!

    Bonyjordan explained to me why the bot flies off the rails the moment it sees it is losing. It runs through let us say the first 50 best moves, sees that they all lose, and is looking at 51 when it runs out of time. It has not yet determined that this also loses, so it makes choice 51 as its move. IMO this behavior could be relatively easily fixed by changing how it manages time. If a favored move loses, give that move some kind of score based on how many choices the opponent has which would keep the win. So a lower score would be a better choice for the bot. Of course the bot would not have time to determine every winning response exhaustively, but it could spend a little bit of extra time to come up with some kind of score. Then when time runs out, instead of playing the move it does not yet know also loses,, it could choose the move that was given the lowest score or one of the lowest scores. Such a move would likely be a better hustling move, giving the opponent maximum opportunity to toss the win. This would also improve bot v bot games, giving us humans a better understanding of where one bot lost the game.

    Another flaw is how the bot handles drawn positions. Basically it will enter a drawing line as if it were winning. Its scoring method does not take draws into account. It is capable of recognizing a draw when one is proposed, because of draw recognition capability which was “built on top” of the engine, but it is possible to trick the computer into a draw or even a loss this way. This would be much more difficult to fix. Maybe the entire deep learning process would have to start over from scratch, and it would not help newcomers.

    Also I would very much like to see standard rules with link removal implemented as an option. And I wonder if the bot could completely solve 18x18. Of course both of these are lots of work.

  • agtoever at 2021-03-27

    Thanks a lot for all the comments and suggestions. I think it’s important to distinguish between the Twixt [bot itself (developed by BonyJordan)](https://github.com/BonyJordan/twixtbot) and [the Python graphical user interface build around that bot (developed by Stevens68 and some contributions from me)](https://github.com/stevens68/twixtbot-ui). I gladly took your suggestions for the user interface and all surrounding functionality, such as keyboard support, hiding policy moves, allow the bot to make “mistakes”, etc. and added them as issues to the GitHub page. Keep an eye out for new releases there.

    As for the discussion about the behaviour of the bot itself: implementing the “standard rules” (e.g. allow link removal) and implementing a “draw” requires changing the bot training software and re-training of the bot. As David already mentions, this is probably a lot of work (and for me: currently not within my skills).

  • stevens68 at 2021-03-27

    Hi, great to hear that some of you installed twixtbot-ui and like it. Good comments and suggestions! 
    @agtoever: thanks for introducing twixtbot-ui here and for transfering the ideas into github issues!
    @add3993: I played around with very low trial sizes like 5, 10, 20. It doesn’t seem to make much difference compared to trials=0. Sometimes I think even any number of trials < 1000 doesn’t make much sense. You could, however edit file constants.py and change TRIALS_RESOLUTION=100 to a lower value. Then you can choose low values in the GUI and see what happens.
    @Tomek: Didn’t have a look at the puzzles but one reason for wrong prediction might be that the network was trained with self-crossing-links=true, i.e. it always “thinks” it’s true when it does an evaluation – despite self-crossing-link=false in your settings. See also sample game scl-issue.T1 where black is going to play N10 which is a winner if scl=true but leads to a draw if scl=false (should play N12 instead). 
    @David J Bush: Interesting ideas. But yes, I also think better handling of draws, different network sizes and also optional link removal would all affect the network layout and the learning process. I currently see no volunteers for that effort.

  • add3993 ★ at 2021-03-28

    Note on processor issues.  People may have heard that recent machine-learning algorithms are sped up by special use of GPUs.  My understanding of this (and please correct any inaccuracies) is that the present Twixtbot is based on the Tensorflow ML framework; which optionally takes advantage of the CUDA GPU-computing procedures for speedup; and these basically only run on machines with Nvidia GPUs (not AMD).

    This might impact the speed of heavy MCTS usage, which does take time; but it has already been noted that MCTS seems to make little difference with few trials (as in, it visibly almost always chooses the same move that was the leading candidate before the MCTS).  The bot is already seemingly very strong without MCTS, and plays extremely fast on my AMD machine.  Finally, unless I’m confused, the base strength without MCTS (or for any fixed amount) should be identical across machines, regardless of hardware.

  • agtoever at 2021-03-29

    @add3993: first a practical question: you asked for keyboard shortcuts. For which GUI operations you were thinking about that? For accessing the buttons (e.g.: <ctrl><z> or <alt><u> or just <u> = undo) or would you also like to be able to play using the keyboard (e.g.: entering <k><1><2><enter> would place a peg at k12)? That last option seems cumbersome for the average user. The first is relatively easily implemented, but I question the value of it, because you already use the mouse to play pegs...

    @add3993: in response to your last post: AlphaZero (as far as I understand it) is composed of two building blocks: a neural network (NN), which needs to be trained (once) and a Monte Carlo simulator. In general, GPU’s are used for training neural networks, because this requires a lot of computation, that can relatively easily be done in parallel and GPUs are really good at that. For the twixt-bot the training of the NN is already done by BonyJordan (https://github.com/BonyJordan/twixtbot) and the trained network is “saved” and loaded every game. On top of the NN giving a “policy”, you can simulate the next moves in the game (this is Monte Carlo simulation) and evaluate it. This combination is what AlphaZero makes so string and unique in setup. Also see for example: https://web.stanford.edu/~surag/posts/alphazero.html. In terms of performance, the MCTS is currently running “single threaded” (using 1 core on your CPU). I think we can make a serious performance improvement there if we run the MCTS multithreaded. Taking the next step would be to run the MCTS on GPU, but that requires a lot of (library and hardware) dependencies and is a lot of work. So maybe we can put running the MCTS multithreaded on the backlog, but given the fact that the trained NN is already quite strong, as far as I’m concerned, this had a fairly low priority. I usually don’t use the MCTS, except in specific cases where running the MCTS might give a better/other insight than the NN policy output.

  • add3993 ★ at 2021-03-29

    @agtoever: Thanks for the clarifications!

    I would love shortcuts for the bottom-row buttons; for toggling the heatmap; and for toggling auto-play for one or both players.  Manual move-entry is cumbersome and not too helpful, I agree.

  • stevens68 at 2021-03-30

    Hi,

    I just pushed a new version of twixtbot-ui:
    * option to display coordinates at mouse cursor position
    * keyboard shortcuts for all menus, buttons, controls in the main window
    (see README.md)

  • stevens68 at 2021-04-06

    A new version of  twixtbot-ui is available (https://github.com/stevens68/twixtbot-ui):

    * option to highlight last move
    * option to hide evaluation info
    (see README.md)


  • add3993 ★ at 2021-04-20

    Talking with several Twixt players, I wonder whether there are some short-term steps to make the software more accessible.  I would like to involve the devs in the conversation while understanding that their time is limited (and again, am already grateful for what we have). 

    The need for a custom Python install is a barrier.  Would it be possible in principle to make a single script that would do it all for you?  Ideally installing a sandboxed version of Python.  I know the instructions given already do a lot very efficiently, but it’s still tricky.

    Failing that, would it be possible in principle to make a giant .zip-ed folder containing all the dependencies for the software?  In combination with a .bat launcher that could be placed on a PC desktop and eliminate need to interact with command line.

    Failing that, I’ve thought about making a virtual machine with pre-installed Twixtbot-ui to send to people, but I think those are pretty big and may still be confusing to use.  A .exe is the ideal, granted I understand this is not easy when relying on existing tools like Tensorflow.

    All of these questions apply more generally to open-source software.  Ordinary people search for software and find a github project that does an amazing thing, but the install is too confusing in its command-line requirements and after downloading a file-directory of stuff, they find they’re not able to use it.  If I had more time I would like to promote accessibility in computing more broadly; as it is I don’t even have the time or skills to do it effectively for abstract games.  But I’ll keep talking about it!

  • add3993 ★ at 2021-04-20

    Just a status report on how it is still tricky to install on Linux.

    Having successfully installed TB-ui on my Windows PC, I tried to do it again with a virtual machine I could share with others (even though this adds some new trickiness and requires a huge VM file).  I used Oracle’s VirtualBox.  Since Windows requires a license, I figured I’d use Linux, and somewhat arbitrarily went with Oracle Linux 64.

    Long story short, I got stuck on some issue with tkinter (part of the GUI tools in Python used to make this project’s front-end).  When launching tbui.py I get a report (from inside PySimpleGUI) of “No module named '_tkinter'” or “named 'tkinter'” (I tried multiple times with Python 3.6 and 3.8).  Such difficulties are common but I wasn’t able to get around it in the time I had.

  • Tomek at 2021-04-20

    For tkinter you have to install Tcl and Tk packages on Linux. `apt install tcl tk` (on ubuntu).

    I’m able to run twixtbot-ui on Windows WSL2 but it still requires some prerequisites and X server to display a window.

  • add3993 ★ at 2021-04-20

    With Oracle Linux (where, I gather, yum is the main pkg manager not apt) I successfully executed “python3.8 -m pip install tcl” and the same with tk, but that didn’t fix it.  Ditto with the commands

    sudo yum install tcl

    sudo yum install tcl-devel

    sudo yum install tk

    sudo yum install tk-devel

  • add3993 ★ at 2021-04-20

    there are some weird discrepancies between how tkinter is referred to in different versions of python (tk, tkinter, Tkinter, etc.) but I don’t know if those are at issue.

  • add3993 ★ at 2021-04-20

    Tried again with an Ubuntu VM but hit the same wall.  I manually installed pip, PySimpleGUI in its Python 3.8, and successfully ran the command Tomek provided (thank you).  I have a package called tk installed but I still get “No module named 'tkinter' ” when I try to run tbui.py (after doing the thing with requirements.txt)

  • agtoever at 2021-04-21

    It’s a good idea to provide some better guidelines on how to get TB-ui to run, although it tricky to provide instructions that run on all platforms (Windows, OSX Linux).

    As for your efforts: I think an Ubuntu vm on Virtualbox is a good (and reproducible) platform.

    As for the errors you see: it might help to understand that there are two distinct package managers here: yum / apt-get to install software packages on your Linux distribution. Python and Tkinter are examples of such packages. Then, within Python, you also need Python packages. Those are installed with the command `pip install -r requirements.txt`, which installs all Python packages required for TB-ui. In the case of Tkinter, you need the Tkinter software package (yum / apt-get) as well as the Python package (tk) for PySimpleGUI to function correctly.

    I’ll see if I can write up some Ubuntu 20.04 instruction (which you could run under a vm) to run TB-ui from a clean install in the next few days...

  • agtoever at 2021-04-21

    By the way, the tkinter Python module is part of the standard Python distribution since Python 3.1. So tkinter should be part of your Python installation. Check `python --version` (make sure it’s 3.6, 3.7 or 3.8). Maybe Python 2 as well as Python 3 is installed, in which case you could check `python3 --version`. To install modules for Python 3, you should do in that case `pip3 install...`

    To check if tkinter works, start Python (`python` or `python3`) and type at the Python prompt (>>>): `import tkinter`. This should work without errors. Next, you could do `tkinter._test()`, which should show a window. You can quit Python by typing `quit()`.

  • agtoever at 2021-04-22

    Under Ubuntu, you need the ‘python3-tk’ package, which includes Tkinter. The following commands will install twixtbot-ui on a clean Ubuntu 20.04 installation. I also created a pull request on the GitHub of stevens68 to include these instructions.


    # Install prequisite packages on the operating system:

    sudo apt install -y git python3 python3-tk python3-pip python3-dev default-libmysqlclient-dev build-essential

    # Download the application from git

    git clone https://github.com/stevens68/twixtbot-ui

    # Install python required Python packages (modules)

    cd twixtbot-ui

    pip3 install -r requirements.txt

    # Run Twixt

    python3 ./tbui.py



  • stevens68 at 2021-05-06

    Hi,

    I created a windows installer for twixtbot-ui:

    https://github.com/stevens68/twixtbot-ui

    (see README.md)

Return to forum

Reply to this topic




Include game board: [game;id:123456] or [game;id:123456;move:20] or [game;id:123456;move:20;title:some text]