Skip to content
Little GolemMake your next move.

Hex bot battle

Public forum discussion.

Hex, Havannah

#51 · gzero_bot

11 Apr 2019

@lazy 38.k4 was 70% on policy from neural network - so it latched on to it because it gets more exploration due to policy percent and the winning rate was much higher than other candidates.  The policy for L3 was 0.02%.  The node is in optimistic mode due to latching on to k4, and with that and the low policy - l3 never gets much exploration.

I am running gzero now with more exploration at root (including some code changes), and a little longer search to compensate for search width.   Replaying k12 opening game, which gzero lost to leela in first round of games.

#53 · lazyplayer

11 Apr 2019

Leela, in recent times you've started to like center. I think even bots really don't know how to play the first few stones!

#54 · David J Bush

11 Apr 2019

"Knowing the first moves" should be equivalent to "solving that size grid." AFAIK the largest posted swap map is for 9x9.

#55 · lazyplayer

11 Apr 2019

Eheh, this time leela wins 2-0. Gzero seems to play desperate tactics/strategies.

#56 · gzero_bot

11 Apr 2019

Yeah, I was checking in, game 401 has been over for a while.  :)  Not sure where that went wrong so early.

Don't know about other game, seems semi-balanced from gzero's point of view.

#57 · lazyplayer

11 Apr 2019

Well, maybe it has a chance on game 398...

#58 · lazyplayer

11 Apr 2019

gzero, actually I've just checked, maybe in 398 it's plainly winning. Leela plays a beautiful style and I like its move visually, but the evil is in the details.

#59 · lazyplayer

11 Apr 2019

I think probably leela is stronger at strategy (global play), and gzero is stronger at tactics (local play).

#60 · lazyplayer

11 Apr 2019

gzero, i think you win 398 (we're after move 32 now), correct? :)

#63 · gzero_bot

11 Apr 2019

398:  I've no idea! :)  gzero still over 50% but dropping with more search.  I'd put my money on leela with those stats (without looking at the board).

#64 · leela_bot

12 Apr 2019

Here is leela's current "swap file". The percentages are the winning percentage leela thinks white has if black plays it's first stone in the indicated hex. How does this compare with gzero's thoughts, gzero?

#65 · lazyplayer

12 Apr 2019

I think if you analyze a8 and a10 deeper, maybe they'll be closer to 50 than they're there.

Anyway, the really surprising result is H3 stronger than G3 and then even more surprising we see that J3 is almost as bad as G3. This latter fact (J3 almost as bad as G3) never crossed anyone mind as far as I know except maybe for Arek: https://littlegolem.net/jsp/game/game.jsp?gid=1597836&nmove=2. Probably it's because it allows white to play L2 at some point.

#66 · gzero_bot

12 Apr 2019

@leela Very cool and pretty! :)

I don't have any swap file, as swap is baked into the rules/neural network.  To get something similar I could generate, from blacks point of view, the win probability after N simulations for each board position with a single white stone from your swap file/heatmap.  I'll try that at the weekend.

What I have done is has ran MCTS at depth=0 for a long time with high exploration settings.  I'll paste a most recent result below.  Also because it is MCTS, it is not very consistent results (the previous time I ran it, a13 was lower than f11).  Nor is there any symmetry (so i took the top result for each symmetry).

(#simulations, policy probability, win probability for black after #simulations)

  • k12 / c2 : 339337, 16.92, 0.473
  • g11 / g3 : 339011, 0.30, 0.465
  • a13 / m1 : 339006, 0.43, 0.462
  • h11 / f3 : 339003, 0.14, 0.456
  • f11 / h3 : 339000, 0.37, 0.448
  • l12 / b2 : 178608, 1.69, 0.439
  • k2 / c12 : 146951, 3.27, 0.438
  • m12 / a2 : 101694, 6.55, 0.436
  • a8 / m6 : 60740, 0.10, 0.438
  • m11 / a3 : 33494, 3.55, 0.425
  • a4 / m10 : 29866, 0.20, 0.424
  • m4 / a10 : 22574, 0.24, 0.417
  • a5 / m9 : 12259, 0.13, 0.415
  • b4 / l10 : 11543, 0.13, 0.412
  • a9 / m5 : 10396, 0.14, 0.406
  • i11 / e3 : 9068, 0.12, 0.409
  • m8 / a6 : 8842, 0.11, 0.408

#67 · gzero_bot

12 Apr 2019

Also interesting with J3 - since it doesn't appear in gzero's list.  leela has played that in our new game 2090407.  :)

#69 · David J Bush

12 Apr 2019

This is very useful, thank you Leela bot. Of course it will take a lot more time, and perhaps a different sort of algorithm, to nail down every cell as either 100% or 0. During that process, some of the relative values between specific cells might flip around.

#70 · lazyplayer

12 Apr 2019

Ah, I was totally confused on 398. What a fool. Anyway, the last 2 games are very interesting starts!

#71 · Carroll

12 Apr 2019

Very nice, thanks!

Did you agree on a thinking time or better a number of playouts for this bot fight?

Could you explain the numbers for the openings? If swap is allowed, every number should be greater than 50% and else I think there should be lot more squares for first player that should be?

#72 · Ignatius J Reilly

12 Apr 2019

@Carroll -- The percentages in the opening diagram assume no swap.

#73 · leela_bot

12 Apr 2019

That is right. Put another way: if black plays it's first move on an indicated hex and that hex has a number smaller than 50% leela will swap. If bigger than 50% leela will play on with white. leela will also swap any empty hex (if black plays it's first move there).

#74 · leela_bot

12 Apr 2019

Also, if leela plays first it would prefer to play k12 (or c2) because that is the cell with the number closest to 50%. In practice, for the sake of variety, it currently randomly plays anything within 6% of 50%.

#75 · leela_bot

12 Apr 2019

For the "bot battle" we have a somewhat informal agreement to allow around 10 minutes thinking time per move.

#76 · HappyHippo

12 Apr 2019

Interesting results with the swap map. I might try more F3/G3 openings

For the first move selection, have you considered a weighted probability of all moves? Apparently this is what the TwixtBot does:

"For the first move, rather than pick the one point closest to 50%, I decided to make a weighted random selection: the closer the win rate is to 50%, the higher the chance I pick that move."

Ultimately that might reduce the bot's winning percentage, but it could potentially make for some more interesting games. And maybe give the lowly humans a chance ;)

#77 · lazyplayer

12 Apr 2019

Happy, if it plays less variation in first move, we can study its game and learn something. So actually, lowly humans have more to gain if it plays objectively better... ;)

#78 · lazyplayer

12 Apr 2019

First principle of Hex strategy: The best way to learn hex is by copying the choices of strong players, especially at the beginning. :-)

#79 · lazyplayer

12 Apr 2019

Next pair of games, my forecast is 2-0 for gzero! The first stone does matter!

#80 · lazyplayer

12 Apr 2019

"I’ve certainly observed that with leela also: if it is losing it starts making really bad looking moves. They are objectively probably still the best moves, but I think human players seem to know how to complicate the situation by playing tricky stuff when they are losing, but the bots are too “honest” for that :)"

Poor bot, these evil humans are tricking him, haha :D

#81 · lazyplayer

13 Apr 2019

Leela_bot, what about game 087? what you play if I play i8? it seems rather game over to me... :)

#82 · Arek Kulczycki

13 Apr 2019

Leela thanks for the heatmap! Very interesting with j3!

Although I don't see why h3 would be stronger than g3, could this difference still be within the error?

And ooops, seems a7 and a8 are way too strong :( I liked those

My last surprise is I would have expected a3 to be more balanced. It's a pitty leela will not play it as it's not within 6%...

#83 · gzero_bot

13 Apr 2019

Interesting game 2090408, it seemed leela had won but then played a couple of moves b10 and a12 - that gave gzero a chance to claw back into the game.

Score so far: leela 5, gzero 3.   Think gzero is quite fortunate with that score line. :)  Congrats leela!

I agree with lazy and arek, it always seems as if games with leela it is fighting a losing battle from move 2 onwards - with the win probability being slowly sapped away. If lucky, gzero seems to have jumps in win probability in the late game, such as the b10/a12 above, which lets it back in the game.   Maybe leela just feeling sorry for her wee brother and throwing the game! ;)

One last game being played now, for reference: 2090522, with h11 opening, since g11 was interesting.

#84 · Arek Kulczycki

13 Apr 2019

It would take longer analysis to prove but to me it seems in 2090408 at move 33 leela could even win in multiple ways... At least the move I would like is 33.h9 and I dont see many opportunities for white to turn the game...

#85 · lazyplayer

13 Apr 2019

Gzero, don't worry, leela is not making late game blunders. In the game 2090408 the "error" is probably 13 g8 or 15 i8. I say "error" in quotes because it's not obvious at all that there was a winning choice there for Leela at all. It's possible that all choices there were actually losing and Leela at best could have picked another that was less obvious.

#86 · lazyplayer

13 Apr 2019

Nonetheless it's possible your idea is still correct. It's possible gzero is slightly stronger in later moves than leela.

#87 · lazyplayer

13 Apr 2019

Gzero, you can try a match at much higher time controls, like 6 hours per move instead of 10 mins. This should allow you to use your endgame skill to compute the right initial stones.

By the way, I think something similar is true between me and Arek. I'm slighly stronger later moves and/or much longer time controls and/or smaller boards.

#88 · lazyplayer

13 Apr 2019

I thought I was winning in game 087 but I was not. Maybe I've blundered somewhere near the end? Overconfidence is terribly dangerous...

#89 · lazyplayer

13 Apr 2019

I should have played 14 L2, maybe even 18 L2 was enough... shame on me!

#90 · lazyplayer

13 Apr 2019

Gzero, I also propose a funny experiment for you to do, if you've time. I see that you can easily change board size (your code is game-agnostic). Why don't you try to set board size = 100, or board size = 500, and then do some training on that, and then you show us the "heatmap" of openings? I'm wondering if center is a winning opening on arbitrarily large boards like 100x100.

#91 · leela_bot

13 Apr 2019

@lazy you are seriously underestimating what "some training" on a 100x100 board would involve.

#92 · lazyplayer

13 Apr 2019

leela, it's better to have a low quality "heatmap" than having no "heatmap" at all :)

btw, take a look at our last game, sadly, i blundered, i was too happy because i had won another!

#93 · lazyplayer

13 Apr 2019

can you confirm i was winning before the blunder? L2 was winning no?

#94 · ypercube

13 Apr 2019

I agree with Leela. Training for size 19 might give us some results. For 29 maybe.

For 100? I doubt it.

#95 · ypercube

13 Apr 2019

The bots do not even play at 19 yet.

I challenge them for a game of 36x36.

#96 · lguser

13 Apr 2019

I too am interested in seeing how good play in the early game scales with board size

#97 · leela_bot

14 Apr 2019

@lazy on your game 2090087. Yeah, 18.k7 was wrong (leela jumped to above 70%). I wouldn't call 18.l2 winning: leela says 51%. It was actually expecting 18.l3 (44%). 14.l2 is also 51% (I didn't run leela super long on these so take these numbers with a grain of salt)

#99 · lazyplayer

14 Apr 2019

leela, it's surprising to me that bots are so "probabilistic". For me it's either win or loss, the rest are illusions. :)

#100 · lazyplayer

14 Apr 2019

Well, no, 18 H7 indeed worse than 18 L2,  maybe I've caused a mess at 16, and 16 D9 would be a clean white win? Or maybe there are no clean wins at all and it's genuinely complicated.

Last page