-
Notifications
You must be signed in to change notification settings - Fork 8
Boost Networks
A little while back the antifish project made a splash by apparently outperforming the equivalent Leela nets. The idea was playing the antifish net (initially just a T30 network) against a dumbed down SF10. This, so went the hypothesis, would allow antifish to outperform SF10, by training against these games at a very low learning rate. And amazingly it seemed to work.
I had a somewhat different hypothesis, however. That was that running supervised learning data from alpha beta engines at very low learning rates could sharpen a network. So I tried my hand at it. Batch size 1024, lr 0.00001 using CCRL data. The sweet spot was 1000 steps.
I tried it on 32930 and my own net, Maddex. It always bumped up the performance by about 30 elo.
Very premilinary at 2+2 on a 1060:
# PLAYER : RATING ERROR POINTS PLAYED (%) CFS(%) W D L D(%)
1 stockfishTB3 : 0 ---- 309.0 494 62.6 100 179 260 55 52.6
2 32930 : -79 36 74.5 190 39.2 74 26 97 67 51.1
3 32930-boost-7000 : -97 36 68.5 186 36.8 64 17 103 66 55.4
4 32930-boost-1000 : -107 44 42.0 118 35.6 --- 12 60 46 50.8
White advantage = 59.37 +/- 10.81
Draw rate (equal opponents) = 58.68 % +/- 2.51
So far, 1+1:
# PLAYER : RATING ERROR POINTS PLAYED (%) CFS(%) W D L D(%)
1 stockfishTB3 : 0 ---- 1014.0 1732 58.5 100 550 928 254 53.6
2 41800 : -47 19 253.0 581 43.5 78 88 330 163 56.8
3 41800-boost-1000 : -57 19 242.5 575 42.2 97 94 297 184 51.7
4 41800-boost-7000 : -84 20 222.5 576 38.6 --- 72 301 203 52.3
White advantage = 62.03 +/- 5.48
Draw rate (equal opponents) = 57.52 % +/- 1.27
My new (old) blog is at lczero.libertymedia.io