[Experiment] Swish

[sethtroisi]

Trying out Swish.

Looks really good!

Code: #799

Inspirations:

• https://arxiv.org/pdf/1710.05941.pdf

Results

Trained 8 networks, 4 baseline + 4 with --use_swish

• Tensorboard shows lower error in all metrics![1]

• Played 40 games for each of the last 2 checkpoints against a fixed external engine (LZ??)

  • Final results show negative elo gain needs more investigation to see if this was unlucky, related to length of training, ...

---

[1]

---

TODO: Assess speed impact