Ethereal Tuning - Data Dump

[Joost Buijs]

I ran a test and got no useful results. That is, the tuning was unable to produce any improvement over Schooners master PST's.

Code: Select all

   1 Schooner2.25-sse                2       8    3000     50%     60%
   2 Schooner-E12_2                  2       8    3000     50%     61%
   3 Schooner1_125_end_2nd           1       8    3000     50%     60%
   4 Schooner-E12_1                 -4       8    3000     49%     59%

The test was compared against my new 1 million epd position filtered training set which I might post. The Ethereal E12 sets above had 80,000 duplicate positions.

I'm running my tuner now with the E12.52 set, it shows some values while it is running, the piece values for the opening phase keep dropping, for the endgame they keep rising, this is usually a bad sign. Maybe it doesn't matter, but I don't like it when the mg/eg values get skewed like this.

[Joost Buijs]

it was marked as rofchade, not rpirofchade, so I guess the full on version, 64x. Significant was not really the hardware it was on, but the absolute dominance of the eval function, they were evaluation crushes not search ones. mobility and positional. not fireworks, which I guess figures, those small nets are not going to be highly knowledge packed for difficult king safety/king tactical stuff, but it looks like the knowledge packing/tuning for standard structural features is more than enough.

Maybe Rofchade uses NNUE, I dont' know. Many engines suddenly have NNUE, probably by just copy-pasting code from Stockfish and adding a precooked network too. Writing optimized code for NNUE and training a network with it is not something you can do in a couple of days, these things take weeks or even months when you have to implement everything from scratch without prior knowledge or experience in this field.

[Joost Buijs]

I ran a test and got no useful results. That is, the tuning was unable to produce any improvement over Schooners master PST's.

Code: Select all

   1 Schooner2.25-sse                2       8    3000     50%     60%
   2 Schooner-E12_2                  2       8    3000     50%     61%
   3 Schooner1_125_end_2nd           1       8    3000     50%     60%
   4 Schooner-E12_1                 -4       8    3000     49%     59%

The test was compared against my new 1 million epd position filtered training set which I might post. The Ethereal E12 sets above had 80,000 duplicate positions.

I'm running my tuner now with the E12.52 set, it shows some values while it is running, the piece values for the opening phase keep dropping, for the endgame they keep rising, this is usually a bad sign. Maybe it doesn't matter, but I don't like it when the mg/eg values get skewed like this.

With the E12.33 set the weights that I get look very similar to the weights I get with my own 4M training set. I wonder what the difference between 12.33 and 12.52 is. I haven't checked the Elo differences yet.

[chrisw]

I ran a test and got no useful results. That is, the tuning was unable to produce any improvement over Schooners master PST's.

Code: Select all

   1 Schooner2.25-sse                2       8    3000     50%     60%
   2 Schooner-E12_2                  2       8    3000     50%     61%
   3 Schooner1_125_end_2nd           1       8    3000     50%     60%
   4 Schooner-E12_1                 -4       8    3000     49%     59%

The test was compared against my new 1 million epd position filtered training set which I might post. The Ethereal E12 sets above had 80,000 duplicate positions.

I'm running my tuner now with the E12.52 set, it shows some values while it is running, the piece values for the opening phase keep dropping, for the endgame they keep rising, this is usually a bad sign. Maybe it doesn't matter, but I don't like it when the mg/eg values get skewed like this.

With the E12.33 set the weights that I get look very similar to the weights I get with my own 4M training set. I wonder what the difference between 12.33 and 12.52 is. I haven't checked the Elo differences yet.

Well, those sets zapped my carefully tuned king attack code down to not very much. Noticeable change in play style.

[jdart]

I also go no improvement with the 12.52 training set over my default set. In fact it was a regression. YMMV.

--Jon

[Joost Buijs]

I've got very strange weights with the E12.52 set, after this I re-tuned with the E12.33 set, now the weights look very similar to the weights I get with my own 4M set. I didn't check performance with self-play yet, but I used these weights during the last blitz tourney. Besides the loss against CoronaVirusChess (due to a questionable position that arose from my opening book) it performed reasonably well.

[D Sceviour]

There are no guarantees that a tuning set can help at arriving at a unique minimum for the generation of PST's. Each test might be converging on a local minimum. There may be at least two major locals. There may be a local minimum for passed pawns, and another one for mating positions or king safety values. Another local might be a combination of these, meaning both are being threatened in a position. The local minimums still have to be defined uniquely for every position using an evaluation function. The evaluator will tell us whether pawn promotions, or mating threats are more important. However, there is no unique regression line that can cover both possibilities. Can NNUE layer design fix this?

[fabianVDW]

I've got >= 40 elo out of this data so far.

[jwilson82]

What methodology did you use for generating these positions? Specifically, how were the games generated and how did you decide which individual positions to extract from each game? I ask because I would like to take a swing at generating my own positions in the future

Thanks in advance

[AndrewGrant]

What methodology did you use for generating these positions? Specifically, how were the games generated and how did you decide which individual positions to extract from each game? I ask because I would like to take a swing at generating my own positions in the future

Thanks in advance

The data generation method is outlined in section 2 of this document:
https://github.com/AndyGrant/Ethereal/b ... Tuning.pdf