CCRL 40/40, 40/4 and FRC lists updated (16th February 2019)

[Rebel]

Just ask what's not clear.

What is 'valuation of move sequences'? What does MIT stand for, and what does it do?

For MIT (Material Imbalance Table) check: - http://rebel13.nl/rebel13/ideas.html

Tell me what's so wrong on the following example:
...
Why should this be forbidden, it's blocking progress.

How do you expect anyone to see that from the output?

anyone ??

You seem to underestimate the folks here.

And I don't underestimate you, you got the point (3 plies deeper) also.

It is what goes on inside that matters. So tell us what you did, and we can tell you whether it should be forbidden and why.

#1. we ??

#2. How it is done for the moment is irrelevant, I have provided a number of possibilities for the CCRL folks what learning from previous games can do and I am pretty sure that list is far from complete.

In the above example search and eval are in full use and regarding AZ and LZ we both know for what the "Z" stands for, don't we?

Yeah, and when Lance Armstrong was winning the Tour de France his muscles and lungs were also in full use. It were the other things that he used that got him into trouble. Likewise in the example above. That the engine can ride uphill faster doesn't mean it is clean...

I liked your creativity and maybe a newbie will nod but remarks like this kill the discussion.

One of us must be missing something. The "Z" indicates the engine was not tuned on any games that it did not generate itself. Why do you think that is funny?

Look at the context, you said - It is absolutely insane to demand engines playing without their eval.

And that's exactly what AZ and LZ are doing, playing with ( Zero) knowledge, no eval. And yeah, I thought that was funny.

[hgm]

For MIT (Material Imbalance Table) check: - http://rebel13.nl/rebel13/ideas.html

Ah, that. This is just tuning of material values beyond the simple model of additive piece values. No individual positions are stored, so CCRL rules allow it, and rightly so.

anyone ??

You seem to underestimate the folks here.

And I don't underestimate you, you got the point (3 plies deeper) also.

And you seem to ascribe them super-natural powers... If you don't want to divulge what you do, fine, but don't bring it up as an example then, because it is really an example of nothing. Everyone knows that engines can improve when you make some changes to them; if that is all you want to show, you are battering an open door.

#1. we ??

Look up 'pluralis modestae'. Or you could prefer to believe that I am a schizofrenic, or of royal blood... Or that indeed I speek for the general reader.

#2. How it is done for the moment is irrelevant, I have provided a number of possibilities for the CCRL folks what learning from previous games can do and I am pretty sure that list is far from complete.

So you showed nothing of relevance, or even anything that anyone here did not know already: chess-engine authors improve their engines based on the games these play, or good quality games played by others (engines or human). Everyone does it, CCRL has allowed it since the beginning of time...

I liked your creativity and maybe a newbie will nod but remarks like this kill the discussion.

Well, they are surely lethal to your point of view. That is what happens to non-viable things. The get killed. You seem to think 'discussion' is a kind of show for entertainment, that should be prolongued as long as possible. But in fact the purpose is to unmask and squash nonsensical ideas, as swiftly as possible.

One of us must be missing something. The "Z" indicates the engine was not tuned on any games that it did not generate itself. Why do you think that is funny?

Look at the context, you said - It is absolutely insane to demand engines playing without their eval.

And that's exactly what AZ and LZ are doing, playing with ( Zero) knowledge, no eval. And yeah, I thought that was funny.

Totally wrong, as I suspected. AZ and LZ are not playing at all with zero knowledge, and no one exept you ever claimed they did. The 'Zero' means they acquired the (vastly superior) knowledge they have from scratch, without any of it being feeded to them from an external source. So the joke here seems to be your appalling misunderstanding of these ground-breaking projects, except that I don't see anything funny about that.

[Rebel]

No!

Our discussion rapidly went down because I am making a case to allow learning now that LZ has entered the Arena and as shown by your answers the only thing you are interested in is to turn each point down by saying - not allowed under the CCRL rules.

What a waste of time.

[Rebel]

http://www.cegt.net/testers/conditions.htm

It seems CEGT does not have learning restrictions.

[Graham Banks]

http://www.cegt.net/testers/conditions.htm

It seems CEGT does not have learning restrictions.

Just because it's not mentioned doesn't mean that they allow learning.

[hgm]

As was pointed out to us, we say we are a standard hardware list but we are not. We have 1CPU and 4CPU engines on there and they play each other to some extent so that the list is joined up. So a "GPU" could be just another variation of that. So the list would be "1CPU", "4CPU" and "GPU". I think CEGT have in fact set aside a couple of standard reference machines for GPU testing for a 2 year minimum period. From their postings, I think the time control for matches is set in the normal way based on the speed of the CPU, and then the GPU is what it is as the standard reference GPU. That seems completely clear and transparent to me.

Well, I think I said this before, but for the CCRL list to remain relevant in a future where flocks of GPU-based engines will be the strongest-playing chess entities for owners of an average gaming PC, you really should find a way to allow GPU usage. Now would be an ideal time to make that transition, as none of the pre-Leela engines is actually able to derive any benefit at all from the presence of a GPU. So if you redefine your standard hardware as 1CPU+1GPU or 4CPU+1GPU (where the GPU could be a commonly used not super-expensive type), all existing results wexcept Leela games would remain valid. And you can benchmark hardware the testers actually use by measuring Leela NPS on such a machine. A CPU-only version of Leela would merely be a curiosity on such a list.

[hgm]

No!

No to what???

Our discussion rapidly went down because I am making a case to allow learning now that LZ has entered the Arena and as shown by your answers the only thing you are interested in is to turn each point down by saying - not allowed under the CCRL rules.

What a waste of time.

What caused it to rapidly go down is that you are injecting all kind of false and non-sensical claims in the discussion. And it is never a waste of time to correct these. Unsuspecting readers might take you seriously otherwise.

Even your summary here is false. As Ray explained above CCRL rules do allow learning, provided that versions that the versions under test are static 'snapshots' taken during the learning process. (Which is an obvious prerequisite to have a definable rating.)

What CCRL rules do not allow is the use of a position database other than EGT. (And IMO it is a mistake to allow EGT, but that is another story.) Whether that position database is constructed by learning or hand-crafted is completely immaterial. And there is a good reason for having that rule: No matter how large and accurate such a database, it will always contain only a negligible fraction of all possible chess positions. So they will always be useless for the general ability of an engine to analyze an arbitrarily chosen position, there just won't be any hits on the database at all. (With the exeption of EGT, because Chess is a convergent game.) The best it can do is mask the stupidity of the engine in the opening phase, where positions close to the start position still have a significant probability of being visited by the search. This is not what the list intends to represent.

[Rebel]

http://www.cegt.net/testers/conditions.htm

It seems CEGT does not have learning restrictions.

Just because it's not mentioned doesn't mean that they allow learning.

Hence I said seems.

[Rebel]

HGM - In this thread I am trying to convince the CCRL folks (and perhaps the CEGT people as well) to review their restrictions on learning, especially when I read: positional learning not allowed. It's not about you or me. And yet as often for reasons that escape me you sooner or later make it that way with your fighting language use and the fun of the discussion goes away, as in this case.

For the CCRL folks
Your FAQ states: positional learning not allowed. And that is what LZ does, only in a much more advanced way.

[Guenther]

HGM - In this thread I am trying to convince the CCRL folks (and perhaps the CEGT people as well) to review their restrictions on learning, especially when I read: positional learning not allowed. It's not about you or me. And yet as often for reasons that escape me you sooner or later make it that way with your fighting language use and the fun of the discussion goes away, as in this case.

For the CCRL folks
Your FAQ states: positional learning not allowed. And that is what LZ does, only in a much more advanced way.

Your wish is completely unrealistic, if you think a bit about it.
The goal of CCRL and CEGT is to establish rating lists and this needs scientific conditions.
With any kind of learning 'during the games' played in the rating process it is impossible to guarantee
entities have the same state always, which would be mandatory for being useful, otherwise each single game
would add noise and this noise would sum up exponentially with each further game.

And I don't understand why you still say LC0 learns. It does not - it has learnt (eval), but it does not learn any further
and remains in its state from beginning to end of the rating games.
No one forbids you to extract info from previous learning and add it to your program, except real moves and positions,
which could be seen as internal book.