tablebase compression / academic integrity

[syzygy]

I found this very interesting paper by the creators of the Lomonosov tablebases:
Compression of underdetermined data in a 7-piece chess table
Unfortunately the paper does not seem to be freely available.

However, I do have a copy of it. Some ideas in it probably have been around for quite some time. But the combination of all the ideas put together including many of the little details... Well, all of that was already known as well, discussed in bits and pieces in various forums since 2008 or so and published in its totality in April 2013:
https://github.com/syzygy1/tb

Interestingly, in this recently published paper the authors make no reference whatsoever to the immediate source of all "their" ideas. They must have forgotten?

[mar]

I know nothing about tablebases, but speaking of general compression
- filters? (png uses filters to achieve better compression, executable compressors convert near calls/jumps to absolute addresses, but this only helps LZ-based compression) - I've no idea about the structure of tablebase data though
- Huffman cannot encode using fractional bits (=> away from theoretical optimum)

What I came across recently is called ANS (a paper by Jarek Duda, http://arxiv.org/abs/1311.2540)
It's static coding similar to Huffman but can encode using "fractional bits" - not really, but with states as context...
(table-version also called FSE by Yann Collet, he also has a working implementation https://github.com/Cyan4973/FiniteStateEntropy and describes it on his blog)
Charles Bloom explains ANS in detail on his blog http://cbloomrants.blogspot.cz/2014/01/ ... ans-1.html
Question is if it'd be worth it (in terms of ratio). While it's probably somewhat slower to decode than Huffman, the table-based version may work pretty fast actually.

[Michel]

I found this very interesting paper by the creators of the Lomonosov tablebases:
Compression of underdetermined data in a 7-piece chess table
Unfortunately the paper does not seem to be freely available.

However, I do have a copy of it. Some ideas in it probably have been around for quite some time. But the combination of all the ideas put together including many of the little details... Well, all of that was already known as well, discussed in bits and pieces in various forums since 2008 or so and published in its totality in April 2013:
https://github.com/syzygy1/tb

Interestingly, in this recently published paper the authors make no reference whatsoever to the immediate source of all "their" ideas. They must have forgotten?

Seems indeed rather shocking. No acknowledment, no reference. Quite surprising since the authors are academics themselves.

You could try to write to the Editor-in-Chief... http://www.allertonpress.com/journals/muk.htm .

[Michel]

A more effective strategy is probably to simply write an article about your own implementation and make it easily available (e.g. at https://github.com/syzygy1/tb or on the arXiv). In it you can point out that essentially the same set of techniques is used in the Lomonosov table bases and that you had multiple interactions with the authors of those in the past. Then Google can do its work.

[mar]

Ah sorry I missed the point, they ripped off your ideas and published them as their own ...

[bob]

I found this very interesting paper by the creators of the Lomonosov tablebases:
Compression of underdetermined data in a 7-piece chess table
Unfortunately the paper does not seem to be freely available.

However, I do have a copy of it. Some ideas in it probably have been around for quite some time. But the combination of all the ideas put together including many of the little details... Well, all of that was already known as well, discussed in bits and pieces in various forums since 2008 or so and published in its totality in April 2013:
https://github.com/syzygy1/tb

Interestingly, in this recently published paper the authors make no reference whatsoever to the immediate source of all "their" ideas. They must have forgotten?

This unfortunately happens. It seems that Computer Science programs sometimes fail to provide at least a modicum of "ethics 101". We've seen this with chess programs, we've seen this with assignments and reports, we've seen this with published papers. The latter when the referees for said journal are not "up to the task" of knowing what has happened with this topic in the past.

That has always been one of my criteria for accepting or refusing to review a paper for publication. I might understand the subject, but if I am not familiar with the subject then I usually decline to avoid letting this very kind of thing slip through a crack.

[syzygy]

Seems indeed rather shocking. No acknowledment, no reference. Quite surprising since the authors are academics themselves.

Yes, I wonder what they were thinking.

You could try to write to the Editor-in-Chief... http://www.allertonpress.com/journals/muk.htm .

Yes, that was my plan.

I do not mind (at all) that they study my code or write about it. But they should not pretend that they came up with everything themselves.