Open Chess Game Database Standard

[dangi12012]

You have participated in the SCID discussion and you're still not aware that SCID can do position queries?

First pretending SCID has no limitations - and when proven wrong changing the topic... I call that behaviour trolling - and gaslighting.
You only try to derail any topic into anarchy.

[Sopel]

You have participated in the SCID discussion and you're still not aware that SCID can do position queries?

First pretending SCID has no limitations - and when proven wrong changing the topic... I call that behaviour trolling - and gaslighting.
You only try to derail any topic into anarchy.

Ironic, considering that I've never said anything even remotely close to "SCID has no limitations".

ps. sorry nguyen for a slight offtopic, I don't want to undermine your efforts, just trying to get dangi's claims straight

[phhnguyen]

Still waiting for it to match even a fraction of SCIDs capabilities

Can you (and/or @Fulvio) tell me what (list) strong points of SCID4 are?

Many people suggested to use, support SCID4. I knew one of the first main strong points is that it has a big and excellent database (I hope one day the authors decide to support other apps by exporting to other formats such as this one). But besides the content (the database), I’m not sure about other things since I’m not its user. I knew it uses its own binary format and it has been developed for a long period thus it has some advantages which are hard for others to match or require a lot of time, effort to fill. Just want to know the current gap between it and our ones and how far we could go, how narrow we could push in the future.

[Fulvio]

Can you (and/or @Fulvio) tell me what (list) strong points of SCID4 are?

The main advantage is the speed of the position search.
In the video I had already posted (
it opens a 6.72 million games database and repeatedly searches the current board position almost instantly, while calculating the stats for the next moves and displaying the best games which reached the position sorted by Black Elo.

EDIT:
Also the small size is interesting.
I think it is the smallest size format for databases that do not contain comments.

[dangi12012]

Can you (and/or @Fulvio) tell me what (list) strong points of SCID4 are?

The main advantage is the speed of the position search.
In the video I had already posted (
it opens a 6.72 million games database and repeatedly searches the current board position almost instantly, while calculating the stats for the next moves and displaying the best games which reached the position sorted by Black Elo.

EDIT:
Also the small size is interesting.
I think it is the smallest size format for databases that do not contain comments.

You know why sqlite is superior dont you? Its language agnostic. sqlite does support almost every language out there as backend. So if someone writes a gui tomorrow in GO or RUST or Java - they can use sqlite instantly.
With SCID4 you are stuck with hard limitations and as we already discussed if you need fast lookup speed you use Tablebase lookup code in C++.

If you use it for the GUI then SQLITE is perfect. SCID4 does not have a turing complete query language attached to it...

[stevenaaus]

Check out Lionel's Scid++
https://sourceforge.net/projects/scidpp/
using CQL

[phhnguyen]

Can you (and/or @Fulvio) tell me what (list) strong points of SCID4 are?

The main advantage is the speed of the position search.
In the video I had already posted (
it opens a 6.72 million games database and repeatedly searches the current board position almost instantly, while calculating the stats for the next moves and displaying the best games which reached the position sorted by Black Elo.

EDIT:
Also the small size is interesting.
I think it is the smallest size format for databases that do not contain comments.

Thanks for the video. I have watched it. Look like it showed two kinds of searching: match exactly a given position and search for (PGN) header (did I miss something?).

As I have posted in some previous pages, matching exactly given positions is just a small part of searching, useful mostly for opening positions but not from middle games. IMHO, it is not hard to implement since we can use some simple techniques such as storing/matching hash keys or even just matching some beginning moves. The challenge is about searching appropriately, say, finding all positions having 3 Queens or given Pawn structures… Do you have any video showing that kind of searching (approximately)?

On the other hand, searching (PGN) header is the (very) strong point of SQL. You may beat it in some very specific searches but in general, you can’t win an SQL engine about the flexibility, coverage, average speed…

[phhnguyen]

Check out Lionel's Scid++
https://sourceforge.net/projects/scidpp/
using CQL

Nice program!!!

I’m not a user of SCID in general nor that app thus I have no idea how good it is. Could you make a brief what is the strong points, how different between that app and Fulvio’s one (SCID4)? Thanks

[stevenaaus]

Lionel has just forked my project, scid vs pc, with the focus being on an uptodate CQL (chess query language) implementation.

CQL is a terribly arcane language, but powerful and getting updates. Lionel is focusing a bit on writing some CQL doco too.

Just seems to me position match is too important to ignore a binary dB, and si4 is pretty good.
But if you need to handle variants, there is also gregors apparently abandoned scidb project, with its own format.

[Fulvio]

As I have posted in some previous pages, matching exactly given positions is just a small part of searching, useful mostly for opening positions but not from middle games. IMHO, it is not hard to implement since we can use some simple techniques such as storing/matching hash keys or even just matching some beginning moves.

Do you already have some benchmarks?
For example, how long does it take in that database to find all the Sicilian games (e4 c5), calculate the statistics, and show the 10 games with a higher elo average that contain that position?

The challenge is about searching appropriately, say, finding all positions having 3 Queens or given Pawn structures… Do you have any video showing that kind of searching (approximately)?

I don't have a video, but you can download it and try (in that database there are 108 games where at least one of the players has 3 queens, even one from Nakamura! And there are 3 games where both have 3 queens!).

On the other hand, searching (PGN) header is the (very) strong point of SQL. You may beat it in some very specific searches but in general, you can’t win an SQL engine about the flexibility, coverage, average speed…

I can agree on flexibility and scalability, but based on the benchmarks you posted even that is 20 times slower.
(and you won't expect the average user to enter SQL queries, right? You will still need to create a more user friendly interface).