hash collisions

[lauriet]

DING, DING, DING.
End of round 10.
You all go back to your corners.

No resolution in site sight. (oops a typo bug)

[bob]

What ?!

Explain yourself.

You said you had a bug in your program. That has to be a lie because you and Chris are claiming the two of you can produce bug-free code. Sarcasm, anyone?

Wrong again.

Released software okay.

Wrong test methodology.

There you go again with your premature judgements.

Was I wrong when I pointed out that a released version of your program had a bug that let it hang or make an illegal move. What does wrong test methodology mean? That it failed to find a bug? So you had a bug that was not caught by testing, so it really wasn't a bug?

[Ovyron]

DING, DING, DING.
End of round 10.
You all go back to your corners.

No resolution in site sight. (oops a typo bug)

Well, after several years of disagreement, these guys finally agreed about whether Rybka was a clone of Fruit or not, so I don't see why they couldn't come to an agreement about bug free software as well.

[Dann Corbit]

If you keep parameter lists small, you can exhaustively test s function for every input bit pattern possible. There are computer systems used for verification. You can also exhaustively test a domain and the check the input range so that the function or method does not violate the domain.

Program proving is not impossible. It's just expensive.

I think too, that the goal of program correctness is something every good programmer strives for.

There's no need to construct strawmem of perfect million line programs and say we can't achieve perfection. The goal is to approach perfection, not to achieve it. If we set goals, they should be achievable goals or we will only find frustration. Logically, we want to write bug free code. But being imperfect means we will make mistakes. So we want to find our mistakes and fix them.

We can make polar statements about impossible extremes, but nothing is gained by that. The reasonable position for debate is , "How much effort should be spent in an attempt to write robust code? There can be sensible disagreement over the sort of effort expended, and debate in that area could be valuable.

IMO, YMMV

[Rebel]

As lone individuals writing chess engines we don't have the resources to do expensive, time consuming test methodologies, beta test procedures before we release.

In writing Amoeba, as a lone individual, I spent 99.9% of my time testing and 0.1% writing code. I have limited financial resources, but I have an invaluable resource: time.

Can you elaborate what the testing is about?

I am mostly testing for enhancements with SPRT; using my own tournament manager.
My development version is also quite different to the release version, as it contains many testing code: perft, epd test, etc.
I also use a language that is probably better at reducing bugs (the D language), as it uses a garbage collector and help to write things in less line of codes.
Is my program bug free? Unfortunately not. Probably half of the Elo of my program came from bugs removal and I guess there are still many of them to suppress.

Year by year your engine will become more and more robust. Tools like MRi may speedup that process.

[Rebel]

Was I wrong when I pointed out that a released version of your program had a bug that let it hang or make an illegal move.

Yes, you were wrong, you must have missed my post that addresses the issue.

What does wrong test methodology mean? That it failed to find a bug? So you had a bug that was not caught by testing, so it really wasn't a bug?

Switching from Arena to cutechess testing without the option restart=on gave me unreliable results.

[Rebel]

If you keep parameter lists small, you can exhaustively test s function for every input bit pattern possible. There are computer systems used for verification. You can also exhaustively test a domain and the check the input range so that the function or method does not violate the domain.

Program proving is not impossible. It's just expensive.

Exactly.

When I write new code or make changes especially when it's complex it may happen I only type a few lines and then test it before continuing Same for a new relative simple but large (say 50 lines of code) routine, I still hack it into small pieces, check and double check. Time consuming and often unnecessary, but it works for me.

[Rebel]

[d]1QqQqQq1/r6Q/Q6q/q6Q/B2q4/q6Q/k6K/1qQ1QqRb w - -

It's a fun position, I tried other engines, in alphabetic order.

Booot6 - ok
Chest - ok
Crafty 24.1 - hangs
Critter 1.4 - hangs
Doch 1.2 - hangs
Fruit 2.1 - hangs
Gideon Pro (1993) - ok
Komodo - ok
Lc0 - ok
Rofchade - ok
Stockfish - ok

[abulmo2]

I won't go into why I don't think garbage collection is the way to manage memory,
but I don't see how D is any safer than C or C++ or Pascal or any other unsafe language.

GC has advantages and drawbacks. In Amoeba, the main advantage I found, is I do not have to worry much about memory management, memory leaks, etc. The main drawback is it make the runtime bigger. Amoeba is not really concerned by program pause, slowdown, etc. that a GC may cause.

As long it allows unchecked references and pointers, it's no longer memory-safe the way Rust, Java or C# are
(I'm not talking about explict unsafe in Rust and C#).
C# has a very complicated way of making sure that references are handled safely and can't escape (extra limitations apply).
If you meant bounds checking on array access, then that's something that can be done easily (either using a wrapper around static arrays or ASAN where available)

Your wrong here. The D language has attributes to make it explicitly safe or not (@safe, @trusted, @system, scope return, ...) https://dlang.org/spec/memory-safe-d.html. Right now it is unsafe by default, but future compiler versions should make code be safe by default. https://github.com/dlang/DIPs/blob/1b70 ... DIP1028.md

EDIT: unless you meant something else than "safer" by "better at reducing bugs", of course

I also meant other things: contract programming, more compact code (thanks to module, GC, ...), unittest, ... You won't see most of that in Amoeba released versions, but the development versions are often cluttered with them.

[chrisw]

[d]1QqQqQq1/r6Q/Q6q/q6Q/B2q4/q6Q/k6K/1qQ1QqRb w - -

It's a fun position, I tried other engines, in alphabetic order.

Booot6 - ok
Chest - ok
Crafty 24.1 - hangs
Critter 1.4 - hangs
Doch 1.2 - hangs
Fruit 2.1 - hangs
Gideon Pro (1993) - ok
Komodo - ok
Lc0 - ok
Rofchade - ok
Stockfish - ok

Ed, where did this position come from originally? Was it part of some “review” done on your program back whenever?

On max move width and max depth, I keep a counter in the source code (debug version) and will print out a debug if my engine exceeds the prior maximum ever found. If it does, I put that as the new maximum ever found back into the source. That plus a margin gives me an actual maximum to use as an array size and defensive bounds check. Saying that, I am not at all confident mine won’t bitch at the movewidth of the above test position.