Engine Crash Detective Story

[emadsen]

I've enjoyed reading all your "war stories" of difficult debugging sessions.

So I had to work backwards, with an assembly-language debugger, to the point where the bad code was generated.

I agree Jon, that has got to be one of the most challenging problems a developer will face.

Well done. Debugging is an under-rated art. It is painful and unrewarding, but true programming skill is learnt debugging, much more so than writing code.

Thanks Lucas! Regarding your code suggestions, I do make the code "scream at me" when the board state is illegal. In the first two lines and the last line of the Board.PlayMove method.

Code: Select all

Debug.Assert(Engine.Move.IsValid(Move));
Debug.Assert(AssertMoveIntegrity(Move));
// ...
Debug.Assert(AssertIntegrity());

This validates the board has a white and black king, the move aspects encoded into the ulong actually match the board state, or'ing the piece occupancies == color occupancy and board occupancy, incremental update of Zobrist key == full recalculation of Zobrist key, etc.

you are missing the potential for generalisation

You are correct about a need to generalize code. My priority is this regard is to eliminate all the color-specific code I have in my evaluation method. Also, remove duplicate code that controls sliding piece move generation and attack counts. Too much copy / paste / renaming of variables there. I need to generalize that code.

It turned out that there was this struct that was passed down, down, down, down way down deep

How frustrating, Dann, when developers misuse data structures (treating a by-val struct like a by-ref class) and create a jigsaw puzzle for the next developer who has to maintain the code.

Banks often keyed an incorrect amount for check that was cashed. It was literally looking for needle in the haystack

Mike, great story. Reminds me of a story my boss told me at my first job out of college. His experience in accounting and computers went back to the 70s. He explained the company would double-check account balances by having the day team do their work on paper, the day mainframe operator run the deck of cards (an accounting calculation), then ensure the two totals matched. Same procedure with the night team. Only the night mainframe run produced a different number. Every other method matched: two by-hand and the day mainframe calculations. Turned out one of the cards was shorter than the others. The day mainframe operator tilted the cards to the left and knocked them on a desk before feeding them into the mainframe. The night operator tilted the cards to the right. The misalignment produced different results for the computerized accounting calculation. Ridiculous!

The bugfix however seems to have another issue because the used history value will be somewhat off now if the bitwise & prevents an overflow.

That's a very good point, Ras. I will fix that. So perhaps I will add a limit check after all. Though I may locate it in the MoveHistory.UpdateValue method and keep the brute-force, unforgiving bit mask code in the Move.Set methods.

In the end, we found out his account number was the binary compliment of Exxon of Florida's and a programming error had caused the problem. Now there's a bug for you.

That is an incredible story, Dann! Thanks for sharing.

[Dann Corbit]

In those days, database activity was roll your own (no ODBC or OLEDB or JDBC access).
The registered customers database (24 million records) and manufacturing (Manman) were on the VAX.
Most of the other database systems were on OS/2 with SQL Server, and SQL Server had a limit of 255 connections.
So all of those database systems needed some kind of pooling system.
Just the tax calculations were utterly enormous. The laws are different in different states, not to mention different countries.
The call system was customers calling about their products or their orders.
Edit: it was sort of a CRM before there was such a thing as a CRM.

[lucasart]

My priority is this regard is to eliminate all the color-specific code I have in my evaluation method. Also, remove duplicate code that controls sliding piece move generation and attack counts. Too much copy / paste / renaming of variables there. I need to generalize that code.

In my experience, the majority of bugs in chess came from copy/paste. One could easily make the argument that copy/paste should be forbidden, because the time spent debugging is often higher than the time saved typing.

When you copy/paste (eg. white code for black case), you are faced with an adverse selection risk:
* if you make a mistake, it will compile, and cause (sometimes rare) problems only at runtime. Hours spent debugging, sometimes much later on in development.
* but if you typed the black version by hand instead of starting with Ctrl+V, and you made a mistake, this would almost always be a syntax error, caught at compile time. Few seconds wasted recompiling, no big deal.

[Rebel]

Remembers me on a bug I had in the 90's which took me about 6 months to find. In those days there was only auto232 and sometimes an engine engine match went rough. The reason, the clock. When it passed 23:59:59 to the new day it started to do strange things, forgot what. Was an easy fix. Called it the 24-hour bug.

[Henk]

Sometimes the ugliest and creepiest code that nobody understands performs best. And when you finally have found and removed the bug it comtained you lose many elo points. (The legacy. You cannot ecape it or you will die)

[Object oriented code is not much better than a (chaotic) network. So best is to avoid it]
[Don't use linq in move generation. Too slow]

[emadsen]

Don't use linq in move generation. Too slow

LINQ for move generation? That's insanity. I avoid it even for line-of-business applications.

[towforce]

Don't use linq in move generation. Too slow

LINQ for move generation? That's insanity. I avoid it even for line-of-business applications.

If your code is in a tight loop that will be run many times, and hence performance is critical, then yes - use SQL.

One criticism of linq I will make is that it makes db access so easy that devs tend to over use it: each access to the database is costly in time, and I am constantly seeing linq users making lots of little accesses instead of a small number of big accesses.

As for saying that SQL is a good language: NO, NO, NO, NO, NO!

Some considerations:

1. you only get one "where" statement per clause, then you must use "and" statements. This makes editing statements unnecessarily annoying. In linq, you can just where where where, then comment out the ones you temporarily don't want

2. SQL is bloody dangerous: you make a mistake in an update statement, which is waaaaaaaay to easy to do in SQL, then you get told that 500 records have been updated, and there's nothing you can do about it. There should be an option for a warning if multiple records will be updated or deleted

3. You are more familiar with SQL than you are with linq or EF, otherwise you'd know that writing fast linq, like writing fast SQL, is a matter of technique

4. the mathematician in me tells me that linq/EF are just much better languages than SQL

5. Linq/EF gets better with every release: more and more, you find the linq/EF query outperforming a similar SQL query

6. Linq has become a very natural language extension now, because it is also used in structures like lists (again, avoid if performance is genuinely important)

7. If you're writing code for a business, the most important thing is maintainability: if you're going to tell me that SQL is more maintainable than linq/EF then IMO you're committing the #1 developer sin: behaving in an "old school" manner.

[Pio]

Don't use linq in move generation. Too slow

LINQ for move generation? That's insanity. I avoid it even for line-of-business applications.

If your code is in a tight loop that will be run many times, and hence performance is critical, then yes - use SQL.

One criticism of linq I will make is that it makes db access so easy that devs tend to over use it: each access to the database is costly in time, and I am constantly seeing linq users making lots of little accesses instead of a small number of big accesses.

As for saying that SQL is a good language: NO, NO, NO, NO, NO!

Some considerations:

1. you only get one "where" statement per clause, then you must use "and" statements. This makes editing statements unnecessarily annoying. In linq, you can just where where where, then comment out the ones you temporarily don't want

2. SQL is bloody dangerous: you make a mistake in an update statement, which is waaaaaaaay to easy to do in SQL, then you get told that 500 records have been updated, and there's nothing you can do about it. There should be an option for a warning if multiple records will be updated or deleted

3. You are more familiar with SQL than you are with linq or EF, otherwise you'd know that writing fast linq, like writing fast SQL, is a matter of technique

4. the mathematician in me tells me that linq/EF are just much better languages than SQL

5. Linq/EF gets better with every release: more and more, you find the linq/EF query outperforming a similar SQL query

6. Linq has become a very natural language extension now, because it is also used in structures like lists (again, avoid if performance is genuinely important)

7. If you're writing code for a business, the most important thing is maintainability: if you're going to tell me that SQL is more maintainable than linq/EF then IMO you're committing the #1 developer sin: behaving in an "old school" manner.

Hi, I agree a little bit of what you are saying but I also agree with Erik.

1) If You want to be able to filter SQL a little bit easier you can either use:
WHERE 1 = 1
AND FilterStatement1
AND FilterStatement2
— AND FilterStatement3

Or use nested sql statements that are easy to use/inspect. Just highlight the parts you want to check and hit F5 in Management Studio

2) There are a couple of simple ways to make sure you don’t update more than one record at a time. You can i) either add instead of triggers on your tables and check that the size of inserted and deleted tables have size 1 or else raise error or ii) forbid the users to modify tables directly and make them use stored procedures where you do the necessary checks. If you don’t want to do separate SP:s for all tables you can provide a generic dynamic sql to do it but then you have to be careful to escape the strings to prevent sqlinjection.

3) It is not always possible to write fast LINQ. Try to write LINQ that can give you the size of a table of billion of records. Of course 99,9 percent of database developers for sql server don’t know it either.

4) LINQ is beautiful but I don’t like EF.

5) LINQ/EF will not outperform native sql because if/when it will do that they will incorporate it into the database engine.

6) I agree LINQ is much nicer and easier to learn than sql.

7) I think you are both right. One big problem with LINQ/EF is that when filtering is done on strings it will be case insensitive when accessing database but not when accessing lists directly for example. That is extremely dangerous.

[towforce]

3) It is not always possible to write fast LINQ. Try to write LINQ that can give you the size of a table of billion of records. Of course 99,9 percent of database developers for sql server don’t know it either.

I am not at work right now, but wouldn't that be entity.table.count(), or something like that?

If the worst comes to the worst, you can send a SQL statement through EF (and linq IIRC) with something like entity.executeSQLCommand(sql query)

[Pio]

3) It is not always possible to write fast LINQ. Try to write LINQ that can give you the size of a table of billion of records. Of course 99,9 percent of database developers for sql server don’t know it either.

I am not at work right now, but wouldn't that be entity.table.count(), or something like that?

If the worst comes to the worst, you can send a SQL statement through EF (and linq IIRC) with something like entity.executeSQLCommand(sql query)

The problem is that count is not a saved value that is updated when rows are added or subtracted. If you have billions of rows I can tell you that is a problem .

In the worst case you send your sql query as you say, the problem is that it takes some time to send the text instead of a procedure call, it takes some more time for the engine to parameterize out the things that can be parameterized and then hash the text that is left to see if there has been something similar executed before so it can reuse the execution plan...