strcpy() revisited

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
mar
Posts: 1992
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: strcpy() revisited

Post by mar » Mon Dec 09, 2013 9:46 am

Rein Halbersma wrote:Avoiding shifting on signed integers is probably a good rule of thumb. E.g. to use shifting on signed integers in order to get cheaper multiplications and divisions is something that your compiler will already do for you (and if it doesn't, it's probably not worthwhile anyway).

But note that not shifting signed integers at all it is slightly more restrictive than necessary. Shifting positive values by valid shift amounts is typically OK, and only if you shift negative signed values do you land in undefined (for left-shifts) or implementation-defined (for right-shifts) behavior.
Yes you're right, it's guaranteed that non-negative representation of signed integers is exactly the same as corresponding representation of unsigned integer of the same size.
I would prefer using an unsigned type in that case though (unless you want to use that value later in an expression that uses signed integers)

bob
Posts: 20471
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: strcpy() revisited

Post by bob » Mon Dec 09, 2013 5:45 pm

Codesquid wrote:
bob wrote:I've grown up in an era of trusting the compiler. You got a bug, it is almost certainly a programming bug.
Undefined behavior in your program is a programming bug.
That era seems to be coming to an end, because a change to a compiler really should not change program behavior, if the original compiler was worth a crap...
See it the other way around: Different compilers expose different bugs in your program, thus increasing overall quality if those bugs are fixed.
Back to square 1. The apple change to libc did NOT fix anything. It broke thousands of existing applications. What is sad is that they COULD have fixed the bug, removed the undefined behavior, and now all those thousands of applications would STILL work, and would no longer fall into any undefined behavior pitfall. They COULD have done that. But they didn't. They simply adopted the draconian "anything undefined simply crashes." They slowed strcpy() down. I suppose they COULD insert code to catch all integer overflows and force a crash when it happens. The minor issue that programs run 3-4 times slower wouldn't bother anyone I assume? I don't REALLY need 20M bps on my 8-core hardware, 5M ought to be more than enough...

syzygy
Posts: 4450
Joined: Tue Feb 28, 2012 10:56 pm

Re: strcpy() revisited

Post by syzygy » Mon Dec 09, 2013 6:34 pm

bob wrote:Back to square 1. The apple change to libc did NOT fix anything. It broke thousands of existing applications. What is sad is that they COULD have fixed the bug, removed the undefined behavior, and now all those thousands of applications would STILL work, and would no longer fall into any undefined behavior pitfall. They COULD have done that. But they didn't.
Indeed back to square 1. The bug was in your code, see C89, section 4.11.2.3 and see C99 and see C11. Apple cannot fix bugs in your code. The most Apple can do is force you to fix it. They did. Maybe they weren't nice to you, but that does not change the fact that the bug was in your code.

All you have to do is admit that your code had a bug. But we already know that this is beyond you.

Rein Halbersma
Posts: 685
Joined: Tue May 22, 2007 9:13 am

Re: strcpy() revisited

Post by Rein Halbersma » Mon Dec 09, 2013 7:03 pm

wgarvin wrote: I think there's probably a market for compilers with a "secure" mode, where they generate slightly less performant code but never perform these optimizations that rely on undefined behavior never happening. You can already get some of that using compiler options, such as -fwrapv (or -ftrapv) or -fno-strict-aliasing. They should take that to the logical conclusion and build us a mode that treats ALL undefined behavior as having some implementation-specific behavior (and bonus points if they spell out clearly what it is and commit to not changing it in the future). Maybe the standards committee or some research group will produce a spec that the compiler vendors could target for it. A lot of developers who make safety-critical embedded devices, avionics, medical equipment, OS kernels, etc. would be better served by a "safer" compiler rather than fastest possible generated code.
In the D programming language they have the @safe annotation, which IIRC works a bit like const, i.e. you can't call non-safe code from within a safe function.

syzygy
Posts: 4450
Joined: Tue Feb 28, 2012 10:56 pm

Re: strcpy() revisited

Post by syzygy » Mon Dec 09, 2013 7:18 pm

I don't know how helpful and complete it will be, but gcc-4.9 will have an UB detector:
UndefinedBehaviorSanitizer (ubsan), a fast undefined behavior detector, has been added and can be enabled via -fsanitize=undefined. Various computations will be instrumented to detect undefined behavior at runtime. UndefinedBehaviorSanitizer is currently available for the C and C++ languages.
http://gcc.gnu.org/gcc-4.9/changes.html

Rein Halbersma
Posts: 685
Joined: Tue May 22, 2007 9:13 am

Re: strcpy() revisited

Post by Rein Halbersma » Mon Dec 09, 2013 7:22 pm

syzygy wrote:I don't know how helpful and complete it will be, but gcc-4.9 will have an UB detector:
UndefinedBehaviorSanitizer (ubsan), a fast undefined behavior detector, has been added and can be enabled via -fsanitize=undefined. Various computations will be instrumented to detect undefined behavior at runtime. UndefinedBehaviorSanitizer is currently available for the C and C++ languages.
http://gcc.gnu.org/gcc-4.9/changes.html
I know which famous chess program I will put to the test :twisted: <<--- that icon is only a small preview of all the demons that will appear

User avatar
Codesquid
Posts: 138
Joined: Tue Aug 23, 2011 8:25 pm
Location: Germany
Contact:

Re: strcpy() revisited

Post by Codesquid » Mon Dec 09, 2013 8:38 pm

bob wrote: What is sad is that they COULD have fixed the bug, removed the undefined behavior, and now all those thousands of applications would STILL work, and would no longer fall into any undefined behavior pitfall
To change undefined into defined, the C-standard needs to be changed.

However, the newly defined behavior might not match what you expect, e.g. they could define the new behavior to be the current undefined one you're suffering from.

They could technically even define it as "If you use strcpy with overlapping strings, bad things happen if and only if you're compiling crafty".

The move forward is to fix the code to use strcpy in only the defined manner. Alternatively you can talk to the standards body and have the standard changed.
Last edited by Codesquid on Mon Dec 09, 2013 8:42 pm, edited 2 times in total.
nanos gigantium humeris insidentes

bob
Posts: 20471
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: strcpy() revisited

Post by bob » Mon Dec 09, 2013 8:41 pm

syzygy wrote:
bob wrote:Back to square 1. The apple change to libc did NOT fix anything. It broke thousands of existing applications. What is sad is that they COULD have fixed the bug, removed the undefined behavior, and now all those thousands of applications would STILL work, and would no longer fall into any undefined behavior pitfall. They COULD have done that. But they didn't.
Indeed back to square 1. The bug was in your code, see C89, section 4.11.2.3 and see C99 and see C11. Apple cannot fix bugs in your code. The most Apple can do is force you to fix it. They did. Maybe they weren't nice to you, but that does not change the fact that the bug was in your code.

All you have to do is admit that your code had a bug. But we already know that this is beyond you.
Never said my code did NOT have a bug. I said it took advantage of a behavior that has worked from day 1 and still works today, except for mavericks which intentionally broke it, without fixing anything at all.

bob
Posts: 20471
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: strcpy() revisited

Post by bob » Mon Dec 09, 2013 8:43 pm

syzygy wrote:I don't know how helpful and complete it will be, but gcc-4.9 will have an UB detector:
UndefinedBehaviorSanitizer (ubsan), a fast undefined behavior detector, has been added and can be enabled via -fsanitize=undefined. Various computations will be instrumented to detect undefined behavior at runtime. UndefinedBehaviorSanitizer is currently available for the C and C++ languages.
http://gcc.gnu.org/gcc-4.9/changes.html
will it catch this one:

a = b + c;

???

with some values of b and c that is undefined. For other values, it is perfectly correct. What WILL it do I wonder? Surely not a warning "this might produce integer overflow which is undefined behavior." Crafty would only produce 20K of those warnings or so...

bob
Posts: 20471
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: strcpy() revisited

Post by bob » Mon Dec 09, 2013 8:44 pm

hgm wrote:
syzygy wrote:Then you should use a compiler that makes a promise that it will deal with integer overflow in a particular way. For gcc, just get used to invoking it with -fwrapv. I'm sure you won't mind the loss in efficiency, especially on loops using an int as loop variable.
Ah, it is good to know that this destructive behavior can be switched off. And of course there won't be any loss in efficiency whatsoever, as I would never write any code that could benefit from 'code-wrecking optimizations'.

I would not write "for(i=1; i>=0; i += i)" when I meant "while(1)". I wonder how many people would... So making the compiler assume this was meant, no matter by which amount of contrived reasoning, doesn't count as sane behavior, IMO.
Actually you can not reliably switch it off. Seems about as many compilers ignore that switch as actually honor it, in the gcc series...

Post Reply