A question about cpu cache

[Isaac]

I've been told that values (such as loop counters) as well as entire asm programs, can fit into the cpu cache and that this implies the program runs faster than if the program was in the RAM for instance.
I know that in general the cpu cache memory is only a few kB or MB but so are chess engines. In particular asmfish is roughly 110 kB if I remember well. Does this mean it can fit into the cpu cache and that it executes faster than on computers that cannot make it fit into the cpu cache?
In order to "put" asmfish into the cpu cache, how would one do this? Open it a few thousands times in a row with a script? I'm guessing not, but I have no knowledge on the topic (as you already noticed).

[hgm]

Caching in modern CPUs is fully transparent. Basically the CPU tries to keep (a copy of) the last so-many used RAM locations in its cache. So when a program, or the data set it uses, is smaller than the cache, the RAM needs never be consulted more than once for every location. If it is larger than the cache, however, parts of it that have not been used for some time will be overwritten by more-recently-used parts of the program.And when the CPU then needs it again, it will have to be read from RAM again (and stored in the cache, at the expense of something else).

So there is nothing you can do as a user to enhance cache use. As a programmer you can make your program small, and ifit cannot be made small enough, at increase its 'locality' (i.e. makesure that things,once accessed, are accessed many times before other stuff (that forced it out of the cache) is accessed.

[Zenmastur]

What HGM says is not entirely correct.

A good paper to read on the subject is "What Every Programmer Should Know About Memory" which can be found on the web.

There are, of course, other papers that deal more directly with the topic of CPU caches but I don't have time right now to look for the proper titles and this one should give you a good intro to the subject.

Regards,

Forrest

[jdart]

Caching applies not just to code but to data.

Modern programs use a lot of memory, especially for the main hash table.

Given current hardware, you can't readily fit the working memory into the processor cache, not even the L3 cache. So you are going to incur overhead when a cache miss occurs and the processor must actually do a memory fetch.

--Jon

[hgm]

What HGM says is not entirely correct.

Indeed, but it is a simplification that suffices to answer his question. Going into technical details of direct mapping, the number of cache ways, the length of a cache line, pseudo-LRU replacement, physical vs logical tagging will only add confusion.