V22.0436 - Prof. Grishman

Lecture 23:   Cache Performance

Two-level cache

Do performance calculations from Gottlieb's web page

Effect of cache size on code

For programs operating on large arrays, there can be a significant speed-up if the array fits in the cache.  For example, if we try the program

#include <stdio.h>
#include <time.h>

#define ARRAYSIZE 200000
#define ITERATIONS 5000

int ray[ARRAYSIZE];
int sum;

main() {
    int iter, i;
    clock_t time1, time2;
    int time;
    time1 = clock();
    for (iter=0; iter<ITERATIONS; iter++)
        for (i=0; i<ARRAYSIZE; i++)
            sum += ray[i];
    time2 = clock();
    time = (time2 - time1) * 1000 / CLK_TCK;
    printf ("CPU time: %i milliseconds\n", time);
    }

with different values of ARRAYSIZE and ITERATIONS (keeping the product, and hence the number of instructions, constant), the program will run faster if the array fits in the cache.  Even if the array size exceeds the cache, we benefit from having the cache fetch an entire line, so we don't get a cache miss on each element.  If we skip array elements

#include <stdio.h>
#include <time.h>

#define ARRAYSIZE 800000
#define SKIPSIZE 4
#define ITERATIONS 5000

int ray[ARRAYSIZE];
int sum;

main() {
    int iter, i;
    clock_t time1, time2;
    int time;
    time1 = clock();
    for (iter=0; iter<ITERATIONS; iter++)
        for (i=0; i<ARRAYSIZE; i+=SKIPSIZE)
            sum += ray[i];
    time2 = clock();
    time = (time2 - time1) * 1000 / CLK_TCK;
    printf ("CPU time: %i milliseconds\n", time);
    }

this benefit decreases.  Large scientific programs can structure their array operations to take advantage of these cache properties.

Virtual memory

Virtual memory provides the next step in the memory hierarchy after main memory: disk memory. The gap in access times, however, is much larger (between 100ns main memory and 10ms disk access time) and this affects the parameters of the design (pages are much larger than cache lines, and hit ratios must be much higher for the machine to work efficiently). As technology changes, we can expect the parameters of the memory levels to change, but the basic idea of a memory hierarchy to remain.



Input-Output

very wide range of device speeds and types (storage devices, computer-computer communication devices,  human interfaces)

Rotating magnetic storage

Input-output: requirements