There is a trade-off between memory speed, cost, and capacity:
A cost effective system must have a mix of all of these memories, which means that the system must manage its data so that it is rapidly available when it is needed. In earlier machines, all of this memory management was done explicitly by the programmer; now more of it is done automatically and invisibly by the system. The ideal is to create a system with the cost and capacity of the cheapest technology along with the speed of the fastest.
If memory access was entirely random, automatic memory management would not be possible. Management relies on:
A cache is a level of memory between main memory and the CPU (registers). The goal with a cache is to get the speed of a small SRAM with the capacity of main memory.
One issue in cache design is the mapping of memory addresses into cache lines. The simples scheme is a direct mapped cache: each word of memory maps into a single line (word) in the cache. Alternative schemese are fully associative (a memory word can be placed in any cache line --- not practical except for the very smallest caches) and set associative (a memory word can be placed into any of a small set of cache lines, typically 2 or 4).
Spatial locality can be taken advantage of by using cache lines ("blocks") consisting of more than one word: when one word is requested by the CPU, the entire block is fetched from main memory.