Computer systems implement a memory hierarchy to balance access speed, storage capacity, and hardware cost.
A. The Memory Hierarchy Pyramid
As you move down the hierarchy:
- Access Speed decreases (Registers are fastest, HDDs/magnetic tapes are slowest).
- Storage Capacity increases (Registers hold bytes, HDDs hold Terabytes).
- Cost per bit decreases (Registers are very expensive, secondary storage is extremely cheap).
B. Cache Memory & Locality of Reference
Cache memory is situated between the CPU and main RAM. It relies on the principle of Locality of Reference:
- Temporal Locality: If a memory location is accessed once, it is highly likely to be accessed again in the near future (e.g., loop variables).
- Spatial Locality: If a memory location is accessed, nearby locations are likely to be accessed soon (e.g., array traversal).
Cache Levels: L1 Cache is built inside each CPU core (smallest, fastest). L2 Cache is slightly larger and placed close to the core. L3 Cache is shared among all CPU cores on the chip (largest, slowest cache level).