Punctual data supply

The premise of this thesis is that the future latency of a traditionally sized first level data cache (DL1) will be at least 3 cycles, and most likely much greater. For the DL1 to continue as an effective mechanism, the data cache hierarchy must include a cache or caches that are smaller and faster than the traditional DL1. A key problem with the conventional cache hierarchy organization is that all load instructions are treated equally. Treating all loads equally implies that all target data are vying for positions in each level of the cache hierarchy regardless of the importance of that data. This thesis proposes a new mechanism to classify loads based on their levels of importance, and to optimize the utilization of the cache hierarchy based on this classification. To this end, this thesis proposes punctual data supply. Punctual data supply is a novel cache hierarchy that focuses on providing a load its data on time. Punctual data supply first classifies loads based on their level of importance, or vitality. An individual load's vitality is defined by when its data is required, and loads are classified into 3 classes: vital (use immediately), semi-vital (use in 2 or 3 cycles), and non-vital (use not for at least 4 cycles). These classified loads based on their vitality are then assigned to caches with appropriate latencies: vital cache (single cycle), semi-vital cache (2 cycles), and the DL1 cache (4 cycles). These smaller caches only contain data for the vital loads, and are therefore more effectively utilized. Overall, a cache organization employing punctual data supply outperforms a conventional cache organization by 10% on a realistic Itanium-2 like machine model.