Conventional Number Systems for DNN Architectures

The two conventional number systems, namely the floating point and fixed point, are commonly used in almost all general-purpose DNN engines. While the FLP representation is usually used for modern computation platforms (e.g., CPUs and GPUs), where high precision is required, FXP is more common in low-cost computation platforms used for applications that require high speed, low power consumption, and small chip area. This chapter introduces these two representations and briefly discusses their utilization for implementing DNN hardware, in order to facilitate a comparison between conventional and unconventional number systems presented in subsequent chapters.