Tail Redundancy And Its Characterizations Of Universal Compression

We completely characterize the asymptotic persymbol average case redundancy of any collection P of i.i.d. distributions over a countably infinite alphabet. We prove that universal compression of length-n i.i.d. sequences from a class P is characterized by how well the tails of of the single letter marginals of P can be universally described. We capture this notion by the tail redundancy of a class P of distributions, and develop on the properties of tail redundancy.