For example, to describe something which can be in any of 8 states, I would need 3 bits of information. This implies that the amount of information needed to describe a compound system is not a function of the amount of information needed to describe each component system. Physicist: The amount of information in a compound system is the sum of the amounts of information needed to describe its components (assuming those components are independent). In the 5 state example it takes 3 bits to specify the state, however the state cannot specify any 3 bit combination. ), according to Shannon’s somewhat ad-hoc experiments, reduces the information from the expected 4.7 bits (log 2 (26)) per letter to only about 2.3 bits per letter.