Q: Is it possible to objectively quantify the amount of information a sentence contains?
For example, one might say that a coin toss generates a single bit of information: the result is either heads or tails. If you’re creative enough, there’s no limit to how much information you squeeze out of this supposedly one-bit sentence. Is it really possible to objectively quantify the amount of information a sentence contains? In an extremely inexact, but still kinda accurate nutshell: the information of a sentence can be measured by how much you’re surprised by it. Unfortunately, there is no completely objective way to quantify exactly how much information a sentence contains.