Origin of scores for letters in Scrabble (in relation to their frequency in a langauge)

Question

In Scrabble, each letter is worth a number of points. Typically, the rarer the letter, the more score it has, see Scrabble letter distribution and score.

Does anyone know how these scores are being assigned? (Or modified, if needed.) Is it based on some gut-feeling, or a formula/algorithm?

As I see, the score is proportional to surprisal (from information theory), i.e. \log(1/p), where p is the frequency of a given letter in a given language. Just I am not sure, if such scoring is planned mathematically, or appeared as some emergent behavior (e.g. other scoring seems unfair, unnatural, etc...).

Answer 1

That is touched upon in the official History of Scrabble by Hasbro, the owner of Scrabble.

Legend has it Butts studied the front page of “The New York Times” to make his calculations for the letter distribution in the game. This skilled, cryptographic analysis of our language formed the basis of the original tile distribution, which has remained constant through almost three generations and billions of games.

I have misplaced (or more likely given away, again) my copy of Everything Scrabble, but I remember bits of what I think I remember reading there (hows that for reliable source?).

From what I remember reading, the letter distribution in Scrabble has continuously slid in favor of people who memorize things like 2-Letter Lists and Q without U words. Even with two well-read players who have never really studied the English language as it relates to Scrabble, the distribution is off-kilter for the normal vocabulary in 2015.

It's not your imagination, there are too many Is. People who know that QI and ZA are words, really tip the value (and risk) of those letters.

Only Hasbro can change Scrabble, but Words With Friends is gaining popularity in part because the tiles more closely represent the way the English language is used today.

Should the tile values change with the times?

Is ZA really a word?

Has X become the best letter in the alphabet?

These are hotly debated subjects.

The history of how the tiles got their distribution seems pretty clear, even if it is described in the official history as a legend. What is less clear is the origin of the point value of individual letters. I think the point value is a part of the same thing, unchanged since the Scrabble name, but that the values were printed on the scorecard, not the tiles. Unfortunately, I have no citation for that, it's one of those things that I read somewhere - or maybe I saw an antique scorecard on display - who knows? But I am fairly certain that the official Scrabble point value per letter has remained unchanged along with the distribution of 100 letters.

Answer 2

The scrabble tile frequencies were almost certainly tweaked based on playtesting as well as letter frequency. The fact that there are only 4 's' tiles was probably intended to decrease the probability of getting the most useful letter to make the game less about simply pluralizing things. Words with Friends tweaked the frequencies further- they added an 'e' and removed an 'i', for example. I imagine that was based on feedback that Scrabble players get too many 'i' tiles.

My theory is that a lexicographic search would inflate the number of 'i' letters appearing in text or a dictionary, e.g. if there are lots of 'ing' and 'tion' words that are common in print but don't come up as often in scrabble due to the probability of getting those other letters. In other words, while 'i' may be 3/4 as common as e in the English, the ability to place that letter in Scrabble is less than that ratio would imply because there are fewer letter sequences in which 'i' can appear.