5

For example, if you play a round of cribbage mathematically perfectly, someone will win because they got better cards on average. But if that difference in scores comes down to 10 points, that could come down to imperfect play or even gambling on a card.

Is it possible to say that cribbage is only for example, 50% skill?

0
2

I can think of two mathematical measures that might be useful, both based around Elo ratings.

  1. Is the correct Elo distribution function. Let's suppose Player B beats Player A 64% of the time, and Player C beats Player B 64% of the time. How often does Player C beat Player A? Generally, speaking, the higher the number, the less of a role luck plays, though you should be suspicious of answers much above 83%. (83% is what you would expect if all player's performance in a game (either because of luck or variance in performance) is normally distributed, all with the same variance)

  2. Is the Elo difference, usually the difference between the top player and an average player. Filtered through the answer to the previous paragraph, this basically means how often the top player beats an average player, except this is frequently hard to directly observe (either because top players never play average ones or because the answer is "always"), so you may need to measure this indirectly through Elo.

In many games, you want to think of a "game" as actually a sequence of games for this to make any sense (just like playing cribbage to 120, as usual, is in some sense a sequence of games).

8
  • 1
    If instead of just the Elo distribution, you have the history of all game results in the community: Measuring skill and chance in games is a research paper by Duersch, Lambrecht & Oechssler from 2017. It is an attempt to rank games on a scale of luck vs. skill. The main idea is to look at how likely a stronger player is to win against a weaker player. In a pure-skill game, a stronger player should always win. In an luck-heavy game, the probability should be close to 50%.
    – Stef
    Nov 30 '21 at 16:31
  • An important thing to mention is that this approach to measuring "luck vs skill" depends on the community of players considered, not just on the rules of the game. For instance, if you look at the game played among professional chess players, you might conclude that chess is a high-skill, no luck game; but if you look at the games played between amateur players in bars, you might conclude that chess does have a fair amount of luck.
    – Stef
    Nov 30 '21 at 16:40
  • 1
    @Stef Which (properly) raises the question of what “luck” means. If you and your opponent are both capable of recognizing and capitalizing on 50% of all possible mistakes, is it “luck” if you win because more of the mistakes made by your opponent happened to fall in the category of those you can exploit than vice versa? And as we think about that, we have to remember that we have implicitly assumed that all mistakes are equal, both in how recognizable they are and in how impactful they are, which is clearly not the case. Goes a long way to showing why this question really is semantic.
    – KRyan
    Nov 30 '21 at 16:49
  • 1
    (Not to say that this set of definitions and the resulting approach to estimating the answer under those definitions is in any way deficient—it’s a great answer and I have upvoted. I just appreciate the food for thought here even more than the actual answer itself.)
    – KRyan
    Nov 30 '21 at 16:50
  • As a refinement of Point (2), one might consider "How long must a session of play be so that an average player loses more than 95% of the time to an expert player?" An average club bridge partnership can expect to get a couple of club wins a year; but will only get a couple of two session wins against the same opposition in a lifetime, and likely never get a 4 session win. Dec 1 '21 at 0:36
1

Compare the Expected value to the Standard Deviation.

The higher the luck factor in the game, the higher S.D(X) - E(X).


For example, lets look at a game of Poker with high and low amount of luck: Assume a player that have a small edge over a Poker table, meaning that her expected value is positive E[X]>0, now

  • In a single round of Poker, the result is very much luck driven. The Standard Deviation is much bigger than the Expected value S.D(X) > E(X).
  • In 1M rounds of Poker, the variance of the sum of results decreases due to the law of large numbers, and expected value is larger than the variance. E(1MX) > S.D(1MX)
6
  • Is this something you can look up somewhere? Nov 30 '21 at 15:23
  • 4
    Variance and expected value are not expressed in the same dimension. So a statement such as "the variance is larger than the expected value" doesn't quite make sense. Perhaps you meant "standard deviation" instead of "variance"?
    – Stef
    Nov 30 '21 at 16:33
  • 4
    I see what you're getting at, but this answer is pretty imprecise with statistical concepts. Playing more hands narrows the variance of the expected value (standard error of the mean), but does not at all change the variance of the value of a hand. You suggest comparing expected value vs. variance to compare skill vs. luck, but then also seem to suggest that you can lower the variance and therefore make a game less luck-dependent simply by playing it many times, which makes no sense. Nov 30 '21 at 16:46
  • @NuclearHoagie, you can lower the variance and therefore make a game less luck-dependent simply by playing a new game: "the sum of several games". Tennis is good example.
    – Cohensius
    Dec 1 '21 at 7:58
  • @Cohensius In the long run, any game will have the more skilled player win more often, making every long series of non-random games effectively devoid of luck. That's not really a characteristic of the game being played, since it it true of every game. If you play enough times, only skill matters, regardless of what you're playing. Dec 1 '21 at 14:22
0

It would very difficult to come up with a metric that depends solely on the game itself, and not the population of players. It's much easier to answer the question "How large a role does luck play, in comparison to the skill difference of players?" than to answer the question "What is an absolute measure of how much of a role does luck play?"

If it's defined in terms of the players, then there are a variety of metrics. One would be to look at the variance of win rates among players; the higher the variance, the less important luck is. However, you'd have to adjust for the fact that players tend to play players of similar skill, which decreases the variance; if two players of exactly equal skill play, then the game will come down solely to luck. We could also look at how much Elo spread there is (different games use different parameters for their Elo assignments, but if you construct Elo scores using the same parameters across games, then comparing the Elo spreads should give some sense of the size of luck).

Not the answer you're looking for? Browse other questions tagged or ask your own question.