What are the standards for algorithmically forecasting metagame evolution under certain conditions of TCG design?

Question

Company X operates a trading card game, periodically designing and releasing new card sets, which seed evolutionary shifts in its competitive metagame. For the purpose of achieving balanced card design, it would be ideal if X could forecast in advance what shifts a candidate new card set would induce, and what new equilibrium metagame state would result. Computer simulations surely play a role here, but I'm not sure how close to solvable this problem is, even under ideal conditions.

Assuming their game's mechanics roughly reflect that of an actual trading card game (30 - 60 card deck size, normal gameplay actions such as playing creatures and spells attacking and so forth, modest design power creep over time, etc.), to what extent can X attempt to accurately forecast the impact of a new set as they design it, given...

1) Set rotation (like Hearthstone's standard format; the number of total cards in the card pool remains relatively constant over time)

2) No set rotation (like the Yu-Gi-Oh! TCG; the number of total cards in the card pool grows large over time)?

Answer 1

It is likely that making this kind of prediction to any useful degree of accuracy is beyond the current state of the art in computing.

First, consider that forecasting the future metagame that uses unreleased cards is essentially equivalent to "forecasting" the current metagame using the current card pool. This actually helps, because that means that you can validate your methodology against every current and past metagame.

So, the task is now to create a simulation that takes the existing card pool, and produces the current metagame. In order to do that, you need to understand and reproduce the reasons that the metagame exists as it currently does. There may be a variety of reasons that a deck may be stronger or weaker in a metagame, including raw deck strength, how easy the deck is to pilot, and how flexibly it can handle different matchups. There are also additional reasons that a deck may appear more or less in a metagame, including card availability, card prices, people's existing decks and collections, and how well it matches popular deck-building heuristics.

In order to understand the strength of any single deck, you would need to be able to pilot it at a pro level to evaluate how it would perform in real top-level tournaments. To my knowledge, there are currently no programs that can play trading card games that well. The cutting edge of research into making computers play games better than humans is probably AlphaGo, and that task is probably simpler because it plays deterministic games on a single pre-defined board.

If you do somehow manage to produce a program that can accurately simulate how any particular matchup would play out, you would need to figure out what matchups are worth testing and evolve that into a stable metagame. This may require independently reconstructing modern deckbuilding strategy, while also allowing for potential rogue decks that exist outside of that strategy. This is the point where you may need to account for the factors like card availability and price. Price is also heavily dependent on metagame popularity, so that would also need to be simulated and fed back into the main simulation. This is also where you would need to evaluate whether a human could and would actually pilot that deck effectively.

Let's say you do manage to put all of that together into a working prediction machine. There are still some potential pitfalls to be aware of. You may miss a rogue deck that is outside of your search space that completely breaks things. Your players may fail to think of a specific deck that features heavily in your simulated metagame, unbalancing the rest of the metagame. You may mispredict whether players would actually be willing or able to use a deck in tournaments, again potentially unbalancing the metagame.

Answer 2

This problem isn't really a solvable one due largely to the size of the problem space.

Consider a game with deck building rules like Magic: The Gathering's standard format (minimum 60 card deck, no maximum size, up to 4 of any given card except basic lands). Imagine you have a 100 card set and have available to you infinite copies of every card.

With just that 100 card set how many decks can you make? Given that there is no maximum deck size, the answer is infinite. But using an infinite maximum deck size isn't really useful, as no one can really play a game with a million card deck. Plus so many of those decks become the same as another deck with one more land in it, they stop being interesting at some point.

So we have to start setting limits on how much of that problem space we are willing to search to find "the best decks" (which will presumably become our meta). So if we are taking Magic as the example here, we can probably limit decks to a maximum size of 300 cards or so (because Battle of Wits exists). Even with a pool of only 100 cards, that's still a ton of decks to consider.

Now you have to program computers to play hundreds of matchups of all these decks to see what is statistically the best. That's hard. Like, really, really hard. Games with perfect information (ie. you know everything that is going on, think checkers) are relatively easy to program AIs for. But even games with perfect information can get tricky. Chess, for example is a game with perfect information and no randomness. Yet making good AIs for that is hard (it can and has been done). And doing so can be time consuming and expensive.

Card games and their AIs are different though. There isn't perfect information (you don't know what's in your opponent's deck, hand or what that face-down card is) and there is randomness (what is your next draw going to be). That makes it really hard to design an AI. It's relatively easy to look at a board and figure out what the best move is to make right now assuming no opponent interaction. But that's how most novice players of any game play and that strategy doesn't take you very far. To add in opponent interaction, probability of getting certain cards, planning longer term, having contingency plans, etc. is going to make this insanely complicated. Oh, and you need to teach computers how to bluff and read bluffs. Good luck with that one.

Oh, and you also have to program in all the rules (without bugs). Depending on your card game, that could be really hard. Any bugs in that programming can invalidate all your test data (think if the AI comes up with a deck that wins 85% of the time in non-mirror matches, that would clearly be the meta deck. Except someone can't program a certain card correctly and that deck wouldn't work out in the real world... oops.)

So at this point (if you've made it this far, both in terms of being able to do this at all and spent the time and money on this) you have an engine that can play and test your game. Now you have to teach the computer about human psychology, metagames, and how they evolve. Metagames evolve usually by figuring out what can topple the current meta. This helps a little in that now you have a starting point for figuring out what to beat and don't have to consider beating every possible deck out there (yay). Now the trick is teaching the computer how to make a meta. Yes you can figure out which deck is statistically the strongest, but one deck does not a meta make. A meta is defined by a lot of rock-paper-scissors type things. In Magic, that's usually some combination of aggro decks, combo decks and control decks. As one deck (or type) rises in popularity (something else you need to program), the meta should change to counteract that by trying to create favorable matchups against the most common deck. Which can change what the most common deck is. Which requires a new decks to counteract that, which changes the meta, which....

Also, you need to deal with some real world simulations here which involve psychology and economics (add those to the programming backlog). People can only realistically make decks with the cards they have. Which means usually having to buy cards. And as cards become more "meta" they become more expensive, leading to people looking for cheaper alternatives or being unable to play the theoretically best deck. Now you have to model supply and demand and things like the time it takes for players to be able to get the cards they need to build these decks. That time and money factor can slow down how fast a meta can shift, meaning that if you are looking at a rotating format your game (and players) may not reach a meta end-state before a rotation happens or a new set comes out and shakes everything up.

You've also need to predict what a human can realistically play and what they will have fun playing. Some people will stick to the "best" deck no matter what but others will get tired of Arcbound Ravager mirror matches and want desperately to play anything else.

This thing can also screw itself over pretty easily. Since it forms a feedback loop (the results of the new meta are fed back in to predict the next meta), any errors, mispredictions or deviations from reality could send it into a spiral or weirder and more wrong predictions.

Oh, and so far we've worked with a small, static set of cards and a static set of rules. Every new set increases the problem space exponentially, rotations require re-evaluation of the meta and new rules or rules changes means new interactions. So more programming... without bugs...

All in all this is going to be an insanely complicated, nigh impossible thing to make. On top of that, it'll likely cost an insane amount to build and maintain which no business will likely want to take on. So I don't think anyone will even try such a thing. It just isn't worth it.