Level 5: Probability and Randomness Gone Horribly Wrong

Readings/Playings

None this week (other than this blog post).

Answers to Last Week’s Questions

If you want to check your answers from last week:

Dragon Die

First off, note that the so-called “Dragon Die” is really just a “1d6+1” in disguise. If you think of the Dragon as a “7” since it always wins, the faces are 2-3-4-5-6-7, so really this is just asking how a +1 bonus to a 1d6 die roll affects your chance of rolling higher. It turns out the answer is: a lot more than most people think!

If you write out all 36 possibilities of 2-7 versus 1-6, you find there are 21 ways to lose to the House (7-1, 7-2, 7-3, 7-4, 7-5, 7-6, 6-1, 6-2, 6-3, 6-4, 6-5, 5-1, 5-2, 5-3, 5-4, 4-1, 4-2, 4-3, 3-1, 3-2, 2-1), 5 ways to draw (6-6, 5-5, 4-4, 3-3, 2-2), and 10 ways to win (5-6, 4-5, 4-6, 3-4, 3-5, 3-6, 2-3, 2-4, 2-5, 2-6). We ignore draws, since a draw is just a re-roll, which we would keep repeating until we got a win or loss event, so only 31 results end in some kind of resolution. Of those, 21 are a loss and 10 are a win, so this game only gives a 10/31 chance of winning. In other words, you win slightly less than 1 time in 3.

Chuck-a-luck

Since there are 216 ways to get a result on 3d6, it is easier to do this in Excel than by hand. If you count it up, though, you will find that if you choose the number 1 to pay out, there are 75 ways to roll a single win (1-X-X where “X” is any of the five losing results, so there are 25 ways to do this; then X-1-X and X-X-1 for wins with the other two dice, counting up to 75 total). There are likewise 15 ways to roll a double win (1-1-X, 1-X-1, X-1-1, five each of three different ways = 15), and only one way to roll a triple win (1-1-1). Since all numbers are equally likely from 1 to 6, the odds are the same no matter which of the six numbers you choose to pay out.

To get expected value, we multiply each of the 216 results by its likelihood; since all 216 die roll results are equally likely, we simply add all results together and divide by 216 to get the expected win or loss percentage.

(75 ways to win * $1 winnings) + (15 ways for double win * $2) + (1 way for triple win * $3) = $108 in winnings. Since we play 216 times and 108 is half of 216, at first this appears to be even-odds.

Not so fast! We still must count up all the results when we lose, and we lose more than 108 times. Out of the 216 ways to roll the dice, there are only 91 ways to win, but 125 ways to lose (the reason for the difference is that while doubles and triples are more valuable when they come up on your number, there are a lot more ways to roll doubles and triples on a number that isn’t yours). Each of those 125 losses nets you a $1 loss.

Adding everything up, we get an expected value of negative $17 out of 216 plays, or an expected loss of about 7.9 cents each time you play this game with a $1 bet. That may not sound like much (7.9 cents is literally pocket change), but keep in mind this is per dollar, so the House advantage of this game is actually 7.9%, one of the worst odds in the entire casino!

Royal Flush

In this game, you’re drawing 5 cards from a 52-card deck, sequentially. The cards must be 10-J-Q-K-A but they can be in any order. The first card can be any of those 5 cards of any suit, so there are 20 cards total on your first draw that make you potentially eligible for a royal flush (20/52). For the second card, it must match the suit of the first card you drew, so there are only 4 cards out of the remaining 51 in the deck that you can draw (4/51). For the third card, there are only 3 cards out of the remaining 50 that you need for that royal flush (3/50). The fourth card has 2 cards out of 49, and the final card must be 1 card out of 48. Multiplying all of these together, we get 480 / 311,875,200, or 1 in 649,740. If you want a decimal, this divides to 0.0000015 (or if you multiply by 100 to get a percentage, 0.00015%, a little more than one ten-thousandth of one percent). For most of us, this means seeing a “natural” Royal Flush from 5 cards is a once in a lifetime event, if that.

IMF Lottery

If you check the comments from last week, several folks found solutions for this that did not require a Monte Carlo simulation. The answer is 45 resources, i.e. the card will stay in play for an average of 10 turns. Since it has a 10% chance of leaving play each turn, this actually ends up being rather intuitive… but, as we’ve seen from probability, most things are not intuitive. So the fact that this one problem ends up with an answer that’s intuitive, is itself counterintuitive.

This Week’s Topic

Last week, I took a brief time-out from the rest of the class where we’ve been talking about how to balance a game, to draw a baseline for how much probability theory I think every designer needs to know, just so we can compute some basic odds. This week I’m going to blow that all up by showing you two places where the true odds can go horribly wrong: human psychology, and computers.

Human Psychology

When I say psychology, this is something I touched on briefly last week: most people are really terrible at having an intuition for true odds. So even if we actually make the random elements of our games perfectly fair, which as we’ll see is not always trivial, an awful lot of players will perceive the game as being unfair. Therefore, as game designers, we must be careful to understand not just true probability, but also how players perceive the probability in our games and how that differs from reality, so that we can take that into account when designing the play we want them to experience.

Computers

Most computers are deterministic machines; they are just ones and zeros, following deterministic algorithms to convert some ones and zeros into other ones and zeros. Yet somehow, we must get a nondeterministic value (a “random number”) from a deterministic system. This is done through some mathematical sleight-of-hand to produce what we call pseudorandom numbers: numbers that sorta kinda look random, even though in reality they aren’t. Understanding the difference between random and pseudorandom has important implications for video game designers, and even board game designers if they ever plan to make a computer version of their game (which happens often with “hit” board games), or if they plan to include electronic components in their board games that have any kind of randomness.

But First… Luck Versus Skill

Before we get into psychology and computers, there’s this implicit assumption that we’ve mostly ignored for the past week, that’s worth discussing and challenging. The assumption is that adding some randomness to a game can be good, but too much randomness is a bad thing… and maybe we have some sense that all games fall along some kind of continuum between two extremes of “100% skill-based” (like Chess or Go) and “100% luck-based” (like Chutes & Ladders or Candyland).

If these are the only games we look at, we might go so far as to think of a corresponding split between casual and hardcore: the more luck in a game, the more casual the audience; the more a game’s outcome relies on skill, the more we think of the game as hardcore.

This is not always the case, however. For example, Tic-Tac-Toe has no randomness at all, but we don’t normally think of it as a game that requires a lot of skill. Meanwhile, each hand of Poker is highly random, but we still think of it as a game where skill dominates. And yet, the game of Blackjack is also random, but aside from counting cards, we see that more as a game of chance than Poker.

Then we get into physical contests like professional sports. On the one hand, we see these as games of skill. Yet, enthusiasts track all kinds of statistics on players and games, we talk about percentage chances of a player making a goal or missing a shot or whatever, sports gamblers make cash bets on the outcomes of games, as if these were not games of skill but games of chance.

What’s going on here? There are a few explanations.

Poker vs. Blackjack

Why the difference in how we perceive Poker and Blackjack? The difference is in when the player makes their bet, and what kind of influence the player’s choices have on the outcome of the game.

In Poker, a successful player computes the odds to come up with a probability calculation that they have the winning hand, and they factor that into their bet along with their perceived reactions of their opponents. As more cards are revealed, the player adjusts their strategy. The player’s understanding of the odds and their ability to react to changes has a direct relation to their performance in the game.

In Blackjack, by contrast, you place a bet at the beginning of the hand before you know what cards you’re dealt, and you generally don’t have the option to “raise” or “fold” as you see more cards revealed.

Blackjack does have some skill, but it’s a very different kind of skill than Poker. Knowing when to hit or stand or split or double down based on your total, the dealer’s showing card, and (if you’re counting cards) the remaining percentage of high cards in the deck… these things are skill in the same sense as skill at Pac-Man. You are memorizing and following a deterministic pattern, but you are not making any particularly interesting decisions. You simply place your bet according to an algorithm, and expect that over time you do as well as you can, given the shuffle of the cards. It’s the same reason we don’t think of the casino as having a lot of “skill” at Craps or Roulette, just because it wins more than it loses.

Professional Sports

What about the sports problem, where a clearly skill-based game seems like it involves random die-rolls? The reason for the paradox is that it all depends on your frame of reference. If you are a spectator, by definition you have no control over the outcome of a game; as far as you’re concerned the outcome is a random event. If you are actually a player on a sports team, the game is won or lost partly by your level of skill. This is why on the one hand sports athletes get paid based on how much they win (it’s not a gamble for them), but sports fans can still gamble on the random (from their perspective) outcome of the game.

Action Games

Randomness works a little differently in action-based video games (like typical First-Person Shooter games), where the players are using skill in their movement and aiming to shoot their opponents and avoid getting shot. We think of these as skill-based games, and in fact these games seem largely incompatible with randomness. There is enough chaos in the system without random die-rolls: if I’m shooting at a moving target, I’ll either hit or miss, based on a number of factors that are difficult to have complete control of. Now, suppose instead the designer thought they’d be clever and add a small perturbation on bullet fire from a cheap pistol, to make it less accurate. You line up someone in your sights, pull the trigger… and miss, because it decided to randomly make the bullet fly too far to the left. How would the player react to that?

Well, they might not notice. The game is going so fast, you’re running, they’re running, you squeeze off a few shots, you miss, you figure you must have just not been as accurate as you thought (or they dodged well). Or if they’re standing still, you sneak up behind them, and you’re sure you have the perfect shot and still miss, you’ll feel like the game just robbed you; that’s not fun and it doesn’t make the game more interesting, it just makes you feel like you’re being punished arbitrarily for being a good shot.

Does that mean luck plays no role in action games? I think you can increase the luck factor to even the playing field, but you have to be very careful about how you do it. Here’s a common example of how a lot of FPSs increase the amount of luck in the game: headshots. The idea here is that if you shoot someone in the head rather than the rest of the body, it’s an instant kill, or something like that.

Now, you might be thinking… wait, isn’t that a skill thing? You’re being rewarded for accuracy, hitting a small target and getting bonus damage because you’re just that good… right? In some cases that is true, depending on the game, but in a lot of games (especially older ones) that kind of accuracy just really isn’t possible in most situations; you’re moving, they’re moving, the head has a tiny hit box, the gun doesn’t let you zoom in enough at a distance to be really sure if you aren’t off by one or two pixels in any direction… so from a distance, at least, a head shot isn’t something that most players can plan on. Sometimes it’ll happen anyway, just by accident if you’re shooting in the right general direction… so sometimes, through no fault of the player, they’ll get a headshot. This evens the playing field slightly; without headshots, if it takes many shots to score a frag, the more skilled players will almost always win because they are better at dodging, circle-strafing, and any other number of techniques that allow them to outmaneuver and outplay a weaker player. With headshots, the weaker player will occasionally just get an automatic kill by accident, so it makes it more likely that the weaker player will see some successes, which is usually what you want as a designer.

Shifting Between Luck and Skill

As we just saw, with action games, adding more of a luck-factor to the game is tricky but still possible, by creating these unlikely (but still possible to perform by accident) events.

With slower, more strategic games, adding luck (or skill) is more straightforward. To increase the level of luck in the game:

• Change some player decisions into random events.
• Reduce the number of random events in the game (this way the Law of Large Numbers doesn’t kick in as much, thus the randomness is less likely to be evenly distributed).
• Increase the impact that random events have on the game state.
• Increase the range of randomness, such as changing a 1d6 roll to a 1d20 roll.
• If you want to increase the level of skill in the game, do the reverse of any or all of the above.

What is the “best” mix of luck and skill for any given game? That depends mostly on your target audience. Very young children may not be able to handle lots of choices and risk/reward or short/long term tradeoffs, but they can handle throwing a die or spinning a spinner and following directions. Competitive adults and core gamers often prefer games that skew more to the skill side of things so they can exercise their dominance, but (except in extreme cases) not so much that they feel like they can’t ever win against a stronger opponent. Casual gamers may see the game as a social experience more than a chance to exercise the strategic parts of their brains, so they actually prefer to think less and make fewer decisions so they can devote more of their limited brainpower to chatting with their friends. There is no single right answer here for all games; recognize that a certain mix of luck and skill is best for your individual game, and part of your job as a game designer is to listen to your game to find out where it needs to be. Sometimes that means adding more randomness, sometimes it means removing it, and sometimes it means keeping it there but changing the nature of the randomness. The tools we gained from last week should give you enough skills to be able to assess the nature of randomness in your game, at least in part, and make appropriate changes.

And Now… Human Psychology

As we saw last week, human intuition is generally terrible when it comes to odds estimation. You might have noticed this with Dragon Die or Chuck-a-Luck; intuitively, both games seem to give better odds of winning than they actually do. Many people also have a flawed understanding of how probability works, as we saw last week with the gambler’s fallacy (expecting that previous independent events like die-rolls have the power to influence future ones). Let’s dig into these errors of thought, and their implications to gamers and game designers.

Selection Bias

When asked to do an intuitive odds estimation, where does our intuition come from? The first heuristic most people use is to check their memory recall: how easy is it to recall different events? The easier it is to remember many examples, the more we assume that event is likely or probable. This usually gives pretty good results; if you’re rolling a weighted die a few hundred times and seem to remember the number 4 coming up more often, you’ll probably have a decent intuition for how often it actually comes up. As you might guess, this kind of intuition will fail whenever it’s easier to recall a rare event than a common one.

Why would it be easier to recall rare events than common events? For one thing, rare events that are sufficiently powerful tend to stick in our minds (I bet you can remember exactly where you were when the planes struck the towers). We sometimes see rare events happen more often than common ones due to media coverage. For example, many people are more afraid of dying in a plane crash than dying in an auto accident, even though an auto fatality is far more likely. There are a few reasons for this, but one is that any time a plane crashes anywhere, it’s international news; car crashes, by contrast, are so common that they’re not reported… so it is much easier to remember a lot of plane crashes than a lot of car crashes. Another example is the lottery; lotto winners are highly publicized while the millions of losers are not, leading us to assume that winning is more probable than it actually is.

What does any of this have to do with games? For one thing, we tend to remember our epic wins much more easily than we remember our humiliating losses (another trick our brains play on us just to make life more bearable). People tend to assume they’re above average at most things, so absent of actual hard statistics, players will tend to overestimate their own win percentage / skill. This is dangerous, in games where players can set their difficulty level or choose their opponents. In general, we want a player to succeed a certain percentage of the time, and tune the difficulty of our games accordingly; if a player chooses a difficulty that’s too hard for them, they’ll struggle a bit more and be more likely to give up in frustrations. By being aware of this tendency, we can try (for example) to force players into a good match for their actual skills – through automated matchmaking, dynamic difficulty adjustment, or other tricks.

Self-Serving Bias

There’s a certain point where an event is unlikely but still possible, where players will assume it is much more likely than it actually is. In Sid Meier’s GDC keynote this year, he placed this from experience at somewhere around 3:1 or 4:1… that is, if the player had a 75 to 80% chance of winning or greater, and they did win exactly that percentage of the time, it would feel wrong to them, like they were losing more than they should. His playtesters expected to win nearly all of the time, I’d guess around 95% of the time, if the screen displayed a 75% or 80% chance.

Players also have a self-serving bias, that probably ties into what I said before about how everyone thinks they’re above-average. So while players are not okay with losing a quarter of the time when they have a 75% win advantage, they are perfectly okay winning a quarter of the time if they are at a 1:3 disadvantage.

Attribution Bias

In general, players are much more likely to accept a random reward than a random setback or punishment. And interestingly, they interpret these random events very differently.

With a random reward, players have a tendency to internalize the event, to believe that they earned the reward through superior decision-making and strategy in play. Sure, maybe it was a lucky die roll, but they were the ones who chose to make the choices that led to the die roll, and their calculated risk paid off, so clearly this was a good decision on their part.

With a random setback, players tend to externalize the event; they blame the dice or cards, they say they were just unlucky. If it happens too much, they might go so far as to say that they don’t like the game because it’s unfair. If they’re emotionally invested enough in the game, such as a high-stakes gambling game, they might even accuse other players of cheating! With video games the logic and random-number generation is hidden, so we see some even stranger player behavior. Some players will actually believe that the AI is peeking at the game data or altering the numbers behind their back and cheating on purpose, because after all it’s the computer so it can theoretically do that.

Basically, people handle losing very differently from winning, in games and in life.

Anchoring

Another way people get odds wrong is this phenomenon called anchoring. The idea is, whatever the first number is that people see, they latch onto that and overvalue it. So for example, if you got to a casino and look at any random slot machine, probably the biggest, most attention-grabbing thing on there is the number of coins you can win with a jackpot. Because people look at that, and concentrate on it, and it gives them the idea that their chance of winning is much bigger than it actually is.

Sid Meier mentioned a curious aspect of that during his keynote. Playtesters – the same ones that were perfectly happy losing a third of the time when the had a 2:1 advantage just like they were supposed to – would feel the game was unfair if they lost a third of the time when they had a 20:10 advantage. Why? Because the first number they see is that 20, which seems like a big number so they feel like they’ve got a lot of power there… and it feels a lot bigger than 10, so they feel like they should have this overwhelming advantage. (Naturally, if they have a 10:20 disadvantage, they are perfectly happy to accept one win in three.)

It also means that a player who has, say, a small amount of base damage and then a bunch of bonuses may underestimate how much they do.

The Gambler’s Fallacy

Now we return to the gambler’s fallacy, which is that people expect random numbers to look random. Long streaks make people nervous and make them question whether the numbers are actually random.

One statistic I found from the literature is that if you ask a person to generate a “random” list of coin flips from their head, they tend to not be very random. Specifically, if a person’s previous item was Heads, they have a 60% chance of picking Tails for the next flip, and vice versa (this is assuming you’re simply asking them to say “Heads” or “Tails” from their head when they are instructed to come up with a “random” result, not when they’re actually flipping a real coin). In a string of merely 10 coin flips, it is actually pretty likely you’ll get 4 Heads or 4 Tails in a row (since the probability of 4 in a row is 1 in 8), but if you ask someone to give you ten “random” numbers that are either 0 or 1 from their head, they will probably not give you even 3 in a row.

This can lead players astray in less obvious ways. Here’s an example, also from Sid’s keynote. Remember how players feel like a 3:1 advantage means they’ll almost always win, but they’re okay with losing a 2:1 contest about a third of the time? It turns out that if they lose two 2:1 contests in a row, this will feel wrong to a lot of people; they don’t expect unlikely events to happen multiple times in a row, even though by the laws of probability they should.

Here’s another example, to show you why as game designers we need to be keenly aware of this. Suppose you design a video game that involves a series of fair coin flips as part of its core mechanics (maybe you use this to determine who goes first each turn, or something). Probability tells you that 1 out of every 32 plays, the first six coin flips will be exactly the same result. If a player sees this as their first introduction to your game, they may perceive this event as so unlikely that the “random” number generator in the game must be busted somehow. If the coins come up in their favor, they won’t complain… but when regression to the mean kicks in and they start losing half the time like they’re supposed to, they’ll start to feel like the game is cheating, and it will take them awhile to un-learn their (incorrect) first impression that they are more likely to win a flip. Worse, if the player sees six losses in a row, right from the beginning, you can bet that player is probably going to think your game is unfair. To see how much of a problem this can potentially become, suppose your game is a modest hit that sells 3.2 Million units. In that case, one hundred thousand players are going to experience a 6-long streak as their first experience on their first game. That’s a lot of players that are going to think your game is unfair!

The Gambler’s Fallacy is something we can exploit as gamers. People assume that long streaks do not appear random, so when trying to “play randomly” they will actually change values more often than not. Against a non-championship opponent, you can win more than half the time at Rock-Paper-Scissors by knowing this. Insist on playing to best 3 of 5, or 4 of 7, or something. Since you know your opponent is unlikely to repeat their last throw, on subsequent rounds you should throw whatever would have lost to your opponent’s last throw, because your opponent probably won’t do the same thing twice, so you probably won’t lose (the worst you can do is draw).

The Hot-Hand Fallacy

There’s a variant of the Gambler’s Fallacy that mostly applies to sports and other action games. The Hot-Hand fallacy is so called because in the sport of Basketball fans started getting this idea that if a player made two or three baskets in a row, they were “running hot” and more likely to score additional baskets and not miss. (We even see this in sports games like NBA Jam, where becoming “on fire” is actually a mechanic that gives the player a speed and accuracy advantage… and some cool effects like making the basket explode in a nuclear fireball.)

When probability theorists looked at this, their first reaction was that each shot is an independent event, like rolling dice, so there’s no reason why previous baskets should influence future ones at all. They expected that a player would be exactly as likely to make a basket, regardless of what happened in the players’ previous attempts.

Not so fast, said Basketball fans. Who says they’re completely independent events? Psychology plays a role in sports performance. Maybe the player has more confidence after making a few successful shots, and that causes them to play better. Maybe the fans cheering them on gives them a little extra mental energy. Maybe the previous baskets are a sign that the player is hyper-focused on the game and in a really solid flow state, making it more likely they’ll continue to perform well. Who knows?

Fair enough, said the probability theorists, so they looked at actual statistics from a bunch of games to see if previous baskets carried any predictive value for future performance.

As it turned out, both the theorists and sports fans were wrong. If a player made several baskets in a row, it slightly increased their chance of missing next time – the longer the streak, the greater the chance of a miss (relative to what would be expected by random chance). Why? I don’t think we know for sure, but presumably there is some kind of negative psychological effect. Maybe the player got tired. Maybe the other team felt that player was more of a threat, and played a more aggressive defense when that player had the ball. Maybe the crowd’s cheering broke the player’s flow state, or maybe the player gets overconfident and starts taking more unnecessary risks.

Whatever the case, this is something that works against us when players can build up a win streak in our games – especially if we tie that to social rewards, such as achievements, trophies, or leaderboards that give special attention to players with long streaks. Why is this dangerous? Because at best, even if each game is truly an independent random event, we know that a win streak is anomalous. If a player’s performance overall falls on some kind of probability curve (usually a bell curve) and they happen to achieve uncharacteristically high performance in a single game or play session or whatever, odds are their next game will fall lower on the curve. The player is probably erroneously thinking their skills have greatly improved; when they start losing again, they’ll feel frustrated, because they know they can do better. Thus, when the streak inevitably comes to an end, the whole thing is tainted in the player’s mind. It’s as if the designer has deliberately built a system that automatically punishes the player after every reward.

Houston, We Have a Problem…

To sum up, here are the problems we face as designers, when players encounter our random systems:

• Selection bias: improbable but memorable events are perceived as more likely than they actually are.
• Self-serving bias: an “unlikely loss” is interpreted as a “nearly impossible loss” when the odds are in the player’s favor. However, an “unlikely win” is still correctly interpreted as an “unlikely but possible win” when the odds are against the player.
• Attribution bias: a positive random result is assumed to be due to player skill; a negative random result is assumed to be bad luck (or worse, cheating).
• Anchoring: players overvalue the first or biggest number seen.
• Gambler’s fallacy: assuming that a string of identical results reduces the chance that the string will continue.
• Hot-hand fallacy: assuming that a string of identical results increases the chance that the string will continue.

The lesson here is that if you expose the actual probabilities of the game to your players, and your game produces fair, random numbers, players will complain because according to them and their flawed understanding of probability, the game feels wrong.

As designers, what do we do about this? We can complain to each other about how all our stupid players are bad at math. But is there anything we can do to take advantage of this knowledge, that will let us make better games?

When Designers Turn Evil

One way to react to this knowledge is to exploit it in order to extract large sums of money from people. Game designers who turn to the Dark Side of the Force tend to go into the gambling industry, marketing and advertising, or political strategy. (I say this with apologies to any honest designers who happen to work in these industries.)

Gambling

Lotteries and casinos regularly take advantage of selection bias by publicizing their winners, making it seem to people like winning is more likely than it really is. Another thing they can do if they’re more dishonest is to rig their machines to give a close but not quite result more often than would be predicted by random chance, such as having two Bars and then a blank come up on a slot machine, or having four out of five cards for a royal flush come up in a video poker game. These give the players a false impression that they’re closer to winning more often than they actually are, increasing their excitement and anticipation of hitting a jackpot and making it more likely they’ll continue to play.

Marketing and Advertising

Marketers use the principle of anchoring all the time to change our expectations of price. For example, your local grocery store probably puts big discount stickers all over the place to call your attention to the lower prices on select items, and our brains will assume that the other items around them are also less expensive by comparison… even if they’re actually not.

Another example of anchoring is a car dealership that might put two nearly identical models next to each other, one with a really big sticker price and one with a smaller (but still big) price. Shoppers see the first big price and anchor to that, then they see the smaller price and feel like by comparison they’re getting a really good deal… even though they’re actually getting ripped off.

Political Strategy

There are a ton of tricks politicians can use to win votes. A common one these days is to play up people’s fears of vastly unlikely but well-publicized events like terrorism or hurricanes, and make campaign promises that they’ll protect you and keep your family safe. Odds are, they’re right, because the events are so unlikely that they probably won’t happen again anyway.

Scam Artists

The really evil game designers use their knowledge of psychology to do things that are highly effective, and highly illegal. One scam I’ve heard involves writing a large number of people offering your “investment advice” and telling them to watch a certain penny stock between one day and the next. Half of the letters predict the stock will go up, the other half say it’ll go down. Then, whatever actually happens, you take that half where you guessed right, and make another prediction. Repeat that four or five times, and eventually you get down to a handful of people for whom you’ve predicted every single thing right. And those people figure there’s no way that could just be random chance, you must have a working system, and they give you tons of money, and you skip town and move to Fiji or something.

What About Good Game Designers?

All of that is fine and good for those of us who want to be scam artists (or for those of us who don’t want to get scammed). But what about those of us who still want to, you know, make games for entertainment?

We must remember that we’re crafting a player experience. If we want that experience to be a positive one, we need to take into account that our players will not intuitively understand the probabilities expressed in our game, and modify our designs accordingly.

Skewing the Odds

One way to do this is to tell our players one thing, and actually do something else. If we tell the player they have a 75% chance of winning, under the hood we can actually roll it as if it were a 95% chance. If the player gets a failure, we can make the next failure less likely, cumulatively; this makes long streaks of losses unlikely, and super-long streaks impossible.

Random Events

We can use random events with great care, especially those with major game-changing effects, and especially those that are not in the player’s favor. In general, we can avoid hosing the player to a great degree from a single random event; otherwise, the player may think that they did something wrong (and unsuccessfully try to figure out what), or they may feel annoyed that their strategy was torn down by a single bad die-roll and not want to play anymore. Being very clear about why the bad event happened (and what could be done to prevent it in future plays) helps to keep the player feeling in control.

Countering the Hot Hand

To counter the hot-hand problem (where a streak of wins makes it more likely that a player will screw up), one thing to do is to downplay the importance of “streaks” in our games, so that the players don’t notice when they’re on a streak in the first place (and therefore won’t notice, or feel as bad, when the streak ends).

If we do include a streak mechanism, one thing we can do is embed them in a positive feedback loop, giving the player a gameplay advantage to counteract the greater chance of a miss after a string of hits. For example, in Modern Warfare 2, players get certain bonuses if they continue a kill streak (killing multiple enemies without being killed themselves), including better weapons, air support, and even a nuclear strike. With each bonus, it’s more likely their streak will continue because they are now more powerful.

Summary

In short, we know that players have a flawed understanding of probability. If we understand the nature of these flaws, we can change our game’s behavior to conform to player expectations. This will make the game feel more fun and more fair to the players. This was, essentially, one of the big takeaways from Sid Meier’s GDC keynote this year.

A Question of Professional Ethics

Now, this doesn’t sit well with everyone. Post-GDC, there were at least a few people that said: wait, isn’t that dishonest? As game designers, we teach our players through the game’s designed systems. If we bend the rules of probability in our games to reinforce players’ flawed understanding of how probability works, are we not doing a disservice to our players? Are we not taking something that we already know is wrong and teaching it to players as if it were right?

One objection might be that if players aren’t having fun (regardless of whether that comes from our poor design, or their poor understanding of math), then the designer must design in favor of the player – especially if they are beholden to a developer and publisher that expect to see big sales. But is this necessarily valid? Poker, for example, is incredibly popular and profitable… even though it will mercilessly punish any player that dares harbor any flaws in how they think about probability.

To this day, I think it is still an open question, and something each individual designer must decide for themselves, based on their personal values and the specific games they are designing. Take a moment to think about these questions yourself:

• Are the probabilities in our games something that we can frame as a question of professional ethics?
• How do we balance the importance of giving the player enjoyment (especially when they are paying for it), versus giving them an accurate representation of reality?
• What’s more important: how the game actually works, or how players perceive it as working?

One More Solution

I can offer one other solution that works in some specific situations, and that’s to expose not just the stated probabilities to the player, but the actual results as well. For example, if you ask players to estimate their win percentage of a game when it’s not tracked by the game, they’ll probably estimate higher than the real value. If their wins, losses and percentage are displayed to them every time they go to start a game, they have a much more accurate view of their actual skill.

What if you have a random number generator? See if this sounds familiar to you: you’re playing a game of Tetris and you become convinced at some point that the game is out to get you. You never seem to get one of those long, straight pieces until just after you need it, right? My favorite version of Tetris to this day, the arcade version, had a neat solution to this: in a single-player game, your game took up the left half of the screen, and it used the right half to keep track of how many of each type of brick fell down. So if you felt like you were getting screwed, you could look over there and see if the game was really screwing you, or if it was just your imagination. And if you kept an eye on those over time, you’d see that yes, occasionally you might get a little more of one piece than another on average over the course of a single level, but over time it would balance out, and most of the time you’d get about as much of each brick as any other. The game was fair, and it could prove it with cold, hard facts displayed to the player in real time.

How could this work in other games? In a Poker video game against AI opponents, you could let the player know after each hand if they actually held the winning hand or not, and keep ongoing track of their percentage of winning hands, so that they know the deck shuffling is fair. (This might be more controversial with human opponents, as it gives you some knowledge of your opponents’ bluffing patterns.) If you’re making a version of the board game RISK that has simulated die rolls in it, have the game keep track of how frequently each number or combination is rolled, and let the player access those statistics at any time. And so on.

These kinds of things are surprisingly reassuring to a player who can never know for sure if the randomness inside the computer is fair or not.

When Randomness Isn’t Random

This brings us to the distinction of numbers that are random versus numbers that are pseudorandom. Pseudorandom literally means “fake random” and in this case it means it’s not actually entirely random, it just looks that way.

Now, most of the things we use even in physical games for randomness are not perfectly random. Balls with numbers painted on them in a hopper that’s used for Bingo might be slightly more or less likely to come up because the paint gives them slightly different weights. 6-sided dice with recessed dots may be very slightly weighted towards one side or another since they’ve actually got matter that’s missing from them, so their center of gravity is a little off. Also, a lot of 6-sided dice have curved edges, and if those curves are slightly different then it’ll be slightly more likely to keep rolling when it hits certain faces, and thus a little more or less likely to land on certain numbers. 20-sided dice can be slightly oblong rather than perfectly round (due to how they’re manufactured), making it slightly less likely to roll the numbers on the edges. All this is without considering dice that are deliberately loaded, or the person throwing the dice has practiced how to throw what they want!

What about cards? All kinds of studies have shown that the way we shuffle a deck of cards in a typical shuffle is not random, and in fact if you shuffle a certain way for a specific number of times (I forget the exact number but it’s not very large), the deck will return almost exactly to its original state; card magicians use this to make it look like they’re shuffling a deck when they’re actually stacking it. Even an honest shuffle isn’t perfectly random; if you think about it, for example, depending on how you shuffle either the top or bottom card will probably stay in the same position after a single riffle-shuffle, so you have to shuffle a certain number of times before the deck is sufficiently randomized. Even without stacking the deck deliberately, the point is that all shuffles are not equally likely.

In Las Vegas, they have to be very careful about these things, which is why you’ll notice that casino dice are a lot different from those white-colored black-dotted d6s you have at home. One slightly unfair die can cost the casino its gambling license (or cost them money from sufficiently skilled players who know how to throw dice unfairly), which is worth billions of dollars just from their honest, long-term House advantage, which is why they want to be very sure that their dice are as close to perfectly random as possible.

Shuffling cards is another thing casinos have to be careful of. If the cards are shuffled manually, you can run into a few problems with randomness (aside from complaints of carpal tunnel from your dealers). A dealer who doesn’t shuffle enough to sufficiently randomize the deck – because they’re trying to deal more hands so they get lazy with the shuffling – can be exploited by a player who knows there’s a better-than-average chance of some cards following others in sequence. And that’s not even counting dealers who collude with players to do this intentionally.

Casinos have largely moved to automated shufflers to deal with these problems, but those have problems of their own; for example, mechanical shuffles are potentially less random than human shuffles, so a careful gambler can use a hidden camera to analyze the machine and figure out which cards are more likely to clump together from a fresh deck, giving themselves an advantage. These days, some of the latest automated shufflers don’t riffle-shuffle, they actually stack the deck according to a randomized computer algorithm, but as we’ll see shortly, even those algorithms can have problems, and those problems can cost the casinos a lot of money if they’re not careful.

The point here is, even the events in physical games that we think of as “random,” aren’t always as random as we give them credit for. There’s not necessarily a lot we can do about this, mind you, at least not without going to great expense to get super-high-quality game components. Paying through the nose for machine-shopped dice is a bit much for most of us that just want a casual game of Catan, so as gamers we have to accept that our games aren’t always completely random and fair… but they’re close enough, they affect all players equally, so we disregard.

Psuedorandom Numbers

Computers have similar problems, because as I mentioned in the introduction, a computer doesn’t have any kind of randomness inside it. It’s all ones and zeros, high and low voltage going through wires or being stored electromagnetically in memory or on a disk somewhere, it’s completely deterministic. And unless you’re willing to get some kind of special hardware that measures some kind of physical phenomenon that varies like some kind of Geiger counter tracking the directions of released radiation or something, which most of us aren’t, your computer is pretty much stuck with this problem that you have to use a deterministic machine to play a non-deterministic game of chance.

We do this through a little bit of mathematics that I won’t cover here (you can do a Google search for pseudorandom number algorithms on your own if you care). All you need to know is that there are some math functions that behave very erratically, without an apparent pattern, and so you just take one of the results from that function and call it a random number.

How do you know which result to take from the function? Well, you determine that randomly. Just kidding; as we said, you can’t really do that. So instead what you do is you have to tell the computer which one to take, and then it’ll start with that one, and then next time you need a random number it’ll take the next one in sequence, then the next, and so on. But because we told it where to start, this is no longer actually random, even though it might look that way to a casual player. The number you tell the computer to start with is called a random number seed, and once you give the computer one seed, it just starts picking random numbers with its formula sequentially from there. So you only have to seed it once. But… this is important… if you give it the same seed you’ll get exactly the same results. Remember, it’s deterministic!

Usually we get around this by picking a random number seed that’s hard for a player to intentionally replicate, like the number of milliseconds that have elapsed since midnight or something. You have to choose carefully though. If, for example, you pick a random number seed that’s just the fractional milliseconds in the system clock (from 0 to 999), then really your game only has 1000 ways of “shuffling”, which is enough that over repeated play a player might see two games that seem suspiciously identical. If your game is meant to be played competitively, a sufficiently determined player could study your game and reach a point where they could predict which random numbers happen and when, and use that to gain an unfair advantage. So we have to be careful when creating these systems.

Pseudorandomness in Online Games: Keeping Clients in Sync

You have to be extra careful with random numbers in an online game, if your players’ machines are generating their own random numbers. I’ve worked on games at two companies that were architected that way, for better or worse. What could happen was this: you would of course have one player (or the server) generate a random number seed, and that seed would be used for both players in a head-to-head game. Then, when either player needed a random number, both machines would have to generate that number so that their random number seed was kept in sync. Occasionally due to a bug, one player might generate a random number and forget to inform their opponent, and now their random number generators are out of sync. The game continues for a few turns until suddenly, one player takes an action that requires a random element, their machine rolls a success, the opponent’s machine (with a different random number) rolls a failure, the two clients compare checksums and fail because they now have a different game state, and both players become convinced the other guy is trying to cheat. Oops. (A better way to do this for PC games is to put all the game logic on the server and use thin clients; for networked handheld console or phone games where there is no server and it’s direct-connect, designate one player’s device to handle all these things and broadcast the game state to the other players.)

Pseudorandomness in Single-Player Games: Saving and Loading

You also have to be careful with pseudorandom numbers in a single-player game, because of the potential for exploits. This is a largely unsolved problem in game design. You can’t really win here, but you can at least pick your poison.

Save Anywhere

Suppose you have a game where the player can save anywhere, any time. Many games do this, because it is convenient for the player. However, nothing stops the player from saving just before they have to make a big random roll, maybe something where they’re highly unlikely to succeed but there’s a big payoff if they do, and keep reloading from save until they succeed. If you re-generate your random number seed each time they reload from save, they will eventually succeed, and they’re not really playing the game that you designed at that point… but on the other hand, they’re using the systems you designed, so they’re not really cheating either. Your carefully balanced probabilities suddenly become unbalanced when a player can keep rerolling until they win.

Save Anywhere, Saved Seed

Okay, so you say, let’s fix that: what if we save the random number seed in the saved game file? Then, if you try to save and reload, you’ll get the same result every time! First, that doesn’t eliminate the problem, it just makes it a little harder; the player just has to find one other random thing to do, like maybe drinking a potion that restores a random number of HP, or maybe choosing their combat actions in a different order or something, and keep trying until they find a combination of actions that works. Second, you’ve now created a new problem: after the player saves, they know exactly what the enemy AI will do on every turn, because once you start with the same random number seed the game now becomes fully deterministic! Sometimes this foreknowledge of exactly how an enemy will act in advance is even more powerful than being able to indefinitely reroll.

Save Points

So you say, okay, let’s limit where the player can save, so that they’ll have to go through some nontrivial amount of gameplay between saves. Now they can theoretically exploit the save system, but in reality they have to redo too much to be able to fully optimize every last action. And then we run into a problem with the opposite type of player: while this mechanism quells the cheating, the honest players now complain that your save system won’t let them walk away when they want, that the game holds them hostage between save points.

Quicksave

Maybe you think to try a save system where the player can save any time, but it erases their save when they start, so they can’t do the old save/reload trick. This seems to work… until the power goes out just as an honest player reaches the final boss, and now they have to start the entire game over from scratch. And they hire a hit man to kill you in your sleep because you deserve it, you evil, evil designer.

Save Anywhere, Limited Times

You give the player the ability to save anywhere, but limit the total number of saves. The original Tomb Raider did this, for example. This allows some exploits, but at least not on every last die-roll. Is this a good compromise?

Oh, by the way, I hope you gave the player a map and told them exactly how far apart they can save on average, and drew some BIG ARROWS on the map pointing to places where the big battles are going to happen, so a player doesn’t have to replay large sections of the map just because they didn’t know ahead of time where the best locations were to save. And then your players will complain that the game is too easy because it gives them all this information about where the challenge is.

Pick Your Poison

As I said, finding the perfect save system is one of those general unsolved problems in game design, just from the perspective of what kind of system is the most fun and enjoyable from the player, and that’s just for deterministic games! When you add pseudorandom numbers, you can see how the problem can get much thornier, so this is something you should be thinking about as a designer while designing the load/save system… because if you don’t, then it’ll be left to some programmer to figure out, God help you, and it’ll probably be based on whatever’s easiest to code and not what’s best for the game or the player.

When Pseudorandom Numbers Fail

Even if you choose a good random number seed, and even if you ignore player exploits and bugs, there are other ways that randomness can go wrong if you choose poor algorithms. For example, suppose you have a deck of cards and want to shuffle them. Here’s a naïve algorithm that most budding game programmers have envisioned at some point:

1. Start with an unshuffled deck.
2. Generate a pseudorandom number that corresponds to a card in the deck (so if the deck is 52 cards, generate a whole number between 1 and 52… or 0 and 51, depending on what language you’re using). Call this number A.
3. Generate a second pseudorandom number, the same way. Call this number B.
4. Swap the cards in positions A and B in the deck.
5. Repeat steps 2-4 lots and lots of times.

The problem here is, first off, it takes an obnoxiously long time to get anything resembling a random shuffle. Second, because the card positions start out fixed, and you’re swapping random pairs one at a time, no matter how many times you repeat this there is always a slightly greater chance that you’ll find each card in its original position in the deck, than anywhere else. Think of it this way: if a card is ever swapped, it’ll swap to a random position, so all positions are equally likely for any card that has been swapped at all. Swapping multiple times makes no difference; you’re going from one random place to another, so you still end up equally likely to be in any position. So any card that’s swapped is equally likely to end up anywhere, with equal frequency, as a baseline (this is good). However, there is some non-zero chance that a card will not be swapped, in which case it will remain in its original position, so it is that much more likely to stay where it is. The more times you perform swaps, the lower the chance that a card will remain in its original position, but no matter how much you swap you can never get that probability all the way down to zero. Ironically, this means that the most likely shuffle from this algorithm is to see all cards in exactly the same position they started!

So you can see that even if the pseudorandom numbers generated for this shuffle are perfectly random (or close enough), the shuffle itself isn’t.

Are Your Pseudorandom Numbers Pseudorandom Enough?

There is also, of course, the question of whether your pseudorandom number generator function itself actually produces numbers that are pretty random, or if there are actually some numbers that are more or less likely than others, either due to rounding error or just a poor algorithm. A simple test for this is to use your generator to generate a few thousand pairs of random coordinates on a 2d graph, and plot that graph to see if there’s any noticeable patterns (like the numbers showing up in a lattice pattern, or with noticeable clusters of results or empty spaces). You can expect to see some clustering, of course, because as we learned that’s how random numbers work. But if you repeat the experiment a few times you should see clusters in different areas. This is a way of using Monte Carlo simulation to do a quick visual test for if your pseudorandom numbers are actually random. (There are other more mathematical ways to calculate the exact level of randomness from your generator, but that requires actual math; this is an easier, quick-and-dirty “good enough” test for game design purposes.)

Most of the time this won’t be an issue. Most programming languages and game libraries come with their own built-in pseudorandom number generation functions, and those all use established algorithms that are known to work, and that’s what most programmers use. But if your programmer, for some reason, feels the need to implement a custom pseudorandom number generation function, this is something you will want to test carefully!

Homework

In past weeks, I’ve given you something you can do right now to improve the balance of a game you’re working on, and also a task you can do later for practice. This week I’m going to reverse the order. Get some practice first, then apply it to your project once you’re comfortable with the idea.

For this week’s “homework” I’m going to go over two algorithms for shuffling cards. I have seen some variant of both of these used in actual working code in shipped games before (I won’t say which games, to protect the innocent). In both cases, we saw the perennial complaints from the player base that the deck shuffler was broken, and that there were certain “hotspots” where if you placed a card in that position in your deck, it was more likely to get shuffled to the top and show up in your opening hand. What I want you to do is think about both of these algorithms logically, and then figure out if they work. I’ll give you a hint: one works and one doesn’t. I’ll actually give you the source code, but I’ll also explain both for you in case you don’t know how to program. Keep in mind that this might look like a programming problem, but really it’s a probability calculation: how many ways can you count the different ways a shuffling algorithm shuffles, and do those ways line up evenly with the different permutations of cards in a deck?

Algorithm #1

The first algorithm looks like this:

1. Start with an unshuffled deck.
2. Choose a random card from all available cards (so if the deck has 60 cards, choose a pseudorandom number between 1 and 60). Take the card in that position, and swap with card #60. Essentially, this means choose one card randomly to put on the bottom of the “shuffled” deck, then lock it there in place.
3. Now, take a random card from the remaining cards (between 1 and 59), and swap that card with position #59, putting it on top of the previous one.
4. Then take another random card from the remaining ones (between 1 and 58), swap that with position #58, and so on.
5. Keep repeating this until eventually you get down to position #1, which swaps with itself (so it does nothing), and then we’re done.

This is clearly different from how humans normally shuffle a deck, but remember, the purpose here isn’t to emulate a human shuffle; it’s to get a random shuffle, that is, a random ordering of cards.

Algorithm #2

The second algirhtm is similar, but with two minor changes.

1. Start with an unshuffled deck.
2. Choose a random card from all available cards (in a 60 card deck, that means a card from 1 to 60). Swap with position #60, putting it on the bottom of the deck.
3. Choose a random card from all available cards, including those that have been chosen already (so, choose another random card from 1 to 60). Swap with position #59.
4. Choose another random card from 1 to 60, and swap with position #58.
5. Keep repeating this until eventually you choose a random number from 1 to 60, swap that card with position #1, and you’re done.
6. Oh yeah – one last thing. Repeat this entire process (steps 2-5) fifty times. That’ll make it more random.

Hints

How do you approach this, when there are too many different shuffles in a 60-card deck to count? The answer is that you start much simpler. Assume a deck with only three cards in it (assume these cards are all different; call them A, B and C if you want).

First, figure out how many ways there are to order a 3-card deck. There are mathematical tricks for doing this that we haven’t discussed, but you should be able to do this just by trial and error.

Next, look at both algorithms and figure out how many different ways there are for each algorithm to produce a shuffle. So in the first case, you’re choosing from 3 cards, then choosing from 2, then choosing from 1. In the second case you’re choosing from 3, then choosing from 3, then choosing from 3. Compare the list of actual possible orderings of the deck, that is, all the different unique ways the deck can be shuffled… then compare to all the different ways a deck can be shuffled by these algorithms. You’ll find that one of them produces a random shuffle (well, as random as your pseudorandom number generator is, anyway) and one actually favors certain shuffles over others. And if it works that way for a 3-card deck, assume it’s similarly random (or not) for larger decks. If you want to go through the math with larger decks, be my guest, but you shouldn’t have to.

If You’re Working on a Game Now…

Video Games

Once you’ve done that, if you’re working on a game that involves computers, take a look at the pseudorandom numbers and how your program uses them. In particular, if you use a series of pseudorandom numbers to do something like deck shuffling, make sure you’re doing it in a way that’s actually random; and if you’re using a nonstandard way to generate pseudorandom numbers, check that by graphing a bunch of pairs of random coordinates to check for undesirable patterns. Lastly, examine how your random number seed is stored in the game state; if it’s a multiplayer game, is it stored separately in different clients or on a single server? If it’s a single-player game, does your save-game system work in such a way that a player can save, attempt a high-risk random roll, and keep reloading from save until it succeeds?

All Games, Digital or Non-Digital

Another thing to do, whether you’re designing a board game or video game, is to take a look at the random mechanics in your game (if there are any) and ask yourself some questions:

• Is the game dominated more by skill or luck, or is it an even mix?
• Is the level of skill and luck in the game appropriate to the game’s target audience, or should the game lean a bit more to one side or the other?
• What kinds of probability fallacies are your players likely to observe when they play? Can you design your game differently to change the player perception of how fair and how random your game is? Should you?