GDC 2010: Sid Meier Keynote - "Everything You Know is Wrong"

Sid Meier (Civilization) gave a great GDC keynote in 2010 where he describes his multi-decade struggle to convey to users ideas like "Stuff that only has a 10% chance of happening still happens sometimes." [VIDEO] Multiply that by a million daily active users and it really works out to "Stuff that has a 1-in-a-million chance of happening will happen to somebody every day". Leading to player reviews like "The game rolled me a 1-out-of-10 SIX TIMES IN A ROW!!! The game is explicitly cheating me!!!" every day. Thus games tend to use random shuffles rather than independent random numbers so as to better align with common misunderstandings about statistics...

This article, and some of the comments here, reminded me of another article (and GDC talk) by Civilization game designer Sid Meier, about his experience with players' perception of probability in games. Sid's talk grapples with the issue "If the game says you have 3-to-1 odds to win a battle, how often do players actually expect to win?" > When designing the combat system in Civilization: Revolution, Sid Meier found himself up against some interesting design problems. His players didn't understand math. In Civ Rev, the strength of units were displayed up front to players before battle to show the odds of victory. For example, an attacking unit might be rated at 1.5 with the defending unit at 0.5. This is a 3-to-1 situation. > Unfortunately, the testers expected to win this battle every time despite there being a 25% chance of losing each time. Sid tweaked the math to make the player win more in this situation. Next, the reverse case was tested. The player had 1-to-3 odds. If they won, the math was functioning properly. They had a slim chance to win and they did. > Sid identified a few cases of interest. When the player was presented with 3-to-1 or 4-to-1 odds, they expect to win. With 2-to-1 odds, the player will accept losing some of the time, but expect to win at 20-to-10, which is just a larger expression of 2-to-1. When the numbers get larger, the perceived advantage grows. [LINK] Here is a link to the actual GDC talk by Sid with the content about probability: [VIDEO] (It

I believe you're thinking of Sid Meier's GDC 2010 keynote "Everything You Know Is Wrong" [1]. The entire talk is interesting, but section on player perception of probability is about 20 minutes in. One takeaway is to be careful about how strength numbers translate into odds. If your strength is 100 and mine is 1, does that mean I have a 1% chance to out-and-out beat you? Your armored tank shouldn't have an even 1% chance of being completely annihilated by my club-wielding warrior (that's somehow still around by then). The later Civ games have taken odds out of the equation, and I think it's for the better. Instead, the amount of damage each unit takes per combat depends on the difference in their strength deterministically. From my own perspective, this is overall more fun than 'randomly' having really strong units lose against weak units occasionally. [1] [VIDEO]

"Civilization Revolution" does this. Say the player loses a 75% roll, then attempts another identical 75% roll. The game will actually give you better odds for the second roll. I think it's silly -- the game basically lies to you about your odds on the second attempt. Sid Meier talked about it at length in this video: [VIDEO] . It starts around 18:25.

The worst part is that even if the algorithm perfectly computed probabilities and was completely open, humans would take away incorrect information. I am reminded of this talk by Sid Meier: [VIDEO] People don't understand probabilities. They don't accept that if someone has a 75% chance of being guilty, they may very well not be. It happened again with FiveThirtyEight when everybody accused their "70% chance of Mrs Clinton winning" of being wrong, when it doesn't preclude Mr Trump winning.