Download Game Playing Characterization

Battle of Botcraft:
Fighting Bots in Online Games
with Human Observational Proofs
Steven Gianvecchio, Zhenyu Wu,
Mengjun Xie, and Haining Wang
Outline






Background
Game Playing Characterization
HOP System
Experiments
Limitations
Conclusion
Outline






Background
Game Playing Characterization
HOP System
Experiments
Limitations
Conclusion
Background
Online Games
 In 2008, online game revenues $7.6B
 about half from massively multiplayer online
games (MMOGs) ex. World of Warcraft (WoW)
 MMOG currency trades for real currency
 players can make real money
 A major problem is cheating
Background
Game Bots
 A common cheat is use of game bots
 able to amass game currency
 cause hyper-inflation
 To combat bots
 process monitors, ex. Warden for WoW
 human interactive proofs (HIPs)
 legal action
Background
Game Bots
 Glider – a popular WoW bot
 controls game via mouse / keyboard APIs
 uses profiles, i.e., configurations and
waypoints
 able to evade Warden
 Blizzard sued MDY (maker of Glider)
 awarded $6.5M
Outline






Background
Game Playing Characterization
HOP System
Experiments
Limitations
Conclusion
Game Playing Characterization
Input Data Collection
 World of Warcraft game
 RUI program (with modifications)
 records user-input events
 converts events to user-input actions
ex. move + move + press + release =
point-and-click
 computes user-input action statistics
Game Playing Characterization
Game Bot
 10 Glider profiles (configurations and
waypoints)
 40 hours
 half with warrior and half with mage
 levels 1 to mid-30s
Game Playing Characterization
Human
 30 humans
 55 hours
>45
100%
90%
home
35-44
women
20s-30s
>10
magic
80%
70%
60%
25-34
5-10
50%
40%
lab
men
10<
2-5
physical
30%
1<
20%
18-24
none
10%
0%
Sex
Age
Gaming
Exp.
Location
Class
Level
 Human
 well fit by Pareto distribution
 Game Bot
 more fast keystrokes
 signs of periodic timing
Keystroke Inter-arrival Time Distribution
 Human
 fewer very short keystrokes
 3.9% shorter than .12 secs
 Game Bot
 36.9% shorter than .12 secs
 more signs of periodic timing
Keystroke Duration Distribution
 Human
 highly-variable speed at all displacements
 Game Bot
 linear speed increases
 high-speed moves with zero displacment
Point-and-Click Speed vs. Displacement
 Human
 decays exponentially
 only 14.1% of movements have 1.0 efficiency
 Game Bot
 81.7% of movements have 1.0 efficiency
Point-and-Click / Drag-and-Drop Movement Efficiency
 Game Bot
 no correlation between speed and direction
Average Velocity for Point-and-Click
 Human
 diagonal, symmetric, and bounded
 diagonals faster than horizontal / vertical
Average Velocity for Point-and-Click
Outline






Background
Game Playing Characterization
HOP System
Experiments
Limitations
Conclusion
HOP System
 A behavioral approach
 human observational proofs (HOPs)
 The idea: certain tasks are difficult for a
bots to perform like a human
 passively observe differences
 HOP-based game bot defense system
 continuous monitoring
 transparent to users
HOP System
 Client-Side Exporter
 transmits user-input actions
 Server-Side Analyzer
 processes and decides: bot or human
HOP System
Neural Network
 Inputs
1. duration
2. distance
3. displacement
4. move efficiency
5. speed
6. angle
# of inputs = # of actions * 7
7. virtual key
HOP System
Neural Network
 Output –
human or bot
Decision Maker
 “Votes” on series of outputs
ex. {bot + bot + human} = bot
Outline






Background
Game Playing Characterization
HOP System
Experiments
Limitations
Conclusion
Experiments
Experimental Setup
 30 human players, 55 hours
 10 Glider profiles, 40 hours
 10-fold cross validation
 test on a bot or human not in training set
 10 different training sets
Experiments
HOP System
1. # of actions (input to neural network)
2. # of nodes (in neural network)
3. threshold x (on neural network output)
> x is bot, <= x is human
4. # of outputs per decision
ex. {bot + bot + human} = bot
Experiments
Configure 1. # of actions and 2. # of nodes
 4 actions with 40 nodes
TPR and TNR vs. # of Nodes and # of Accumulated Actions
Experiments
Configure 3. threshold and 4. # of outputs
 threshold 0.75 with 9 outputs per decision
TPR and TNR vs. Threshold and # of Accumulated Outputs
Experiments
Detection Results
 Configured System
 4 actions, 40 nodes, threshold 0.75, 9 outputs
 Glider – avg. true positive rate of 0.998
 Humans – true negative rate of 1.000
True Positive Rates for Bots
Experiments
Decision Time
 # of action * time per action
 avg. 39.60 seconds
Decision Time Distribution
Experiments
Detection of Other Game Bots
 MMBot in Diablo 2
 different bot, different game
 without retraining the neural network
 MMBot – true positive rate of 0.864
 Humans – true negative rate of 1.000
Outline






Background
Game Playing Characterization
HOP System
Experiments
Limitations
Conclusion
Limitations
Experimental Limitations
 Size
 30 not enough
 Lab vs. Home
 mostly in-lab
 Character equipment / levels
 Other bots and games
Limitations (cont.)
Potential Evasion
 Interfere with client-side exporter
 block user-input stream
 manipulate user-input stream
 Mimic human behavior
 replay attacks
 model human user-input
Conclusion
 Game Play Characterization
 95 hours of user-input traces
 bots behave differently than humans
 HOP System
 exploits behavioral differences
 compared to HIPs, HOPs are transparent and
continuous
 detects 99% of bots with no false positives
 raises the bar for attacks
Questions?
Thank You!
Questions?
Thank You!
Questions?
Thank You!
Experiments
System Overhead
 Memory
 per user = 4 actions * 16 bytes + 16 outputs *
1 bit = 66B
 server with 5,000 users = 330KB
 CPU – P4 Xeon 3.0Ghz
 95 hours of traces in 385ms = ~296 hours/sec
 server with 5,000 users = ~1.4 hours/sec