E-Sim Optimal Equipment Selection

• Project name: E-Sim Optimal Equipment Selection
• Class: 4IT495 Simulation of Systems (WS 2014/2015)
• Author: Andriy Zhubryd
• Model type: Monte-Carlo Simulation
• Software used: Microsoft Excel

Introduction

E-Sim is free strategic world simulation. It uses the Earth, real-life countries with their current borders as the strategic map. Player can become a citizen of any country available and take an active part in its development. Currently, 58 countries are available in 2 servers ‘primera’ and ‘secura’. New countries are introduced to the game as soon as the significant number of its real-life citizens start playing the game. Czech Republic is represented in the ‘primera’ server only and, unfortunately, is completely conquered by Poland. Slovakia is more lucky – it’s represented only on ‘secura’ server and is currently successfully occupying Austria and Germany. Also, there’s special server ‘suna’ where only Africa is represented and all players are citizens of African countries. Few days ago new server ‘oriental’ was open – Far East countries only. Each country uses their native currency, but also unified currency ‘gold’ is used in the game.

E-Sim has four main modules every player can enjoy:

• Economic – players can work as an employees, manage companies, become CEO’s of stock companies, trade, import and export goods, exchange currency. Economic module is thoroughly constructed: demand and supply are all natural, admin do not directly interfere with production. Sometimes to balance economy new items of higher quality (therefore more expensive) are introduced, but direct use of so-called ‘buy-bot’ is not used (‘buy-bot’ was widely used in another game with similar concept – eRepublik – surplus was bought and destroyed by admin). Every country have native region with some bonus to the productivity in specific branch;
• Political – game has 3 kinds of elections: party leader elections, congress elections and presidential elections, where citizens decide who will rule their country. Congress can create new laws: change taxes, declare war or peace treaty, grant citizenship to foreigners, print national currency from gold. On the higher (international) level country representatives communicate with other countries, form alliances to defend themselves;
• Military – every player is the fighter, who can defend his country, his friends or conquer enemies. Fighters unite into military unit to increase discipline, concentrate their fire and become more powerful as the nation. War is necessary in this game and during 866 days of ‘secura’ server 16110 battles took place (on average 18.6 battles per day; battle consists of 2-hour rounds and it can last from 16 to 30 hours);
• Media – journalism, newspapers etc. E-Sim has well-developed module for publishing newspapers, which helps another 3 modules. Journalists can gain money if people get interested with their articles and subscribe to their newspapers.

We will focus on the military module and simulate different sets of equipment to maximize damage player can do in one day.

Problem definition

Let’s look closer at the military module of E-Sim. Battle for any region consists of 2-hour rounds. To win the battle one side should win 8 rounds. During the round both sides deal some damage and side with more damage in the end of 2-hour interval wins the round. Here comes the first rule of battle: damage, dealt in the very last minutes of the round is more valuable that in the beginning of the round. On average player can fire all their damage in 1.5-2 minutes, so the last 2 minutes of the round are the most intense. Important detail, which later resulted in the drastic changes of military module, is that the best fighter (with the most damage dealt) of the round (on each of the sides) is winning ‘Battle Hero’ medal and gets paid 5 gold (this sum will later be compared with costs of weapons and to explain strategy of players).

Damage dealt by the individual depends on 2 characteristics: strength and rank. Strength is gain every day during the training so it solely depends on how many days player was actively playing this game. It doesn’t depend on how much money player can afford to spend. Rank is being gained through fighting: it depends only on the cumulated damage player have done during his whole ‘life’. So this characteristic depends how actively player fights and how much money he spends on the weapons. Rank is quantified as some coefficient and multiplied by strength – this forms the basic hit value. Real battle damage will be modified with different bonuses: weapon bonus, military unit order bonus, location bonus. Experienced fighters always try to use all bonuses: they flight to specific region to gain the location bonus, they set military unit order on that battle. The most significant bonus is weapon bonus where most of the money are spent. Game has weapons of 5 quality levels (q1,q2,…,q5 notation; q5 – the best). If you fight without weapons you get -50% of your basic hit, with q1 weapons it’s +20% and with q5 weapons bonus is +100%. Full hit with q1 weapons costs around 7-8 gold, with q5 – 25 gold, average pay from work is 2 gold, so you can imagine how much you should work to fight with q5 – own stocks, companies, trade.

To sum up, players could easily and precisely calculate their damage in the battle. This was comfortable, but some players exploited this in the bad way to get a lot of money so admin was forced to change hit calculation. I would like to explain you how precise calculation of damage was exploited. As I mentioned earlier, in every round ‘Battle Hero’ medal with 5 gold prize is granted for players with the highest damage on each side. If few players (for example 10) get the same highest amount of damage on one side – 10 medals will be granted and everyone of 10 will get 5 gold each. Generally, anyone can take part in battle so competition is too high to make deal with everyone and take ‘group Battle Hero’. But there is very special kind of battle – Civil War – where only citizens of the country involved can fight. This was exploited by one small country (Malaysia, which had the population of about 60 people) – in the chat they were coordinating their damage and a lot of people (up to 10 in one round) got the ‘Battle Hero’ medal and 5 gold. It became a real gold mine for the small group of people. Civil Wars in Malaysia became very frequent and there was no way to stop it.

To destroy this opportunity admin introduced a lot of changes to the military module and made it random: players couldn’t precisely calculate their damage. For example, basic hit of 1000 damage became an uniformly distributed value between 800 and 1200 damage – on the average it was the same, but you couldn’t predict you exact damage.

With randomization of basic hit value, admin also introduced new parameters usually available in MMORPG games: change of miss (zero damage), critical strike (double damage) and chance to avoid damage (deal damage, but does not spend hit). These parameters make hit value even more volatile and unpredictable. To influence these parameters admin created equipment pieces. 5 different slots were available for equipment: body armor, helmet, googles, weapon upgrade and offhand (later this number was increased to 8, but with another properties – economical – so we don’t consider them in damage simulations). There are 5 levels of quality (later level q6 was created, but it’s too expensive and rare to consider in simulation) of equipment, which determine interval in which values of bonuses are determined. Each piece has 2 bonuses, which can be of different kind. There are 7 kinds of military bonuses from equipment. 2 of them are proved to be not very efficient for developed players, so they were removed from the simulation. The rest 5 bonuses and their intervals for q5 items (q5 is the optimal equipment set for developed players) are:

• Increased damage (4-6%)
• Increased maximum damage (8-12%)
• Increased critical chance (4-6%)
• Reduced miss chance (6-7.5%)
• Chance to avoid damage (4-6%)

Goal of the research is to find the best equipment set that maximizes the expected damage.

Method

To test what equipment set should perform better Monte-Carlo simulation will be used in MS Excel. Through numerous repetitions of fights in any equipment set possible average result can give an image how player should select equipment set.

Also, expected damage of every equipment set will be calculated and compared with Monte-Carlo simulation result. The last method of this research will be marginal utility approach: how single equipment property on average influences total outcome.

Model

Every player has the same starting probabilities of miss, critical hit and chance to avoid damage. They are 12.5% for miss and you can decrease it to 0; 12.5% for critical hit and can be increased till 40%; 5% chance to avoid damage and can be increased till 40% with equipment. These restrictions and starting values mean that there’s no need of acquiring 3 parameters which decrease miss chance because 2 properties of reduction of miss chance will cover from 12% to 15%, which in most cases will be higher than starting 12.5%. Restriction with critical hit chance suppose that on average there’s no need of getting more than 6 properties for increasing critical hit: parameter will be between 36.5% and 48.5%, which is in most cases higher than 40%, so it will be rounded down to 40%. For change of avoid parameter's logical maximum is 7 properties. There is no upper limit for increasing damage and maximum damage.

We are going to simulate full one-day fight of player, equipped with 5 q5 pieces of equipment. 5 pieces of equipment provide 10 properties and restriction for maximum number of some properties were mentioned before (those restrictions do not forbid using higher number of properties, but makes their bonus inefficient). We will use Monte-Carlo simulation in Microsoft Excel with 200 simulations for every possible equipment set (622 combinations). Each one-day full hit simulation requires generation of 185 uniformly distributed random variables: 5 variable for equipment set and 3 variables for every of 60 hits during the day. This means that all simulations require 622x185x200 ~ 23 mil generation of random variables. Simulation took around 50 seconds in Microsoft Excel on the laptop with Intel core i5 laptop.

Minimum number of hits is 150, but chance to avoid damage can is influencing this number. If we acquire maximum probability of avoiding damage (40%) average expected number for hits will be around 250. Game provides players with choice to either hit with 1 hit or do ‘berserk’ of 5 hit at the time. As higher damage in short period of time in the end of the round is required, experienced players hit berserk and, therefore, 150 hits transform into 30 berserks. We give the upper limit of 60 berserks per day to take into account the influence of chance to avoid damage. Important note is that events of miss, critical hit and avoiding damage are independent, which means that avoid and miss can happen simultaneously (player will just waste the weapons).

Firstly, we should generate all possible equipment sets. Total number of properties should be 10, restrictions for miss, avoid and critical were mentioned before. In my experience, Microsoft Access can be used for quick and easy generation of such table: Cartesian product of the same table 5 times with some additional condition can be formed in one SQL query. For our conditions, 622 different combinations are available. Algorithm of simulation is next:

• Select equipment set;
• Generate values of equipment properties;
• Calculate player’s parameters;
• Simulate one-day full fight.

This algorithm is repeated for every possible equipment set 200 times – simulation of 124 400 one-day full fights.

---

Another way to find the best equipment set is the use of mathematical expected values: means of distributions. Expected damage is calculated as Basic damage, modified with bonuses coefficients: Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Expected Damage = Basic Damage \times Miss \times Avoid \times Critical \times Increase Damage \times Increase Max Damage}

With equipment sets we can influence all variable except Basic Damage. We should find formulas for calculation of the average value of those variables to optimize the Expected Damage:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Miss = 1-p_{miss}}

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Critical = 1+p_{crit}}

Avoid can be described as the geometric distribution so its average will be the following:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Avoid = {1 \over {1 - p_{avoid}}}

Calculations of multiplier for Increase Damage and Increase Max Damage show why second property is always more efficient:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Increase Damage = 1 + bonus_{IncDam}}

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Increase Max Damage = {{0.8 + 1.2 \times (1 + bonus_{IncMaxDam})} \over 2} = 1 + 0.6 \times bonus_{IncMaxDam}}

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle bonus_{IncMaxDam} = 2 \times bonus_{IncDam} \Rightarrow }

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Increase Max Damage = 1 + 0.6 \times 2 \times bonus_{IncDam} = 1 + 1.2 \times bonus_{IncDam} > 1 + bonus_{IncDam} = Increase Damage}

---

The last on the list of our analysis is marginal utility approach. This approach will consider how adding one more property to equipment set will change damage comparing to the value before adding property. Let’s illustrate it on the chance of avoid example. Starting value of avoid is 5% and every additional property adds on average 5%. So with 0 properties we have 5%, with one it increases to 10%, with 2 to 15% etc. Let’s calculate multipliers and marginal utility:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Avoid_0={1 \over {1 - 0.05}} = 1.052 }

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Avoid_1={1 \over {1 - 0.1}} = 1.111 }

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Avoid_2={1 \over {1 - 0.15}} = 1.176 }

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle MarUtil_1={{Avoid_1 - Avoid_0} \over Avoid_0} = 5.60 \% }

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle MarUtil_2={{Avoid_2 - Avoid_1} \over Avoid_1} = 5.85 \% }

Using this table of marginal utility, we can choose the most efficient equipment set to maximize expected damage.

Results

After finishing all the calculations, we sorted possible equipment sets by average damage due to simulation. In the table, we showed 15 best sets we got from simulations:

Top 15 equipment sets according to simulation

Rate of set	Increase damage	Increase max damage	Decrease miss	Increase avoid	Increase critical	Simulated damage	Decrease %
1	0	2	1	7	0	292517	0,00%
2	0	1	2	7	0	291523	0,34%
3	0	2	2	6	0	291142	0,47%
4	1	0	2	7	0	290177	0,80%
5	1	1	2	6	0	289716	0,96%
6	1	1	1	6	1	287812	1,61%
7	0	2	1	6	1	286385	2,10%
8	1	1	1	7	0	286329	2,12%
9	0	1	2	6	1	286327	2,12%
10	0	0	2	7	1	286213	2,15%
11	0	1	1	7	1	286121	2,19%
12	0	3	1	6	0	285608	2,36%
13	1	2	1	6	0	285353	2,45%
14	0	3	2	5	0	284587	2,71%
15	2	0	1	7	0	284579	2,71%

From the simulation’s results we can make next conclusions:

• The most valuable properties are decrease of miss chance and increase of chance to avoid damage. Number of avoid property appearance is changing between 6 and 7, decrease of the miss chance – between 1 and 2.
• Increase of damage and critical hit chance properties aren’t as significant as others – their appearance is changing between 0 and 1.
• Increase of maximum damage may have some significant influence on the damage. This is a very important note because it has an empirical and economical explanation underneath, which will be shown later.

Now let's look at the 15 worst sets of equipment:

Worst 15 equipment sets according to simulation

Rate of set	Increase damage	Increase max damage	Decrease miss	Increase avoid	Increase critical	Simulated damage	Decrease %
608	4	1	0	0	5	237752	18,72%
609	5	0	0	0	5	237194	18,91%
610	8	0	0	0	2	237170	18,92%
611	0	2	0	2	6	237123	18,94%
612	2	2	0	0	6	236693	19,08%
613	2	0	0	2	6	236559	19,13%
614	9	0	0	1	0	235717	19,42%
615	3	0	0	1	6	235393	19,53%
616	9	1	0	0	0	235271	19,57%
617	9	0	0	0	1	234817	19,73%
618	6	0	0	0	4	234621	19,79%
619	8	2	0	0	0	234021	20,00%
620	4	0	0	0	6	233105	20,31%
621	10	0	0	0	0	233059	20,33%
622	3	1	0	0	6	230839	21,09%

Here, on the contrary, we see higher appearance of increase damage and critical hit properties and lower values of others. Decrease miss chance property has only zero values, which proves that it has high influence on the damage.

Overall, results are interesting. They clearly show which properties you should pay attention to and which should be left unnoticeable. And it proved main rule of every player – decrease miss chance. Though it varies between 1 and 2 in the simulation, every player has an equipment set where miss chance is fully covered. About other properties there wasn’t exact leader, but by simple calculations it’s obvious that increase of maximum damage is better that just increase of damage. Thorough calculations and analysis of expected influence will be shown in the next part.

---

From the expected damage approach we gain the next best equipment sets:

Top 15 equipment sets according to optimization

Rate of set	Increase damage	Increase max damage	Decrease miss	Increase avoid	Increase critical	Simulated damage	Decrease %
1	0	1	2	7	0	298125	0,00%
2	0	2	1	7	0	296888	0,42%
3	1	0	2	7	0	295313	0,94%
4	1	1	1	7	0	294237	1,30%
5	0	0	2	7	1	293750	1,47%
6	0	1	1	7	1	293471	1,56%
7	2	0	1	7	0	291586	2,19%
8	0	2	2	6	0	290769	2,47%
9	1	0	1	7	1	290702	2,49%
10	0	3	0	7	0	290391	2,59%
11	0	3	1	6	0	288731	3,15%
12	0	0	1	7	2	288641	3,18%
13	1	1	2	6	0	288173	3,34%
14	1	2	0	7	0	287930	3,42%
15	0	2	0	7	1	287875	3,44%

As it can be seen, best combinations are practically the same as from simulation results.

---

Using this table of marginal utility, we can choose the most efficient equipment set to maximize expected damage.

Marginal utility of properties

Property \ Appearance	1	2	3	4	5	6	7
Increase damage	5,00%	4,80%	4,50%	4,30%	4,20%	4,00%	3,80%
Increase max damage	6,00%	5,70%	5,40%	5,10%	4,80%	4,60%	4,40%
Miss	7,70%	6,10%	0,00%	0,00%	0,00%	0,00%	0,00%
Critical	4,40%	4,30%	4,10%	3,90%	3,80%	3,60%	3,50%
Avoid	5,60%	5,90%	6,30%	6,70%	7,10%	7,70%	8,30%

Only one property has an increasing trend – chance to avoid damage property. It has not the highest increase from the start, but increasing trend makes it extremely efficient comparing to other properties. Also, utilities of reduce miss chance property are very high, so they should be definitely included in the optimal set of equipment.

According to the table, the best set of equipment should consist of 7 appearances of avoid property, 2 appearances of reduce miss chance property and the last property should be the increase of maximum damage because avoid and miss properties are used fully and increase of maximum damage has the maximum utility of the rest. This combination is the best according to the optimization by expected damage approach and second best (with only 0.34% difference) according to the simulation results. Funny thing is that the best combination according to the simulation is the second best in the mathematical optimization.

Conclusion

In any simulation, any model or optimization we always omit some aspect, some part of the reality. Here we try to maximize damage, but we neglect the fact that one of the properties increases costs of one-day full fight.

Chance to avoid damage allows player to hit again, but for bonus hits it requires weapons as well. Unfortunately, this ‘expensive’ property turned out to be the most efficient. If we use that strategy, our expenses on weapon will increase by 67%: q1 fight will cost 12-13 gold and q5 – around 40 gold. I showed average salary and prize from ‘Battle Hero’ medal, so you can understand that this is very high price and players tend not to choose avoid property. Moreover, avoid increases time needed to deal the full damage, which slightly decreases the efficiency of your fight.

Practically every player has miss chance covered – it’s intuitive decision, but it turned out to be the most efficient. As players try not to use avoid property, next one in line of the most efficient is increase of maximum damage. And there’s good empirical prove that it’s efficient because players with the most cumulative damage in game have 7-8 appearances of increase of maximum damage property (2 appearances of course go to the reduce miss chance property). So if we take into account the cost of the fight, our optimal equipment set is the same as in the best soldiers of the game, which means our results are empirically proven. Although this set gives around 90% of damage provided by the avoid set, money and time play important role here.

Overall, we can say that Monte-Carlo simulation was successful and result from it can be used by players when they consider forming their set of equipment.

Code

Excel simulation file

Report of simulation in pdf and docx.