New loot system

[Akkadius]

Originally Posted by ChaosSlayerZ

In NEW system if you saying that probabilities are removed, how can you replicate the system above where I want a situation where npc may or may not drop anything at all even before we get to the items?

I know this is easy to over think and I did it too when I was question Cavedude on changing the system before we bring in the data that I parsed from Magelo.

Think of it as a header.

loottable_entry:

mysql> select * from loottable_entries WHERE loottable_id
 = 1000 limit 1;
+--------------+-------------+------------+-------------+-----------+---------+
| loottable_id | lootdrop_id | multiplier | probability | droplimit | mindrop |
+--------------+-------------+------------+-------------+-----------+---------+
|         1000 |        2014 |          1 |          25 |         1 |       0 |
+--------------+-------------+------------+-------------+-----------+---------+

You have a 25% probability of rolling into the lootdrop:

Inside that lootdrop (2014):

mysql> select * from lootdrop_entries where lootdrop_id = 2014;
+-------------+---------+--------------+------------+--------+----------+----------+------------+
| lootdrop_id | item_id | item_charges | equip_item | chance | minlevel | maxlevel | multiplier |
+-------------+---------+--------------+------------+--------+----------+----------+------------+
|        2014 |    3329 |            1 |          1 |   8.25 |        0 |      127 |          1 |
|        2014 |    3342 |            1 |          1 |    8.5 |        0 |      127 |          1 |
|        2014 |    3353 |            1 |          1 |   8.25 |        0 |      127 |          1 |
+-------------+---------+--------------+------------+--------+----------+----------+------------+

Each item in this lootdrop now is rolled individually. In other words, all three of these items could drop at the same time but they are all rolled individually instead of ONE roll a random number 1-100 and it being (3329 - 8.25%) - (3342 - 8.5%) - (3353 - 8.25%)

Now it is a roll for each item:
3353 - 8.25 (Rolled 10, well shucks)
3329 - 8.5 (Rolled 4, nice it dropped!)
3342 - 8.5 (Rolled 94, no cigar)

Originally Posted by ChaosSlayerZ

Also, when you say that items will now "roll individually" - I do know that all %% on items were screwed up and place in list was affecting proper percentages, but "roll individually" sounds somewhat vague - do items total still have to add up to 100%?

No items do not have to add up to 100% as they are rolled individually, the existing % and drops for PEQ will be audited against the script I am building to dump the new data and audit the existing data.

Make sense?

[cavedude]

Each item is rolled against its own chance now. Before, the items were rolled against all the others in the group which meant only 1 could drop (without multiplier) and the table had to equal 100%. In the new system, the table no longer needs to equal 100%, the table no longer will always drop an item (without probability, the table will essentially always roll true), and the table could potentially drop all of its items.

Probability was a silly redundant system and there was no reason why it couldn't have been combined into chance which is exactly what I've done. If you look at the SQL in the commit, you'll see I multiplied probability by chance to come up with the new chance percentage. The resulting chance is the same percentage as the old system, only it's now using 1 value instead of 2. So for example, if the old system had a probability of 20% and a chance of 50%, the total chance the item would drop is 10%. (20% chance the table would even be picked, and then you'd split that in half since the item only had a 50/50 shot of being picked.) 10% is now what the chance column reflects for that item in the new system. Let me convert your example for you to the new system to visualize it for you:

loottable_id 12345 would have inside of it:

lootdrop_entries 26, mindrop 0, droplimit 1, multiplier 1
lootdrop_entries 85, mindrop 0, droplimit 2, multiplier 2

lootdrop_entries 26 would have inside:
rusty mace 6.6%
club 23.1%
short sword 3.3%

lootdrop_entries 85 would have inside:
bronze bracer 61.6%
chainmail 15.4%

This replicates both the drop rates of the old system (multiply probability by chance after converting to decimal) and the drop behavior as well (lootdrop 26 will only drop 1 item max, lootdrop 85 will drop 2 items max.)

[cavedude]

Akkadius: Probability is no longer used, I haven't dropped it yet because I was still using it as a reference. Everything checks out so it'll be gone in the next revision of the DB. We're using straight percentages now, so the percentages you see on Magelo are the exact ones we'll merge into the DB.

[Akkadius]

Originally Posted by cavedude

Akkadius: Probability is no longer used, I haven't dropped it yet because I was still using it as a reference. Everything checks out so it'll be gone in the next revision of the DB. We're using straight percentages now, so the percentages you see on Magelo are the exact ones we'll merge into the DB.

I know we're using straight percentages, it was my fault for not thinking twice on that. Had to hurry and finish the post so I could finish configuring some things. MY BAD CD!

[ChaosSlayerZ]

ok I got it now thank you guys!

The multiplication of probability by % chance was the part I was missing.
I guess the new system was done for the sake of optimization, thought old one was a bit more straight forward (new system maybe a bit harder to explain to a newbie)

One more question abotu exmaples.

Suppose in old system I have following:

lootdrop_entries 85, multiplier 1, prob 100%

lootdrop_entries 85 would have inside:
item1 10%
item2 10%
item3 10%
item4 10%
item5 10%
item6 10%
item7 10%
item8 10%
item9 10%
item0 10%


so only 1 out of 10 items could drop each time with equal chance (old system RND buginess aside)

how this end up looking in new system?

[sorvani]

the probability in loottable_entries is always 100% now. All tables always drop. there is no field to edit that % in the editor anymore. I would have to look at CD's code again to tell you if it is even used, since the field is still in the DB

When you create a lootdrop the code then randomly picks one of the items in the lootdrop_entries table and starting at that point it then goes one at a time through each item rolling the % chance.

if no item hit's it's individual % then no item is dropped. the min/max numbers affect as cavedude described.

In your example that loot drop converts exactly the same with the addition of a droplimit= 1 (the conversion sets all drop limit's to 1 unless the lootdrop had a mulptiplier originally)

then a random number is rolled form 1 to 10. (result 6 for example) the code then rolls the % listed for item6. if it hits, it checks the total dropped against droplimit and since it is now 1 it will end the loop. if item6 did not drop it would roll item 7, 8, 9, 10, 1, 2, 3, 4, 5 until one dropped or the loop ran out.

[cavedude]

It's actually a LOT more simple than the old way once you grasp it. But yes, there is a learning curve involved coming from the old way.

That example is simple, you just use mindrop and droplimit to force 1 item to drop every time (since probability was 100%) So:

lootdrop_entries 85, mindrop 1, droplimit 1, multiplier 1

lootdrop_entries 85 would have inside:
item1 10%
item2 10%
item3 10%
item4 10%
item5 10%
item6 10%
item7 10%
item8 10%
item9 10%
item0 10%

Since probability was 100%, none of the percentages will change of course. No more and no less than 1 of those 10 items will drop.

But changing it up, let's say the old probability was 80%. Then the same example would look like:

lootdrop_entries 85, mindrop 0, droplimit 1, multiplier 1

lootdrop_entries 85 would have inside:
item1 8%
item2 8%
item3 8%
item4 8%
item5 8%
item6 8%
item7 8%
item8 8%
item9 8%
item0 8%

Notice mindrop is now 0 because we are no longer guaranteed an item, and all the percentages dropped because multiplier is 20% lower. But, they all equal 80% - the old probability

Let's have some fun and show off what the new system can do.

Same example above, but:
mindrop 0, droplimit 0: 0 to all 10 of those items will drop.
mindrop 5, droplimit 0: 5-10 of those items will always drop.
mindrop 1, droplimit 2: 1-2 of those items will drop.
mindrop 0, droplimit 0 and all the items have a 50% chance of dropping: 0-10 items can drop, but most of the time 5 will (since "probability" is now 500%.)

[cavedude]

Originally Posted by sorvani

the probability in loottable_entries is always 100% now. All tables always drop. there is no field to edit that % in the editor anymore. I would have to look at CD's code again to tell you if it is even used, since the field is still in the DB

I mentioned above probability is still in the DB as I was still using it to verify everything. But, we're good and it'll be gone in the next revision as it's no longer used.

[ChaosSlayerZ]

ok i think I am getting a grasp of the new concept now.

From what I see if all item in the lootdrop have identical chance to drop, then it doesn't matter what % they are given as long as its equal.

In other words:
if mindrop 1, droplimit 1, multiplier 1

then
item1 10%
item2 10%
item3 10%
item4 10%
item5 10%
item6 10%
item7 10%
item8 10%
item9 10%
item0 10%

is equal to:

item1 66%
item2 66%
item3 66%
item4 66%
item5 66%
item6 66%
item7 66%
item8 66%
item9 66%
item0 66%


but next question arises about efficiency:

you said that the new code will "keep rolling" until it roll the needed value.
so if items all set to 10% and NONE fall within 10% on the first roll, the system has to roll again and again until one of them does. Of course the chance of that is small - but isn't it a strain on the system?
Theoretically the rolling can go on indefinitely, isn't it?

Another issue arises when more than 1 item suddenly satisfies drop parameters but with droplimit=1, how does system chooses which item drops if say all 10 have been rolled successfully?

Not to criticizing all the hard coding you put into it, but wasn't old system a bit simpler/faster? Since its only rolled once on the item list?
I know that the RND was bugged, and item order affected RND in a bad way, but wasn't fixing RND a more straight forward solution, than the a whole new approach?

Of course, I am not the person to judge the code on its performance, but just trying to think out loud and understand the reasoning beyond the rewamp

[Akkadius]

That was a concern I had for Cavedude at the time but the way he had implemented poses no real performance hit whatsoever.

[cavedude]

From what I see if all item in the lootdrop have identical chance to drop, then it doesn't matter what % they are given as long as its equal.

It doesn't matter what the other items are doing, the only percentage that matters is the current item's chance. But, if neither droplimit or mindrop are set, then the percentage most certainly matters. If it's set to 1%, you'd be lucky to get a single item to drop. It it's set to 90%, you have a good chance of every item dropping. But, not every item has to be equal, and they don't all have to equal 100%... You could have 50 items in a lootdrop table with chances ranging from 0.01% to 100% if you wanted. In that case, yes the NPC could drop all 50 items if you're super lucky and you haven't set a droplimit.

droplimit = 1 the first item that rolls successfully drops, the remaining items aren't even rolled. As I said in the first post the order which each item is rolled is randomized each time, so when you do set a droplimit, the same item won't drop over and over again because it's first in the list.

There isn't a performance hit, PEQ showed no difference in time to boot zones. That was a concern of mine at first too, but KLS believed that if done right, we wouldn't see an impact provided we didn't have super long unlimited lists. It turns out he was correct. So, if you have a long list of 100+ items, be sure to set a droplimit so the server doesn't have to iterate through every item. (Although, the code does set a droplimit for lists 100+ automatically.) Rolling will never go indefinitely, but setting a high mindrop or not limiting long lists certainly can increase the rolls. Basically, if you're smart about your setup you'll be fine. In your example, if all the items failed they are only re-rolled if a multiplier is set to 2 or above or mindrop is 1 or above. If neither are true, then the loop ends and the table drops nothing.

Fixing RNG has nothing to do with this... This new system is far more powerful and indeed simpler than the old one. Seriously, the old system was unnecessarily complicated and backwards. Probability is gone, there is no longer a need to have multiple lootdrop tables, those tables no longer need to equal 100%, and we now have a method to have complete control over how many items drop, whether that be a specific number or a range without fiddling with multipliers or multiple tables.

[sorvani]

Originally Posted by ChaosSlayerZ

From what I see if all item in the lootdrop have identical chance to drop, then it doesn't matter what % they are given as long as its equal.

No it matters. 10% != 66%

lootdrop 20 has mindrop=0 droplimit=1 and multiplier=1

This means that no matter how many items are in the lootdrop that between 0 and 1 of them will drop.

lootdrop 20 has 5 entries.

position 1: itemid 13071 = 90%
position 2: itemid 13072 = 50%
position 3: itemid 13073 = 50%
position 4: itemid 13074 = 50%
position 5: itemid 13075 = 1%

First, the code will roll a random number between 1 and 5.
This will determine which item gets rolled first.
Say the result is 2.
The code then looks at the % for the item in position 2 and rolls against that %.
If it hits, the item is dropped and the loop terminates, no more items are rolled.
If it does not hit, then the code moves on to position 3 and rolls against the % for that item.
This continues with positions 4 then 5 then loops back around to position 1.
If at this time no item has yet dropped, the loop terminates because there is no minimum drop requirement for the lootdrop.

Now, let us change the % for position 1 to 100% and assume that the random starts on position 3. If position 3, 4 or 5 hit, then that means position 1 will not even get rolled even though it is 100%. Similarly, if the random started on position 1 and it was 100% then none of the other 4 positions would even get rolled. the loop would be stopped.

Is that all clear?


Now if you have an item (or more than 1) that you DO want to be 100% Make a separate table for the 100% drop(s).

Lootdrop 21 contains 3 items that should ALWAYS drop.
Set mindrop=3, maxdrop=3 and multiplier=1

Position 1: item123 = 100%
Position 2: item456 = 100%
Position 3: item789 = 100%

No matter where the random 1 to 3 starts this loop each item will drop.


Now you have a loot drop where the mob always drops 2 items from a list of 3 (and can dupplicate a drop)

lootdrop 22 contains 3 items, set mindrop=2, maxdrop=2 multiplier=1

Position 1: item123 = 40%
Position 2: item456 = 40%
Position 3: item789 = 20%

The random 1 to 3 will start at 3 and roll against the 20% for that position.
Then it rolls against hte 40% for position 1 then against the 40% for position 2.
If only 1 of the 3 items dropped, the mindrop=2 has not yet been met so the loop will continue. pos 3, 1, 2 until a second item drops.

[trevius]

I think I am understanding this new system, but haven't been through the code that supports it yet. I do have one concern though based on the stuff I have been reading in this thread. I have seen a few statements in this thread saying that the old system had to total 100% for the chance in the lootdrop_entries for any given lootdrop_id. This is not true, and if I am understanding the new system and SQL provided for converting it, this could break loot drops for any server that was not following that rule.

The way the old system worked with chance is that it totals the chance from all items with the same lootdrop_id in the lootdrop_entries table. It then rolled the chance to drop for each item against the total from all entries for that lootop_id. So, whether you have 10 items all set to 10% chance or 10 items all set to 100% chance, the result would be the same.

Here is an example:

item1 10%
item2 10%
item3 10%
item4 10%
item5 10%
item6 10%
item7 10%
item8 10%
item9 10%
item0 10%

So, the total chance is 100 in this case. It would then roll 1-100 for each item (starting at a random item location in that list) until one of the rolls was less than or equal to 10. This gives it 2 points of randomness when choosing the drops, where the first point is the random point that it starts in the list and the second point is the rolling of each item until it finds a winner.

Here is another example:

item1 100%
item2 100%
item3 100%
item4 100%
item5 100%
item6 100%
item7 100%
item8 100%
item9 100%
item0 100%

In this case, the total is 1000. So in the old system it would then roll 1-1000 for each item, effectively giving each of them a 10% chance to drop, the same as the first example.

I regularly started using values of 100 for chance for lootdrop_ids that were all supposed to have the same drop rate. My reasoning was that it means there is no math involved at all when setting drop rates, because you don't have to manually add up the totals and verify they total up to 100. I don't have to set 5 items each to 20, or 10 items each to 10 or 7 items each to 14 (which would actually be 5 set to 14 and 2 set to 15 to total 100) to make equal drop rates.

My concern is mainly with the SQL to convert the chances. If I am understanding the SQL from Cavedude correctly, my lootdrops that are set to 100 would now each have a 100% chance to drop (assuming probability of 100 was set for them). So, in the new system, it would always pick the first item it rolls for. The only saving point is that the new system should still be picking an item from the list to start rolling at by random, so there would still be a random factor in it. The problem is that there wouldn't be 2 random factors in cases like this like I mentioned previously.

I haven't messed with it yet, but I think it is probably possible to write a different query to do the conversion from the old system to the new that would take this issue into account. It would also correct any issues with loot drops where someone didn't do their math correctly when trying to total 100 in the chance field for a lootdrop_id.

I would guess we can use SUM() to total up the chance field for each lootdrop_id, and then do chance * 100 / total for each individual entry for that lootdrop_id before doing the rest of the math that cavedude provided for calculating the new chance values. For example, if the total is 1000 for 10 items that are each set to 100, SUM() would get 1000. So we would do chance * 100 / 1000 to get a value of 10 for each entry, which is what the chance should have been if following the concept of totaling them all to 100.

Here is another example to break it down and show a more complex result:

item1 50%
item2 50%
item3 20%
item4 20%
item5 60%

The SQL to convert this would then need to total those chances up to 200, and then it would do chance * 100 / 200 that was set there for the following results:

item1 50% (50 * 100 / 200 = 25%)
item2 50% (50 * 100 / 200 = 25%)
item3 20% (20 * 100 / 200 = 10%)
item4 20% (20 * 100 / 200 = 10%)
item5 60% (60 * 100 / 200 = 30%)

So then they would all total 100% chance.

Since SUM() has to be ran in a select and you can't select from a table while updating the same table, I think we would just need to create a temporary table to store the SUM() totals for each lootdrop_id and the id of the lootdrop_id, then we can do the rest of the math in a second query for the updates while selecting the sum values from the temp table. Then just drop the temp table.

The SQL should be something like this (untested):

CREATE TEMPORARY TABLE temp_table (
lootdrop_id INT(11) PRIMARY KEY,
chance_total INT(11) DEFAULT 0
);

INSERT INTO temp_table ( temp_table.lootdrop_id, temp_table.chance_total ) (SELECT lootdrop_entries.lootdrop_id, SUM(lootdrop_entries.chance) FROM lootdrop_entries GROUP BY lootdrop_entries.lootdrop_id);

UPDATE lootdrop_entries, temp_table SET lootdrop_entries.chance_total = (lootdrop_entries.chance_total * 100 / temp_table.chance_total) WHERE lootdrop_entries.lootdrop_id = temp_table.lootdrop_id;

DROP TEMPORARY TABLE temp_table;

This would need to be ran prior to the conversion cavedude committed, which is this:

update lootdrop_entries lde
inner join loottable_entries lte ON lte.lootdrop_id = lde.lootdrop_id
SET lde.chance = (lte.probability/100)*(lde.chance/100)*100;

I think that should resolve any issues with converting from the old system to the new. The only thing that might need to be accounted for is any lootdrop_ids that are set to a total chance of 0, as I think that might cause an error when trying to divide by 0.

Again, this SQL is untested, so do not run this until it has been tested and verified to work as intended!

If I am misunderstanding something with the new system, please let me know

FYI, this is what ChaosSlayer meant by setting them all to 10% or all to 66% is the same thing in the old system. As long as they were equal values, the chances were equal no matter what.

[namini]

Just when I thought I had my head wrapped around the old system. Heh. Guess I'll move onto tradeskills and faction portions of customizing. I trust in those with the experience, but, may I ask if its safe to assume that this change will break Allaclones display of "probability" as well as "loot table" categorizing when displaying an NPC's loot?

[trevius]

Originally Posted by namini

Just when I thought I had my head wrapped around the old system. Heh. Guess I'll move onto tradeskills and faction portions of customizing. I trust in those with the experience, but, may I ask if its safe to assume that this change will break Allaclones display of "probability" as well as "loot table" categorizing when displaying an NPC's loot?

Yes, AllaClone will require a fairly minor fix for it to properly pull the new drop rate system. I don't want to side-track this thread, but hopefully Akka and I will get some time soon to fix the clone for the new field changes. We still have quite a lot of work to do on the new stuff we have been working on for AllaClone, but we could always put out an update that doesn't include the stuff we are still working on.