Loot and merchant lag (The Linux version)

[Rikon]

Possible Solution:: In the database under rule_value, "Character:EnableDiscoveredItems" is set to true for ruleset 10. Change to false to alleviate this issue.

----

I wasn't paying attention when I necro'd a thread (http://www.eqemulator.org/forums/showthread.php?p=241947) about Loot lag in the Windows server section. Considering I am running Linux, I thought having it's own thread here would be wise. Here are some the pertinent info.

Server: Linux Debian 8.1
MySQL: 5.5.43-0+deb8u1

Symptom::
When users go to loot corpses, they can only loot one or two items before it lags out and they have to re-loot. A similar, and probably related, issue occurs when dealing with vendors.

The source I compiled from was taken from direction followed HERE (http://wiki.eqemulator.org/p?EQEmu_on_Debian_7_Virtualbox#8-install-eqemu-source-files) on the Wiki. I.E. git://github.com/EQEmu/Server.git

My DB was populated from PEQ (http://peqtgc.com/releases)

I've been busy with work, but I can try to get a video capture if it would even help.

[Uleat]

I don't believe a video is necessary..the symptoms are descriptive enough to understand the problem.

As to the cause, that part is problematic due to your resources being inaccessible.


To start, I would double-check your logs for any database-related messages.

I'll look through the linux threads to see if I can find anything similar to this issue and update what I find a bit later.

[Rikon]

I do have full access to the db and the server. My thanks in advance for your assistance. It is appreciated.

[Uleat]

What does this query return?
SELECT VARIABLE_NAME, VARIABLE_VALUE FROM information_schema.GLOBAL_VARIABLES WHERE VARIABLE_NAME = 'max_connections'

[Akkadius]

I wasn't paying attention when I necro'd a thread (http://www.eqemulator.org/forums/showthread.php?p=241947) about Loot lag in the Windows server section. Considering I am running Linux, I thought having it's own thread here would be wise. Here are some the pertinent info.

Server: Linux Debian 8.1
MySQL: 5.5.43-0+deb8u1

Symptom::
When users go to loot corpses, they can only loot one or two items before it lags out and they have to re-loot. A similar, and probably related, issue occurs when dealing with vendors.

The source I compiled from was taken from direction followed HERE (http://wiki.eqemulator.org/p?EQEmu_on_Debian_7_Virtualbox#8-install-eqemu-source-files) on the Wiki. I.E. git://github.com/EQEmu/Server.git

My DB was populated from PEQ (http://peqtgc.com/releases)

I've been busy with work, but I can try to get a video capture if it would even help.

There is a rule that pertains to discovered items, if this rule is enabled this can slow looting down as each item loot is a hit to the DB because it wasn't written the best (Sorry Trevius I love you man). I'm not sure if he added memory caching to each of the lookups or not. This can definitely do it.

It wouldn't hurt to enable MySQL query debugging to see what is actually taking up the most query time.

Example:

http://wiki.eqemulator.org/p?Logging_System_Overhaul#logs-set-gmsay-example-enabling-mysql-query-debugging-to-gmsay

[rhyotte]

Cool to know. Thanks.

[Shendare]

I see Character:EnableDiscoveredItems . Appears to be set to true for ruleset 10, false for the others by default.

[Rikon]

There is a rule that pertains to discovered items, if this rule is enabled this can slow looting down as each item loot is a hit to the DB...

I believe this was the issue. I'm going to do more testing and report back.

[provocating]

Are you saying that turning on discovered items is lagging you out? You have a much deeper problem if that is the case. If something is that wrong with the MySQL server this is just a band aid to a problem that will creep up again. You need to start running something like MyTop and see what is holding everything up.

[Rikon]

Are you saying that turning on discovered items is lagging you out? You have a much deeper problem if that is the case. If something is that wrong with the MySQL server this is just a band aid to a problem that will creep up again. You need to start running something like MyTop and see what is holding everything up.

I spoke too soon. The problem still exists.

The error when hitting 'loot all' on a corpse is as follows:

Error: Corpse::Lootitem: BeingLootedBy != client

Checking out zone/corpse.cpp:

LootingItem_Struct* lootitem = (LootingItem_Struct*)app->pBuffer;

if (this->being_looted_by != client->GetID()) {
client->Message(13, "Error: Corpse::LootItem: BeingLootedBy != client");
SendEndLootErrorPacket(client);
return;

So is this client-based? Am I reading this right?

[provocating]

Just curious what version of clients do you have a mix of?

[Rikon]

I have no idea what the users have, but I have UF.

[provocating]

You really need something to monitor those queries. On Linux it is easy, not sure about Windows. I read there is MyTOP for Windows.

[Rikon]

You really need something to monitor those queries. On Linux it is easy, not sure about Windows. I read there is MyTOP for Windows.

How do I do it on linux

[Akkadius]

The logging system does all of this with MySQL queries....

[N0ctrnl]

MyTOP.

Also, while it's on my mind, you did turn your journal off on the client, yes? I know that can cause issues sometimes.

[Rikon]

hmmm. I may have an issue. #log produces nothing.

[Akkadius]

For the 3rd time, the logging system reveals all, it's an extremely powerful tool and will tell you all you need to know to troubleshoot your issue.

http://wiki.eqemulator.org/p?Logging_System_Overhaul

It's #logs:

http://wiki.eqemulator.org/p?Logging_System_Overhaul#command-logs

http://wiki.eqemulator.org/l/wa/images/log_system/logsys_log_set_command.PNG

[provocating]

I was not saying it does not reveal what queries are running. I was saying there could be other queries the MySQL engine could be running that are taking a long time to run. MyTop will show the long running queries and queries that may not even be related to EqEmu. You cannot do that with #log

[Akkadius]

I was not saying it does not reveal what queries are running. I was saying there could be other queries the MySQL engine could be running that are taking a long time to run. MyTop will show the long running queries and queries that may not even be related to EqEmu. You cannot do that with #log

It's enough to troubleshoot the issue and visually see what queries are running when loot is hanging, it doesn't need to be much more complicated than that. Coming from someone whose troubleshot many of these issues.

[Rikon]

Which log is most viable? I'm not seeing anything under the MySQL one--nothing that would indicate an issue. It shows the table updates and queries, but nothing seems out of place.

[Rikon]

I'm not seeing anything that would cause an immediate alarm. I'm really at a loss. The server is so laggy, it's unplayable, and I don't know the cause. The server's RAM and Processor all look fine. No idea on this one.

I'm willing to grant GM status to a vetted member here to take a look. I'm not sure how I could repay, but we can chat.