NET ERROR Flooding my Error Logs

[trevius]

Starting yesterday, I noticed that I was getting a HUGE error log in one of the files called eqemu_debug_zone_xxxx.log and logged into the game and used the #iplookup and found it was a player from china that was causing the error. It was his first day playing on my server. This caused 100MBs of logs within a few hours. I edited my router settings to narrow down allowed ports and thought it would stop the error from hitting the server. I told the guy he needed to figure out what was causing it, or I would have to block him completely from my server. So, he logged off kindly, and I continued to get the error from his IP. Finally I rebooted the server with the new router settings in and it was back to normal. I deleted the log file, but made note of his IP for future reference if needed.

Today, I check my log files and now that file is up to 250MBs! This time it is coming from a player that has played on my server for a few days and I have never seen the error from him before now. He was on at the same time yesterday when the other log issue was happening, but it wasn't from his IP.

So, this is the error I am getting:

[02.07. - 23:42:29] [NET__ERROR] 24.167.163.206:1619: Tried to write a packet beyond the end of the queue! (40899 is past next out 40898)
[02.07. - 23:42:29] [NET__ERROR] 24.167.163.206:1832: Tried to write a packet beyond the end of the queue! (40815 is past next out 40814)
[02.07. - 23:42:29] [NET__ERROR] 24.167.163.206:2382: Tried to write a packet beyond the end of the queue! (40223 is past next out 40222)

It is those 3 lines about 50 times per second each!

Does anyone have a clue what that is? Is it dangerous, or can I ignore it? And if I can ignore it, can you please tell me exactly what I need to do to turn it off? I wish there was an easy way to turn off and on these debugs, but as far as I can tell, there is not. I don't need a debug to run unless I see a problem! Waste of system resources, IMO. There are too many debugs with too much information that is hard to read and even harder to understand.

If anyone has seen this error and knows what it means, I would like to know. Google came up with nothing from searching it.

[Knightly]

I can't tell you why it happens, but if you comment out the following line in common\EQStream.cpp it will stop:
if(CompareSequence(NextOutSeq, seq_send) == SeqFuture) {
//_log(NET__ERROR, _L "Tried to write a packet beyond the end of the queue! (%d is past next out %d)" __L, seq_send, NextOutSeq);
sitr=SequencedQueue.end();

However, someone else can probably tell you what it means and how to actually stop it from happening (the above will just tell you how to ignore it).

In looking over that portion of the code, it looks like what is happening is that the server is receiving the packets out of sequence. In this case, it is expecting the response for 40898 and got back the response for 40899. So the server is wondering where 40898 went since it doesn't have that information.

I would expect this to happen on a high latency network where a lot of packets are getting dropped. However, I may be barking up the wrong tree.

[trevius]

Makes sense, cause it is mostly only from Chinese players. I need to get that debug turned off. Guess I will have to figure out how to setup the source and compile it all instead of using the simple win32 version. Just wish those debugs were turned off by default.

[trevius]

Here is another example from today. The thing that gets me is that some of them are the same exact message. It isn't incrementing like you would think an error like this should:

[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)

[trevius]

BTW, incase anyone else finds the debugs more than annoying, I *think* I have found a simple work-around for it.

Simply rename your "Log" folder in your eqemu directory to "Log2" or even just delete it. The debugs can't make logs if they don't have a folder to go into.

Why didn't I think of this before lol? Well, I originally considered it, but I figured they would just create another folder, so I didn't even bother.

The best part is that it is really quick and easy to turn off and on. All you do is rename your log folder back and restart your server.

Sure, you will get an error or 2 about the folder being missing so debugs and logs can't work, but I don't think it is a big deal. It seems to have reduced lag on my server by a lot so far as well. I wish there was an easy graceful way to disable and enable logs/debugs, but for now, this should work for anyone.

Hope someone finds that info useful.

[Congdar]

This error started happening on my server a few days ago. The account/player in question did not seem to be doing anything strange, and they claimed to not be experiencing any lag.

The real problem is that even after they log off, the error continues to fill the logs... I mean.... they're gone but their ip and error keep filling up the log files making them huge!

Is there a better way to handle out of order packets than this?

[kiirdiir]

I can't tell you why it happens, but if you comment out the following line in common\EQStream.cpp it will stop:
if(CompareSequence(NextOutSeq, seq_send) == SeqFuture) {
//_log(NET__ERROR, _L "Tried to write a packet beyond the end of the queue! (%d is past next out %d)" __L, seq_send, NextOutSeq);
sitr=SequencedQueue.end();


Here is another example from today. The thing that gets me is that some of them are the same exact message. It isn't incrementing like you would think an error like this should:

[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)
[02.09. - 01:33:57] [NET__ERROR] 219.77.151.159:1193: Tried to write a packet beyond the end of the queue! (2781 is past next out 5625)


Maybe it was meant to be:

if(CompareSequence(NextOutSeq, seq_send) > SeqFuture) {
_log(NET__ERROR, _L "Tried to write a packet beyond the end of the queue! (%d is past next out %d)" __L, seq_send, NextOutSeq);
sitr=SequencedQueue.end();

Just throwing it out there, passing through trying to find solutions to other problems. :)

[Congdar]

The compare can only return SeqInOrder, SeqFuture or SeqPast, so > probably isn't correct and ignoring it doesn't seem right either... is the packet stuck in the queue somewhere because this is currently an endless loop even after the account has logged off. Is there some way to clear this packet?