Possible Login Server Bug

[Dralanna]

There is a thread going on in the General Forum regarding this issue. (http://www.eqemulator.net/forums/showthread.php?t=22521) It was suggested that we put it here. I'm copying many of the relevant posts here:

Originally Posted by zuperzip

If any of the information below is not solid please let me know what criteria needs to be met to make the information concrete and useful. I would be happy to answer any questions or perform further testing. Contact me by PM or admins may use email.

The eqgame.exe error has been visited many times on this and other forums. For many players of EQEmu there is a certain cause for this error at a certain stage of the login process that can block logins for an hour or more at a time, off and on throughout the day.

I'm going to call this error "SSSE" for Server Select Screen error. There are a number of causes for the eqgame.exe error message and it is necessary to focus on when it occurs on order to define the SSSE.


--- When the SSSE strikes ---
The SSSE occurs when the "LOGIN" or "QUICK CONNECT" button is pressed either with mouse button or tab/enter keys. If you use the "LOGIN" button the SSSE strikes immediately so you never get to press the "PLAY EVERQUEST!" button on the server select screen. Players find themselves trying to get to the "PLAY EVERQUEST!" button as quickly as they can by mouse or tab/enter but fail. A slowed mouse pointer is a symptom that the eqgame.exe error is about to pop up. Often players conclude it is a mouse driver problem until they try using the tab/enter keys but fail. It is also true that if the error does not strike immediately, sitting at the server select screen and doing nothing may eventually result in the SSSE. If the "QUICK CONNECT" button is used the SSSE will strike without displaying the server select screen. Once loading begins for the character select screen the SSSE will not occur.

The above is followed by the generic eqgame.exe error message which reads "eqgame.exe has encountered a problem and needs to close. We are sorry for the inconvenience." In the same window there's also the standard Microsoft message with options to "Send Error Report" or "Don't Send". This is on Windows XP.


--- The SSSE cause ---
Recently I performed a series of packet captures and analyzed them. Searching the forums shows "neville" also did some packet captures. I'm not sure if anything further was done with his results.

http://www.eqemulator.net/forums/sho...5&postcount=21
http://www.eqemulator.net/forums/sho...6&postcount=22

What the results show connects a specific packet to the SSSE. That packet contains the survey information for the client. You may recognize this survey since occasionally it pops up during login, "Is this thing on?......... 1) Yes, 2) No, 3) What thing?, 4) Stop wasting my time!". This will be referred to as the survey packet.

Though the survey rarely displays during login the survey packet is always sent with one exception I'll mention later*. In my captures a survey packet arrives as either the 3rd or 4th packet from the server and never causes a problem. This survey packet arrives before the server list is sent for the server select screen. Anytime a second survey packet is sent to the client an SSSE occurs immediately.

A second survey packet can be sent at the tail end of the login process following the list of servers. This makes it impossible to log into your selected server even using the "Quick Connect" button. Generally repeated tries will eventually get through but may take 20 minutes and 40 login attempts or more.

A second Survey packet may also be sent at some point after the initial login while sitting at the server select screen. This survey packet also arrives at the tail end of the list of servers, but in this case it's the server list update. The delay between login and when this second survey packet is sent may vary from a few seconds to hours. When I was running tests on March 20th a second survey packet arrived with delays between 2.5 minutes and 34 minutes over five captures.


What follows is one of the second survey packets as it appears in the capture. It is always 233 bytes in size:

0000 00 0b 6a 58 fb f5 00 30 ab 07 79 7d 08 00 45 00 ..jX...0..y}..E.
0010 00 db 00 00 40 00 34 11 64 c2 0c a4 14 02 c0 a8 ....@.4.d.......
0020 00 02 17 6e 04 30 00 c7 47 b2 00 09 00 17 29 00 ...n.0..G.....).
0030 00 00 00 00 00 00 00 00 00 00 01 00 55 53 00 45 ............US.E
0040 4e 00 01 00 00 00 49 73 20 74 68 69 73 20 74 68 N.....Is this th
0050 69 6e 67 20 6f 6e 3f 00 04 00 00 00 01 00 00 00 ing on?.........
0060 00 00 00 00 01 00 00 00 00 01 00 00 00 01 00 00 ................
0070 00 00 00 00 00 59 65 73 00 02 00 00 00 00 00 00 .....Yes........
0080 00 01 00 00 00 00 01 00 00 00 02 00 00 00 00 00 ................
0090 00 00 4e 6f 00 03 00 00 00 00 00 00 00 01 00 00 ..No............
00a0 00 00 01 00 00 00 03 00 00 00 00 00 00 00 57 68 ..............Wh
00b0 61 74 20 74 68 69 6e 67 3f 00 04 00 00 00 00 00 at thing?.......
00c0 00 00 01 00 00 00 00 01 00 00 00 04 00 00 00 00 ................
00d0 00 00 00 53 74 6f 70 20 77 61 73 74 69 6e 67 20 ...Stop wasting
00e0 6d 79 20 74 69 6d 65 21 00 my time!.


For comparison a harmless first survey packet shown below is 241 bytes in size.

0000 00 0b 6a 58 fb f5 00 30 ab 07 79 7d 08 00 45 00 ..jX...0..y}..E.
0010 00 e3 00 00 40 00 34 11 64 ba 0c a4 14 02 c0 a8 ....@.4.d.......
0020 00 02 17 6e 04 30 00 cf 2b f6 00 03 04 00 15 00 ...n.0..+.......
0030 01 bf 00 09 00 01 29 00 00 00 00 00 00 00 00 00 ......).........
0040 00 00 02 00 55 53 00 45 4e 00 01 00 00 00 49 73 ....US.EN.....Is
0050 20 74 68 69 73 20 74 68 69 6e 67 20 6f 6e 3f 00 this thing on?.
0060 04 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 ................
0070 00 01 00 00 00 01 00 00 00 00 00 00 00 59 65 73 .............Yes
0080 00 02 00 00 00 00 00 00 00 01 00 00 00 00 01 00 ................
0090 00 00 02 00 00 00 00 00 00 00 4e 6f 00 03 00 00 ..........No....
00a0 00 00 00 00 00 01 00 00 00 00 01 00 00 00 03 00 ................
00b0 00 00 00 00 00 00 57 68 61 74 20 74 68 69 6e 67 ......What thing
00c0 3f 00 04 00 00 00 00 00 00 00 01 00 00 00 00 01 ?...............
00d0 00 00 00 04 00 00 00 00 00 00 00 53 74 6f 70 20 ...........Stop
00e0 77 61 73 74 69 6e 67 20 6d 79 20 74 69 6d 65 21 wasting my time!
00f0 00 .


* The one exception is some accounts are never sent survey packets. These accounts do not receive the survey packet near the beginning of the login process or at anytime after and so never experience the SSSE. Currently I know of three such accounts, one I have tested myself but do not own. The other two are owned by someone who experiences SSSEs on his other accounts. Both owners would be interested in helping rid the EQEmu community of the SSSE.

I believe firewall settings may somehow help prevent or allow this problem to occur but right now this is nothing more than a hunch. I do not have packet captures showing differences between different firewalls and their settings.


Below is older information that I gathered over the past months and may be extraneous now. I'm including it in case it may be of some value.

--- Gathered information about the SSSE ---
1) Many hours may go by with no accounts/players experiencing SSSEs or at least the frequency of the SSSE is very low.

2) When there is a high frequency of SSSEs all accounts/players that can be affected by the SSSE will have great difficulty logging in. Repeated login attempts will eventually get past the SSSE.

3) While sitting at the Server Select screen an SSSE may occur immediately when there is an update in the server list.

4) Some accounts are completely immune to the SSSE. This is true even when an immune account is used on different computers in different global locations where the SSSE does occur and when it is occurring with a high frequency.

5) Some players never experience the SSSE.

6) For two weeks this past February I visited a different location 140km away using a different DSL modem and ISP. I took one of my EQEmu computer boxes with me but all other equipment used was from the other location. I often loaded up 4 EQEmu accounts and switched accounts to play different characters. For that two weeks I never experienced an SSSE even though other players often mentioned in game how harsh the SSSE was being to them. A couple days before I left home and the day after I returned home I experienced the SSSE on that computer box.


--- What I think ---
SSSEs are not related to drivers or settings on the client system. It is caused by data sent by the login server or server select server to the EQ Titanium client. The the frequency with which the data is sent raises and lowers in an irregular pattern. For some accounts this data is never sent at anytime. Some firewalls filter out this data preventing client crashes.

[Dralanna]

More quotes

Originally Posted by zuperzip

I have found that responding to the survey stops the login server from sending you the survey packet. The survey shows itself very rarely so when it does come up and asks, "Is this thing on?" you need to select "Yes" and click "Submit". I did this and now that one account is no longer sent the survey packet and so no longer gets the SSSE.

Originally Posted by Vastlee

Any follow up on this?

This exact thing you described, as opposed to the random eqgame.exe crash that others seem to get mixed up with this, has been happening to me for over a year now. I have 6 accounts. 2 of them are as you stated, "Immune" to it.

One night I actually counted 36 retries before it let me through, and retries seem to be the only thing that actually gets you through. Not rebooting, not logging other accounts in, nothing. And the # of retries seems to be 100% random.

This is the first post I've seen on the subject that identifies the problem specifically. The unfortunate part is that there are several things that cause eqgame.exe to crash that when someone asks about it on the boards 10 other people pop up with another "Oh yeah mine too" story... only it isn't the same thing.

I really hope that someone at least takes notice of this. Running the game 20 times really really sucks nuts.

P.S. The error I'm talking about is 100% exactly as described above. I implore anyone who does not have the exact same symptoms to please not post. I agree with the above that it is something that is being sent from the login server that is causing the problem, which means if we can't get those who maintain that to take notice then nothing will ever get done. So PLEASE

P.S.S. Is there some way to force the survey thing to come up? If clicking YES to it is all I have to do I'd be happy to. I just never see it come up.

Originally Posted by Erich

I don't think the "I have been bugged about the survey" flag is client side. If it was then it wouldn't be an account that is immune but rather a entire computer. I'm willing to bet that anyone with an account that is immune will tell you that it's immune on any computer that they use while an account that is not immune will not be immune on any computer.

I'm betting that the "I have been bugged about the survey" flag is server side and if it can be turned on for all accounts this problem will go away.

Originally Posted by Vastlee

Agreed, I have 5 machines. The accounts that are "immune" are so on all machines.

Originally Posted by Darkonig

I was getting on just fine for months and then one of my characters had the survey show up. I responded as suggested above and now periodically have the crashes on several characters. If a character is having the problem on one of my machines and I try to log him in on a different machine the account still has problems even tho other accounts on the same computers work fine. It is definitely something the login server is doing. I have never had the problem from the minilogin server.

I grabbed a couple packet dumps from when the one account was crashing excessively (more than 20 times in a row and on multiple computers), but haven't had a chance to review it yet.

Originally Posted by Darkonig

Ok, just went thru the dumps and found like expressed above. On the ones that crashed, there was a survey packet of size 241 was received, then the server list, then 1 or more additional survey packets of size 233.

Even using quick connect, there was a survey packet size 241 sent, then the server list, then additional servey packets of size 233. They are received regardless of whether the client choses to display them.

On the characters that got in fine, there were no survey packets at all.
So, we either need to get the login code changed to quit sending survey packets, or build a filter to remove them from the stream when sent.

I am going to try the latter approach and see if it resolves the problem. I will have it notify me of any filtered survey packets to verify if it makes a difference. I would assume that altering the data stream to remove those packets would not be any more of a legal issue than using a different login server to begin with.

Originally Posted by zuperzip

There is a work around that a player by the name of Blasticus on PEQ recently discovered and told us about. At one point I thought accessing a webpage while simultanously logging in would prevent an SSSE but couldn't get it to work the majority of the time. Blasticus tried using a speed test and found it worked very well and so have I. Here is what works for me, open the EQ client and type in your username and password in windowed mode so it is ready to go. Now find any speed test website with your webbrowser (I've been using performance.toast.com or infospeed.verizon.net) and get ready to start the speed test. Start the speed test then switch to the EQ client and quickly log in.

What's important is you 1) start the EQEmu authentication process and 2) get past the server select screen _while_ the speed test is downloading its sample.

It also apparently works if you do the speed test on another system that uses the same router as your EQEmu system.

Hair brained? It sure is... but it works so well for me during a meteor shower of SSSEs that it's more than just dumb luck. I suppose if I want to figure out why it works I'll need to put my nose back into packet sniffing.

Originally Posted by Dralanna

I'm guessing that people are having luck with the speed test trick because it uses a lot of bandwidth and is probably delaying the survey packet long enough for you to login.

Did anyone trying this also do a packet capture?

Originally Posted by Dralanna

Not sure if anything has changed recently, but yesterday 3 of my accounts got the "is this thing on?" survey... to which I (obviously) said yes. Now those accounts no longer crash. The remaining accounts continue to crash at the server select screen.

Originally Posted by Darkonig

4 of my accounts yesterday got a successful survey. The survey pack contains the question Is this thing on? and 4 possible answers. With responses of submit and skip. So, when I stated before that you always got a survey packet of one size, then list of servers, then 1 or more additional survey packets I was incorrect. The actual transmission is a survey packet, then the list of servers, then zero or more additional survey packets less an 8 byte custom udp header. If no bogus survey packets follow then you get the survey displayed, if 1 or more bogus ones arrive then you crash.

I answered all 4 the same way, as above. One of them crashed again several hours later when I went to log it back in. So, answering the survey does not guarantee the login server will not ask again.

As to the other poster about cannot packet sniff the login server, all of the survey packets are identified as coming from the login server IP and Port with appropriate sequence numbers to suggest they really did.

Originally Posted by Vastlee

Of all my accounts (9 active now) only 1 got prompted with the survey. (Is this thing on) and I answered "Yes" too. The account still drops though.

I hope that the surveys were sent out as an attempted response by the devs to this thread. I'd be happy just to know someone is aware and looking into it, even if the first attempt didn't work. If the survey pops up again I might try answering something else and see what happens.

Good work to all the non-dev people for providing useful information.

Originally Posted by Vastlee

I had the same thought as you Dralanna, but the 2 accounts I have that never crash have never seen the Survey pop up, and the 1 I got the other day still crashes. I'm still fairly sure it has something to do with the survey but I'm not sure that answering it solves the problem (Or has any effect). I also considered the thought that maybe it is the answer that makes or breaks you. I answered "Yes" to it this time. If it ever comes up again I'll try skipping it and see if that changes anything.

Although, it seems the easiest answer would be for the login server to simply stop sending the packet that causes the crash.

Originally Posted by Dralanna

With all due respect, Angelox.... I am convinced this is a bug with the login server. I have several accounts, when I captured packets every single one got the packet listed in the first post at one time or another. When I got that packet, I crashed. When I didn't get that packet I didn't crash. When the survey finally popped up on 3 of my accounts and I answered correctly those 3 accounts never got that packet again, and subsequently have not once crashed at the server select screen. Now, to rule out the possibility of a client or computer issue, I've logged my accounts (both the bugged and the unbugged) on using multiple machines, OSs, Connection types.... anything I could change and consistently the bugged accounts stayed bugged and the unbugged accounts stayed unbugged.

[Vast]

Thanks Dralanna, hopefully this will catch some attention now.

[Vast]

I'd like to note that every account that the survey has come up on and I selected "Yes" to the question "Is this thing on" is now fixed and immune to the bug.

This is definitely survey related. If they could just do a round of survey's for people to answer or stop them all the way it would solve this problem.

[Vast]

Anyone seen this? I am still very much affected by this as I have been for a year now, and OOC is full nightly cursing 3SE bug, so I'm not the only one.

[Vastlee]

I'd like to note something here. Last night I was one of the rare few who was on when the login servers came back up. When they did I started logging in and was extremely surprised when all of my characters logged in on the first go. This is something that hasn't happened.

I was thrilled, I assumed that during this down time someone had addressed this issue, but to make sure it wasn't just an amazing coinky dink I logged in and out a total of 13 times and every single time I got in fine.

So I go to bed and get up this morning to find that it's back. I tried to remember what was different. The one thing I noticed is that not a whole lot of the servers had reattached to the login server. So there is a possibility that one server name or number of servers in the list is directly tied with the survey.

Anywho, there appears to be a missing X variable along with the survey one. Hopefully someone will catch this soon.

[Frosty311S]

so for about a month straight I have not been able to log into my account I have spent one day counting about 150 - 250 log in attemps and every single of them got the SSSE crash and I have no been able to log in for a month straight how I am still here and still wanting to play is a mystery to me I should have just gone to SoD because they never had a log in issue but I would like to try something a lot better then there

[Vast]

150-250? I've never come close to touching that and I've been dealing with this issue for over a year. 30-40 I've done, but that was top end and I've always eventually gotten in. You might consider the possibility that it isn't the same issue as the above posted.

[John Adams]

Y'all are way more patient than I am. If I can't get through after 3 tries, I'm off finding something else to do (read: I switch my server to minilogin tee hee hee).

[xulok]

What is minilogin? This bug is driving me bonkers.

Also, has anyone figured out how to filter out this particular packet with a firewall? I something alluding to it in the thread, but nothing definitive.

[patyrn2k1]

To answer xulok's question concerning minilogin, it's a standalone login server you would have on your local pc that allows access to a standalone eqemu server that you have on your network. When this bug pops up for me ( as it does all the time ) I just log into my own world and play around for a bit, as to your other question, I have no idea if one's firewall could filter this packet out, my quess is that it can't be done with the run of the mill FW, probly need some solid FW software to do this with.

[xulok]

Is it Sony code that is sending the packet or is it community code?

[a6equj5]

I just changed the eqlsUIConfig.ini file section below to Show=0 and it fixed the problem for me after trying to log in 20 times or so.

[PollWindow]
INIVersion=1
XPos640x480=0
XPos1280x1024=320
YPos640x480=0
YPos1280x1024=272
Show=0
BGTint.red=255
BGTint.green=255
BGTint.blue=255
BGType=1
Fades=true
Delay=2000
Duration=500
Alpha=255
FadeToAlpha=255
Locked=true

Hopefully I'm not posting where I shouldn't be. I'm new here, but it was a definitive fix for me related to the poll window.

[zuperzip]

As of yesterday the last of the 6 accounts I often play received the Survey. I responded "Yes" then clicked "Submit" now none of those accounts get the SSSE. The reason I'm posting this is because of when it happened. I was in game with one of my accounts talking in guildchat and heard about someone losing a character to a random disconnection from PEQ, this seems semi-common the last couple days. As they logged their character back in it received the Survey. Also another person who had been fighting through the SSSEs for 20 minutes or so announced that they also just received the Survey. Quickly I fired up another EQ client and logged in my 'uncured' account. Sure enough I got the Survey. I found none of us had used the Quick Connect.

This was the only time I've heard of more than one player getting the Survey at roughly the same time. It could be it was just random luck but it's worth chasing on a small scale. Here are my untested recommendations.

1. If you receive the Survey quickly notify others in your guild so they may try to do the same thing I did. I'm guessing you've got one or two minutes max to catch the wave. If you find similar results post back here and we'll start announcing in /ooc and IRC.

2. Don't use Quick Connect when attempting to catch the Survey.

Originally Posted by xulok

Is it Sony code that is sending the packet or is it community code?

Xulok, as far as I can tell It's all between the player and the EQEmu login server.

[John Adams]

Originally Posted by a6equj5

Hopefully I'm not posting where I shouldn't be. I'm new here, but it was a definitive fix for me related to the poll window.

You're safe. That's an editable data file, so it should be no more illegal than changing eqhosts.txt If anyone else tries this fix and it works, please let us know. Though chances are, you got lucky. This has been a sinister bug for far too long.