Past, Present and Future

[Doodman]

I just wanted to post about the event over the last few weeks in regards to the stability (or lack thereof) of the site and the login server.

The site and loginserver were running pretty well over the last few months or more until a few weeks ago. Starting the the site was completely unusable and the loginserver was crashing all the time. Seems odd that after months of running well it all fell apart.

The site was being hammered by numerous bots (bad bots, ignoring robots.txt, etc) and was slaughtering the site. I spent a lot of time over the last couple of weeks watching logs, banning IPs and implementing defenses against the attacks. I finally resorted to, as I'm sure you noticed, making the forums require logging to even be viewed. This seemed to turn the tide for the site.

The loginserver was running pretty well for a long time, until someone in the community found a buffer overrun bug in the loginserver and decided to exploit the fact that they could make it crash. It wasn't a random crash. It was crashing in the same spot, from the same user, sending the same information. I hastily implemented a fix to prevent the attack, which ended up fixing the issue but introducing the "incorrect password" issue that was seen for a day or so. That is also now fixed. The login server has been up (except for a restart by me) w/o crash for day in a half. Which, sadly, considering the past few weeks is quite a bit.

During this time (mainly because of the DoS attack) it became painfully obvious that the box we signed up for 2 years about is showing its age. The machine is was undersized as far as CPU and RAM and badly need to be upgraded to newer hardware.

We've asked KLS to investigate a new server (based on donations) and to take the lead in being responsible for the server. The rationale here is several-fold.

1) I don't do anything with EQEmu except some minor maintenance on the server.
2) I'm too busy with other things to give it the attention it needs
3) Since I don't do anything with EQEmu, I'm done footing the bill personally.

I know people are frustrated with the level of service over the last few weeks (or more). Trust me, so am I. I did what I could to keep it under control when it was happening and it seems to have paid off.

I know an alternate loginserver has been offered and, frankly, I don't blame anyone who moves to that server. I think it is a bad idea to have two separate loginservers, but there is nothing I can or will do about it. You are free to make your own choices.

I won't speak for the other admins, but I'm not going anywhere. It's just the time for changing of the guards.

I'll be here if needed and will do what I can to help support the project. Even though I don't work with EQEmu anymore, my heart still belong to EQ and EQEmu. Feel free to PM me or catch me on IRC is you need anything.