Lets Make a Mega Server!

Glasswalker · #1 11-05-2002, 01:06 AM

Question:

Why don't we all try and organize a mega distributed server system? with even 10 computers distributed over the net we could run 2 zone servers a piece and have a huge server farm running (in comparison to the current offerings) hehehe... if we could get some kind of permenant distributed server system up, and allow people to sign up as zone servers and link in, anyone running a zone server stable for more than a week gets gm status... we could work out the details later, all I am saying is that it would be easy with this emu design to develop a distributed network... if everybody that runs a server now could dedicate 2 zone servers we could have a huge server running... even if they could pitch in 1 zone server, the machine would be able to handle many users per zone, and we could still have tons of zones up...

if anyone is interested in this send me an email at: glasswalker@cogeco.ca

I would like to hear who might be interested...

oh and I have about a dozen friends that would run servers if they see some interest first... the problem is they don't want to be the first... so if we can pull together the server will grow, and we will attract more users too

TheClaus · #2 11-05-2002, 02:16 AM

Not sure if that would work because of the database server being hit across the network. Good luck though.

Glasswalker · #3 11-05-2002, 02:27 AM

Possibly if mysql could be set up to run on each of the servers running... But to simply replicate changes, similar to the way that windows 2000 domains replicate the active directory over a WAN link...

DeletedUser · #4 11-05-2002, 04:01 AM

You could properly startup the zones, it would take a while, you send about 15 megs of information, but I believe there is a bit of information when the zone first starts up that stalls the server (Which then the client disconnects because it does not want to wait)

TheClaus · #5 11-05-2002, 04:59 AM

15 megs. GOOD GOD!!!

I will be happy to have my own private eq for friends.

DeletedUser · #6 11-05-2002, 05:17 AM

items are 360 bytes each, 32000 items, 11520000 bytes roughly 11.25 megs just right in items. Funny thing is, at one time we use to send it to each zone, now emumemshare only requests it once, made a big difference

Trumpcard · #7 11-05-2002, 06:02 AM

So, what the hell is IPCs ?????

DeletedUser · #8 11-05-2002, 06:52 AM

You only know if you were in the chat room last night

Glasswalker · #9 11-06-2002, 01:04 AM

Yes I know the db data the zone file requests is huge... but what if there was a mysql server on each zone server... that way there is a copy of the database locally, no network based copying... and changes to the database are replicated to all the zone servers to the world server and back again, to sync up the data every 5 minutes or something... and if only the changed data is synced that keeps traffic to a minimum... This is how win2K handles active directory shares between domain controllers, it can't have every user authenticating and such back to a single server, the WAN traffic would be huge... so it has local domain controllers for each lan, and these sync up the changed data with one another every once in a while... The active directory can sometimes be as big as 50-100meg but it replicates fine over dialup with no real big performance hits... if microsoft can do it I am sure we can at least make a stab at it?

Trumpcard · #10 11-06-2002, 01:54 AM

Theoriticially it wouldnt be hard to set up, there are a few ways to do it.

1st, start off with 2 servers, each using the exact same database, each running 2 zones a piece. With mysql clustering/replication, you could keep these in sync, but this would be a bit slow, one acts as a master, the other a slave, so data updated on the slave might not be immediately available on the master. The better solution would be a 3rd log database that acts as a replicator, it gets posted the sql transactions on the other servers, which get immediately sent to the other servers in the clusters, so its pretty immediate in terms of data transfer. The mysql pipe is pretty thin, so a few mysql write transactions a second arent going to kill you in terms of bandwidth. Especially if you consider what changes in mysql, the data that is going to change is going to be player related mostly, so thats about the only data you need to pass around. You're not going to be passing around npc data, zone, items, etc, all that will be replicated initially when you set up the cluster. You could change those directly in the log server, and push it out to the servers. This kind of replication is pretty common.

Heres some information on a simple master/slave replication.
http://www.ls-l.net/mysql/

Its shouldnt be hard to set up, and it does have the benefit of allowing you to focus all your bandwidth on a single zone or two.
If the zoneserver is optimized well enough, it should be able to handle the load of 20+ players in a zone, but thats just an educated guess.

The hardest part would be getting the people together at the same time, and getting them organized. It's like herding cats...

Glasswalker · #11 11-06-2002, 03:56 AM

Interesting... I was pretty sure you could do replication in SQL...

This is good... so it would not be a huge bandwidth hog on the servers to do this...

if you organized it as:

1 machine for worldserver
1 machine for sql master
and then 1-2 zones per member server from there in...

Interesting... we will have to look into this further

Oh, and also... have you ever actually tried to herd cats? it isn't easy... hehehehe... it is funny... but not easy...

I figure if it is realistic to do, we could get a domain name up for the server, and a basic website... then hand out subdomains to "members"

People would have to take a 1 week "application" phase to get member status... during that time a simple script is run on the server to ping the host and simulate bandwidth spikes over time... as long as the host stays reliably up for a week or so, then they can join the serverfarm as member... after another week as member server, they then (if stable) get Lead GM status...

This way we can weed out the newbies that want to join in for the fun of it but can't run a stable server...

If this works it could be kickass...

Combine that with a new db update, and I almost have my db filter utility ready which will accurately filter out all dupe spawns automatically...

I think realistically by the time basic pathing is done on the emu, we could also have a 90% complete DB filtered out for dupes, and running on a decent 24/7 serverfarm for everyone to enjoy!

Now we need to rally some supporters

hehehe...

here kitty kitty kitty

hehehehehe

Trumpcard · #12 11-06-2002, 05:16 AM

Actually, the best way to do it would be each mysql server configured as a dual master/slave.

They would push data around to one another... Each would be set as a slave to all of the other masters, and each of the other masters would be one of it's slaves. The connection map on this thing would look like something from a spirograph....

The more you have, the more bandwidth your going to consume in the traffic obviously, but until you get past 4 or so, I don't think it would be to bad.

Thats pure conjecture though, but once again, pretty minor cofiguration setup change on the part of the sql servers.

PhunkyBeard · #13 11-11-2002, 08:05 AM

ooo... cool...

Zetareticuli · #14 11-12-2002, 07:42 PM

That would take determination and teamwork, but in the long run would be worth it. I am all for it and will contribute as much as possible. Just let me know.

Trumpcard · #15 11-12-2002, 11:10 PM

Once there is a good, clean database that everyone agrees on as a base, I'd be willing to try it out. Im using it like that now on my linux/windows machines, I run a mysql server on my windows box, and its easier for me to replicate the linux server than it is to connect to my linux mysql server using eqadmin.