NAT problem understood

[theCoder]

I think...

After playing around trying to log into my brand new 0.3.0 install, here's what I've found.

The login server at eqlogin.eqemu.net tends to ignore the worldaddress (from the .ini file), if that's ever sent. It always reports back my external IP to my EQ client (confirmed through packet sniffing).

There are no problems with the EQEmu world or zone servers. At least as far as letting the users log in (I haven't gotten past that point yet). The world server, as far as I can tell, services login requests correctly. The problem is not what's being sent, or where it's being sent to, but where it's being sent from.

Here's what I see is happening:
1) The EQ client correctly gets my world server external IP address from the login server as X. It sends a login request to the world server at X.

2) The world server receives the request, processes it, and returns a response. Since the world server knows that the client is on the internal network, it uses the internal inteface (IP address Y) to send the response (actually, this is Linux's doing, but it's still right).

3) So far so good. But here's the problem. Everquest uses UDP to send data, which is a connectionless protocol. Because it's connectionless, applications that use it have to examine the source address of each packet to determine what the packet is for. So, when the EQ client sends a packet to IP X, it expects a response from X, not Y. I imagine (though I don't know for sure) that the the EQ client is just dropping the packets from IP Y, since it doesn't know that they're really from X.

So what are the solutions?

Running your own login server is one solution, but it's not great. First, you can't do that, since even minilogin isn't being released anymore. Second, that means that you can't connect to the official login server. A stop-gap measure at best

It would theoretically be possible to write a NAT module for EQEmu like the module for FTP. This module would fix the IP address so that it was the internal IP (Y) instead of the external IP (X). A little bit better solution, but hard to do (I don't even know where to start on that)

Perhaps modify the world (and zone?) server so that it always says it's from IP in the send is X, even if it's using the internal interface. I don't know how much work this would entail. The downside to this approach is that it would most likely require that the server be run as root (I don't think normal users would be able to do this).

Change the login server on eqlogin.eqemu.net to return the worldaddress field instead of the connection source. This would mean that external people couldn't play on my server (since my worldaddress is currently Y), but they can't anyway, since port 9000 isn't open externally.


Anyone have any other ideas? I'm not sure what else to do, and right now I can't even play on my own server. I'm going to try to look into the third option, since it seems the easiest for me to do (I think the fourth is the easiest, but I can't do that).

[theCoder]

I think I've found a way to make the world server work... I'll post a patch later tonight when I get some time...

[theCoder]

I believe that this will work. I haven't been able to test it fully (I know that the world modification works, and the zone modification is the same so it should work) because eqemu.net says my server is locked... I'll have to look into this tomorrow...

To install the patch, run from your eqemu directory:
patch -p1 < EQEmu-0.3.0-NAT.patch
(adjusting the file name for wherever it's actually saved, of course).

The patch file only modifes the net.cpp files in the respective world and zone directories. Let me know if there are any problems.

Note: because of limitations of this message board, I had to rename the file to add a .zip extension. It really is a g'zipped tarball of a patch file.

[Trumpcard]

If this works, Im going to buy you a beer...

[theCoder]

The NAT patch does work, at least for me logging into my server. I opened the ports up on the firewall, so others should be able to try it also (though it would have always worked for everyone on the outside). The server name is "theCoder test" and it mentions the NAT patch.

Basically, all it does is bind the daemons to a specific interface instead of INADDR_ANY. This apparently causes Linux to send response packets back through that interface so it picks up the correct from IP address. The world server picks up the IP address from the worldaddress line in LoginServer.ini (the ini parser is buggy, so remove the account= and password= lines from the file to get it to work). The zone server uses the address passed in on the command line. Both servers use gethostbyname, so the address can either be the actual external IP, or a name that resolves to the IP. Note that it has to be the external IP (the one that eqemu.org sees), not the internal one.

[Trumpcard]

Awesome! Works like a champ! Would these changes cause any problems in the windows version, it would be great to get this tested out across the board then get it merged into the source...

1st time Ive been able to logon to my own linux server... Awesome work Coder!!

[theCoder]

I tried to put in the necessary ifdef's to make the code work on Windows (based on other code I saw in the EMU). I haven't tried this, however, but it _should_ work.

It would be nice if this made it into the next release, but if not, I'll be sure to put out a patch file for it.

[Kyraxe]

Hi,

I just switched over my old P2 to Redhat 7, and it will be working as my NAT. Can I also have the EQEmu server on this machine, and if so what do I have to do with your file in order to set this up so that my 3 other home machines can connect to it?


Cheers,
Kyraxe

[Lurker_005]

There is another way around the Nat problem, set your external IP or name that your using to resolve to 127.0.0.1 in your host file

You can set an ip to an ip can't you? or do you have to use a hostname (dynamic hostname if need be)?

Anyhow this is reported to solve the problem without patching. You computer connects like it is all set to 127.0.0.1 and everyone else connects using your external IP.

[theCoder]

To answer Kyraxe's question, there are two solutions that I know of. I think the easiest solution is to use the patch I made (btw, it does still work for 0.3.1, patch just says the lines are off, but it still applies correctly). See one of my previous posts for how to apply it.

If you use the NAT patch, then all you have to do is put your external IP address as the worldaddress (in the LoginServer.ini and as a command line argument to the zone processes). That's all you have to do.

The other solution (which I recently learned from Hogie on the IRC channel) is to use domain names to your advantage. I think this is what Lurker was trying to say, but obviously, you don't really want to use the loopback (127.0.0.1) since you aren't going to be running the EQ client on a Linux machine :). The DNS solution is faily easy to do for Linux computers (/etc/hosts is well defined), but you'll have to hunt around on your Windows box to find it's hosts file (mine is in windowssystem32driversetc, but I've seen it in other places on other installs). The other thing you'll need is a domain name for your IP address (goto www.dhs.org if you don't have one -- you can get a subdomain for free there). What you need to do in this solution is use the domain name as the world address. The login server at eqemu.net will report that name back to the EQ client when it wants to connect (when I wrote the patch, I mistakenly thought it only sent back the IP address). To allow your internal hosts to connect to your server, you need to modify their hosts file to override the DNS setting. So add a line like the following to each internal EQ client machine's hosts file:

int.ern.al.IP domainname.dhs.org

filling in the numbers for the IP address (for example, I use 192.168.1.1) and your correct domain name.

Personally, I think the NAT patch is a little cleaner solution, but I'm biased. Unfortunately, the NAT patch only works if your emu server is running on your NAT computer. People like cduckman (another thread) who want to run their emu on another internal machine would benefit more from the DNS solution. So take whichever solution works best for you.

[Kyraxe]

Thanks for the replies. I am going to try and set up my Redhat machine today and see if I can get this going. =)

[DeletedUser]

Herm, the loginserver should be using the worldaddress (assuming the linux version of world sends it right). This issue is exactally why it was added, so you could use the hosts file to have the dns entry point to your internal address from internal machines.

[theCoder]

Quagmire, I think the Loginserver is working just fine. The reason I thought that it didn't use the worldaddress (besides the fact that it's closed and not open like the rest of this project) is that I tried putting in my internal IP in the worldaddress field, but that didn't work. I later found out that the world server's INI parsing routine is broken for fields that have no data (like account= and password=). Those cause the next item to not be read. So simply removing them from the LoginServer.ini file makes it work. And the DNS solution would (I think) work also.

However, I like pointing my domain name at my external IP address and the computer's name at the internal IP address, so I'm going to stick with the NAT patch for now. Really, having more solutions to a problem isn't a bad thing at all :)

[Kyraxe]

Hey Codeman, I got it setup using the domain name and the hosts file. However I am still unable to connect to my server, I get a 1017 error (and no I don't have a firewall running) Would this be caused by me messing up one of the IP entries somewhere? Any ideas? =)



Thanks

[theCoder]

Kyraxe,

First verify that on the EQ client computer, the name resolves correctly. Go to a command prompt and run 'ping domainname' (domainname being your domain name of course). It should say 'Pining domainname [x.x.x.x]'. Make sure that x.x.x.x is your internal (not external) IP address.

Also, make sure that you don't have any account= or password= lines in your LoginServer.ini file on your server. You don't need them, and last I checked, they screwed up the parser so that it would read the worldaddress field. When you start up the world server, it should say

World server listening on: domanname:9000

If it doesn't then something is wrong.

If both of those check out, then I can't think of anything off the top of my head that's wrong...

[Trumpcard]

Coder,
Are any changes necessary to the NAT server host file? I'm trying out this technique, but havent been successful with it yet.

So, in a nutshell, you just override your external dns name on the client box to your internal ip, thats it?

[TheClaus]

Trumpcard,

I got this to work by doing the following. I changed my HOSTS file in windows to the following.

192.168.1.1 theclaus.kicks-ass.org

Then I opened a cmd prompt and tested it by pinging theclaus.kicks-ass.org. It should come back as 192.168.1.1. Then I used this script file that theCoder made and modified it. After that I tested and been running fine all weekend. Attached is the eqemud script. You'll notice that I have the zones are going to the localhost for the world. That is cause they are on the same machine. It was orginally setup to go to the $PUB_IP on both but couldn't connect on the inside. With this setup both myself and outside people can connect fine. Don't forget to CHMOD 755 file so it will execute. Then when I run my server I do a ./eqemud start

[theCoder]

The NAT patch was written so that people didn't have to change any DNS entries. However, it will only work if and only if the world and zone servers are running on the NAT server. The NAT patch makes the EMU listen on only the external interface (actually, whichever interface you specify, but you want the external one). NAT servers have to have at least two interfaces, but the login server can only record and give out one of those. If the world server specifies to the login server that it is on the internal one, external clients cannot connect. If the world server says it's on the external interface, internal EQ clients get confused because of how the UDP packets are returned when the EMU listens on both interfaces. By forcing the EMU to only listen on the external interface, the UDP packets get formed the way the EQ client expects them and everything works.

What the NAT patch won't help with is an EMU server behind a NAT server. The only way I know of to solve that is with port forwarding on the NAT server and DNS work on the EQ clients (note that you shouldn't need to mess with the hosts file on the NAT server).

A better solution would be to have a separate entry in the conf file specifying the bind address the EMU should use (0.0.0.0 will bind to all interfaces). As soon as I get some time (I'm in the midst of moving across the country right now, so I shouldn't even be reading this board :)), I'm going to look into that, and also look into setting up autoconf and making a more "linux friendly" version.

[Trumpcard]

Very cool, thanks... I'll try it out again when I get in this afternoon. Nice thing about this script is that it will fit nicely
into rc3.d or /etc/runlevels.

Just got finished installing gentoo linux this weekend on my primary partition (left RH7.3 on my secondary).

I can't begin to tell you how much faster it is! My KDE performance is probably 50% faster. It helps to recompile glibc and kde with the newest binutils (2.12+ that has the new combreloc added in).

I pretty much built everything from scratch, you'd be amazed at having things compiled from the ground floor with the right compiler options. (i686 makes a huge difference, most everything these days in prepackaged formats ends up with i386).

It took me no time to get the majority of my functionality shifted from my RH partition over to the gentoo one. I may end up moving everything over eventually. For now dual boot will work, but I have a feeling that she will be going to way of the dodo before too long. The emerge portage system is really cool too, it makes system updates a snap. Give the distro a shot if you guys get the chance, I bet you won't be disappointed. I had showeq back up and running in a couple of hours after got the system stable. (had to rebuild my kernel a few times).

I was able to port my websites, database servers, mail configuration over in the course of a days time.

Its definitely easier getting showeq running correctly on gentoo than it is on RH7.3 (I was having issues with the new one colocating gcc 3.0.4 and 2.96)

I got the tip about Gentoo from Ratt, I'll have to send him a thank you note, this distro rocks! One odd thing, that I've actually come to like is they do not provide a telnetd daemon, it's sshd or nothing baby.. Makes it harder for me to get to it from work where we only have a telnet proxy, but I can just telnet to my prod boxes then ssh from there.

Very slick...

[theCoder]

Thanks for the heads up on gentoo. If I ever get some new hardware, I'll give it a try.

I'm not sure why you can't ssh out of your work box, though. I can't imagine that someone would port block outgoing ssh connections but allow outgoing telnet connections, but I guess stranger things have happened... If that's the case, set up sshd to listen on port 23 (the telnet port) and then tell your ssh client (I'd recommend PuTTY on windows) to connect on port 23.

One thing you shouldn't do is ssh from a telnet connection. Anyone can listen in on the telnet side almost eliminating the security of ssh (unless the telnet connection is on a secure line).

[Trumpcard]

Well, we only have a telnet proxy server, you actually telnet to the proxy, then you telnet from it, you're not being transparently passed through it like you would through a http/ftp proxy..

Makes me miss the old days at IBM of having a socks-i-fied connection...

[btuch]

Does anyone have a NAT patch for the 0.3.7 or 0.3.8 patch? Been looking through the source and it looks like the entire section was re-written.

[TheClaus]

You shouldn't need it. If you look at the entire post it shows how to get it working without the patch. The way I have done it is to put theclaus.kicks-ass.org (pub_ip) into my HOSTS file under Windows and then pointed it to my internal ip on my network 192.168.1.1

That should work.

[btuch]

TheClaus,

I have all the servers set to run on my external IP, on a Linux box runnin NAT/MASQ. Set up this way, others can connect to my outside IP, but I cannot connect from my internal IP. I always get an error in the log files that I have been disconnected....

From TheCoder's post:

The NAT patch was written so that people didn't have to change any DNS entries. However, it will only work if and only if the world and zone servers are running on the NAT server. The NAT patch makes the EMU listen on only the external interface (actually, whichever interface you specify, but you want the external one). NAT servers have to have at least two interfaces, but the login server can only record and give out one of those. If the world server specifies to the login server that it is on the internal one, external clients cannot connect. If the world server says it's on the external interface, internal EQ clients get confused because of how the UDP packets are returned when the EMU listens on both interfaces. By forcing the EMU to only listen on the external interface, the UDP packets get formed the way the EQ client expects them and everything works.

I am having that very problem, where the world server seems to be bound to both my internal and external IP addresses. This is why I'm looking to patch the code. However after going through the source, it looks like a lot has changed since v0.3.0. Hopefully someone can post a patch, or some instructions on where to place code to get the world server to bind to only the IP address specified.

Its not a DNS issue, but an IP issue with the world server binding to both IPs I think.

-Brian

[Trumpcard]

Heres my take on it...

The code should support binding to an interface, rather than using INADDR_ANY, this is a case thats going to happen. 50% or more of linux users are going to be using their linux box as a firewall/masq. box and trying to play from the internal lan. It also cuts off other potential binding issues that could happen when using multihomed machines.

I'd like to see a command line option passing in which interface to use into zone on startup, should be a simple change combined with the changes Coder made.

On the otherhand, there is a workaround, even though I wasnt able to get it to work, you should try it first. I still think that this is a valid change to put in that stems off potential problems in the future though.

[btuch]

Trumpcard,

I see your post, I'll try the script when I get home.

Thanks!

-Brian

[Trumpcard]

Read the whole thread, the instructions are in there...

Basiclly, you give your linux box a dns name from a free server (dyndns.org , freedns, etc). The you change your windows host file on your internal network to map

LINUX DNS NAME INTERNAL IP

So internal requests to your masq box will go to the internal interface rather than going out and back in.

[btuch]

Didn't work.

I can connect to the login server, but then the world server log gives "Bad/expired session key: ls#1".

So....hopefully someone can come up with the patch for the >=0.3.8 to tell the world server to bind specifically to 1 IP, and not both.

-Brian

[TheClaus]

I'll setup a linux server tonight and see if I can get it to work. Basically what Trump and I said should work though.

[btuch]

I think I found the area to put TheCoder's patch into the 0.3.8 release...

In the file ~/world/console.cpp for the line:

address.sin_addr.s_addr = htonl(INADDR_ANY);

and put a // in front of it.

Then below that line paste the lines (without the --snip--):

--snip--
// bind only to the address specified as the world address
hostent* bindaddr = gethostbyname(net.GetWorldAddress());

if (bindaddr != NULL)
{
#ifdef WIN32
memcpy ((char FAR *)&(address.sin_addr), bindaddr->h_addr, bindaddr->h_length);
#else
memcpy ((char*)&(address.sin_addr), bindaddr->h_addr, bindaddr->h_length);
#endif
}
else
{
// could not resolve world address... bind to all interfaces then
cout << "Error: could not resolve world address " << net.GetWorldAddress() << endl;
cout << "Listening on INADDR_ANY" << endl;
address.sin_addr.s_addr = htonl(INADDR_ANY);
}
--snip--

I tested this, and the world server will now bind to the IP specified in the LoginServer.ini

The only changes I made were I took out the references to "this->" and added in "net.".

I'll test it when I get home from my internal LAN.

-Brian

[btuch]

woot!

Adding in the code above allowed me to connect from outside and inside my firewall....

/cheer

-Brian

[TheClaus]

Okay now explain what you have in your loginserver.ini and what your zone ips are pointing too. I am curious and would like to try this out myself.

[btuch]

I am running Login/World/Zone all on the Linux box....

Where it says "my.domain.com" is where my domain name is, which is a public IP.

LoginServer.ini:
--snip--
[LoginServer]
loginserver=my.domain.com
loginserver2=my.domain.com
loginport=5999
loginport2=5999
locked=false
worldname=Testing
worldaddress=my.domain.com
account=
password=

[WorldServer]
Defaultstatus=Public
Unavailzone=

[LoginConfig]
ServerMode=Standalone
ServerPort=
UplinkAddress=
UplinkPort=
UplinkAccount=
UplinkPassword=
--snip--

And in the eqemud script from before, the function launching the zone servers looks like this:

--snip--
PUB_IP=my.domain.com

# syntax startzone zonename port
startzone() {
echo -n " Starting $1: "
#should this not specify the zone name anymore?
./Zone '.' $PUB_IP $2 $PUB_IP > $1.log 2> $1.errors &
echo_success
echo
}
--snip--

Forgot...my eqhosts.txt file points at "my.domain.com:5999" for both inside and outside the Linux box.

-Brian

[Trumpcard]

How are you running a loginserver on linux?

[btuch]

Ah,

I downloaded the old login source posted on www.eqemu.org.

It is the login source....just an old version.

The only problem I have with it is...sometimes it takes several tryes to get into the world server, because the world log says:

Bad/expired session key: ls#1

I am looking at the source to try to fix this.

It usually takes about 2-10 tries for the world server to get in. And to create an account I followed the instructions from the .txt file with the Login binary for windows. My Linux box is also running the MySQL server.

-Brian

[Trumpcard]

Maybe we can convince image and co. to merge this change into the baseline so we don't have to reapply it. Verify that it doesnt cause any problems on the windows version, and I think it's a good solution.

Image, any chance of this happening? Pulling in this change will make it more flexible, I would think this would keep the problem from occuring on the windows side as well. No more worries about where your putting your server, and where youre trying to log in from.

[btuch]

I shot an email off to image about it, hoping he would do the same you suggested ;)

-Brian

[Trumpcard]

I hated it never got rolled in when theCoder worked on it the first time, it was a problem that was driving me nuts, and I never could get the dns override technique to work right for me. Its a simple change, just didnt work. I had a weird shit development server at the time, so no telling.

I'll harass him about it on IRC when I get on tonight. You ought to jump on IRC and join up with the dev team Btuch, you've obviously got some code saavy! The more the merrier!

[btuch]

The only problem I've run into so far is I cannot "telnet my.domain.com 9000" from the machine now to get into the console for the world server. Dunno why. Gotta look at it some more. It takes the login/passwd but then says its wrong...

I'll try to pop in there tonight or Friday ;)

-Brian

[penfold1972]

Silly question, but is your login/password in the account table of MySQL? I seem to always forget to add it when I drop my database and rebuild. BTW, a decent GUI for mySQL and Gnome can be found at: http://www.gnome.org/softwaremap/projects/gmyclient

It's a far cry from character editing and other EQEMU functions available in the admin tool, but for some easy database tweaking (clearing tables, data edits, etc.) I've fallen in love with it.

[btuch]

ahhhhh!

Thanks, fergot to check there. Some how the password field for the accound got cleared.

Can now log into the world server console.

I have always been partial to http://www.phpmyadmin.net/ ;)

Thanks for the tip!

-Brian

[penfold1972]

Yeah, I have some projects to start playing with the phpmyadmin and phpmyedit stuff... just kinda buried in other little projects at the moment. I gave a halfhearted attempt to get it running a couple of weeks ago, but I ran into some glitches, so I went back to my MySQL projects. Who said Attention Deficet Disorder was a bad thing? :)

[btuch]

Image and the other Devs,

Can you please add this into the world code if possible so the world server only binds to one IP address?

In the file ~/world/console.cpp for the line:

address.sin_addr.s_addr = htonl(INADDR_ANY);

and put a // in front of it.

Then below that line paste the lines (without the --snip--):

--snip--
// bind only to the address specified as the world address
hostent* bindaddr = gethostbyname(net.GetWorldAddress());

if (bindaddr != NULL)
{
#ifdef WIN32
memcpy ((char FAR *)&(address.sin_addr), bindaddr->h_addr, bindaddr->h_length);
#else
memcpy ((char*)&(address.sin_addr), bindaddr->h_addr, bindaddr->h_length);
#endif
}
else
{
// could not resolve world address... bind to all interfaces then
cout << "Error: could not resolve world address " << net.GetWorldAddress() << endl;
cout << "Listening on INADDR_ANY" << endl;
address.sin_addr.s_addr = htonl(INADDR_ANY);
}
--snip--

Thanks!

[Trumpcard]

Good solution... If it doesnt find the world address, resort to any incoming..

Image, dont make me beg here.. lol.... I finally got my linux server rebuilt, I want to actually run a server again and be able to log onto it !

[Trumpcard]

Hmmm.. This code change doesnt work for me quite.. The world server reports it's listening on that right interface, but still doesnt get through..

There are at least 2 other places than specify INADDR_ANY,

TCPConnection.cpp and EQNetwork.cpp .

I think they need to be modified to bind properly as well.

[btuch]

Hmmm...I'll have to look at that. What I posted fixed the NAT problem I was having. I'll look at the code and see if I can modify it, unless you would like to; which might be better since you would be able to tell if it fixed your problem ;)

I'll look at it anyways.

[Trumpcard]

Originally, the only place it would bind the interface was in the 2 seperate net.cpp files. After version 0.3.0, these binds were yanked out of net.cpp and moved to console, EQNetwork, and TCPConnection. This looks like some merging that Quagmire did but not sure. I would think that both the zone and worldservers would need to bind to the interface, console seems to be where world does, but where is zone getting its connection?

Its not quite as easy to move the logic into EQNetwork and TCPConnection as it was console, so I need to check what additional includes are needed to get it in. (If this is really even the problem).

The behavior Im seeing is that once you login, and try to pick the worldserver in the server list, I'll get 10-15 Logged in: LS#106 Trumpcard messages, this is happening in client.cpp. The login in wrapped in a big while with a switch.. The log on is never completing so it never gets to sending the list of guilds..