Finding Opcodes

**trevius** · #1 07-02-2008, 10:33 PM

Hmm, I may have answered my own question about using the logs to find opcodes. To watch them as they come in, simply open a terminal window in Linux and type:

Code:

cd /home/eqemu/server/logs
tail -f *.* | grep OpCode

You could probably add more stuff to grep for, but this seems to be a start at least lol. I already see some opcodes coming from my dozens of players that are unknowns. Now, if I can get a test server setup so that it is only me that is on, I can start testing and see if my idea works for defining opcodes. There are too many people on my server to use the system for testing without giving them all the boot for a while, so I will have to get a second one running.

Unfortunately my other PC is windows only, and I don't know how to do a tail in windows so I can grep for OpCodes only. So, I may have to build my other PC as dual boot to Debian Linux. Unless I can figure out how to run another server from the same PC I run my main server on. So, I will have 1 that only I can use. Maybe if I make another directory and set the config to use different zone ports... I dunno, but I don't think it would talk right on port 9000 with both server sessions wanting to use the same port to communicate with the login server.

**trevius** · #2 07-02-2008, 11:39 PM

Looks like I might be able to do this on my server even with players on it after-all. This tail shows the name of the char that sent the opcode, so if I am testing, I will just look for my own.

It has already found a few unknown opcodes from my players and these seem to be somewhat common.

OP_Unknowns:
0x6a5f
0x45ff
0x7085
0x3b21
0x1241

I will do some testing tonight to see if I can force it to produce more opcodes and hopefully get some of the unknowns defined. I will let you all know of any progress I make. If this actually works, it is far easier than I could have ever expected lol. Maybe I was thinking too hard, but I imagine that if it was this easy, someone else would have been using this method already...

**AndMetal** · #3 07-03-2008, 12:51 AM

I took a quick peek into the source, and it looks like this is where those OpCode errors you found are spit out:

zone/client_packet.cpp:

Code:

  375 	case CLIENT_CONNECTED: {
  376 		ClientPacketProc p;
  377 		p = ConnectedOpcodes[opcode];
  378 		if(p == NULL) {
  379 			char buffer[64];
  380 			app->build_header_dump(buffer);
  381 			mlog(CLIENT__NET_ERR, "Unhandled incoming opcode: %s", buffer);
  382 			if(app->size<1000)
  383 				DumpPacket(app->pBuffer, app->size);
  384 			else{
  385 				cout << "Dump limited to 1000 characters:\n";
  386 				DumpPacket(app->pBuffer, 1000);
  387 			}
  388 			break;
  389 		}
  390 
  391 		//call the processing routine
  392 		(this->*p)(app);
  393 		break;
  394 	}

You could relatively easily put some code in there to spit out the OpCode info to the terminal (cerr is used elsewhere in the source). Then again, looking just above the above code, someone already has:

Code:

  337 	#if EQDEBUG >= 9
  338 		cout << "Received 0x" << hex << setw(4) << setfill('0') << opcode << ", size=" << dec << app->size << endl;
  339 	#endif
  340 
  341 	#ifdef SOLAR
  342 		if(0 && opcode != OP_ClientUpdate)
  343 		{
  344 			LogFile->write(EQEMuLog::Debug,"HandlePacket() OPCODE debug enabled client %s", GetName());
  345 			cerr << "OPCODE: " << hex << setw(4) << setfill('0') << opcode << dec << ", size: " << app->size << endl;
  346 			DumpPacket(app);
  347 		}
  348 	#endif

So, you could set EQDEBUG=9 to get console output of what looks to be all of the OpCodes. This is done in the zone makefile before compiling:

Code:

   15 DFLAGS=-DEQDEBUG=5 -DCATCH_CRASH -DNO_PIDLOG -DSHAREMEM -DSPELL_EFFECT_SPAM -DFIELD_ITEMS -DCOMBINED -DAPP_OPCODE_SIZE=2 -Di386

If this works, I might put up a VM server to use locally to dig out some OpCodes. It sure beats packet sniffing + creating/fixing a program to decode it all

Then, we'll just need to figure out the packet structures if we don't have them already.

**KLS** · #4 07-03-2008, 06:54 AM

I put some code into the server just to see how the code works further. To analyze live packets I believe you would need something to analyze the entire stream; or at least the session request since right now it appears the client receives a key from the server used to encode and decode.

Right now the emu always sends back: 287454020 which is (0x11223344), I actually added some logging to see what it was since the code has no commenting and can be kinda hard to follow at times, so I guess I got that part right the first time.

Basically:

Client creates protocol packet of OP_SessionRequest

Code:

pragma pack(1)
struct SessionRequest {
	uint32 UnknownA;
	uint32 Session;
	uint32 MaxLength;
};
pragma pack()

Server replies with a session responce

Code:

pragma pack(1)
struct SessionResponse {
        uint32 Session;
	uint32 Key;
	uint8 UnknownA;
	uint8 Format;
	uint8 UnknownB;
	uint32 MaxLength;
	uint32 UnknownD;
};
pragma pack()

Client -> OP_SessionRequest -> Server
Client <- OP_SessionResponce <- Server

the uint32 key is what we use to decode the packet if the flag for encoding (0x04) is set in format, and clearly the session would be the session id, format is the bitfield to store the encode and compression flags for the stream.

I'm still trying to understand it myself though, wtb code commenting -.-.

Oh yeah, also if we're trying to find a client -> server opcode that's pretty simple because you can just have it dump to log or terminal over the emu as stated above but a lot of the ops we're missing aren't client -> server and the ones we are is because the functionality hasn't been implemented and if it were finding the opcodes wouldn't be an issue obviously.

**trevius** · #5 07-03-2008, 07:58 AM

Here is what I have found so far just from running the tail I mentioned above. I am sure some things I have tested must be sending opcodes in another format or something, because they don't seem to send the server anything even though I think they would be unknowns. One example would be when you right click an adventure merchant. Unless the opcode is just assigned to the wrong OP_ name in the .conf file for it already. Then, it wouldn't be getting logged by my tail, because it is currently only able to see opcodes that error for one reason or another.

Errors (unhandled):

0x381d - OP_WeaponUnequip2 - unequip weapon 2 or unequip weapon 1
0x63da - OP_WeaponEquip2 - equip weapon 2
0x6c5e - OP_WeaponEquip1 - unequip weapon 1
0x63da - OP_WeaponEquip2 - equip weapon 1
0x6c5e - OP_WeaponEquip1 - equip weapon 1
0x6f0c - OP_Bandolier - Added and named new bandolier name - same when bandolier is used to swap weapons
0x1ee9 - OP_BazaarSearch - /Bazaar Search or clicking "welcome" on the /bazaar window
0x5891 - OP_RaidInvite - Invited a player with /raid - got some packet info as well
0x6f82 - OP_LFPCommand - Shown on the player when they accepted group invite
0x7f9d - OP_Report - /report playername
0x5306 - OP_Feedback - /feedback and filled out a feedback and hit send. Got some packet as well

I know that unhandled doesn't mean there are problems. But, in some cases, it might be useful to have the notes I made just in case someone decides to write some code to handle them.

Unknowns:

0x524e - Begin /trader mode or Closed /trader window even if Begin Trader mode hadn't been started (on off toggle?)
0x6a5f - Comes in after not moving the mouse for exactly 15 minutes. And comes in again as soon as the mouse is moved again - Toggle Auto-AFK?
0x19d8 - /viewpetition command ran
0x5fc7 - Clicked "View Stats" on the Adventure Window - Same for clicking the Refresh button there
0x230a - Clicked "Leaderboard" on the View Stats Window for LDoN Adventures - Same for clicking the Refresh button there
0x48fe - Used the "Who" button in the friends window - Same for typing /who all friends or friend
0x224c - /zone command
0x35e8 - Right click a player in /becomenpc mode in bazaar (basically trader mode on)
0x3d05 - /veteranReward
0x5eba - Used Shift+T to open the Titles window

And some of these probably aren't considered issues, because there are no systems in place for them. But, I am sure someone would love to get bazaar working, and Titles, and the petition manager window would be nice too. Also, I didn't even know EQ had an auto-afk feature, but that one opcode really seems to be auto AFK. It comes in exactly 15 minutes every time after the mouse hasn't moved on that window and comes in again as soon as it is moved again. That would be a cool feature to have IMO.

Notes:

Starting trader mode by opening the /trader window and setting items for sale in a Trader's Satchel didn't make my character show up on the /bazaar search window to be listed under "Traders". BUT, using /becomenpc on myself changes my name to add "Trader" in front of it the way that trader mode is supposed to on Live. Then, once I had /becomenpc on, I now showed up as a trader in the /bazaar search window. It still wouldn't list the items I had for sale from the search results though.

I will continue looking for more opcodes and I am going to mess around with other versions of EQ as well to see how difficult it would be to get them working for the emu. If most of the opcodes to get another version of EQ working can be attained with the tail I have been running, then it really shouldn't be all too tough. I still need to check into those IDA Pro scripts as well to see what they do and if they are still useful in any way.

I guess some progress is better than nothing. Might as well get as many opcodes in place as possible, so if someone wants to write code to get the systems working, they can do so without worrying about opcodes.

**KLS** · #6 07-04-2008, 04:34 AM

the SWGemu people have a pretty decent writeup of the basic UDP transmission protocol sony uses for apparently all their MMOGs. Including a section on encryption, definitely easier to understand the code after reading it. I think it may differ in the footer a bit from the protocol sony uses in EQ, but it's similar enough to give a good idea of what's going on.

http://trac2.assembla.com/swgemu/wiki/Packets

**KLS** · #7 07-05-2008, 04:41 AM

It's really really close actually, I wrote a small collector today that can identify separate streams and reveal their relevant information. Can't decode packets yet but hey that stuff's hard for an afternoon of work. Trying to devise a system for combined packets doesn't sound very fun at least since they can be combined as many times as the client wants apparently.