Starting Work on SoF Opcodes/Structs

[trevius]

Well, I guess I was looking too hard for the spawn struct size lol. I simply converted the 897 to HEX, which is 381, and then searched the SoF eqgame.exe code for 381 and found this:

.text:00481860                 mov     eax, [esp+arg_4]
.text:00481864                 sub     esp, 80h
.text:0048186A                 cmp     eax, 381h
.text:0048186F                 push    esi             ; char
.text:00481870                 jz      short loc_4818DC
.text:00481872                 push    offset aErrorCorruptAd ; "ERROR: Corrupt addplayer, disconnecting"...
.text:00481877                 call    sub_645680
.text:0048187C                 mov     eax, dword_98AF10
.text:00481881                 add     esp, 4
.text:00481884                 mov     esi, 1
.text:00481889                 mov     [eax+38E98h], esi
.text:0048188F                 mov     ecx, dword_98AF10
.text:00481895                 push    0FFh
.text:0048189A                 call    sub_4BF9E0
.text:0048189F                 mov     ecx, dword_98AF10
.text:004818A5                 mov     dword_925C8C, 106h
.text:004818AF                 mov     [ecx+5C0h], esi
.text:004818B5                 mov     ecx, dword_761C6C
.text:004818BB                 push    offset dword_907EF0
.text:004818C0                 call    sub_43D670
.text:004818C5                 push    offset aErrorInReceive ; "ERROR: in receive add player, LEN failu"...
.text:004818CA                 call    sub_645680
.text:004818CF                 add     esp, 4
.text:004818D2                 xor     al, al
.text:004818D4                 pop     esi
.text:004818D5                 add     esp, 80h
.text:004818DB                 retn

And, when I changed the Titanium spawn structure size and logged in, it didn't crash, but it did time out and put out this debug log from the normal EQ debug file:

2008-12-15 06:08:03	Zone Connect -- 2 -- Sending MSG_EQ_ADDPLAYER
2008-12-15 06:08:06	Zone Connect -- 3 -- Received MSG_SEND_PC
2008-12-15 06:08:06	Zone Connect -- 4 -- Received MSG_EQ_ADDPLAYER
2008-12-15 06:08:07	ERROR: Corrupt addplayer, disconnecting.
2008-12-15 06:08:07	
2008-12-15 06:08:07	Networking: Connection Closed [0] with 0 pending bytes.
2008-12-15 06:08:07	ERROR: in receive add player, LEN failure.
2008-12-15 06:08:07	
2008-12-15 06:08:07	disconnected at C:\EQ\EverQuest\EverQuest.cpp:17233 (char. select) g_world == NULL
2008-12-15 06:08:07	
2008-12-15 06:08:07	Character is Trevazar.
2008-12-15 06:08:07	
2008-12-15 06:08:10	nError is TRUE at C:\EQ\EverQuest\EverQuest.cpp:20261

So, it sounds like those guys at SEQ might be correct with that spawn file after-all. Now, I just need to figure out what is being done on SoF that is causing a crash. Anyone know of a debug program that could be run on Everquest?

[OscarGrouch05]

Anyone know of a debug program that could be run on Everquest?

Yes i use w32dsm89
will allow you to read it in asm lang plus debug the program all in one.
send email addy and i'll be happy to link ya to them don't want to post
anything againts the rules.
What i'm doing is looking at the code with
w32dsm89
etu-dasm-32/16 bit disassembler v 2.22 alpha
i really think etu-dasm would help you out
more as it will allow you read more info
in english so you'll understnad it better.

i been following up on what your doing.
what i'm intrested in is getting the expanshions
to read 15-15 etc. i noticed your not letting
us know what ver of peqserverpack.. 80) as well i
change the opcode OP_EnterWorld=0x7cba
put in patch_Anniversary.conf and opened
fos got into char slect screen.
i was using the newest ver
ActivePerl-5.10.0.1004-MSWin32-x86-287188
PEQUpdatePack-4.0-1129Rev233
mysql-5.0.51a-win32

[trevius]

When you say you want expansions to display 15 out of 15, do you mean at the server select? I don't think there is anything we can do about that. I am pretty sure we would have to adjust the login server source code to fix that. That isn't an option, because no one has access to it that is active around here these days. Titanium shows the wrong number of expansions, and SoF shows 0 of 0 expansions. Luckily, that has no effect on how the actual server handles your connection, it is just a display thing.

The version of code or database I am running doesn't really matter, but I am using one of the latest revisions from the SVN, R238. My database is about a year old PEQ one that has been updated by me for my custom server. The changes I am making should be able to work on any version once they are all done. If I can get it working, I will submit any needed changes for database tables once we get to that point, but for now, I am doing everything without relying on database changes.

My current status is that I worked all night last night trying to duplicate the same crash on Titanium by throwing off the structure format by increasing or decreasing the size of certain pieces of the playerprofile structure. I couldn't once get it to crash the way that SoF is, and it actually made it in game almost no matter what I changed, but it did make things wrong like plat, levels, etc, because the structure info wasn't aligned properly. I also tried to remove all of the opcodes in Titanium that I don't have correct for SoF yet, and that didn't cause a crash either.

The spawn struct seems like it should be ok, and the player profile looks like it should be very close to correct, so I am still trying to figure out what is causing the crash. I am wondering if something else was added to the playerprofile struct that we don't know anything about. There were 1032 packets added to the end of the structure sometime between the Titanium to Anniversary time period. I have no idea what that 1032 is supposed to be for, but maybe that is the cause of the crash, since we aren't sending anything at all for it.

[trevius]

After spending hours working to get to the next step towards getting in game, I finally made some progress.

[Fri Dec 19 06:01:40 2008]00131:Zone Connect -- 0 -- Received MSG_ZONE_ADDRESS
[Fri Dec 19 06:01:40 2008]00132:Zone addr [192.168.1.102:19997] received...
[Fri Dec 19 06:01:41 2008]00133:ZONING
[Fri Dec 19 06:01:41 2008]00134:Networking: Connection Closed [0] with 0 pending bytes.
[Fri Dec 19 06:01:41 2008]00135:Networking: using port [1969].
[Fri Dec 19 06:01:41 2008]00136:Networking: Connection Established [1]
[Fri Dec 19 06:01:41 2008]00137:Connected to 192.168.1.102:19997...

[Fri Dec 19 06:01:41 2008]00138:Zone Connect -- 2 -- Sending MSG_EQ_ADDPLAYER
[Fri Dec 19 06:01:43 2008]00139:Zone Connect -- 3 -- Received MSG_SEND_PC
[Fri Dec 19 06:01:43 2008]00140:Zone Connect -- 4 -- Received MSG_EQ_ADDPLAYER
[Fri Dec 19 06:01:43 2008]00141:Received our Player from zone. MSG_EQ_NETPLAYERBUFF is next.
[Fri Dec 19 06:01:43 2008]00142:Player = Trevazar, zone = Loading
[Fri Dec 19 06:01:43 2008]00143:MSG_TIME_STAMP received.

[Fri Dec 19 06:01:43 2008]00144:MSG_TIME_STAMP received. (Items inc).

[Fri Dec 19 06:01:51 2008]00145:fatal error in main thread Code = c0000005  ADDR=0x004e59f9

[Fri Dec 19 06:01:51 2008]00146:EAX=22447600  EBX=22447600  ECX=22447600  EDX=00000001

[Fri Dec 19 06:01:51 2008]00147:ESI=0013a53c  EDI=224476e0  EBP=30347c39  ESP=0013a4f0

[Fri Dec 19 06:01:51 2008]00148:Raw Stack

It turns out that it was the spawn struct causing that after-all. The EQ debug info isn't very helpful (but much better than nothing!), because it appeared to be crashing at the point of getting the player profile. After messing with the player profile, no matter what I did to it, I couldn't duplicate the crash on Titanium.

I think I am probably only a couple smaller structs away from getting in game now. All of the big ones should be ok enough to get in at least, I think. On to solving the next crash point problem :P

[trevius]

Just to test, I made a new level 1 character (on a Titanium Client) and deleted all of his items and attempted to log him in with SoF. So far, I got further than ever

[Sat Dec 20 04:48:12 2008]00129:Initializing character select UI.
[Sat Dec 20 04:48:12 2008]00130:Resetting game UI.
[Sat Dec 20 04:49:01 2008]00131:Zone Connect -- 0 -- Received MSG_ZONE_ADDRESS
[Sat Dec 20 04:49:01 2008]00132:Zone addr [192.168.1.102:20253] received...
[Sat Dec 20 04:49:01 2008]00133:ZONING
[Sat Dec 20 04:49:01 2008]00134:Networking: Connection Closed [0] with 0 pending bytes.
[Sat Dec 20 04:49:01 2008]00135:Networking: using port [1574].
[Sat Dec 20 04:49:01 2008]00136:Networking: Connection Established [1]
[Sat Dec 20 04:49:01 2008]00137:Connected to 192.168.1.102:20253...

[Sat Dec 20 04:49:02 2008]00138:Zone Connect -- 2 -- Sending MSG_EQ_ADDPLAYER
[Sat Dec 20 04:49:12 2008]00139:Zone Connect -- 3 -- Received MSG_SEND_PC
[Sat Dec 20 04:49:12 2008]00140:Zone Connect -- 4 -- Received MSG_EQ_ADDPLAYER
[Sat Dec 20 04:49:12 2008]00141:Received our Player from zone. MSG_EQ_NETPLAYERBUFF is next.
[Sat Dec 20 04:49:12 2008]00142:Player = Ztestius, zone = The Nexus
[Sat Dec 20 04:49:12 2008]00143:MSG_TIME_STAMP received.

[Sat Dec 20 04:49:12 2008]00144:MSG_TIME_STAMP received. (Items inc).

[Sat Dec 20 04:49:12 2008]00145:Item done, MSG_WEATHER_EVENT received.

[Sat Dec 20 04:49:12 2008]00146:Initializing zone.
[Sat Dec 20 04:49:12 2008]00147:Initializing world.
[Sat Dec 20 04:49:12 2008]00148:Verifying world files.
[Sat Dec 20 04:49:12 2008]00149:Attempting to load nexus.EQG.
[Sat Dec 20 04:49:12 2008]00150:Loading zone specific files.
[Sat Dec 20 04:49:12 2008]00151:Loading nexus_obj2
[Sat Dec 20 04:49:12 2008]00152:Loading nexus_obj
[Sat Dec 20 04:49:12 2008]00153:Loading nexus_chr2
[Sat Dec 20 04:49:12 2008]00154:Loading nexus_chr
[Sat Dec 20 04:49:12 2008]00155:Loading nexus
[Sat Dec 20 04:49:13 2008]00156:Loading objects
[Sat Dec 20 04:49:13 2008]00157:Loading lights
[Sat Dec 20 04:49:13 2008]00158:Initializing cameras.
[Sat Dec 20 04:49:13 2008]00159:Initializing lights.
[Sat Dec 20 04:49:13 2008]00160:Initializing visual effects.
[Sat Dec 20 04:49:13 2008]00161:Initializing target indicator.
[Sat Dec 20 04:49:13 2008]00162:Initializing player path.
[Sat Dec 20 04:49:13 2008]00163:Performing post-load operations.
[Sat Dec 20 04:49:13 2008]00164:Initializing precipitation system.
[Sat Dec 20 04:49:13 2008]00165:World initialized: nexus
[Sat Dec 20 04:49:13 2008]00166:Requesting zone data.
[Sat Dec 20 04:49:13 2008]00167:Resetting cameras.
[Sat Dec 20 04:49:13 2008]00168:Resetting vision.
[Sat Dec 20 04:49:13 2008]00169:Resetting overlays.
[Sat Dec 20 04:49:13 2008]00170:Zone initialized.
[Sat Dec 20 04:49:13 2008]00171:Creating INI files.
[Sat Dec 20 04:49:13 2008]00172:Requesting AA data.
[Sat Dec 20 04:49:13 2008]00173:Requesting Skill rank data.
[Sat Dec 20 04:49:13 2008]00174:Loading UI.
[Sat Dec 20 04:49:13 2008]00175:Deactivating previous UI.
[Sat Dec 20 04:49:13 2008]00176:Unloading previous textures.
[Sat Dec 20 04:49:13 2008]00177:Resetting UI.
[Sat Dec 20 04:49:13 2008]00178:Resetting character select UI.
[Sat Dec 20 04:49:13 2008]00179:Resetting game UI.
[Sat Dec 20 04:49:13 2008]00180:UI Reset.
[Sat Dec 20 04:49:13 2008]00181:Initializing UI.
[Sat Dec 20 04:49:13 2008]00182:Initializing UI manager.
[Sat Dec 20 04:49:13 2008]00183:Initializing fonts.
[Sat Dec 20 04:49:13 2008]00184:Parsing UI XML.
[Sat Dec 20 04:49:13 2008]00185:Parsing INI file .\UI_Ztestius_stormhaven.ini
[Sat Dec 20 04:49:13 2008]00186:INI file .\UI_Ztestius_stormhaven.ini loaded.
[Sat Dec 20 04:49:14 2008]00187:Loading default UI resources.
[Sat Dec 20 04:49:14 2008]00188:Initializing game UI.
[Sat Dec 20 04:49:14 2008]00189:Parsing INI file .\Ztestius_stormhaven.ini
[Sat Dec 20 04:49:14 2008]00190:INI file .\Ztestius_stormhaven.ini loaded.
[Sat Dec 20 04:49:15 2008]00191:Loading Icons.
[Sat Dec 20 04:49:16 2008]00192:Resetting viewport.
[Sat Dec 20 04:49:16 2008]00193:Game UI Initialized.
[Sat Dec 20 04:49:16 2008]00194:Entering main loop.
[Sat Dec 20 04:49:16 2008]00195:DoMainLoop - Just after packet processing()
[Sat Dec 20 04:49:16 2008]00196:DoMainLoop - Just after actor creation()
[Sat Dec 20 04:49:16 2008]00197:Loading game sounds.
[Sat Dec 20 04:49:16 2008]00198:Requesting initialization data.
[Sat Dec 20 04:49:16 2008]00199:DoMainLoop: just before first while(!EverQuest.ReceievedWorldObjects).

I think I just need to track down a few more opcodes and I should be able to at least get a naked level 1 in game. Then, I will have to work out if it is AAs or Items or something else causing the crash when I try to log in my GM character that I have done most of the testing with. Still making progress, and getting much better at finding opcodes in the assembly code. I still can't find every one of them using the comparing method, but I can probably find about 50% that way

[Yeormom]

Trevius is our hero.

[trevius]

While it is good to know some people are following this post, I think it would be best to keep the clutter down if possible. I am hoping that I can get it working enough that others might be able to start assisting me at some point and it is easier to read if there is less clutter. I do appreciate the cheering on (it is alot of work so far), but this thread isn't really the place for it. Besides, until it is at least somewhat usable, there isn't much to cheer about. There is a good chance that we may never have enough info to get SoF fully functional. I am trying to stay positive about it though lol.

I have already learned a ton over the past couple of weeks just playing with the different sources I have available to me to get them all frankenstiened into something that will at least let us log all of the way in. If I could read the assembly code better, it may eventually be possible to pull everything we need straight from there, but I am still a good ways from being able to understand it enough to do that.

Currently, I am trying to understand the item serialization code. I have the full itemlist for SoF, and I think it should be very close to accurate. But, until I know how to set the serialization up so that it works with my new list, I can't load items. Other than that, I already have a large amount of the bare minimum opcodes needed to log in. There are only a few more I need to have a complete list. Here is what I have so far:

Opcode Name=Titanium Op - SoF Op
OP_SendLoginInfo=0x4dd0 - 0x6c3c V
OP_GuildsList=0x6957 - 0x04FB V
OP_LogServer=0x0fa6 - 0x129A V
OP_ApproveWorld=0x3c25 - 0x7FC0 V
OP_EnterWorld=0x7cba - 0x1340 V
OP_PostEnterWorld=0x52A4 - 0x1AEE V
OP_ExpansionInfo=0x04ec - 0x0A1B V
OP_SendCharInfo=0x4513 - 0x6040 V
OP_World_Client_CRC1=0x5072 - 0x7a9e V
OP_World_Client_CRC2=0x5b18 - 0x3795 V
 - 0x22cf - size 2056 - Some new CRC from SoF.  This shouldn't be required to use at all.
 - 0x43ba - size 2056 - Some new CRC from SoF.  This shouldn't be required to use at all.
OP_AckPacket=0x7752 - 0x4D38 V
OP_WorldClientReady=0x5e99 - 2EBD V
0x58FB - size 8 - the Auto-AFK opcode - This isn't in Titanium opcodes, but does exist.  Not required though.

Log into a zone:
OP_EnterWorld=0x7cba - 0x1340 V
OP_MOTD=0x024d - 0x5711 V?
OP_SetChatServer=0x00d7 - 0x71B8 V
OP_SetChatServer2=0x6536 - 0x32CC V
OP_ZoneServerInfo=0x61b6 - 0x18B1 V
OP_WorldComplete=0x509d - 0x2486 V
OP_PlayerProfile=0x75DF - 0x0FEB V
OP_ZoneEntry=0x7213 - 0x737E V
OP_TimeOfDay=0x1580 - 0x7274 V
OP_TributeUpdate=0x5639 - 0x71Fb ???
OP_TributeTimer=0x4665 - 0x4775 ???
OP_CharInventory=0x5394 - 0x08A3 V
OP_TaskDescription=0x5ef7 - 0x0Af9 ???
OP_TaskActivity=0x682d - 0x2E60 ???
OP_CompletedTasks=0x76a2 - 0x5855
OP_Weather=0x254d - 0x70A5 V

OP_ReqNewZone=0x7ac5 - 0x3CDC NEW from IDA 0x5417 V
OP_NewZone=0x0920 - 0x5D22 V
OP_CustomTitles=0x2a28 - ???
OP_SendAATable=0x367d - 0x6F05 V
OP_UpdateAA=0x5966 - 0x45D2 V
OP_RespondAA=0x3af4 - 
OP_SendTributes=0x067a - 0x28C1??? From Tail 0x6764
OP_TributeInfo=0x152d - 0x4775  From Tail 0x3F33
OP_SendGuildTributes=0x5e3a - 0x6774
OP_SpawnAppearance=0x7c32 - 0x10B7 NEW from IDA 0x50EC V? 0x0000?
OP_ReqClientSpawn=0x0322 - 0x45D2 NEW from IDA 0x014C V
OP_SpawnDoor=0x4c24 - 0x68C8 NEW from IDA 0x3547 V???
OP_GroundSpawn=0x0f47 - 0x33E5 ???
OP_SendZonepoints=0x3eba - 0x2992 ???
OP_PetitionCheckout=0x0000 - Shouldn't be required as we don't have it in Titanium either.
OP_SendAAStats=0x5996 - 0x3518 ??? - I don't know if this is used in Titanium
OP_SendExpZonein=0x0587 - 0x3703 V?
OP_AAExpUpdate=0x5f58 - 0x3518 ???
OP_ExpUpdate=0x5ecd - 0x581A ???
OP_RaidUpdate=0x1f21 - 0x00AF ???
OP_GuildMemberList=0x147d - 0x317C ???
OP_GuildMOTD=0x475a - 0xD677 ???
OP_SetServerFilter=0x6563 - I don't think this is required.
OP_ClientReady=0x5e20 - NEW 0x2854 V
OP_PetitionUpdate=0x0000 - Shouldn't be required as we don't have it in Titanium either.
OP_WearChange=0x7441 - 0x25F0 V?
OP_HPUpdate=0x3bcf - 0x62F6 ???
OP_WeaponEquip2=0x63da - skip this as it is unhandled in Titanium and comes from the client each time you zone
OP_WeaponEquip1=0x6c5e - skip this as it is unhandled in Titanium and comes from the client each time you zone
OP_ManaChange=0x4839 - 0x0659 V?
OP_SpecialMesg=0x2372 - 0x1126 V?
OP_TGB=0x0c11 - 0x0137 V?
OP_ClientUpdate=0x14cb - 0x5Cf3 V?
OP_FloatListThing=0x6a1b - I don't have this yet, but it comes from the client and should be able to get it from the tail once I can get in game with SoF

The V means it has been verified either from the client, or from IDA or both. This is about 95% sure to be accurate. The V? means I verified it from IDA, but the code isn't an exact match, so I am only 50-75% sure it is accurate. The ??? means I was either unable to find it at all in IDA, or that I found it and it doesn't look like a match to me, they are likely to be wrong.

I think if we can get all of those, I may have enough opcodes correct that we can at least log in a naked character all of the way. Once the item serialization is done for SoF, we should be able to login geared characters as well. I think the main opcode I need to be accurate now is the doorspawn opcode. I think that also sends objects in the zone and seems to be where the naked test character is getting hung up at.

Like I said, as soon as I can get any character logged in all of the way, I will submit my changes to the SVN so others can assist with it if they want. What I will probably do is set it so that the Anniversary files (that I am using for getting SoF to work currently), will not be used by default on new builds. It would be disabled by a simple define and could be easily enabled by anyone who wanted to mess with it. I will post how to enable it if I can get to that point. It will just mean a simple 1 line code change to enable it (uncommenting a #define). The reason to leave it disabled is so you don't have players trying to use SoF and crashing zones if there are issues with it that cause crashes.