Ahh, thanks, I was thinking that equip_chest2 was probably texture. I will try to get that going tonight too if I can find it.

And yeah, I figure that switching the item structure from being serialized as a long string into being serialized as a binary structure is very simple stuff. But don't forget that I am still quite a noob and without a good example to work from for something that big, I just don't know what to do. That is why I was asking for a little help with it. I posted the structure that it should be above. Lots of those will just basically be unknowns until we can build the rest of the stuff to pull those fields from the database. Adding that other stuff in won't be a problem once the basics are working. I just don't know how to make it do that serialize in binary instead of a string! If you could do that for me, it would help in a huge way!

Note that right now, the clientinventory encode is set to just send an empty item packet until the real item structure is in place.

Anyway, like I said before, I figured changing items from a string to binary would be simple, I just don't know how to do it. Does that scare you people that just spent 6$ on SoF? :D

Items are a really huge factor holding back SoF from being done. KLS, if it doesn't take much time, do you think you could possibly make the change in the SoF patch files so it sends it as binary instead? I think once I see how you write it, I should be able to move stuff around if needed to get it functioning. I don't think you have a copy of SoF yet, so you probably can't test it. Don't worry about messing up the SoF patch file and updating the SVN with them as long as it doesn't break the compile.

I think I could figure out how to do it, but each time I try, I wind up getting stuck and having to revert back to the SVN revision.

I can write something up, I don't have the client... 6 dollars... >< so whether or not it works is anyone's guess.

One thing to remember when working with the patches is it doesn't matter what form the data coming in is in, all that matters is what we send. Right now the data comes in as an internal structure and goes out as a string, it shouldn't be *too* hard to go from internal structure to eq structure.

Well, the main thing I couldn't figure out is how to write a different version of the structure for items. In SoF, many of the fields are changed from int8 to int32 when looking at the structure in item_structs.h and I don't know if just putting a struct for it in SoF_structs.h would work with that like all other normal structs. I don't fully understand why it gets it's own struct file as apposed to being with all of the other structs.

The other important thing to keep in mind when writing it is that all strings like name, lore, charmfile and such need to be variable length. Basically, it sends the string and then it sends 00 after the string to signify the end of the string. Other than that, it should all just be normal binary. Would that mean instead of using "char" for the string fields, we would use "varchar", or would it be more like "char field[1];" to make them variable length? I don't know how to make them variable length...

If you do get something written up for it, KLS, I would really appreciate it!

The goal is to make the new SoF server continue working with titanium I assume?

Yes, it all works perfectly fine together right now and there should be no reason to change that. Since the patch files system were added back when Titanium was added to the emulator, it has always been compatible with both the 6.2 client and Titanium. And now it will be compatible with those 2 as well as with SoF.

BTW, I got the SVN updated this morning with my current SoF patch files and a few more changes. Keep in mind that the SoF_structs.h file is pretty sloppy in a few places that are still being worked on heavily, but once things are finalized, I will clean it all up.

Most importantly, I was able to get the clientupdate stuff almost completely worked out last night! I think spawn position updates should be accurate, at least enough to use for now. I know that X, Y, Z and heading at least are all accurate and the delta stuff probably is as well, but I haven't tested that fully yet. For the client position updates that get sent to the server, I have X Y and Heading all in the right spots now, so Z is the last major part to figure out. The deltas and animation will still need to be verified, but they aren't as important.

I also found a few more things in the spawn struct like flymode, LFG flag, hair and a couple more. I haven't narrowed those down to exact positions yet, because they aren't really what I am looking for and they can be added in at any time after the game is playable. I must say that it is pretty cool changing the unknowns and logging in to see what has changed. So far I have found quite a few fields that aren't even known in Titanium, so maybe we can make use of them starting with SoF.

I still wasn't able to figure out how to get #zone working yet. Either the opcode I have for requestclientzone is wrong, or the structure is wrong. I am going to compare it to a log from EQLive from ShowEQ and should be able to find what I need.

Wouldn't happen to have a packet of a server->client item would you?

Databasce Converter
 

Do you need a Database Converter Professional Edition to help you convert svn to peq etc i'll upload it. This is the best i can do at this time to help you out.

Assuming that the packets in SoF are the same as live, or very close, here is one from EQLive:

Wow, I am surprised that it actually posted that whole item packet. That was pretty big lol.

Anyway, here is how I am breaking down a single item from those examples:

And that should all be fairly well explained in the packet structure I posted for it in this thread earlier here:

After reviewing it a bit more, it looks like there was already code to handle the click/proc/worn/focus/spell section properly. And also one for handling the augments section, so those extra structs here are probably not needed, if it is just set to work the same as the current item structure.

Also, I am only pretty sure of the structure up to the end of the click/proc/worn/focus/spell section, so everything after that is just a guess that will probably need to be moved around later after we can test and see what is what. It should be pretty easy to figure out unknown fields just by changing them and seeing what gets changed.

Well, zoning via normal zone points from one zone to another is working. Any other type of zoning via commands isn't yet, though. It looks like the client is now expecting to get the zonechange structure from commands instead of the requestclientzonechange structure like Titanium uses. Using commands, I get a message that it was unable to find the player name, and requestclientzonechange doesn't send a player name, but zonechange does. I am guessing that the best way to handle this for the / commands usable by GMs would be in the SoF.cpp file by doing an encode on those opcodes. But, for # commands, that is handled by the server and doesn't get an opcode from the client, so I am not sure what the best way to handle those would be yet.

I am going to keep messing with it and see what happens. First off, I will see if I can verify for certain that it is looking for the zonechange struct now. If it is, then we will have to figure out a way for SoF to handle those commands differently. Off the top of my head, the only easy solution I can think of would be to make alternate zone related commands for SoF. I would prefer not to have to do that, but there may not be a choice unless someone can come up with a better way to handle it.

charter name aspected
 

[Fri Jul 25 2008]00197:Initializing character select UI.
[Fri Jul 25 2008]00198:Resetting game UI.
[Fri Jul 25 2008]00199:Zone Connect -- 0 -- Received MSG_ZONE_ADDRESS
[Fri Jul 25 2008]00200:Zone addr [199.108.3.107:21509] received...
[Fri Jul 25 2008]00201:ZONING
[Fri Jul 25 2008]00202:Networking: Connection Closed [0] with 0 pending bytes.
[Fri Jul 25 2008]00203:Networking: using port [1133].
[Fri Jul 25 2008]00204:Networking: Connection Established [1]
[Fri Jul 25 2008]00205:Connected to 199.108.3.107:21509...

[Fri Jul 25 2008]00206:Zone Connect -- 2 -- Sending MSG_EQ_ADDPLAYER
[Fri Jul 25 2008]00207:Zone Connect -- 3 -- Received MSG_SEND_PC
[Fri Jul 25 2008]00208:Received our Player from zone.
[Fri Jul 25 2008]00209:Received MSG_EQ_ADDPLAYER, Player = *charter name here *, zone = Skyfire Mountains
[Fri Jul 25 2008]00210:MSG_TIME_STAMP received.

[Fri Jul 25 2008]00211:MSG_TIME_STAMP received. (Items inc).

I took out my charter name where you see *Charter name here* is where it goes

LOL, after working on the client update position stuff over and over trying to figure out why what I was seeing wasn't matching up with what I was setting for the packet structure, it looks like I finally got it all working perfectly as far as I can tell. Apparently, the logs from the server can't always be trusted. It was telling me that the client was sending 36 byte packets for clientupdates, but it is actually sending 40. It was also re-arranging all of the blocks in the packet for some reason, which made working out the structure nearly impossible. Luckily, I just tried using the exact structure from ShowEQ around that time and it works flawlessly! I dunno what the logging is doing to cause me all of that trouble, but at least it works now :D

I am still working on getting zone commands to work. I think I can just redo the requestclientzonechange structure for SoF, but I need to figure out how to get the client's character name in the SoF.cpp encode for it. I have tried a few things, but none of them seem to work yet.

I am also still working on the ground spawn structure, because something is definitely wrong with it. Objects spawn about 100X bigger than they are supposed to. I have tried adjusting all of the unknown fields and none of them are making any difference, so I am guessing the known fields are probably ordered wrong or in the wrong place.

I also still need to narrow down a few more fields in the spawn structure, but that is mostly just cosmetic stuff like armor, tint, texture and size.

Something is still missing from the AA stuff that is keeping them from loading, but I think it should be sending them correctly, so I am guessing it is maybe just the list of AAs that isn't going out properly yet.

Most of the rest of the stuff to work on will probably be best to wait until items are working. I could start working on the action stuff for fighting, but without being able to load items, it will probably just keep causing crashes. Once items are working though, I don't think the action structures will be hard to get going.

The /who all related structures also need work, but I think I can probably figure that out by looking at EQLive packets. I just haven't checked that yet.

This will probably take a while to get working, I'm not sure on the size of the item struct... which will probably take a lot of time in itself. I think I've got the item header right though.

It's very tedious -.-.

And yet, every time someone makes a post about progress, a hundred thousand voices cry out in bliss and tremors of joy.

Quick question guys,
for the most part is this already enabled? I was going to check this out when I have time, Just was wondering if this is set to go with normal server compile or is something else needed to be defined before compile?

It is disabled on the SVN revisions for now by default. Mainly because we don't want everyone to try using it on public servers and causing zone crashes due to any unknown bugs. Once it is fairly bug free and complete, it will probably be enabled by default.

For now, you just have to edit the /common/patches/patches.cpp and uncomment the #define line at the top to enable SoF. Then compile and it should be enabled.

Thanks trevius,
I hope i get time to mess with this coming up here, Your work on this has been amazing :)

I'm going to remove the define next time I commit fyi, it's pretty pointless to have it in there. Any server op that finds a patch causes a problem or doesn't want a certain client to be able to connect can just remove the patch register line.

And in the same sense if you want it off by default just comment out the code that registers it.

btw I'm very lost on items atm (I really wish I had a packet log from this client).

What do you need for a dump? I got my client today so could provide you with something in the next day or so.

LOL, tell me about it! One single packet log from 9/7/07 that had a person loging in a single time would have made this project 100X easier lol.

The only thing that has gotten me this far is help from the ShowEQ source and EQLive logs from ShowEQ and the rest has been just "best guess" work. The good thing is that even if something is wrong in the packet, most structures will still let you get in game. Then, once in game, I just check what is wrong and mess with the structure until it looks correct.

For items, if you can get one single item to load at all, I think getting the rest of it working won't be too bad. I am more than happy to tweak the items serialization. I just need something to go by whether there is just an encode being done in the SoF.cpp, or a whole new way of handling those packets. Once I see how you are doing it, I think I should be able to figure it out from there.

Here is what I would do:
1. Start with a character with just 1 simple items (cloth cap maybe) on them.
2. Try logging into the game with the serialization you have setup for it (shouldn't need to be perfect just yet).
3. If the client hangs, crashes or disconnects on logging in, then check the EQ dbg.txt file in the everquest/logs directory.
4. It should say something about loading the item and then have a number next to it. That number should be the slot it thinks the item is in.
5. If that slot is wrong, then slot_id is in the wrong spot in the serialization. Try moving it until it picks up the correct slot ID that the item is in on the character.
6. If it reports the correct slot, then we know we should be good up to that point. I imagine once it does that, it may actually load all of the way in game.
7. I don't think there were any new fields added or changed since SoF was released, so the total packet size should be the same.
8. Try inspecting the item if you get this far, and if the name doesn't show up, then name probably needs to be moved a bit.
9. The item might not show up at all if the icon field is in the wrong place. The position of this field is probably one of the most important for starting work on this. Really, I can't imagine that the position of the icon field could vary more than 20 bytes from where it currently is in the structure I provided. If anything, it may be sooner. I doubt it would be later than where it is on EQLive.
10. Once Icon and Name are showing up properly, it should just be a matter of checking item stats and verifying that they all line up properly. Maybe some super item with a little of each stat could be set just to show what fields are what.

I am very willing to help with the guess work of lining up the structure. I only need a working system to use to send binary instead of a string. Once that system is in, feel free to update it on the SVN and just leave the code in that sends the inventory as a 0 sized packet until we get the inventory stuff worked out. The 0 sized inventory packet is forced in the SoF.cpp for now in the clientinventory encode I think. Otherwise it tries to send the string and crashes horribly.

She needs the item packets from the server to the client. Unfortunately, that is something we have to make and cannot collect with the emulator/SoF client together. I have provided an example from EQLive (and can send more examples if needed), and I imagine that SoF should be the same or very similar to EQLive considering no new fields have been added as far as I know. The only things I can think of that might have been added may be herioc resist stats, but I can't really verify that. I am just going by what I see on 13th floor and other info I find here and there.

So far I've gotten: Client crash (but I figured it out), then created an empty item then moved some fields around and the item disappeared completely!

Ahh, sweet! At least it sounds like you are making some progress. If you can get an empty item, the rest shouldn't be too bad to work out. I think the only other hard part might be getting augs/bags working properly. But, then again, an aug/bag should just be items within items. Once the other fields are correct, it should just be figuring out where the item within an item should be placed.

Any way you could update the SVN with the version you had with an empty item in it? I am guessing that means you actually saw an icon for the item? If you made it that far, I am pretty sure I could get most of the rest of it worked out in a night or 2. As long as you leave the encode on the clientinventory to send a 0 size packet, then it shouldn't effect anyone who is just trying to log into SoF normally without items. I can just comment that part out so it actually tries to load items while I work on it.

Ahhh, I see. That's tough sledding uphill both ways.

I've got this now:
http://i8.photobucket.com/albums/a2/...e/EQ000001.jpg

I've purposely filled it with strings lacking null terminators to try to find the name area, and succeeded... sorta I still gotta nail down which one it is but it's a start.

Anyone wants to inform me what that green background means I'd greatly appreciate it. I've obviously got slot correct, but the rest of the header seems to be hit and miss. I have a feeling it's very different from the live packet you linked.

If i remember correctly it has something to do with evolving items.

Well, I would look at how the Titanium items headers are and compare that with the live one I posted. I imagine it is somewhere in between. If you have the slot ID right, then it should just mean that you have to add an int 32 or int16 extra into the header, so it doesn't pick up part of the header where it thinks name should begin. I don't know where that * is coming from, maybe you are sending the number 42 as an int8 which as a string gets converted to a *?

Oh yeah, and before someone tries it and can't figure out what happened, I wanted to mention that right now, you need to keep your character level at level 75 or less when using SoF. Anything over 75 (via #level) will result in setting your character's hps to 5 for some reason, even though everything else like skills and others seem to be ok. Then, if you try zoning at any level over 75, it seems to crash the client every time right after the zone loads. I think it is actually crashing when the spawns are sent, so it is probably an issue with the spawn struct that is causing it. I really hope that there isn't a restriction on the client that is causing the level 75+ issue. The actual eqgame.exe was created before SoF was released and was ran on Live when the max level was still 75. But since one of the key features of SoF was a max level of 80, I am betting that it isn't a restriction of the client, and instead, an issue with a struct that can be fixed. Maybe there is just a new field that needs to be set to allow a player to exceed level 75. Though, I would think at least with GM mode on, it should work ok, but it doesn't. I am sure we can figure it out at some point.

I have some live packet traces from July 2008 when I was working on the Pet Buff window. Maybe they will have some Item packets that are closer to the SoF structs than the current live packets. I'll upload them when I get home tonight in case they are of any use.

EDIT: Here: http://www.rama.demon.co.uk/EQLive-Jul24-2008.rar

Inventory begins at:

The green background was used for 2 things in EQ(at least) that I can think of. The first and the one you are interested in, is to indicate an evolving item. Matters not if it was fully evolved only that it does evolve. The second was in the bazaar in trader mode. Once an item was set with a price and added to the "selling" list the background turned green.

The "*" in the name indicated that is was a starting item given to you or something like that. The initial weapon, and backpack in the tutorial were like that and the burlap shirt given by the Guildmaster when you turned in your note all had the asterisk. There might be more, I think it had to do with making them not sellable to merchants, but I'm not 100% sure of that.

Yeah the * was just part of the string I was brute forcing. I'm pretty sure something in the header just wasn't getting set right. As I didn't actually change anything but the values in the header to get the name to show up... and no matter what I change icon seems to stay the same so I don't think I'm close to it yet.

And incidentally I found a packet log from a little after sof, but not same client version. It's pretty similar to derisions and both match up with my expectations of where things will be so we'll see.

I wouldn't mind seeing that packet log if possible Or at least I need a single SendAA packet to see what I am doing wrong with AAs. I have tried many different variations on the struct and it should be setup right or very close to right. I haven't been able to get it to display anything. I am almost wondering if it may be an issue with the UI not having the correct version of the AA window to display tabs properly. Though, it looks like EQLive's AA window to me, so it should be the same. The base AA before any effects added onto it should be 104 bytes long for SoF (it is on Live). I can get it to log in with as little as 96 bytes long, but it seems to crash alot more. The issue may just be that I have to fill in the database with the expansion number and tab information that each AA is supposed to be from. On Live, they replaced the "type" field, which just tells the AA window which tab to display the AA on, into 2 fields. One field that tells it which tab to display on (1-4) and one field that tells it which expansion it is from. I think 0 means it is a tradeskill, but 3 to 15 are the other expansion values I see, so I am guessing they mean Velious to Current. It is pretty easy to read the packet for SendAA from Live, as little has changed from Titanium.

I am also working on the /who all struct and I think I know what needs to be done. I have the 3 related structs all built, but I need to get the encodes/decodes working correctly for them. I have the encodes/decodes basically built, but there are a couple issues with them. At least now I get the "search returned no results" messages or whatever.

For Icon to show up, I think it needs null terminators after strings for Item Name, Lore, and the ITxxx strings. Then, the few int8s after that should be correct. The only one I would wonder about is the Price field. It looks to me like it is actually an int64, but that is the first and only time I have seen anything that might use int64 in EQ, so I don't know if that is correct or not. Maybe the change to int64 came in more recently, so you might want to try it as an int32. I recall reading a recent EQLive patch message that says they corrected an issue with selling stacks of items that totaled over 4million PP or so. So, maybe that was when they changed it from an int32 to int64. That is all just speculation though. I imagine the older log files you guys have would confirm that pretty easily.

I've got this now:
http://i8.photobucket.com/albums/a2/...EQ000003-1.jpg
I used Derision's packet as a start since his was a bit easier to read than mine. The one I wrote still has sequence # and size on it so it can get a little hard to read esp when you're reading a lot of hex etc etc.

Something in the item header if not set correctly makes the item not show up which was what was throwing me off.

Right now item is like this:

Obv everything's not the same as middle of 2008 client but it was a bit closer than the current client.

Again:
http://i8.photobucket.com/albums/a2/...e/EQ000004.jpg

I think the itemtype field has changed which is why it appears like that but I'm not sure. I get to start plugging in values all over and see what they do.

I just noticed that your items are showing up in the old item UI window like Titanium. I prefer the old one (Titanium version) to the new window for item stats on Live. I thought it was already changed in SoF though. Or, is there an option somewhere that lets you decide which one to use? I know I saw a screenshot of an item from SoF when it was being beta tested, and it had the new item window. Maybe they still used the old one when that particular build was made?

Also, I looked at the SendAA packets from Derision's ShowEQ log and it looks like it lines up the same as EQLive does now. Same size and layout of the structure as far as I can tell. Here is an example of one of those packets without any extra effects on it:

One thing to help alot with items is the 13th floor:
http://eqitems.13th-floor.org/

Just search for the item you are trying to break down and then look at the raw data. It should help a ton when lining up the fields.

I also use HexVis alot or Windows Calculator for converting the hex easily to see if it matches 13th floor.

That last uint8 is Item Type. It is 00 for normal items, 01 for bags and should be 02 for books. This should be done on the Serialization because I think it tells the client how treat the item when you right click it. Since you can't read books or use clickies if an item is inside a bag, it should be because the item type isn't sent when they are inside a bag, so the client knows only to show the item stats.


Just from looking at the log Derision posted, here is my best guess for item Serialization (pretty much matches with what KLS has so far):

Here is the best breakdown I can figure for the beginning of the Item Struct:
Most of the rest of it should line up with the structure breakdown I posted for EQLive items. I can work out the rest of it if needed. I will definitely check into these logs more later tonight.

It's probably my UI, I use default_old from titanium. To save room I only change out the files that need to be between my versions of the client. So while titanium UI default was overwritten my default_old folder wasn't.

Maybe that has something to do with this not looking right, good catch on that sir... I probably woulda never noticed otherwise.

You realize you saved me a week of pointless work because I wouldn't of caught that for a week:
http://i8.photobucket.com/albums/a2/...e/EQ000005.jpg

I had to dl an itemwindow from eqinterface cause apparently I overwrote mine and didn't back it up =(

Item fields seem fairly accurate to derision's log and sorta accurate to the live log tho the live one's is different in spots, but that's to be expected after a year.

Now I gotta put it all into a structured format, right now I'm just pushing ints and bytes onto my data stream then nail down the exact size to be able to send more than one at a time.

Nice! Glad I could help lol :)

Another thing I noticed from Derision's log is that his first item, Jboots, reports in the serialize as being in slot 30. For Titanium, slot 30 should be cursor I think, but I doubt he had them on his cursor when he logged in. My guess is that it was actually in the bottom right main inventory slot (like you show in your screenshots, KLS). If that is so, then I am guessing that all slots got bumped up one when the Power Source slot was added, because in Titanium, that slot is slot 29, not 30. Oddly, I don't see a power source slot in the UI that comes with SoF, but it should already be in since Power Source was added with Burried Sea. I see the power source slot just fine when I log into Live now. Either way, I am betting that Power Source got added as being slot 22 and that just bumped all of the main inventory slots up by 1.

Also, I finally figured out a major graphics issue I was having in some of the new zones. Apparently, it isn't a good idea to have min and max clip plane for fog 1 2 and 3 set to the same values lol. I don't know how many hours I wasted trying different graphics settings and different video drivers versions. Either way, the new zones now show up perfectly to me. I will post the SQL to update/add them all if Cavedude hasn't already gotten them all added yet. His would probably be much better than mine anyway. I am just using 1 example and copying it for all zones, so they all have the same fog settings and what-not.

Yeah slot 30 is what I have the boots set to in that SS. The items on character seem to work fine I'm guessing the bag slots all got bumped up one.

Also I don't see a power source slot on the default ui that came with sof either =/

After trying about 5 different UIs that came out around the time that SoF was released and that all work with SoF (though some have buggyness), I am at the conclusion that maybe something needs to be sent from the server for the Power Source slot to show up. None of the UIs I tried had a power source slot. Since it came out with Burried Sea (one expansion before SoF), I have to assume that it should definitely be working when SoF came out. All of the UIs I tried looked like they had an empty area of the inventory window that could be used for the power source slot, but none of them actually had it.

Here is a preview of one of the UIs I tried that shows the Power Source slot in the UI:

http://www.eqinterface.com/downloads...ew.php?id=9182

But, when I actually tried that UI, it wasn't there. That UI hasn't been updated since 10/5/2007, so I know it wasn't added after SoF came out. I know it isn't a big issue for now, but at some point, I will probably try to figure out what needs to be sent to load that slot in the UI. It is probably something in the player profile I am guessing.

Whether the Power Source slot appears is governed by the info sent in OP_ExpansionInfo. The struct has changed, rather than just a uint32, it is now 64 unknown (unused from a look in IDA) bytes, followed by the expansion bitfield.

I have this sorted in my Working Copy and it will come with the Character Creation stuff when I commit that.

http://www.rama.demon.co.uk/SoFInventory.jpg