I think you have been spending too much time around Mort. You are starting to post like him and not making any sense :P

I think i'm going to start crying in a corner now -- feel free to delete our three posts to remove clutter, i'm not sure what I was smoking at that time if you know what I mean~

I would actually recommend creating a branch, like I did with loading the spells via the database. That way, you can upload changes that may sorta work so that others can work on it also, but won't affect the trunk ("stable", if you want to call it that). Then, once it's a little more stable/accurate, we can update the branch back into the trunk.

Given that the patch files are pretty much self contained I don't think it really needs a new branch. At least not unless you need to modify core parts of the server to make it all work, in which case yes I'd agree with a SoF branch.

Any modifications of the core files that are needed shouldn't affect the other clients. So far, the only core file changes I have made are to add a new opcode. The only other change I currently have planned for core files is to add a second way to handle the sendexpzonein opcode. Since Titanium sends the same op and expects it right back, the current files work fine for Titanium. They also work for SoF, but the order in the EQ Debugs is a bit off, which might be a concern. So, I plan to copy the function in client_packet.cpp for sendexpzonein and make a different version for SoF. That won't effect how the other clients work, only how it handles SoF logins. Basically, instead of sending the sendexpzonein and getting it back from the client, SoF will send the send the sendexpzonein and send back the new worldobjectssent opcode and handle it by sending that same opcode back to the client again.

Sorry if that sounds a little confusing, but I have it tested on my test box and it works fine and doesn't change anything about how the other clients currently work. At this time, I don't think another branch is needed really. I will just leave it set so you have to uncomment a #define to enable using/testing SoF. So, by default, servers won't be able to use it unless they make that change. This is really only because of the possible crash issues with the SoF client that should probably be avoided for now on non-test servers.

As KLS said, since the patch files are handled in their own separate way, the SVN updates I make for SoF should be ok.

Right now, I am still working on and testing some structure changes to see if I can get more stuff working. I think figuring out the clientupdate structure is an important one to focus on for now. It is a pretty small struct (only 36 in size), but it uses some split signed/unsigned blocks that I don't really understand yet.

Here is the structure of it that I am currently working on:
The actual packet example from that looks like this:
Really, once the spawn struct and clientupdate structs are worked out, I think that should have most of the struct work done. I am sure there will still be more little things to work out in the other structs, but after that, it will mostly be finding missing opcodes. I still haven't worked out a way to find all missing opcodes just yet, but I have been verifying more new ones almost every day.

If I only had a packet collect from 9/7/2007 or shortly after that, it would make this whole process ALOT easier lol.

That's not really what I was talking about, that all really falls under patch files. For example when we wanted to fix inter-zoning we had to actually modify how you zone as 6.2 sent a different series of packets overall than titanium and so required changes to both the patch files and the zone server. When I talk about "core changes" that's things that aren't contained to a certain version and could potentially affect other versions. =p

Just do us a flavor and make SoF commits contain only SoF related changes! :)

As if I had time to make any other changes while I am working on SoF. This thing is like a full time job that I don't get paid for lol.

I will probably be doing an SVN commit for SoF tonight. There is alot more progress since the initial release. Spawns now show up (missing names though), you can actually move around in the zone instead of being stuck in place, doors spawn and ground spawns spawn (but still need work), and a few new opcodes are verified.

Items are a big one that needs work, but those shouldn't be too bad once I know how to do the serialization like a structure so it sends it the way it needs to. I think it is almost to the point where other people might actually be able to start helping out on it soon. I am probably going to try to write a beginner's guide in the wiki for packet structures and maybe some opcode stuff.

You just described the entire EQemu project!

I do so enjoy documentation~

Back to working on the item serialization again, I broke down items into as close to an exact structure as I can figure out. This is for EQLive, but no new fields have changed from SoF to Live as far as I know, so it should also apply to SoF. This structure may not be 100%, but I think it is at least 95% accurate. This is based off of what I could find from 13th floor and packet collects from EQLive.

Since items are handled in a different way than any other struct, I am not exactly sure how to implement this properly. Currently, the emulator sends items as a long string. For SoF, they need to be sent like any other normal structure. Right now, I can't figure out how to add these structs in without having to mess with the item serialization for other clients. I will keep working on it, but it is a bit above my skill level right now. For starters, I need to figure out how to change the serialization from using MakeAnyLengthString and instead for it to send the actual data as it is written in the struct.

I was able to get Targeting working finally and also able to get names to show up now too. The only other major thing to get working from the spawn structure is size. Most of the rest isn't really required and could be done later at any point. These are a couple of major things that were holding me back from making much more progress, so having those working will help alot to getting the game playable.

The actual process I used to find those settings is a bit of a pain, but seems to actually work fairly well. Basically, I took all of the unknowns in the spawn structure and have been setting each one to send 1s for all fields in them. So, by doing this for each section of unknowns 1 at a time, I can compile the changes and then start the server and log in to observe if there were any differences and what they are.

Here is the actual spawn encoding that I am using to test:

The section noted in blue are the new fields I found and am forcing them to send all spawns with a certain setting in that field. The section noted in green are the actual settings to test the unknown fields. I just uncomment the line of the unknown section I want to test. There are some notes next to some of them as to what I noticed when testing them. There are also numbers next to some as well, which correspond to the section in orange below that. Basically, I tried picking the unknowns that were smallest first. I was doing more than 1 at a time to reduce the number of times I would have to recompile. The numbers tell which ones I turned on for each test number I ran. So, the ones that have 1 next to them were the first ones I tried and the notes in orange below are what I saw as a result from that first test. The notes could have been done better, but at least it was something to reference to help keep track of what was being done and being seen.

The reason I am doing a little write-up on how I did this is so others may be able to use the same technique in the future to help refine the structures even further. Other than being able to figure out the structure from reading the IDA output, which is well above my skill level, this guess and test technique is the only other way I could think of to figure out these needed structure fields.

The same technique should be able to work fine for figuring out the rest of the fields. I think it should be fairly easy to figure out stuff like beard and beard color, face, etc with this technique. So, almost anyone could help refine this list of field positions in the structure.

I will most likely be doing a write-up wiki page on what I know about working with packet structures. It really isn't all too complex, but it is important to understand a few key things that took me a while to figure out on my own.

I will probably do an SVN update with the new targeting and name stuff tonight, or very soon. It is finally getting to the point where it is almost playable. I think getting items working is probably one of the biggest things left to do. I have the item structure worked out, but may need help in finding the best way to implement it.

I was finally able to try attacking a mob for the first time in SoF, and of course it caused EQ to crash :P I am pretty sure the crash was due to the fact that items hadn't been loaded though.

For a while, I was a bit concerned that we would never be able to get SoF functioning well enough to actually use as an upgrade to Titanium. Now, I am fairly confident that all possible issues can be resolved and that it will almost certainly be completed and at least as functional as Titanium is now. It is all really just a matter of time and work. I am really hoping that once I have the game somewhat playable and most systems functioning, that others will start assisting in the project. I know most people think this stuff is well beyond their skill level, but if I can learn it, most people with technical skill can as well. And, I think that if I write a couple of wiki pages to help people understand what to do, it will make a huge difference in who can start helping.

So much like myself, you too have little room for your other hobbies. :(

Just wanted to remind you that AugSlotUnk; is now known as AugSlotVisible.

Thanks. Yeah, I noticed that from 13th floor back when I first started working on serialization. I went ahead and changed it in the structure I posted above so it should be accurate now.

Using the same method I described before, I was able to find the helm field in the spawn structure and also in the illusion structure last night. I think this method could even be useful for finding more stuff in Titanium if we needed it. Even though it is a bit of a pain, it isn't really too hard to do. It just takes time.

I also wanted to mention that I was changing through races last night and noticed that there are considerably more global races in SoF than there are in Titanium. There are a few new races as globals that aren't in Titanium at all, including a dragon model with 5 textures! And also many of the models that exist in Titanium are now set as globals as well like Shilskins, all Pirates (LoY) and at least a few others. This is great for custom servers that want to use a variety of races in custom zones and don't want to require any changes on the client side. I also saw some new global races that were objects. One was a big floating skull with flies and a centipede crawling around it and some of the others looked like runes and floating things. The new races look really cool from what I have seen so far. I am really excited to be able to start making content using them.

The last major thing I want to find from the spawn structure is model size. Other than that, I do need to figure out why texture doesn't seem to work either. I don't see texture as one of the fields in Titanium, so I am guessing that it comes from the appearance packet instead of from the spawn struct.

Items and spawn/client position updates are still the 2 most important things that need to be done right now. Once those 2 are complete, I think the rest will be just filling in the little stuff and working out the details to refine it. Once I can fight and kill an NPC, I think it will be ready for player beta testing. The only things that I think are holding that up right now are the items and position updates I already mentioned. Basically, things are getting really close to being playable.

If anyone is interested in helping, let me know. I could use help from someone with more experience in packet structures. I don't quite understand what unions are. Most importantly, I don't understand how signed and unsigned fields work. Here is an example of the client position update structure from Titanium and it uses signed and unsigned, which seems to combine a into an int32 somehow, but I don't understand exactly how...

That struct is pretty straight foward, but I don't understand the sections in green and how those actually work. The client update comes from the client, so it is easy to look at them and see what is being changed and what the order is. I just don't know how to figure out how to write those green section types.

Here is the struct I am working on for SoF so far:

As you can see, I only have a small bit of that structure verified. I am really just guessing with the sections in green since I have no clue how to figure that part out yet. If someone can explain how those signed and unsigned with sizes work, that may help. All I know is that they all have to add up to 32 for each section of them.

Unfortunately, the positioning stuff is one of the few things that SoE seems to completely switch around almost every time they patch. The only good thing is that it is a fairly small packet, so even just guessing, it shouldn't be way too hard to get it set correctly.

I will probably try to get an update onto SVN tonight. I know there isn't much reason to update it regularly for SoF changes at this point, but when I make major breakthroughs, I think it is worth it. The main reason I am doing the updates at all is that hopefully at some point, it will be working enough that someone else might realize that they could start helping so we could get this expansion done much quicker. I think once people start realizing that SoF will actually get completed, they might be more willing to put some of their own time into helping. Up until recently, I think it was questionable if it would ever work or not, but at this point, I am very confident that it will be able to fully replace Titanium (though Titanium will still be usable of course).

I still don't have an ETA on when it will be playable or widely used on servers, but it is getting close to time to maybe consider picking up that last copy of SoF at your local retailer if you happen to see a copy and have an extra 20$ laying around that you don't mind spending :P At least anyone interested in testing or helping out with the SoF development should definitely get a copy if they find one. Of course, don't blame me if for some freak reason it never gets finalized!