Starting Work on SoF Opcodes/Structs

[Zeice]

Ok it won't let me edit now, but I went and pulled some IDs from the live spell file.

Hand of Tenacity is still 9809, but Hand of Temerity is 14300. So the numbers did change from SoF assuming that that spell with ID 14282 was supposed to be Hand of Temerity. Granted this is just one spell, and one example, but something does seem to be off.

[trevius]

Yeah, I imagine that they may have been moving spells around still at the point when they made the CD/DVDs for it. Level 80 didn't go live until the day that SoF was actually released, but I am sure they must have been running it on the test/beta server for a while at least.

There is another issue with level 76+ as well that I already mentioned once. For some reason, everything works perfectly for 76+ accept that it sets your hps to 5 max (before gear stats get added). The client is doing this for some reason, but I am unsure why. It is probably something that can be fixed in the Player Profile I hope.

Any issues with spells isn't too big of a deal. Many servers have a max level of 70 as it is, and even if they go up to 75, they will still mostly be covered with spells. But yeah, most likely it sounds like some spell adjustments will probably need to be made. That isn't a show-stopper though, it can be worked-around fairly easily. You can't go by the current EQLive spell file anymore, as they changed it around a TON so that they only give the client spells that they need to know about how to cast, so focuses and worn effects and stuff probably aren't in there anymore. It just reduces their file size to save bandwidth on updates I imagine.

I think I still have a Live spells_us.txt file that has the full 20kish spells in it. I will probably use that to make a new spell file for SoF that will work with Titanium and SoF and include spells up to level 80. Then, if he wanted, Cavedude could add it to PEQ so people could get it. Though, it would mean that someone would have to host the file for download so all SoF members could get it. Though, it will most likely only affect players that are level 75+.

There is another issue too that needs to be dealt with on spells. If a SoF player buffs a Titanium player with a spell ID that the Titanium users doesn't see (over 10k spell ID), it can cause some very buggy situations. My thought for a solution might be to expand on my idea to make quest globals usable for flagging spells. In this case, we could have all SoF users hail a mob in an SoF zone to verify that their account has SoF. Once they do that, it would give them access to be buffed by the level 70+ buffs. Basically, there would be an extra check when buffs are cast to see if the client is flagged for SoF or not. Really, this could be expanded to be used for limiting certain buffs to certain players, but I think getting rid of this possible bug would be a priority use for it. Just an idea anyway.

[trevius]

FYI, for anyone who is getting hacker entries for SoF users reporting that they are using /kill, you can ignore that. Apparently the /kill opcode is set to what should actually be the bankerchange opcode I think. Your players may report getting a reported for hacking message, they can ignore that as well.

I will get this corrected in the next update. Just wanted to make people aware so they don't freak out if they suddenly see a bunch of /kill hack entries in the hackers table

[trevius]

Just wanted to post to say that just because there haven't been any updates on the SVN for the past week doesn't mean I have stopped working on SoF. Xinu and I have been working on some of the few main things that are still not functioning properly like grouping and buffs and others. We just don't have anything new that is completely fixed yet, so I don't see a reason to update. Though, we have gotten a couple new encodes/decodes done right I think. Once the fixes are ready, I will get the SVN updated.

[trevius]

Finally got AAs loading into the AA window with the help of Xinu. Turns out the only problem is that the RespondAA packet was sending each AA in the list as 8 bytes, and they are now 12 each, the same as in the Player Profile now. Also, the first int32 in the packet needs to be the total AA spent, but that isn't working correctly just yet. AAs still aren't usable, but this is a big step in the right direction

[whiteone]

We are getting there one step at a time.

[KLS]

Trev, PEQ was having a crash with SoF inventory loading the other day. But the stack trace completely befuddles me. You guys haven't gotten any odd crashes with it have you?

[trevius]

Odd, I haven't had any crashes on Storm Haven that were caused by SoF that I know of. Though, I wouldn't be surprised if there are still some item issues or something that could cause a crash. I will keep an eye out for them. Mostly been on my test server lately though and haven't had much time on the server itself.

[KLS]

#0  0x4007aa3d in std::__num_base::_S_format_int(std::ios_base const&, char*, char, char) () from /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/libstdc++.so.5 
(gdb) bt 
#0  0x4007aa3d in std::__num_base::_S_format_int(std::ios_base const&, char*, char, char) () from /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/libstdc++.so.5 
#1  0x4007aabb in std::locale::_Impl::_M_remove_reference() () 
   from /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/libstdc++.so.5 
#2  0x4007a008 in std::locale::operator=(std::locale const&) () 
   from /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/libstdc++.so.5 
#3  0x400714b2 in std::ios_base::_M_init() () 
   from /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/libstdc++.so.5 
#4  0x4006f248 in std::basic_ios<char, std::char_traits<char> >::init(std::basic_streambuf<char, std::char_traits<char> >*) () 
   from /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/libstdc++.so.5 
#5  0x082440bd in SoF::SerializeItem(ItemInst const*, short, unsigned*, unsigned char) (inst=0x4485ec70, slot_id=1, length=0xffffffff, depth=0 '\0') 
    at istream:106 
#6  0x08242469 in SoF::Strategy::Encode_OP_CharInventory(EQApplicationPacket**, EQStream*, bool) (p=0x400bf4e5, dest=0x44a06700, ack_req=true) 
    at ../common/patches/SoF.cpp:773 
#7  0x08245181 in StructStrategy::Encode(EQApplicationPacket**, EQStream*, bool) const (this=0xffffffff, p=0x1, dest=0xffffffff) 
    at ../common/StructStrategy.cpp:22 
#8  0x08234ec8 in EQStreamProxy::FastQueuePacket(EQApplicationPacket**, bool) ( 
    this=0x1, p=0x400bf4e5, ack_req=255) at ../common/EQStreamProxy.cpp:36 
#9  0x08234e87 in EQStreamProxy::QueuePacket(EQApplicationPacket const*, bool) 
---Type <return> to continue, or q <return> to quit--- 
    (this=0x4480a188, p=0x400bf4e5, ack_req=true) 
    at ../common/EQStreamProxy.cpp:30 
#10 0x080d78c8 in Client::QueuePacket(EQApplicationPacket const*, bool, Mob::CLIENT_CONN_STATUS, eqFilterType) (this=0x1, app=0x44822a40, ack_req=true, 
    required_state=1074525413, filter=4294967295) at client.cpp:649

There's the trace from a PEQ crash cavedude posted. It seems very odd; almost if the STD library is crashing for some reason. I might rewrite it using void* instead of sstreams. Those were just the lazy way to do things back when I was still trying to create the entire format anyway.

[trevius]

Hmm, I can't make much of that debug. At least I think the item struct is just about as final as it is going to get minus maybe some of the new fields that may need to be moved around at some point after they are actually set to pull from the database so they can be tested.

I was able to correct the issue with some items showing as stacking that shouldn't be. I just set the stacksize field to check the IsStackable bool before setting stacksize higher than 0. It seems in this case that charges and stacksize seem to work interchangeably, but if I am wrong, that may need to be adjusted again. Though, stack sizes and item charges seem to be correct now as far as I can tell. I also identified one of the unknown fields in the serialization that was set to always send ff ff ff ff. According to the EQLive packets I was comparing it to, that is the number of current charges an item has. In most cases, that would be 00 00 00 00 for normal items, and ff ff ff ff for clickies with unlimited charges. But, for potions or clickies with limited charges, that should be where the current charges are set on the item. Unfortunately, since clickies aren't working yet, I can't test to see if that updates properly or not just yet. I think it will, though.

It seems that there may have been some changes to some of the systems that aren't yet fully functional. I am not completely sure yet, but I think group update may have changed how it gets sent. It also seems that for deleting stacked items like food/drink or ammo, there are 2 new packets that are sent that seem to only contain the slot that the item is in. First, the client sends an 8 byte packet that only has the slot ID in it and then the server replies with a 12 byte packet that also only has the slot ID in it. Maybe this is some kind of extra confirmation that the item exists before the server tries to delete it. It could be that opcodes already exist for these 2 packets, but I can't find them if so. I also don't see that same thing happening in Titanium. In Titanium, it either just sends the consume packet or it gets sent the delete packet depending on if it is food/drink or ammo.

So, there may need to be a bit of coding to finish off some of the last issues with SoF, but I don't think it will be all that much. At least we have made some nice progress with Items and AAs over the past couple of days Still knocking out 1 issue at a time, but the list is slowly shrinking.

[trevius]

Last night, I was able to get selling to merchants working properly. Just had to adjust for the special slot changes that KLS setup in the item struct and do an encode/decode for selling items.

Also, I got trading working from PC to PC/NPC. Though, it will bug your character if you cancel a trade and require you to zone or relog to fix it.

I also corrected a stacking issue introduced in Revision 376 that was causing stackable items that weren't in a stack to show up as being non-stackable.

I thought I did a commit of those changes last night, but I must have forgotten to. I will get it on the SVN tonight.

I am pretty sure that AAs are close to being usable now too, but I just need to figure out why it isn't using the encode I set on it when it sends the packet. It always wants to send a 1920 byte sized packet, which is the same that it sends in Titanium. I am guessing that it is just not using the structs at all and is being forced to send that way in the AA.cpp file. Maybe I can figure out how to set it to use the structures like all other normal packets do. I am sure that respondAA is why AAs aren't being updated after purchasing them until you zone currently. Once respondAA is working properly, that should fix that problem. I am hoping it will also make AAs actually start working too, and it just might. Other than that, it may be that SendAAStats may be needed for AAs to actually work. More testing needed on that one though.

Big thanks to Xinu for helping me by collecting opcodes and logs from Live to make this work alot easier. He has gotten me everything I asked for so far and does it quicker than I would have

[trevius]

The SVN has been updated with the updates in the previous post.

Last night, I was able to get the RespondAA packet working almost 100%. This means that we can now buy AAs and they update in the AA window properly without requiring you to zone. It turned out I had the wrong opcode set for it this whole time. It is really odd, because IDA seemed to be pretty clear on what the right one was for Titanium, SoF and Live. Finally figured it out from a packet collect by Xinu. The RespondAA is now sending the correct packet format and size (3604 bytes) for SoF. The only problem I am still having with it is that RespondAA is now responsible for sending the total Spent AA points for the AA window. I have tried a few ways of getting or counting the spent AAs, but everytime it results in the wrong amount. So, for now, I have it hard set to just display 500 spent. Any ideas of how to get the total AAs spent for this would be appreciated. Here is the current encode I have being done:

ENCODE(OP_RespondAA) {
	ENCODE_LENGTH_EXACT(AATable_Struct);
	SETUP_DIRECT_ENCODE(AATable_Struct, structs::AATable_Struct);
	/*
	int aa_spent_total = 0;
	int k;
	for(k = 0; k < structs::MAX_PP_AA_ARRAY; k++) {
		if (emu->aa_list[k].aa_value !=0)
			aa_spent_total += emu->aa_list[k].aa_value;
	}
	*/
	eq->aa_spent = 500; //aa_spent_total;

	int r;
	for(r = 0; r < structs::MAX_PP_AA_ARRAY; r++) {
		OUT(aa_list[r].aa_skill);
		OUT(aa_list[r].aa_value);
		eq->aa_list[r].unknown08 = 0;
	}
	FINISH_ENCODE();
}

I was hoping that getting RespondAA working would make AAs actually start working properly, but it didn't seem to change anything in that aspect. It does seem that some of the ones that have hotkeys for them work for creating hotkeys. Also, most AA effects are handled serverside anyway, so as long as we buy them, they will work. The ones that I am mostly trying to figure out are the innate stats and planar power ones to increase stat caps. None of those seem to be working. Though, I am pretty sure those were changed due to the restructuring of the AA tables for the new window format. For example, many AAs that were improvements to previous AAs were combined into 1 AA that just allowed higher training in the same AA. This includes innate stats, resists, defensive, hps, etc. Because of the big overhaul on SoF AAs, for them to be correct, we will want to make new AA tables specifically for SoF. The best way to populate those tables would be by getting the packet collectors working for Live that can parse AA packets and fill in a table with the info from them. Then, just log in 1 of each class and collect them all and fill in the entire AA table(s) that way.

At least for now, AAs should be functional enough to get by with. But, I think this is still a fairly high priority. Some will work, but a fairly large amount of them won't until new tables are created.

If I can get the Total AAs Spent field working properly, I will get this update on the SVN later tonight. Otherwise, I may hold-off until it is working to get the correct AA Spent Total.

[KLS]

Just don't break AAs for other clients =p

[trevius]

I think everything would pretty much have to be done directly from the SoF patch files, so I don't think that would have a chance to break anything (I hope) for other clients. Though, I really have no idea how I would handle pulling all new AA table information into the SoF.cpp properly to use for an encode. Someone with more database/code experience might have to help with that one. I can handle getting most opcodes, encodes/decodes and packet structures fixed, but I am not so great at coding/database stuff just yet lol.

It may just be that the increasestats_struct isn't setup correctly. Though, I am not exactly sure what that struct is for, but I would guess that it could be related to increasing stats from AA purchases.

And I haven't broken anything for other clients that I didn't correct almost immediately. At least I as far as I am aware of! :P

[trevius]

The list of what needs to be fixed in SoF is growing smaller daily Last night, I got the NPC/PC trading bug fixed from when the window is closed as well as the tradeskill container bug when the window is closed. Live sends 2 new opcodes when those windows are closed and Titanium doesn't send them. I had to add 2 new opcodes for this to work. I should probably have this updated on the SVN later tonight.

I also got food consumption working and right click items are almost working. There are some new opcodes being used on Live for when an item is right clicked to be used. I haven't figured out exactly what is being sent in all cases, but in most cases, the client is just sending the slot that the item is in and then the server replies with that same slot. It also sends the entity ID of the client's current target. I assume this is some kind of verification that the item exists or something. I added 2 new opcodes for this as well and some handling for it. That got food/drink consumption working, but item right click effects still aren't actually casting their effects yet. It did stop the client from being bugged when trying to use right click effects though. From Live, it seems that these new item verification packets actually trigger the server to cast the effect. In Titanium, right clicking an item is just like casting a spell. The client sends the castspell packet and the server handles it from that point. Since it seems like SoF uses the new way from Live, it sounds like we will have to set the item verification (that is what I am calling it for now anyway) handling to cast the spells. I don't think that should be too hard, but I am open to any code suggestions

I also think there may need to be something done to stop right clickable items with recast delays from showing up as grey. I am sure there is either some new opcode that is supposed to tell the item how long the recast countdown is, or the recast delay countdown may be something that goes into one of the unknowns in the item serialization (this is my guess).

The item verification packets seem to be pretty straight forward in most cases. The odd thing is that certain items send "23 fa ff ff" when clicked instead of sending the slot id. I'm not quite sure what that is supposed to be, but it doesn't seem to be anything to do with the slot. I saw that same thing from one of my live packet collects and also on my test server from clicking an item with charges. It will probably take more investigation before I can get clickies all working properly, but at least it is getting closer. I may put the beginning work of these new opcodes and opcode handling on the SVN tonight as well.

Other than AAs, which I think need new AA tables just for SoF and coding to handle that, I think grouping is really the last major thing that needs to be fixed in SoF to have it almost as fully functional as Titanium is. I imagine raids will also need work too, but groups need to be working before we can get to working on raids. Either way, it is almost all done!