Out of ideas

[provocating]

I am perplexed on this one. I love difficult problems but I am running out of ideas on where to look, maybe someone will have some ideas where else I can look.

Basically you never get to the character screen, never even see the loading bar. Crash log....

[Wed May 20 08:12:13 2015]00043:WorldAuthenticate: Initiating Login.
[Wed May 20 08:12:13 2015]00044:WorldAuthenticate. I got a message of type 0x5b0b (23307).
[Wed May 20 08:12:13 2015]00045:WorldAuthenticate. I got a message of type 0x6f79 (28537).
[Wed May 20 08:12:13 2015]00046:WorldRPServer message: server name Legacy of FrostStone

[Wed May 20 08:12:13 2015]00047:WorldAuthenticate. I got a message of type 0x86c7 (34503).
[Wed May 20 08:12:13 2015]00048:Networking: Connection Closed [0] with 0 pending bytes.

These are the conditions.

Only happens on underfoot, Titanium and ROF2 do not do this.
Seems to happen on creation of 5th character, but not always? Makes no sense but that is what happens.
Deleting 5th character solves it
Adding 6th character manually solves it

[provocating]

To further add to the strangeness of this issue. If I move the 5th character to another account, like my admin account then both accounts are fine. When I move that character back to the original account it crashes.

If I leave the newly added character to the admin account but then create a new character on the crashing account, it appears fine. I have checked all starting items, they all exist. The spawn location for the character definitely exist, since it only happens on Underfoot that is another oddity. I have seen one or two post here from searching that hint at the same problem but no one ever reported back that they figured it out, I definitely will report back the resolution.

[NatedogEZ]

The only time I see this is if there is a corrupt character on the account.. i still have no idea how the corrupt character comes into existence, except for crashing while creating a new character and then that account is now bugged till the character is removed.. or they log in with RoF2 (the character wont be bugged on this client)

But not sure how to replicate the client crash upon character create though :(

[provocating]

Well at least it is not my imagination. I am going to keep looking into it, because it is not just any character that does it. So it is almost like it is the starting city, bind location, something like that.

I would say that is would just be a fluke, but I created a 5th character while I was playing and noticed the crash for me. Since we only have 5 players while in Beta for LoF I need to get this resolved.

[provocating]

Well a human cleric did not do it as the 5th character, but a barbarian rogue did make it crash. I think this has to do with character create combinations. Since I have been working for a while getting original character start set, this is probably on me screwing something up. I will check to make sure all my ducks in a row.

[Drajor]

Dump and diff every OP_SendCharInfo payload if you have not already.

[provocating]

Drajor can you explain that in a little more detail?

[Drajor]

Sure! Add a call to FileDumpPacketHex at the end of the UF encoder for OP_SendCharInfo (here (https://github.com/EQEmu/Server/blob/master/common/patches/uf.cpp#L2197)). Collect a good send (where there are 4 characters or less), then collect a bad send (on creation of the 5th character). Use a program like winmerge to have a look at the differences in what is being sent.

[provocating]

Sure! Add a call to FileDumpPacketHex at the end of the UF encoder for OP_SendCharInfo (here (https://github.com/EQEmu/Server/blob/master/common/patches/uf.cpp#L2197)). Collect a good send (where there are 4 characters or less), then collect a bad send (on creation of the 5th character). Use a program like winmerge to have a look at the differences in what is being sent.

Beautiful, pretty sure I understand that so I will give it a shot.

[Uleat]

If you do a barbarian rogue in the 6th slot, does it still work as normal or do you crash?


Also, is the crash an actual server crash? If so, please post the server crash log.


There's a LOT of packet data that is set based on observed packets collected during the client's active cycle.

It could be that we're limiting things without knowing..but, it would take some digging to figure out if that's the case and
what needs to be done.

[provocating]

I have spent some time on the problem, but not as much as I liked. This is what I have found recently. This also happens on the latest source.

I can create the exact same Male Barbarian Rogue, same starting location and diety, same facial features and depending on the name, depends on if it crashes or not. The server never crashes, just the client. Now get this if on this persons account I have 5 characters and create this 6th character with a name I know will crash, nothing happens. If I delete any character so that named character is now the 5th, the client crashes.

So temporarily we have a dummy character on his account so he can play with this new Rogue called Bingham. This is the strangest thing I have seen on one of my servers yet. I have run into strange problems but not one I could not pinpoint what is causing it. So I have to figure it out, I love a problem like this.

So far names that caused it

Bingham
Binghan
Chronos

I am not sure how the name is related but if I create a character with a name of Zalbat, Tester no crashes. Maybe a sorting issue? A lower alphabet name would fall to the top. That is all I thought of so far. Maybe it is coincidence? I still have a lot of testing to do. Depending on what goes on this weekend, I may have time to spend on it.

[Uleat]

I sent you a PM on this..let me know privately if that case is true since it's exploitable.

[Uleat]

Can you post some of the UF packet dumps, provocating?

[provocating]

I will try to Friday.

[provocating]

I pm'd both of you directly about the packet dump.

This is what I have found by trial and error. It has nothing to do with class, race, diety etc. I was incorrect in my assumption it was. It has to do with the naming of the characters and possibly what that does to the size of the packet.

When I add the 5th character if it is 7 characters, the UF client crashes. If I rename it with one less letter with HeidiSQL no crash. Then if I add a letter to any other character on the account it crashes.

So as a theory I did the exact same thing on my GM account. I made 5 dummy characters with the exact same lengths of names each. Instant crash.

How is that for an odd one?

I just tried it on both of my servers, which have a slightly different code base. I also updated my test server with the latest source, same exact thing.

[Shendare]

Hm. Wonder if it's a serialization issue, then. Total number of characters in character names going over a certain accidental limit based on buffer size?

[provocating]

I have no idea Shendare, this surpasses my knowledge of C++. Some things I can do and figure out but when you get into the streaming of data from client to server, that is past me. I will get there one day and this has been a great learning experience.

I will not post the magic combination here, but give it to the power that be.

EDIT: Just to be sure it is not my server that has something screwed up, I did it on PEQTGC and it crashed, 5 characters with the same combination. It boils down to the total character length of the characters names being a certain number.

[provocating]

Drajor and Uleat I sent both of you a packet dump

[Uleat]

I just committed a change that may address this issue.

Give it a shot and see it that helps.


EDIT: That will clean up my coding..if there's still an issue, it should be a little easier to trace.

[provocating]

Roger that. Probably will be Saturday before I can test it, but I definitely will. Thanks for looking into it, also excited to see what the change was.

[Uleat]

Since the buffer was an allocation and not an initialization, it would be possible to have the character at eq_cse.Name[<strlen> + 1]
not be 0...

And since I was taking name strlen from the eq buffer and not emu buffer, it could have thrown off the length of the entry.


If you still have issues, let me know and I'll look at the server generation code.

[provocating]

One of my servers is working off January source, and until I can move some of my custom things over, I am going to have to keep using that source. These are the lines I will have to change if I am thinking right.

bufptr += strlen(emu->name[r]);

And add this, like you did, which both will have to be done differently. The entire function has changed.

eq_cse->Name[0] = '\0';

This is the current function in total.

ENCODE(OP_SendCharInfo)
{
ENCODE_LENGTH_EXACT(CharacterSelect_Struct);
SETUP_VAR_ENCODE(CharacterSelect_Struct);

//EQApplicationPacket *packet = *p;
//const CharacterSelect_Struct *emu = (CharacterSelect_Struct *) packet->pBuffer;

int char_count;
int namelen = 0;
for (char_count = 0; char_count < 10; char_count++) {
if (emu->name[char_count][0] == '\0')
break;
if (strcmp(emu->name[char_count], "<none>") == 0)
break;
namelen += strlen(emu->name[char_count]);
}

int total_length = sizeof(structs::CharacterSelect_Struct)
+ char_count * sizeof(structs::CharacterSelectEntry_Struct)
+ namelen;

ALLOC_VAR_ENCODE(structs::CharacterSelect_Struct, total_length);

//unsigned char *eq_buffer = new unsigned char[total_length];
//structs::CharacterSelect_Struct *eq_head = (structs::CharacterSelect_Struct *) eq_buffer;

eq->char_count = char_count;
eq->total_chars = 10;

unsigned char *bufptr = (unsigned char *)eq->entries;
int r;
for (r = 0; r < char_count; r++) {
{ //pre-name section...
structs::CharacterSelectEntry_Struct *eq2 = (structs::CharacterSelectEntry_Struct *) bufptr;
eq2->level = emu->level[r];
eq2->hairstyle = emu->hairstyle[r];
eq2->gender = emu->gender[r];
memcpy(eq2->name, emu->name[r], strlen(emu->name[r]) + 1);
}
//adjust for name.
bufptr += strlen(emu->name[r]);
{ //post-name section...
structs::CharacterSelectEntry_Struct *eq2 = (structs::CharacterSelectEntry_Struct *) bufptr;
eq2->beard = emu->beard[r];
eq2->haircolor = emu->haircolor[r];
eq2->face = emu->face[r];
int k;
for (k = 0; k < _MaterialCount; k++) {
eq2->equip[k].material = emu->equip[r][k].material;
eq2->equip[k].unknown1 = emu->equip[r][k].unknown1;
eq2->equip[k].elitematerial = emu->equip[r][k].elitematerial;
eq2->equip[k].color.color = emu->equip[r][k].color.color;
}
eq2->primary = emu->primary[r];
eq2->secondary = emu->secondary[r];
eq2->tutorial = emu->tutorial[r]; // was u15
eq2->u15 = 0xff;
eq2->deity = emu->deity[r];
eq2->zone = emu->zone[r];
eq2->u19 = 0xFF;
eq2->race = emu->race[r];
eq2->gohome = emu->gohome[r];
eq2->class_ = emu->class_[r];
eq2->eyecolor1 = emu->eyecolor1[r];
eq2->beardcolor = emu->beardcolor[r];
eq2->eyecolor2 = emu->eyecolor2[r];
eq2->drakkin_heritage = emu->drakkin_heritage[r];
eq2->drakkin_tattoo = emu->drakkin_tattoo[r];
eq2->drakkin_details = emu->drakkin_details[r];
}

bufptr += sizeof(structs::CharacterSelectEntry_Struct);
}

FINISH_ENCODE();
}

[provocating]

Compiling now, I will report back.

Yeah, still crashing before character select.

[Uleat]

I pushed a memset as well in the charsel-gen code..but, that likely won't help your case.


Let me look over what you have in this encode.

I'm assuming that you're still using the Ti-based character select struct? (pre-variable toons per client version)


EDIT: Yourmemcpy(eq2->name, emu->name[r], strlen(emu->name[r]) + 1);essentially does the same thing as the Name[0] = '\0' - assuming that the server gen code has a proper nullterm.

Still looking over it..I'm not seeing an immediate problem..but, might need to compare the structs to make sure something isn't off there.


EDIT2: Does your UF::structs::CharacterSelectEntry_Struct have its name field declared as char name[1] or char name[0]?


EDIT3: Ok, just did a diff of crash/no crash hex dumps after adding in the 'ppy' to the crash one - there was no difference.

The packet structure appears to be properly filled in the crash version and I'm just not seeing anything in the encode function that would cause this.

Have you tried running it without the scope declarations? I'm wondering if the wrong struct isn't being referenced for size...

[provocating]

Uleat are you getting your client to crash on your end? I am just wondering if you are able to replicate the problem yourself?

[Uleat]

Nothing here..hence my somewhat random approach.

I'm just going over potholes in the server code and reviewing what you posted above.

[provocating]

It always does it for me using UF when the character name count with all characters in the account hits 38. Tried it on two servers with the same result, one being PEQTGC.

[Uleat]

Ahh, kk..last I tried was 7-char names. I'll take a look at the 38 count.


EDIT: I did one better..I created an account and used all of your names...

Yes, it crashed..so, it is definitely reproduceable in the base code.

[Uleat]

[Sun May 24 22:25:07 2015]01870:fatal error in main thread Code = c0000005 ADDR=0x2e4080c0

Buffer overrun...

[provocating]

How did you get a buffer overrun? Where did that present itself?

[Uleat]

I created an issue for the root cause of this: https://github.com/EQEmu/Server/issues/418

I (or someone) will push a fix once one is found and tested.

[Shendare]

Good detective work! Hopefully it can be figured out and put to rest.

[provocating]

I thank Uleat for being persistent.

Overall it is not much of a pain, small percentage of players will have this happen.

[Uleat]

Ok, just committed a fix for this.

Keep an eye out for any new problems since this change occurs in the handler for all out-going packets.

[Uleat]

I re-opened this issue.

It did create other problems...

[Shendare]

D'oh! Thanks for continuing to work on it!

[Uleat]

I'll keep looking at it..probably because I'm a masochist.

But, it will probably take someone more familiar with the netcode to figure this one out completely.

It's definitely the right area!

[Noport]

Unload your customized UI files reference Directx reload your video card drivers. Your Directx file MSVCR80.dll took a dump..

[provocating]

Unload your customized UI files reference Directx reload your video card drivers. Your Directx file MSVCR80.dll took a dump..

Why would it only do it with an account with a combination of character names equaling 38 characters?

[Noport]

Reason: problem is your Resolution settings
Goto Underfoot folder click and run OptionsEditor make sure Resolution match your display settings. Check the UI your using might not support that many characters.

[Uleat]

I think that I discovered the error of my ways...

Keeping it in a branch until it can be more fully tested: [branch merged and deleted]

(That is accessible through git.)

[Uleat]

Ok, this fix is now implemented in the base code.

Please report any server-based packet issues or newly occurring client crashes as a new issue.