Why bitshift to get to the x,y,z co-ords

[moofta]

OK, so I have been hitting my head against the packet structs trying to update EQExtractor2 to match Live. There has been some trial and error, and also some blatant ripping off of ShowEQ (who seem to have it working ), and of course also ripping off Derisions work. PASt that, I must admit this is stretching my ability, but please be gentle :P

I can basically get enough of OP_PlayerProfile working, and OP_ZoneEntry is going OK until I get to the co-ordinates. Here's is the part of the ShowEQ spawnStruct relating to locs:-


union
{
struct
{
unsigned pitch:12;
signed deltaX:13; // change in x
unsigned padding01:7;
signed z:19; // z coord (3rd loc value)
signed deltaHeading:10; // change in heading
unsigned padding02:3;
signed x:19; // x coord (1st loc value)
signed deltaZ:13; // change in z
unsigned heading:12; // heading
signed deltaY:13; // change in y
unsigned padding03:7;
signed animation:10; // velocity
signed y:19; // y coord (2nd loc value)
unsigned padding04:3;
};
int32_t posData[5];
};


In the below code Postion1 relates to posData[0], Position2 to posData[1] and so on.



float XPos = Utils.EQ19ToFloat((Int32)(Position3) & 0x7FFFF); //Moofta-Verified
float YPos = Utils.EQ19ToFloat((Int32) (Position5 >> 10) & 0x7FFFF); //Moofta-Verified
float ZPos = Utils.EQ19ToFloat((Int32) ((Position4 >> 13) & 0x7FFFF));
//heading is definitely NOT Position3 unlike last patch that worked
float Heading = Utils.EQ19ToFloat((Int32)(Position3) & 0x3FF);


As you can see, X and Y use a bitwise AND (0x7FFFF), which is then converted using EQ19ToFloat (more on that later). That makes a kind of sense to me, since in the struct x,y,z are using the first 19 bits (at least that's how I read the 19 of "signed y:19;" ).

1. What doesn't make sense to me, is the bitshift of >> 10 or 13. Can anyone explain this to my tiny little brain? Is it because they padded out 10/13 bits then stored the number using the last 19? Presumably, therefore, the maximum shift would be 13 if we're using 19 bits?

2. I don't understand the EQ19ToFloat method really. If the 19th bit of EQ19Value is 0, it divides the value by 1<<3 (which is always 8? If so, why not use use 8 ). If it is 1, then it subtracts from EQ19Value the difference between the max value of 19 bits (0x7FFFF) +1. Why? NFC.


public static float EQ19ToFloat(Int32 EQ19Value)
{
if ((EQ19Value & 0x40000) > 0)
EQ19Value = -(0x7FFFF - EQ19Value + 1);
return (float)EQ19Value / (float)(1<<3);
}


I basically bruteforced the remaining Position* s from no bitshift up to >> 13 and no dice. I even thought f*ckit and bruted the bitwise AND all the way up to 32 bits. No dice. So I'm missing something, and I suspect it's my incomplete knowledge causing that. Any ideas/comments/welcome (other than "give up, retard" that is).

Hepl em!!1

[paulyc]

For one thing, it appears the code doesn't quite match the struct, specifically on calculating the z position. According to that struct the z bits are Position2 & 0x7FFFF. Heading calculation is clearly incorrect so I'm going to guess the ZPos is too (or for a different version of the packet?).

How to figure out where the bits are? Group the bit fields into 32 bit integers since the type of each bit field is int.
First 32 bits is posData[0] Position1
unsigned pitch:12;
signed deltaX:13; // change in x
unsigned padding01:7;

Second 32 bits, posData[1] Position2
signed z:19; // z coord (3rd loc value)
signed deltaHeading:10; // change in heading
unsigned padding02:3;

z is listed first so it is simply the lowest 19 bits of this 32 bit int, posData[1] & 0x7FFFF

Third posData[2] Position3
signed x:19; // x coord (1st loc value)
signed deltaZ:13; // change in z
again, x listed first, lowest 19 bits, posData[2] & 0x7FFFF

Fourth posData[3] Position4
unsigned heading:12; // heading
signed deltaY:13; // change in y
unsigned padding03:7;
nothing here

Fifth posData[4] Position5
signed animation:10; // velocity
signed y:19; // y coord (2nd loc value)
unsigned padding04:3;
animation is stored in the lowest 10 bits, y is stored in the next 19 bits, that's why it needs to be shifted right by 10. (posData[4]>>10) & 0x7FFFF

[paulyc]

Next question, EQ19ToFloat. The largest positive value we can store in a 19-bit integer is 0x3FFFF and the largest (magnitude) negative value we can store is 0x40000. But remember, our value is stored into only the lowest 19 bits of the 32-bit variable EQ19Value. Stored as a 32-bit int, 0x40000 is not a negative number at all, to properly store a negative 19-bit value into 32-bits the high bit needs to be sign extended all the way to the left, 0xFFFC0000.

So the first line of this function tests if the value is supposed to be negative, if it is, the second line does a little bit of trickery to sign extend the 19-bits into 32 (same as 0xFFF80000 | EQ19Value), now we have a proper signed native int.

The last line converts that properly sign extended 32-bit integer into a 32-bit float. Yes, (1<<3) is the same thing as 8, I think it is written like this to make it more clear what is going on. Whats going on is the lowest 3 bits of this integer are a fractional part. For example EQ locs are shown with one digit decimal value after the point, doesnt map perfectly to the range of .0 to .9 since 3 bits is 7 values so theres a little rounding error but

000 = .0
001 = .125
010 = .25
011 = .375
100 = .5
101 = .625
110 = .75
111 = .875

[paulyc]

Next question, EQ19ToFloat. The largest positive value we can store in a 19-bit integer is 0x3FFFF and the largest (magnitude) negative value we can store is 0x40000. But remember, our value is stored into only the lowest 19 bits of the 32-bit variable EQ19Value. Stored as a 32-bit int, 0x40000 is not a negative number at all, to properly store a negative 19-bit value into 32-bits the high bit needs to be sign extended all the way to the left, 0xFFFC0000.

So the first line of this function tests if the value is supposed to be negative, if it is, the second line does a little bit of trickery to sign extend the 19-bits into 32 (same as 0xFFF80000 | EQ19Value), now we have a proper signed native int.

The last line converts that properly sign extended 32-bit integer into a 32-bit float. Yes, (1<<3) is the same thing as 8, I think it is written like this to make it more clear what is going on. Whats going on is the lowest 3 bits of this integer are a fractional part. For example EQ locs are shown with one digit decimal value after the point, doesnt map perfectly to the range of .0 to .9 since 3 bits is 7 values so theres a little rounding error but

000 = .0
001 = .125
010 = .25
011 = .375
100 = .5
101 = .625
110 = .75
111 = .875

By the way, this type of number format, which has some integer bits and some fractional bits, is known as fixed-point. It is not as commonly used as floating-point and not typically implemented in hardware, which is why this function exists, to convert the value into a format that the CPU can natively manipulate.

So to summarize, EQ19ToFloat takes a 19-bit fixed-point number, which has 16 integer bits and 3 fractional bits, and converts it to a 32-bit floating-point number.