OpenEQ Progress report #1: 10/2/04

[daeken_bb]

Well, development has been moving a bit slowly on the last few days, but it's picking up now. I'm rewriting and redesigning a decent portion of the zone loading code now, as jbb has figured out the final major parts of the placeable object code. (THANK YOU!)

I wrote a new linked-list implementation which is extremely powerful, and wrote wrapper macros around it to simplify its use as much as possible.

What follows is our agenda for what needs to be done in the next week or two. If you have any interest in working on any of these, please reply here so that everyone can see it and we don't have multiple people working on the same things.

1) Work on figuring out the character model formats and also the animations in the new zones.
2) Work on making the WLD convertor output a fully packaged EQG complete with proper placeable objects and all.
3) Write a decent UI implementation to replace my half-assed one.
4) Rewrite the zone loading code. I'm currently working on this.


Thanks to everyone who has helped out

Happy Hacking,
Lord Daeken M. BlackBlade
(Cody Brocious)

Edit: I usually spell implement 'impliment', even though I know the proper spelling of it... freaky :P

[kathgar]

You are probably better off using STL instead of a custom linked list. The original way that EQEMu stored entities was horrible and was a pain in the ass to change when I noticed how horrible it was.

EDIT: For the love of god it is implement

[daeken_bb]

Originally Posted by kathgar

You are probably better off using STL instead of a custom linked list. The original way that EQEMu stored entities was horrible and was a pain in the ass to change when I noticed how horrible it was.

EDIT: For the love of god it is implement

I would use the STL, but we're pure C, not C++

Not to mention that whole hatred of the STL and everything associated with it :P

[kathgar]

Yeah, that would be a problem. Still, linked-lists are not the best choice. If you are going to store all mobs(npcs+players) you probably want them stored in a r-b binary tree based on id instead of a linked-list. Hell if you want store them in a linked list but have a way of referencing them better than iterating through the linked list.

[daeken_bb]

Originally Posted by kathgar

Yeah, that would be a problem. Still, linked-lists are not the best choice. If you are going to store all mobs(npcs+players) you probably want them stored in a r-b binary tree based on id instead of a linked-list. Hell if you want store them in a linked list but have a way of referencing them better than iterating through the linked list.

Well, that stuff is quite a while down the road still, but if we go with LLs for that, everything will be abstracted with macros. The LL implementation I wrote was for placeable objects, so it's fairly minor for now

[jbb]

I'm less hopeful about being able to figure out the character models and animation details. I can't decide if I should look at that, the new zone lighting information, or making a loader dll for my engine for old style zones first.

[daeken_bb]

Originally Posted by jbb

I'm less hopeful about being able to figure out the character models and animation details. I can't decide if I should look at that, the new zone lighting information, or making a loader dll for my engine for old style zones first.

The character model data should be very simple... the animation might be a bit more complex, though. I might work on that sometime soon, but I dunno yet.

The lighting data should be very simple and pretty much trivial to RE.
As for making a loader DLL... sounds like *cough* fun :P

[jbb]

The lighting data should be very simple and pretty much trivial to RE

I've not been able to make any sense at all of the LIT files yet, althouth I've only spent half an hour looking at them.

[Windcatcher]

Not to worry...I've been working on mob animation and I believe I've figured it out...

In the 0x12 fragment, we *thought* that the fields were like this:

RotateDenominator
RotateXNumerator
RotateYNumerator
RotateZNumerator
TranslateXNumerator
TranslateYNumerator
TranslateZNumerator
TranslateDenominator

Where they were either 4 bytes or eight bytes, depending on a flag.

The translate stuff is okay, but the rotation stuff isn't. Actually they are Euler parameters, and the fields should be like this:

E0
E1
E2
E3
TranslateXNumerator
TranslateYNumerator
TranslateZNumerator
TranslateDenominator

You animate a mob in this way: from the Euler parameters E0...E3, form a 3x3 transformation matrix. The matrix describes a rotation around an arbitrary axis. Then, instead of rotating around X, Y, and Z as before, multiply the matrix by the (X, Y, Z) position vector for the vertex:

V' = VM

Then translate as before. The matrix multiplications, like the rotations we did earlier as well as the translations, are cumulative. Here is an example code snippet (this routine is actually nested in a larger one and not all variables it uses are declared here):

    Procedure CalculatePieceInfo(Frame: Single; Index,LastIndex: Integer; XOfs,YOfs,ZOfs: Double);
    // ------------------------------------------------------------------------------------------
    // Routine for calculating skeleton piece information as soon as the skeleton has been
    // loaded.  The idea is to precalculate information for each piece but calculate final
    // (X, Y, Z) vertex values on the fly with information stored here.  It requires way too
    // much RAM to cache final vertex values for each piece of each skeleton so this is the
    // next best solution.
    //
    // Frame ranges from 0 to 1 and represents the animation frame.  Generally this routine
    // will always be called such that frame references a unique frame (e.g. an animation with
    // three frames will be called only three times).
    //
    // Index is the piece index as taken from the Data10 tree.  0 is the first piece index.
    //
    // LastIndex is like Index but is the index for the previous piece, or -1 if we are working
    // on the first piece in the skeleton tree.
    //
    // XOfs, YOfs, and ZOfs are the cumulative translations from the last piece we did in the
    // skeleton tree, or (0, 0, 0) if we are working on the first piece.
    // ------------------------------------------------------------------------------------------
    Var
      I           : Integer;
      D12         : Data12;
      XOfs1       : Double;
      YOfs1       : Double;
      ZOfs1       : Double;
      FrameNum    : Integer;
      E0,E1,E2,E3 : Single;
      Len         : Single;

    Begin
      If (Index <= High(Pieces)) Then
      Begin
        D12      := Pieces[Index].D12;

        // Figure out the actual frame from the animation value we passed 

        FrameNum := Round(Frame * (D12.Size1 - 1));

        // Calculate the rotation angle of this piece

        If (D12.Flags And 8) <> 0 Then
        Begin
          // The Data12 structure contains a vector in the form of signed 16-bit values.
          // The Euler parameters represent an arbitrary rotation around a unit normal
          // (see http://mathworld.wolfram.com/EulerAngles.html).
          //
          // We can to convert them to properly scaled floating-point values by dividing
          // by the length of the given vector (essentially normalizing the values).

          E0 := D12.Data4[FrameNum].E0;
          E1 := D12.Data4[FrameNum].E1;
          E2 := D12.Data4[FrameNum].E2;
          E3 := D12.Data4[FrameNum].E3;

          Len := Sqrt(Sqr(E0) + Sqr(E1) + Sqr(E2) + Sqr(E3));
          E0  := E0 / Len;
          E1  := E1 / Len;
          E2  := E2 / Len;
          E3  := E3 / Len;
        End
        Else
        Begin
          // The Euler parameters are passed as 32-bit floating-point values.  No
          // normalization is necessary.

          E0 := D12.Data8[FrameNum].E0;
          E1 := D12.Data8[FrameNum].E1;
          E2 := D12.Data8[FrameNum].E2;
          E3 := D12.Data8[FrameNum].E3;
        End;

        // Get the transformation matrix values from the Euler parameters, transposing the matrix because
        // of the way we multiply things (my multiply does V' = MV instead of V' = VM, and my matrix
        // multiplication routines do M' = M1 x M instead of M' = M x M1)

        Pieces[Index].A11 := Sqr(E0) + Sqr(E1) - Sqr(E2) - Sqr(E3);
        Pieces[Index].A21 := 2 * (E1 * E2 + E0 * E3);
        Pieces[Index].A31 := 2 * (E1 * E3 - E0 * E2);
        Pieces[Index].A12 := 2 * (E1 * E2 - E0 * E3);
        Pieces[Index].A22 := Sqr(E0) - Sqr(E1) + Sqr(E2) - Sqr(E3);
        Pieces[Index].A32 := 2 * (E2 * E3 + E0 * E1);
        Pieces[Index].A13 := 2 * (E1 * E3 + E0 * E2);
        Pieces[Index].A23 := 2 * (E2 * E3 - E0 * E1);
        Pieces[Index].A33 := Sqr(E0) - Sqr(E1) - Sqr(E2) + Sqr(E3);

        // Make this matrix cumulative with the one from the previous piece

        If LastIndex >= 0 Then
        Begin
          M1 := T3x3Matrix.Create(Pieces[Index].A11,Pieces[Index].A12,Pieces[Index].A13,
                                  Pieces[Index].A21,Pieces[Index].A22,Pieces[Index].A23,
                                  Pieces[Index].A31,Pieces[Index].A32,Pieces[Index].A33);
          M2 := T3x3Matrix.Create(Pieces[LastIndex].A11,Pieces[LastIndex].A12,Pieces[LastIndex].A13,
                                  Pieces[LastIndex].A21,Pieces[LastIndex].A22,Pieces[LastIndex].A23,
                                  Pieces[LastIndex].A31,Pieces[LastIndex].A32,Pieces[LastIndex].A33);
          M1.Multiply(M2); // M1 = M2 x M1

          // Store the matrix parameters for this piece.  It's too expensive from a RAM standpoint
          // to save every vertex for every frame of every skeleton, so we're going to save the
          // information we need to create the transformed vertices on the fly.  Translation information
          // will be saved below.

          Pieces[Index].A11 := M1.M[1,1];
          Pieces[Index].A12 := M1.M[1,2];
          Pieces[Index].A13 := M1.M[1,3];
          Pieces[Index].A21 := M1.M[2,1];
          Pieces[Index].A22 := M1.M[2,2];
          Pieces[Index].A23 := M1.M[2,3];
          Pieces[Index].A31 := M1.M[3,1];
          Pieces[Index].A32 := M1.M[3,2];
          Pieces[Index].A33 := M1.M[3,3];
          M1.Free;
          M2.Free;
        End;

        // Get the translation vector for this piece

        If (D12.Flags And 8) <> 0 Then
        Begin
          XOfs1 := D12.Data4[FrameNum].MoveXNumerator / D12.Data4[FrameNum].MoveDenominator;
          YOfs1 := D12.Data4[FrameNum].MoveYNumerator / D12.Data4[FrameNum].MoveDenominator;
          ZOfs1 := D12.Data4[FrameNum].MoveZNumerator / D12.Data4[FrameNum].MoveDenominator;
        End
        Else
        Begin
          XOfs1 := D12.Data8[FrameNum].MoveXNumerator / D12.Data8[FrameNum].MoveDenominator;
          YOfs1 := D12.Data8[FrameNum].MoveYNumerator / D12.Data8[FrameNum].MoveDenominator;
          ZOfs1 := D12.Data8[FrameNum].MoveZNumerator / D12.Data8[FrameNum].MoveDenominator;
        End;

        // Rotate the translation vector using the matrix transformation from the previous skeleton piece

        If LastIndex >= 0 Then
        Begin
          M2 := T3x3Matrix.Create(Pieces[LastIndex].A11,Pieces[LastIndex].A12,Pieces[LastIndex].A13,
                                  Pieces[LastIndex].A21,Pieces[LastIndex].A22,Pieces[LastIndex].A23,
                                  Pieces[LastIndex].A31,Pieces[LastIndex].A32,Pieces[LastIndex].A33);
          M2.Multiply(XOfs1,YOfs1,ZOfs1);
          M2.Free;
        End;

        // Add the cumulative translation from all previous pieces as we walked the skeleton tree

        XOfs := XOfs + XOfs1;
        YOfs := YOfs + YOfs1;
        ZOfs := ZOfs + ZOfs1;

        // Save the translation information in the piece so we can animate on the fly

        Pieces[Index].XOfs := XOfs;
        Pieces[Index].YOfs := YOfs;
        Pieces[Index].ZOfs := ZOfs;

        // Recursively calculate for any skeleton pieces that this piece references

        For I := 0 To D10.Data1[Index].Size - 1 Do CalculatePieceInfo(Frame,D10.Data1[Index].Data[I],Index,XOfs,YOfs,ZOfs);
      End;
    End; // CalculatePieceInfo

Then to animate, take the given vector from the Data36/Data2C/Data37 structure, multiply by the stored matrix, and add the stored translation vector for the piece that vector goes with. If anyone has any questions I'd be happy to discuss it. I've tested this with loads of models and it works really well.

WC

[daeken_bb]

Very cool WC, but that's WLD-only, right?

Have you taken a look at the mob animation for the new file formats?

[Windcatcher]

Nope...I'm only tackling WLD at the moment...I plan on looking at the new stuff at some point, but I have a lot on my plate at the moment (more than you know, ) I have a strong suspiction that mathematically the new stuff will be similar.

WC

[daeken_bb]

Originally Posted by Windcatcher

Nope...I'm only tackling WLD at the moment...I plan on looking at the new stuff at some point, but I have a lot on my plate at the moment (more than you know, ) I have a strong suspiction that mathematically the new stuff will be similar.

WC

Well, I will try to figure out the new animations within the next few weeks of no one else does

I'm working on some stuff alongside OpenEQ, so I can't devote all of my time to it right now, otherwise I'd be working on that stuff already hehe