Some New Map Files

[gernblan]

Maybe I'm dumb but someone please explain to me what the "- 2" or "+ 19" mean in the offsets that were already hacked.

The wiki says that this offset will be on a 16 byte boundary but those do change that, no?

Examples:

} else if(string("ter_guildhall.ter") == zone_name) {
buffer = ter_orig + 0x3080 - 2;
} else if(string("ter_guildlobby.ter") == zone_name) {
buffer = ter_orig + 0x4190 + 19;

So if the offset for guildhall is 0x3080 as stated in the wiki example... why subtract 2 from it?

...and why add 19 to guildlobby's?

I at least need a basic understanding of these values and why they are the way they are (and different from the wiki) before I can really sit down and hack away at these zones for us.

[uncommon]

If you look at the various example in the ter.cpp file it's not always on a 16 bit boundary.
About the guildhall example, 0x3080 - 2 is obviously the same as saying it's 0x307e (i tried and it works)... personnaly that's how i found the barter.eqg opcode (it's 0x56AD), i was sure it was around 0x56B0 but that one didn't worked so i tried 0x56B0 - 1 then 0x56B0 - 2 etc... till it worked. Probably an horrible way to find it but it worked.

I hope that's clear ? Sorry my english sucks.

[gernblan]

VERY clear, thank you!

May I ask what fhalls was too?

As we find them, let's post the offsets so that we can all have them, please?

Ok, knowing what you just told me, I'll going to try to work on another zone.

[uncommon]

For sure, let's do that, fhalls.eqg is 0x2230.

[KLS]

There's a handful of zones up to depths that I can't load and render properly atm, but it's just a matter of finding the offsets at this point. Should probably look for a more general solution, for version 2 EQGs but it's frustrating..

There's a freaking chunk of unknown data before the vertex data, somehow related to the material layers and a potential chunk of unknown data after the index data.. making finding the start of the vertex data pretty much impossible without offsets or identifying the unknown data before the vertex data.

[gernblan]

Quite right KLS.

I'm working on it feverishly. I am thinking of a brute force method.. maybe writing a perl script to cycle the offsets, compile azone, execute it.

Would be ugly but it just may work...

That being said, my perl isn't up to par yet but I'm going to try anyway.

[gernblan]

I'm having a hell of a time writing that script... my perl just isn't good enough yet.

Anyone up to taking a stab at this? It just may help get more zone maps made.

What the script needs to do:

(first you'd manually run azone on the zone you want to hack to get the .ter file name (since it's not always the same as the zone shortname), then you'd go into the ter.cpp file and add an offset line for that zone)

1) using a loop that increments the offset value you want to try (loop range of, say 0x0500 to EOF maybe?) ...load the ter.cpp file and at the offset location for the zone you want to hack, change it to the loop value, then save it back out.
2) compile azone
3) run azone on the zone you are hacking
4) trap the output to see if it gave a segfault. If it did, loop again. if it did NOT crash, double check for a .map file for that zone. If one exists, exit, reporting the offset found.

SO, part of this is manual and the script would have to be tweaked some each time because a new line like:

else if(string("ter_hive.ter") == zone_name) {
	  //drachnidhive - not working yet
	  buffer = ter_orig + 0x2B36;
  }

... would have to be added. The example above I added to try to hack drachnidhive.

Anyone up for banging out a perl script that can do this? I promise I'll hack the living heck out of it and then post the ter.cpp with the correct offsets for the zones I am able to make maps for (as well as the maps themselves) if you do.

So, the short of it is that the perl script would help change the offset in ter.cpp in a brute force way and be smart enough to know when it worked.

Then the user manually adds another line to ter.cpp like the above, changes the perl script to point to the offset location in ter.cpp (would be great if the script parsed that line and you could just tell it what line it was on!)... then the user runs the brute force script on the next zone... etc.

Is this idea stupid? Anyone think that it will not work? If not, why not?

Thank you.

[Derision]

Originally Posted by KLS

Should probably look for a more general solution, for version 2 EQGs but it's frustrating..

I don't know whether the Dev's worked out the formula for calculating the offsets for version 2 EQGs, but I searched and couldn't find anything relevant after this thread.

I've been messing about trying to get OpenEQ to load version 2 EQGs, and came across this old thread:

http://www.eqemulator.net/forums/arc...p/t-21615.html

Based on that, I knocked together the following bit of Python to calculate the offsets (excuse my Python, but I'm new at it ):

import struct, posixfile, socket, zlib, pdb, sys



for eqgfilename in ["broodlands", "guildhall", "guildlobby", "harbingers", "stillmoona", "stillmoonb", "thenest", "thundercrest"]:
    filenames = []
    files = []
 
    eqgfile = file(eqgfilename + '.eqg', 'rb')

    block = eqgfile.read(12)
    (offset, magic, unknown) = struct.unpack('L4sL', block)


    eqgfile.seek(offset, posixfile.SEEK_SET)

    block = eqgfile.read(4)

    dir_count = struct.unpack('I', block)



    for i in range(0, dir_count[0]):
        block = eqgfile.read(12)
        (crc, fileoffset, filesize) = struct.unpack('LLL', block)


        if crc == 0x61580AC9:

            CurrentPos = eqgfile.tell()
	    eqgfile.seek(fileoffset, posixfile.SEEK_SET)
	    InflatedLength = 0
	    uncompressed = ''
   	    while InflatedLength < filesize:
	        block = eqgfile.read(8)
	        (deflen, inflen) = struct.unpack('LL', block)

	        block = eqgfile.read(deflen)
	        uncompressed = uncompressed + zlib.decompress(block)
	        InflatedLength = InflatedLength + inflen

	    eqgfile.seek(CurrentPos, posixfile.SEEK_SET)
            fncount = struct.unpack('L', uncompressed[0:4])	

	    pos = 4
	    for j in range(0, fncount[0]):
	        fnlen = struct.unpack('L', uncompressed[pos:pos+4])

	        fmt = str(fnlen[0]) + 's'
	        fname = struct.unpack(fmt, uncompressed[pos+4:pos+4+fnlen[0]])
                fname = fname[0].strip('\x00')

                if fname[len(fname)-4:len(fname)] == '.ter':
                    wantedfile = fname

	        filenames.append(fname)
	        pos = pos + 4 + fnlen[0]
        else:
            files.append((eqgfile.tell() - 12, fileoffset))


    for i in range(dir_count[0] - 2, 0, -1):
        for j in range(0, i):

            if files[j][1] > files[j+1][1]:
                tmp = files[j]
                files[j] = files[j+1]
                files[j+1] = tmp



    for a in range(0, dir_count[0]-1):
        if filenames[a] == wantedfile:

	    eqgfile.seek(files[a][0], posixfile.SEEK_SET)
            block = eqgfile.read(12)
            (crc, fileoffset, filesize) = struct.unpack('LLL', block)

	    eqgfile.seek(files[a][1], posixfile.SEEK_SET)
	    break

    uncompressed = ''
    inf = 0

    while inf < filesize:
        block = eqgfile.read(8)
        (deflen, inflen) = struct.unpack('LL', block)

        block = eqgfile.read(deflen)
        uncompressed = uncompressed + zlib.decompress(block)
        inf = inf + inflen

    (magic, version, list_len, obj_count, vert_count, tri_count) = struct.unpack('4sLLLLL', uncompressed[0:24])


    ter_tmp = list_len
    pos = 24
    while pos < ter_tmp:
        strlen = 0 
        while uncompressed[pos+strlen] != chr(0):
            strlen = strlen + 1
        fmt = str(strlen) + 's'
        (strvar) = struct.unpack(fmt, uncompressed[pos:pos+strlen])
        pos = pos + strlen+1

        strlen = 0 
        while uncompressed[pos+strlen] != chr(0):
            strlen = strlen + 1
        fmt = str(strlen) + 's'
        (strval) = struct.unpack(fmt, uncompressed[pos:pos+strlen])
        pos = pos + strlen+1



    pos = 24 + list_len
  
    for b in range(0, obj_count):

        (index, name_offset, another_name_offset, property_count) = struct.unpack('LLLL', uncompressed[pos: pos+16])

        pos = pos + 16
        for a in range(0, property_count):
            pos = pos + 12

    print "Offset for EQG Zone %-20s (TER: %-22s) is %8X" % (eqgfilename, wantedfile, pos)

For the EQG zones hardcoded in the currently downloadable version of azone, it spits out the same offsets:

Offset for EQG Zone broodlands           (TER: ter_broodlands.ter    ) is    382B7
Offset for EQG Zone guildhall            (TER: ter_guildhall.ter     ) is     307E
Offset for EQG Zone guildlobby           (TER: ter_guildlobby.ter    ) is     41A3
Offset for EQG Zone harbingers           (TER: ter_harbingers.ter    ) is     1178
Offset for EQG Zone stillmoona           (TER: ter_main.ter          ) is    71948
Offset for EQG Zone stillmoonb           (TER: ter_easterntemple.ter ) is     E33E
Offset for EQG Zone thenest              (TER: ter_abyss01.ter       ) is    CA244
Offset for EQG Zone thundercrest         (TER: ter_stormtower01.ter  ) is    6222C