Some New Map Files

[uncommon]

For sure, let's do that, fhalls.eqg is 0x2230.

[KLS]

There's a handful of zones up to depths that I can't load and render properly atm, but it's just a matter of finding the offsets at this point. Should probably look for a more general solution, for version 2 EQGs but it's frustrating..

There's a freaking chunk of unknown data before the vertex data, somehow related to the material layers and a potential chunk of unknown data after the index data.. making finding the start of the vertex data pretty much impossible without offsets or identifying the unknown data before the vertex data.

[gernblan]

Quite right KLS.

I'm working on it feverishly. I am thinking of a brute force method.. maybe writing a perl script to cycle the offsets, compile azone, execute it.

Would be ugly but it just may work...

That being said, my perl isn't up to par yet but I'm going to try anyway.

[gernblan]

I'm having a hell of a time writing that script... my perl just isn't good enough yet.

Anyone up to taking a stab at this? It just may help get more zone maps made.

What the script needs to do:

(first you'd manually run azone on the zone you want to hack to get the .ter file name (since it's not always the same as the zone shortname), then you'd go into the ter.cpp file and add an offset line for that zone)

1) using a loop that increments the offset value you want to try (loop range of, say 0x0500 to EOF maybe?) ...load the ter.cpp file and at the offset location for the zone you want to hack, change it to the loop value, then save it back out.
2) compile azone
3) run azone on the zone you are hacking
4) trap the output to see if it gave a segfault. If it did, loop again. if it did NOT crash, double check for a .map file for that zone. If one exists, exit, reporting the offset found.

SO, part of this is manual and the script would have to be tweaked some each time because a new line like:

else if(string("ter_hive.ter") == zone_name) {
	  //drachnidhive - not working yet
	  buffer = ter_orig + 0x2B36;
  }

... would have to be added. The example above I added to try to hack drachnidhive.

Anyone up for banging out a perl script that can do this? I promise I'll hack the living heck out of it and then post the ter.cpp with the correct offsets for the zones I am able to make maps for (as well as the maps themselves) if you do.

So, the short of it is that the perl script would help change the offset in ter.cpp in a brute force way and be smart enough to know when it worked.

Then the user manually adds another line to ter.cpp like the above, changes the perl script to point to the offset location in ter.cpp (would be great if the script parsed that line and you could just tell it what line it was on!)... then the user runs the brute force script on the next zone... etc.

Is this idea stupid? Anyone think that it will not work? If not, why not?

Thank you.

[KLS]

Based on my EQG loader I got sidetracked last week with:

arena (works)
arena2 (works)
tutoriala.eqg (works)
tutorialb.eqg (works)
lavastorm.eqg (works with manual offset: 0x2db00+24)
highpasshold.eqg (works)
freeporteast.eqg (works)
freeportwest.eqg (works)
freeportsewers.eqg (works)


Depths of Darkhollow
westkorlachb (works)
westkorlacha (works)
corathus (works)
corathusb (works)
drachnidhiveb (works)
dreadspire (works)
illsalina (works)
westkorlachc (works)
drachnidhivea (works)
drachnidhivec (works)
illsalin (works)
nektulosa (works)
shadowspine (works)
eastkorlacha (works)
corathusa (works)
westkorlach (works)
illsalinb (works)
drachnidhive (works)
illsalinc (works)
eastkorlach (works)


Dragons of Norrath
guildhall (works with manual offset: 0x3080-2)
delvea (works with manual offset: 0x39bb60+4)
stillmoona (works with manual offset: 0x71944+4)
thenest (works with manual offset: 0xCA244)
stillmoonb (works with manual offset: 0xE340-2)
barter (works with manual offset: 0x56AD)
broodlands (works with manual offset: 0x382BB-4)
guildlobby (works with manual offset: 0x4190+19)
thundercrest (works with manual offset: 0x62230-4)
delveb (works with manual offset: 0x38BF0)

Omens of War
anguish (works)
dranikcatacombsa (works with manual offset: 0x16b0+8)
dranikcatacombsb (works with manual offset: 0x12a0+15)
dranikcatacombsc (works with manual offset: 0x16b0+8)
dranikhollowsa (works)
dranikhollowsb (works)
dranikhollowsc (works)
draniksscar (works)
harbingers (works with manual offset: 0x1178)
provinggrounds (works)
chambersa (works)
chambersb (works)
chambersc (works)
chambersd (works)
chamberse (works)
chambersf (works)
causeway (works)
riftseekers (works)
draniksewersa (works with manual offset: 0x1410+19)
draniksewersb (works with manual offset: 0x1410+19)
draniksewersc (works with manual offset: 0x13b0+30)
bloodfields (works)
fhalls (works)
dranik (works)
wallofslaughter (works)

Not including any version 4 EQGs and just eqgs I found on my system. There are special conditions for version 1(fhalls) and version 3(anything in DoD) that don't let them load with the version 2 loader correctly (though you can get fhalls to work with an offset). I'll try to post my code once I clean it up a bit.

[uncommon]

Nice, thank you KLS.

What are version 4 EQGs btw ? PoR zones and stuff like this ?

[KLS]

commonlands, southro, northro, newest nektulos are examples. Not sure PoR are v4 I suspect they're v3 and everything past v4.

[gernblan]

Thank you KLS, looking forward to it!

That along with some sort of brute force script should help this situation a lot.

Like I said, I am willing to spend the time brute forcing these if someone would be kind enough to drum up a script to do so. I'm just not "good enough" to make one that works... I've been trying since I first mentioned it.

[Derision]

Originally Posted by KLS

Should probably look for a more general solution, for version 2 EQGs but it's frustrating..

I don't know whether the Dev's worked out the formula for calculating the offsets for version 2 EQGs, but I searched and couldn't find anything relevant after this thread.

I've been messing about trying to get OpenEQ to load version 2 EQGs, and came across this old thread:

http://www.eqemulator.net/forums/arc...p/t-21615.html

Based on that, I knocked together the following bit of Python to calculate the offsets (excuse my Python, but I'm new at it ):

import struct, posixfile, socket, zlib, pdb, sys



for eqgfilename in ["broodlands", "guildhall", "guildlobby", "harbingers", "stillmoona", "stillmoonb", "thenest", "thundercrest"]:
    filenames = []
    files = []
 
    eqgfile = file(eqgfilename + '.eqg', 'rb')

    block = eqgfile.read(12)
    (offset, magic, unknown) = struct.unpack('L4sL', block)


    eqgfile.seek(offset, posixfile.SEEK_SET)

    block = eqgfile.read(4)

    dir_count = struct.unpack('I', block)



    for i in range(0, dir_count[0]):
        block = eqgfile.read(12)
        (crc, fileoffset, filesize) = struct.unpack('LLL', block)


        if crc == 0x61580AC9:

            CurrentPos = eqgfile.tell()
	    eqgfile.seek(fileoffset, posixfile.SEEK_SET)
	    InflatedLength = 0
	    uncompressed = ''
   	    while InflatedLength < filesize:
	        block = eqgfile.read(8)
	        (deflen, inflen) = struct.unpack('LL', block)

	        block = eqgfile.read(deflen)
	        uncompressed = uncompressed + zlib.decompress(block)
	        InflatedLength = InflatedLength + inflen

	    eqgfile.seek(CurrentPos, posixfile.SEEK_SET)
            fncount = struct.unpack('L', uncompressed[0:4])	

	    pos = 4
	    for j in range(0, fncount[0]):
	        fnlen = struct.unpack('L', uncompressed[pos:pos+4])

	        fmt = str(fnlen[0]) + 's'
	        fname = struct.unpack(fmt, uncompressed[pos+4:pos+4+fnlen[0]])
                fname = fname[0].strip('\x00')

                if fname[len(fname)-4:len(fname)] == '.ter':
                    wantedfile = fname

	        filenames.append(fname)
	        pos = pos + 4 + fnlen[0]
        else:
            files.append((eqgfile.tell() - 12, fileoffset))


    for i in range(dir_count[0] - 2, 0, -1):
        for j in range(0, i):

            if files[j][1] > files[j+1][1]:
                tmp = files[j]
                files[j] = files[j+1]
                files[j+1] = tmp



    for a in range(0, dir_count[0]-1):
        if filenames[a] == wantedfile:

	    eqgfile.seek(files[a][0], posixfile.SEEK_SET)
            block = eqgfile.read(12)
            (crc, fileoffset, filesize) = struct.unpack('LLL', block)

	    eqgfile.seek(files[a][1], posixfile.SEEK_SET)
	    break

    uncompressed = ''
    inf = 0

    while inf < filesize:
        block = eqgfile.read(8)
        (deflen, inflen) = struct.unpack('LL', block)

        block = eqgfile.read(deflen)
        uncompressed = uncompressed + zlib.decompress(block)
        inf = inf + inflen

    (magic, version, list_len, obj_count, vert_count, tri_count) = struct.unpack('4sLLLLL', uncompressed[0:24])


    ter_tmp = list_len
    pos = 24
    while pos < ter_tmp:
        strlen = 0 
        while uncompressed[pos+strlen] != chr(0):
            strlen = strlen + 1
        fmt = str(strlen) + 's'
        (strvar) = struct.unpack(fmt, uncompressed[pos:pos+strlen])
        pos = pos + strlen+1

        strlen = 0 
        while uncompressed[pos+strlen] != chr(0):
            strlen = strlen + 1
        fmt = str(strlen) + 's'
        (strval) = struct.unpack(fmt, uncompressed[pos:pos+strlen])
        pos = pos + strlen+1



    pos = 24 + list_len
  
    for b in range(0, obj_count):

        (index, name_offset, another_name_offset, property_count) = struct.unpack('LLLL', uncompressed[pos: pos+16])

        pos = pos + 16
        for a in range(0, property_count):
            pos = pos + 12

    print "Offset for EQG Zone %-20s (TER: %-22s) is %8X" % (eqgfilename, wantedfile, pos)

For the EQG zones hardcoded in the currently downloadable version of azone, it spits out the same offsets:

Offset for EQG Zone broodlands           (TER: ter_broodlands.ter    ) is    382B7
Offset for EQG Zone guildhall            (TER: ter_guildhall.ter     ) is     307E
Offset for EQG Zone guildlobby           (TER: ter_guildlobby.ter    ) is     41A3
Offset for EQG Zone harbingers           (TER: ter_harbingers.ter    ) is     1178
Offset for EQG Zone stillmoona           (TER: ter_main.ter          ) is    71948
Offset for EQG Zone stillmoonb           (TER: ter_easterntemple.ter ) is     E33E
Offset for EQG Zone thenest              (TER: ter_abyss01.ter       ) is    CA244
Offset for EQG Zone thundercrest         (TER: ter_stormtower01.ter  ) is    6222C