EQEmulator Forums

EQEmulator Forums (https://www.eqemulator.org/forums/index.php)
-   General::General Discussion (https://www.eqemulator.org/forums/forumdisplay.php?f=586)
-   -   Anyone interested in client development? (https://www.eqemulator.org/forums/showthread.php?t=33432)

pfyon 02-04-2012 10:30 PM

Thanks, Tabasco. I was running into difficulties getting it to compile until I read the error messages and noticed it was complaining about x64 (this is a 32 bit machine). Just an FYI to anyone else who downloads the tarball, make clean first.

PixelEngineer 02-05-2012 08:06 PM

Did you get it up and running Pyfon? Got a screenshot?

pfyon 02-05-2012 08:59 PM

I haven't been able to get it working, probably due to the same issue that Tabasco et al ran into. I tried an s3d file from titanium, sof, and sod without luck.

Code:

pfyon@family-kubuntu:~/lanterneq/Lantern$ ./Lantern
24

pfyon@family-kubuntu:~/lanterneq/Lantern$

Code:

pfyon@family-kubuntu:~/lanterneq/Lantern$ cat log.txt
<---> Logfile Initialized on 2-5-2012 at 19:56:27
<---> Preferences from lantern.ini have been read.
<---> Loading zone: crushbone
<---> Screen width: 800 - Screen height: 600
<-!-> This is not a valid PFS archive
<-!-> Could not initialize the S3D file!
<-!-> Fatal Error: Unable to load zone data!
<---> Fatal Error: Unable to initialize Lantern!

Code:

pfyon@family-kubuntu:~/lanterneq/Lantern$ cat lantern.ini
crushbone
800
600


Tabasco 02-05-2012 09:25 PM

I had to do a lot of fixes for 64 bit that I have yet to publish. I'll get those committed tomorrow along with some of the header fixes so we can hopefully have a single source tree that builds without any modification for platform or architecture.

You might be able to quickly fix your copy by looking at the global datatypes include.
Code:

typedef unsigned int uint;

typedef unsigned char uchar;
typedef int8_t int8;
typedef uint8_t uint8;
typedef int16_t int16;
typedef uint16_t uint16;
typedef int32_t int32, dword;
typedef uint32_t uint32;

http://midian.homelinux.net/ltscreen1.png

KLS 02-05-2012 10:06 PM

The mirrored zones sounds very similar to an issue I ran into with my .eqg renderer because sony inverts x and y axis.

pfyon 02-05-2012 10:19 PM

Thanks again tabasco, that did fix that issue. I also had to change line 96 of S3D.cpp from

Code:

memcpy(&file_count, meta_block, sizeof(long));
to

Code:

memcpy(&file_count, meta_block, sizeof(int32));
Now I'm sitting at a black window from Lantern without any useful output.

Code:

pfyon@family-kubuntu:~/lanterneq/Lantern$ tail log.txt
<---> Logfile Initialized on 2-5-2012 at 21:16:32
<---> Preferences from lantern.ini have been read.
<---> Loading zone: gfaydark
<---> Screen width: 800 - Screen height: 600


Tabasco 02-05-2012 10:30 PM

With SOIL I haven't had any texture issues, but I haven't tested in 64-bit yet.
What do you get if you try to move around (sometimes 0,0,0 isn't ideal) or what do you get when you run something from mesa-utils like glxgears, or glxinfo, or try to load a different s3d?

KLS, I noticed some strange stuff when translating 2d map files to locs inside a web-based editor. OpenGL tends to use a Y-Up model, but every engine wants to apply its own translation anyway. In addition to that, they use one system for coords and another entirely for labels. I can only imagine what other idiosyncrasies we'll find inside their archives.

lerxst2112 02-06-2012 12:02 AM

It may not help that by default Direct3D is left-handed and OpenGL is right-handed with regards to coordinate systems. Not hard to account for if you know the format the data is loaded in and the destination system you want it to fit, but certainly a common pitfall.

Taurinus2 02-06-2012 04:04 PM

Any thoughts toward a sound lib? FMOD? irrKlang?

What threading model do you plan to adhere to?

Also do you have a design doc laying around?

I applaud your drive and progress on this project.

PixelEngineer 02-06-2012 05:15 PM

Quote:

Originally Posted by Taurinus2 (Post 206898)
Any thoughts toward a sound lib? FMOD? irrKlang?

What threading model do you plan to adhere to?

Also do you have a design doc laying around?

I applaud your drive and progress on this project.

I think when the time comes, I will try FMOD. Looks like what a lot of people use today in the industry.

As for threading, I honestly have no idea yet. There is a design doc (readme.txt) on the SVN and it just lays out some goals I want to accomplish.

I probably won't be committing anything for a week or so while I add support for shaders and lighting. More than anything I am just reading about the OpenGL pipeline and trying to understand everything before I move on. After all, this is a project for learning.

Great to see it running on Linux and go ahead and commit that whenever you get the chance.

Cheers!

Tabasco 02-06-2012 05:56 PM

Revision 29 should build and function on linux 32 and 64 bit.

PixelEngineer 02-06-2012 06:14 PM

Quote:

Originally Posted by Tabasco (Post 206902)
Revision 29 should build and function on linux 32 and 64 bit.

Fantastic. Thank you Tabasco. Also, for those who don't know, Tabasco is now a contributor to the project.

pfyon 02-06-2012 10:10 PM

Excellent, works out of the box. Performance shot up once I installed the nvidia proprietary driver too ;).

Vexyl 02-07-2012 12:44 AM

Results
 
Revision 29 compiles/runs with no errors on Fedora 13.

Code:

[user@host ~]$ uname -a
Linux host 2.6.34.9-69.fc13.i686.PAE #1 SMP Tue May 3 09:13:56 UTC 2011 i686 i686 i386 GNU/Linux

`g++ --version`
Code:

g++ (GCC) 4.4.5 20101112 (Red Hat 4.4.5-2)
...

`make --version`
Code:

GNU Make 3.81
...


I did take a few screenshots while I was testing it, but didn't want to put all 5 in one post so here are the links:

Unrest: http://i.imgur.com/6wgCs.jpg

Mistmoore 1 (Overview): http://i.imgur.com/EH4QD.jpg

Mistmoore 2: http://i.imgur.com/eUgH5.jpg

Highkeep 1 (Front entrance): http://i.imgur.com/UGJ7o.jpg

Highkeep 2: http://i.imgur.com/V6XJB.jpg

That last one is interesting because the establishment plaque is upside down, and the left and right textures adjacent to the plaque are blurry.

Edit: Going to put up some neriak screenshots just for fun.

pfyon 02-07-2012 09:29 AM

I also noticed in gfay that all the trees' leaves were upside down.

Tabasco 02-07-2012 11:01 AM

This is indeed the case, and as you said, pretty easy to fix.

Quote:

Originally Posted by lerxst2112 (Post 206872)
It may not help that by default Direct3D is left-handed and OpenGL is right-handed with regards to coordinate systems. Not hard to account for if you know the format the data is loaded in and the destination system you want it to fit, but certainly a common pitfall.

The resolution should now load from the ini file and you can change camera speed with +/- now as well.

PixelEngineer 02-07-2012 06:44 PM

As for the flipped textures, there is a flag in the WLD header that denotes if the zone is old or new. I assumed that would allow us to flip or not flip the texture across the y axis. Unfortunately, the classic and the titanium Greater Faydark both say they are old zones so it won't be that simple. My next idea is just to check the bit depth on the textures when they load. If the depth is 8 bit, we know it's a classic zone and 24 bits, a newer version zone. Again as I mentioned, I will be learning shaders for the next week or so and then I am going to tackle both per vertex and per pixel lighting in GLSL.

Awesome work Tabasco.

Cheers!

PixelEngineer 02-13-2012 08:36 PM

As I continue to progress through my study of shaders, I had a fun idea last night. For me, seeing Lantern grow and accumulate a number of DLL's in the directory has been a little depressing. I downloaded a Game Boy Advance emulator yesterday and it had only 1 file; the executable.

I am going to try as I progress to actively reduce the amounts of external dependencies and DLL's so that quite possibly Lantern can simply be an executable. It could as a first run, create the .ini.

Thoughts?

Taurinus2 02-13-2012 09:09 PM

My thoughts:

In order to achieve this on Windows, you would need:

1) Static linkage
1a) Source for every lib you use to achieve this.

Sure, lib developers/vendors make pre-built libs available, but depending on them means that the show is over once development ends (or you move on to another environment that they do not support) and no source is supplied. You also have to keep crt linkage in mind (cannot mix them).

Even being feasible it is simply not worth the effort IMO.

Secondly, why?

Your data dependencies number in the thousands already so it would seem pointless to me.

PixelEngineer 02-13-2012 09:24 PM

Just an idea I was kicking around in my head. Obviously it would not be possible with more advanced libraries like FMOD but in terms of smaller libraries where I only use 1-2 functions from them, it might be useful.

Again, not something I am set in doing, just an idea. :D

And thank you for the great reply!

Cheers!

Speedz 02-13-2012 11:40 PM

Made a small mod for a live client like feel to movement.

camera.cpp line 53

if(ks[SDLK_UP] || ks[SDLK_w] || SDL_GetMouseState(NULL, NULL)&SDL_BUTTON(1))

Speedz 02-13-2012 11:52 PM

I hope you guys don't mind my type of submissions. I too am just learning. At this point I can do simple adds, and whatnot. But to do some grand feature addon is a bit beyond me. I like to make simple apps or make aesthetic type changes or addons. I have a local working copy with a bunch of changes that haven't made it to the svn. But thats ok if they are not needed.

As I do this I will get a grasp of C+. But my main experience is C# and I have just a basic grasp of that with the help of google and microsoft help links.

PixelEngineer 02-14-2012 01:27 AM

Quote:

Originally Posted by Speedz (Post 207202)
I hope you guys don't mind my type of submissions. I too am just learning. At this point I can do simple adds, and whatnot. But to do some grand feature addon is a bit beyond me. I like to make simple apps or make aesthetic type changes or addons. I have a local working copy with a bunch of changes that haven't made it to the svn. But thats ok if they are not needed.

As I do this I will get a grasp of C+. But my main experience is C# and I have just a basic grasp of that with the help of google and microsoft help links.

Don't be discouraged at all. What I would recommend is looking at a simple fragment. How about 0x21, the BSP tree. Look in the source for my handling of that fragment and compare it to the WLD documentation: ftp://ftp.mirrorservice.org/pub/sour...1.1/wlddoc.pdf

Then, pick a fragment that is not implemented, any one. Try to create a struct out of it. Try to implement a loading function mimicking the style that have already been created. And yes, I appreciate your contributions. Don't get down on yourself.

Cheers!

Taurinus2 02-16-2012 11:26 AM

PixelEngineer,

I know you aren't there yet but I would like to point you toward boost::asio for your networking. Boost as a whole is excellent and highly recommended.

pfyon 02-16-2012 07:32 PM

I was thinking about getting into the network side of things (as I have little interest in GUIs, but lots of interest in network code) and was thinking about using boost for cross-platform threading.

Taurinus2 02-16-2012 11:07 PM

Go for it.

When it came time to learn the asio library I decided to do a fun project to bang out some code with. I wrote a CLI client/server remote tool for my private server that features a handful of opcodes and a single port being utilized on either end. A modest project, but one that was finished in one sitting with all planned features.

Also, PixelEngineer, now that I am thinking of it, you will need a tighter timing mechanism because you are going to need to track your frame deltas. I say this because I foresee you needing to preempt certain operations if you do not have enough time to complete for that frame. Let's say for instance you arrive at some processing logic and you see that your delta is 15ms, you know that you must defer and return immediately (60hz = 16.6ms/frame). This assumes single-threaded execution with regard to that particular logic and your rendering but I believe that the principle will still be applicable in any case.

PixelEngineer 02-18-2012 01:30 AM

Great suggestions guys. You are welcome to start on the server/network component. :D

Also, a small update:

- Added support for fragment 29, which deals with regions with water, lava, PVP, etc.
- Added rendering support (temp) for these different regions.
- Plus/Minus keys now do not effect look sensitivity.
- F1 now takes screenshots
- Split, and moved the various text files (CHANGELOG, BUGS, LICENSE, README)

The underwater effect needs some tweaking and the specifics might be found in some of the unknowns in this fragment. I mainly added the fog effect so I could test the accuracy of the implementation.

http://i.imgur.com/CMFH7.png

It also implemented the PVP zones. As someone who has never been in Qeynos, can someone verify that the red X's were not areas that you would be in the PVP area? Only the green highlighted area is denoted in the zone files as the PVP area.

http://i.imgur.com/mOrOk.jpg

Thanks!

Derision 02-18-2012 05:44 AM

If you go in game, to the locations you have marked with an X, and then move back toward the centre of the arena, the client automatically generates the mesage 'You have entered an Arena (PvP) area', so I would say you have it right.

PixelEngineer 02-18-2012 06:02 AM

Quote:

Originally Posted by Derision (Post 207356)
If you go in game, to the locations you have marked with an X, and then move back toward the centre of the arena, the client automatically generates the mesage 'You have entered an Arena (PvP) area', so I would say you have it right.

Fantastic, thank you.

ableard 03-05-2012 02:40 PM

So I have picked his up out of ccuriosity. I have VC 2010, typical versions running of eqemu, no opengl.
We I run in VS the window pops a nd closes. I get no signficant error or messages. Do I need to obtain any opengl distributables?
I have added .s3d file into debug folder, changed ini.
thanks

Taurinus2 03-07-2012 04:55 PM

OpenGL is a standard lib but you do need drivers that provide opengl acceleration.

With VS, you will find that the working directory for targets launched from within VS is the project root and not the target directories.

Putting the .s3d and configuration file into the debug/release folder should work if you invoke lantern from shell or console.

PixelEngineer 03-07-2012 11:16 PM

Can you copy/paste your log.txt file here or if it's huge, PM me with it. I will post the fix here.

PixelEngineer 03-08-2012 06:04 AM

Is anyone here experienced in reverse engineering files? I have been going through and documenting every type of file the client uses. There is one file however, which seems to stump me. The file is denoted by:

<shortname>_sounds.eff

This file is a counterpart to the other file <shortname>_sndbnk.eff which lists files that are either emitted or looped. This file is in plain text.

I am not that experienced in reverse engineering and wanted to know what you guys recommend. I assume the file contains things like x, y, z positions, sound file name, volume, radius and that sort of thing.

It's possibly ZLIB compressed or just a bizarre file format. It has no discernable header so I doubt it's a specific file format. Searches for .eff bring up EverQuest effect format but there are no specifics.

Any help/advice would be greatly appreciate. (hint hint Windcatcher)

:D

Cheers!

Taurinus2 03-08-2012 11:50 AM

Installing an api hook against the client and logging seek/read against files of that extension would be a start. That's half the picture, anyway. The other half is a memory scanner that locates what was just read in the client's memory. You can then trace execution from that point.

ableard 03-08-2012 07:53 PM

thanks for the reply.

Taurinus2 the .s3d is indeed in /debug

Log file

<---> Logfile Initialized on 3-5-2012 at 10:57:24
<---> Preferences from lantern.ini have been read.
<---> Loading zone: gfaydark
<---> Screen width: 640 - Screen height: 480
<-!-> Could not load the PFS archive!
<-!-> Could not initialize the S3D file!
<-!-> Fatal Error: Unable to load zone data!
<---> Fatal Error: Unable to initialize Lantern!

PixelEngineer 03-10-2012 03:45 AM

Quote:

Originally Posted by ableard (Post 207930)
thanks for the reply.

Taurinus2 the .s3d is indeed in /debug

Log file

<---> Logfile Initialized on 3-5-2012 at 10:57:24
<---> Preferences from lantern.ini have been read.
<---> Loading zone: gfaydark
<---> Screen width: 640 - Screen height: 480
<-!-> Could not load the PFS archive!
<-!-> Could not initialize the S3D file!
<-!-> Fatal Error: Unable to load zone data!
<---> Fatal Error: Unable to initialize Lantern!

If you're in Windows, the .s3d should be in the same folder as the SLN. Try moving it there. The error there (so glad I did error logging) is from the .s3d not being found.

PixelEngineer 03-10-2012 03:46 AM

Quote:

Originally Posted by Taurinus2 (Post 207911)
Installing an api hook against the client and logging seek/read against files of that extension would be a start. That's half the picture, anyway. The other half is a memory scanner that locates what was just read in the client's memory. You can then trace execution from that point.

Thank you.

Do you have a guide on how to do this?

Derision 03-10-2012 08:52 AM

Quote:

Originally Posted by PixelEngineer (Post 207898)
Is anyone here experienced in reverse engineering files? I have been going through and documenting every type of file the client uses. There is one file however, which seems to stump me.

The file is denoted by:

<shortname>_sounds.eff

I had a look a this, since it piqued my interest.

First of all, since you suggested it may be ZLIB compressed, I wrote a quick program to read one of the files and then move through it byte by byte and attempt a ZLIB decompress.
That failed, so it wasn't zlib compressed.

The next thing I did was open up eqgame.exe in IDA and found where the string %s_sounds.eff was referenced and looked at the dissasembly.

I could see that it reads the file 84 bytes at a time, and indeed, the few files that I looked at were all multiples of 84 bytes.

Next thing I did was write a quick program to read one of the files, 84 bytes at a time, and print out each group of 4 bytes as both an int and a float.

I could see from my output some floats that looked very like groups of co-ordinates to me, so I went in game and did a #goto to some of them to verify
there appeared to be a sound emitted in that location (some were obvious, like camp fires or steam geysers).

Anyway, this is what I figured out:

Code:

Offset
00                int8 Unknown[12]
12                int32 Sequence Number ?                // Increments by one for each entry in the file
16                float Y
20                float X
24                float Z
28                float Radius?                        // Radius from position that sound can be heard ?
32                int8 Unknown[52]


PixelEngineer 03-12-2012 03:13 AM

Quote:

Originally Posted by Derision (Post 207976)
I had a look a this, since it piqued my interest.

First of all, since you suggested it may be ZLIB compressed, I wrote a quick program to read one of the files and then move through it byte by byte and attempt a ZLIB decompress.
That failed, so it wasn't zlib compressed.

The next thing I did was open up eqgame.exe in IDA and found where the string %s_sounds.eff was referenced and looked at the dissasembly.

I could see that it reads the file 84 bytes at a time, and indeed, the few files that I looked at were all multiples of 84 bytes.

Next thing I did was write a quick program to read one of the files, 84 bytes at a time, and print out each group of 4 bytes as both an int and a float.

I could see from my output some floats that looked very like groups of co-ordinates to me, so I went in game and did a #goto to some of them to verify
there appeared to be a sound emitted in that location (some were obvious, like camp fires or steam geysers).

Anyway, this is what I figured out:

Code:

Offset
00                int8 Unknown[12]
12                int32 Sequence Number ?                // Increments by one for each entry in the file
16                float Y
20                float X
24                float Z
28                float Radius?                        // Radius from position that sound can be heard ?
32                int8 Unknown[52]


Absolutely gorgeous. Thank you so much. I have IDA in my toolbox but I don't really have a great understanding of how to use it. Did you read a guide on it or something?

That leaves virtually no files in the original EverQuest that are unknown now. I am in the process of interviewing for a job so that can explain my absence. I am still however, working on this project and am redoing the way the zones load with WLD classes with several levels of inheritance for their specific purpose.

Thank you again Derision. Very much appreciated.

Taurinus2 03-12-2012 03:38 PM

You payed 1000USD for a license to software you don't know how to use? How did you get your developer confirmation for it?:???:

If you got it through an unauthorized channel (would seem the case) then I can say with high confidence that you are root-kit'd (by someone other than yourself).


All times are GMT -4. The time now is 07:36 AM.

Powered by vBulletin®, Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.