Working on 0.8.0 compatibility

[Windcatcher]

It's been a long time since I've been active in the community, but I've decided to bite the bullet and try to switch SimpleClient to the newest svn. I'm very nearly there, near enough that I feel comfortable posting about it. Here's my current status:

- 0.6.1+ netcode is working
- World server starts and SC can create characters
- My 0.5.5 DB has been largely converted to 0.8.0, including pathing (I need a little help with the guilds table, though, I'm missing some fields).
- Spells have been upgraded to Titanium format and imported to the DB using GeorgeS's tool.
- Upgraded ClearLogin so it can talk to the latest SVN.
- Upgraded SC so it has the Titanium opcodes and structures (with the exception of serialized items for the moment)

To do:

- Working on zoning in. It looks like I'm going to have to create an arena zone with ID 189 since the server seems to want to move characters to that when all else fails. I can probably throw something together tonight.

- Once I get zoning in working, I need to upgrade SC so it can understand Titanium serialized items. It looks like the server won't serialize items until it actually sends them, so I don't have anything to work with until I can zone in.

- Package up the next version of OpenZone. SimpleClient and OpenZone have been upgraded so they can support dynamic lighting with fragment shaders. This meant that I had to expand the .XWF file format, so a release of SimpleClient will have to come with a release of OpenZone. I had wanted to include support for bumpmapping, but that will have to come at some point in the future. When it does, it will involve another file format change.




Now...

Since working on SimpleClient has been so cumbersome for me, I've decided that I want to release the source code. I'm announcing this early because I know that there are concerns about cheating, so I want to give the admins a chance to say, "no, keep it private". If that's the case, though, then YOU will have to provide a private SVN where you can store it, because that's just too much for me to handle. At some point, if that doesn't materialize I'm just going to release the lot publicly, so if you want it kept private it's in your best interests to set something up. I've basically reached the point where I don't want SC to languish anymore.

[pfyon]

As far as cheating goes, I imagine that anything you could modify simpleclient to do, MQ2 can already do.

[Amphitryon]

Thank you THANK YOU!!! :) You just made my day!

as far as cheating goes, Im not worried and nobody else should be either (since its not like the entire planet does not already know of things like MQ, ShowEQ, packet injectors, and a host of other cheating methods.

ITs up to GMs and admins to pay attention if they want to combat cheating, be willing to knock some heads together so to speak.

Besides, there is some major precedent for releasing the code now...The game Ryzom..well last month they released their server code, client code and dev tools as open source...yea the actual production game code. I think getting SC into more developer hands will be a boon for EQEmu, and realy enable/empowered those who want to make custom content free of SOE EULA entanglements.

At any rate I am so glad to hear this news...I have already invested a lot of effort into learning SC, OZ and a host of other tools, for the project I am working on..great news indeed!

This code should not be made private...if admins wind up sitting on it, its never going to grow

Also...it wasnt that long ago someone managed to decompile and publish all the source for the Eve Online client..you know what,it did not matter,,it did not increase cheating, nor did it give undue advantages (since the important stuff is server side). As such having the SC sources public, likely will NOT change anything as far as cheating goes and we will finally see a LOT more custom zones being published.

[bristle]

i as well think WC does a great service by releasing the client. there is a lot to learn. maybe i will not remove the delphi 6 after all.

[Windcatcher]

Definitely keep your Delphi 6 handy. I'm still putting together an arena zone to make the server happy, but it's almost ready for me to use. Then I need to get zoning in working and upgrade to the 0.8.0 item serialization layout. Once that's done, maybe a day or two to test and then it will be ready to post.

Have any admins had a chance to think about how to handle the code? Public or private?

Assuming that this gets posted publicly, I can just post a link to a ZIP file for starters, but a public SVN makes the most sense in the long run. That's not something I know how to handle (or care to).

[Secrets]

Definitely keep your Delphi 6 handy. I'm still putting together an arena zone to make the server happy, but it's almost ready for me to use. Then I need to get zoning in working and upgrade to the 0.8.0 item serialization layout. Once that's done, maybe a day or two to test and then it will be ready to post.

Have any admins had a chance to think about how to handle the code? Public or private?

Assuming that this gets posted publicly, I can just post a link to a ZIP file for starters, but a public SVN makes the most sense in the long run. That's not something I know how to handle (or care to).

I talked with KLS about making a client after we talked about this in PMs.

I believe we'd be fine with it, as said previously in this thread, anything SC can be modified to do, MQ2 can already do.

I would go ahead and post a zip, I can also get an SVN repository for you set up if you'd like. (and for anyone else who wants to develop on it.)

I don't think security is a concern for the EQEmu dev team at this time, at least that's what KLS thinks.

[GeorgeS]

That's excellent news, WC

As a backup, I'll host the source here on this forum site where the server resides. It will therefore always be available for people.

GeorgeS

[bristle]

ok i dont know delphi but i have look at it and it seems nice by comparison to certain language (c++). but do you need a verison that support enterprise edition? also i think delphi support .net in some form. do you need enterprise for that?

basically, can i use a free delphi 6 to do everything, including .net? the think with .net is that i can use other language too (in theory anyway).

[Windcatcher]

The free version of Delphi 6 should be able to compile SimpleClient, though I haven't tried it, as I bought the Pro version when it came out. However, I don't think I'm using anything that only comes with the Pro version.

[bristle]

i will go through and recompile all the sources files and see how im doing. of course with delphi 6 free version. i would spend more for the pro, but then i would get the newest one, and i dont know what that costs.

[bristle]

i will go through and recompile all the sources files and see how im doing. of course with delphi 6 free version. i would spend more for the pro, but then i would get the newest one, and i dont know what that costs.


well i dont think i have the delphi 6 any more. there is a free pascal which i may use. i guess the delphi 6 was available for a short time only. now they have 2010 architect that cost thousands of dollars. with the free pascal it should do it though.

[Windcatcher]

Just a little update...I haven't yet gotten to getting zoning in working yet, because I've been throwing together a little arena zone...

http://i48.tinypic.com/2rf98v6.jpg

Most of the hard work is done; once I'm satisfied with it I'll see about zoning in.

[Secrets]

Just a little update...I haven't yet gotten to getting zoning in working yet, because I've been throwing together a little arena zone...
Most of the hard work is done; once I'm satisfied with it I'll see about zoning in.

Just as an FYI, it is defaulting to that zone because 189 is the "Mines of Gloomingdeep", which is the new EverQuest tutorial. You can change where you are going in the rule_values table, by setting the entry "World:TutorialZoneID" to a different value.

[Windcatcher]

That's good to know. I've finished my arena zone, at least enough that it's usable, and I've updated Peshara Highlands so you can get to the arena. One question I have is, is a tutorial zone necessary? In other words, when you create a character, does the server automatically send you to whatever zone is the tutorial zone, regardless of the start_zones value for that race/class combo? I can throw together a tiny tutorial zone, but I'm wondering if I need to. That said, if I have to create one, is there some special way of handling exiting from the tutorial zone so that the server knows to route the character to the correct starting zone?

As a side note, one benefit of making this arena zone is that I had to add several new features to OpenZone, such as greater control over individual polygons (especially vertex normals). In my opinion, the arena zone is quite stunning.

[Secrets]

That's good to know. I've finished my arena zone, at least enough that it's usable, and I've updated Peshara Highlands so you can get to the arena. One question I have is, is a tutorial zone necessary? In other words, when you create a character, does the server automatically send you to whatever zone is the tutorial zone, regardless of the start_zones value for that race/class combo? I can throw together a tiny tutorial zone, but I'm wondering if I need to. That said, if I have to create one, is there some special way of handling exiting from the tutorial zone so that the server knows to route the character to the correct starting zone?

As a side note, one benefit of making this arena zone is that I had to add several new features to OpenZone, such as greater control over individual polygons (especially vertex normals). In my opinion, the arena zone is quite stunning.

You can disable it entirely afaik in the rule_values table. Tutorialbutton or something like that. You'd have to look through it all.

As for the rest, i'm looking forward to seeing the arena zone! :D

[Windcatcher]

The Arena at sunset.

http://i50.tinypic.com/ftohmh.png

This is still on my 0.5.5 server; I'm about to move to the 0.8.0 server one I take care of a couple more small details.

[Windcatcher]

Started working on zoning in to the 0.8.0 server. It took a few hours to get it connecting to the zone server, but I eventually found that the problem was the port byte order. 0.5.5 clients expect it in network order, Titanium clients expect it in Intel order. Adding a configuration switch solved that problem. I then had to fix a couple of Titanium struct definitions and make my netcode just a little more robust. I'm now connecting to the zone server, but not yet getting ingame. The zone server eventually crashes after a few minutes, and the client doesn't reach the fully zoned-in state.

I noticed that the world server defaults to using a tutorial zone with ID 189 if there is no setting, so the easiest thing to do was just make a bare zone for it with a zone exit for the tutorial. I'll see how it works whenever I get zoning in working.

So I've still got lots of work to do, but I'm making some progress.

[Windcatcher]

The very first zone in to my 0.8.0 server:

http://i48.tinypic.com/kc05eq.png

I still need to handle its item serialization scheme, but now we're cooking with gas :D

[trevius]

Very nice work, Windcatcher! Congrats on getting in-game. That is quite a step :) I am sure everything else falls into place much smoother once you are able to get in. I know for sure that when trying to get new EQ clients to work with EQEmu, the getting in-game part was normally the biggest step.

[Windcatcher]

I have items mostly loading, to the point that I can get a merchant list when I right-click on a merchant NPC. I still need to fix something in the way I'm handling Spawn_Structs: something is bugged somewhere, but I don't think it will be too hard to fix. I've also fixed a ton of bugs in the client, to the point that I'm very happy with the stability.

That said, I decided to take a little detour to work on OpenZone a little bit. Why?


http://i50.tinypic.com/xdhv8i.png

BUMPMAPPING!!!!

I still need to work on it some more and will have to update the zones, upgrade SimpleClient to support it, etc., but the hardest stuff is done.

I'd also like to add support for multitexturing, but I've wanted to get bumpmapping in for a LONG time.

[Akkadius]

I have items mostly loading, to the point that I can get a merchant list when I right-click on a merchant NPC. I still need to fix something in the way I'm handling Spawn_Structs: something is bugged somewhere, but I don't think it will be too hard to fix. I've also fixed a ton of bugs in the client, to the point that I'm very happy with the stability.

That said, I decided to take a little detour to work on OpenZone a little bit. Why?


http://i50.tinypic.com/xdhv8i.png

BUMPMAPPING!!!!

I still need to work on it some more and will have to update the zones, upgrade SimpleClient to support it, etc., but the hardest stuff is done.

I'd also like to add support for multitexturing, but I've wanted to get bumpmapping in for a LONG time.

This is nuts, and since you are getting back to OpenZone. I was importing some Quake 3 maps quite some time ago and trial and error told me that maps that are too big simply won't load or export. I had to use very small maps to do anything with them, is there a reason for this? Or perhaps an explanation? I have a system made for CTF and having these maps would be sweet.

Otherwise phenominal work Windcatcher, I'm sure someone wants your babies...

[Windcatcher]

I'm slowly making progress. I think I've got bumpmapping perfected in the engine, and I've been slowly upgrading some of the mesh library objects and zones to use it. I might take a look at the shader this afternoon to see if I can speed it up, but otherwise it's working. My dynamic lighting shader broke use of tinting like infravision and such, but I don't think that will be hard to fix.

On the SimpleClient front, I've fixed the weirdness I was seeing with player and mob positioning. It was the old reversed-X-Y issue rearing its head, and all it took was correcting my Titanium packet definitions file. I also have it using my upgraded engine, though now I have to upgrade the client so it actually uses all the extra bumpmapping stuff. That isn't trivial, but it's otherwise pretty straightforward.

[Windcatcher]

Hmm, getting bumpmapping working in the client was easier than I thought...

http://i49.tinypic.com/9uqcyv.png

Next step is to fix infravision...

[Windcatcher]

This is nuts, and since you are getting back to OpenZone. I was importing some Quake 3 maps quite some time ago and trial and error told me that maps that are too big simply won't load or export. I had to use very small maps to do anything with them, is there a reason for this? Or perhaps an explanation? I have a system made for CTF and having these maps would be sweet.

Otherwise phenominal work Windcatcher, I'm sure someone wants your babies...

I haven't looked at the Quake import code in ages, and I didn't write it to begin with but only ported it. I'd need a sample Q3 map that isn't working for you to see what's going on.

[Lothrian]

any chance we can get the 1021 version of sc that works with 0.5.x eqemu? all the download links are broken

[Tharkun]

This is spectacular work and I have a question and the answer is probably something that is far off (development wise).

Mostly I'm wondering if there are plans to have things customizable that currently are limited by the SOE client. For example, you can change the AA's around but since the client wont let you earn AAExp until lvl 51, there is a limit to the customization of AA's. As well as other features that are coded to the client.
Also I'd be curious to the how...

I understand that right now your probably more concerned with getting the client as functional as Titanium.

[Windcatcher]

any chance we can get the 1021 version of sc that works with 0.5.x eqemu? all the download links are broken

Patience :D

Infravision is fixed, so now just about all that's left is to re-export my updated zones, tweak some mob positions, and do some final testing. I'm probably going to pay for this, but I'd love to get this and OpenZone 8.7 out the door over the long weekend. In making the arena zone I had to add a bunch of new features to OpenZone, so some of the time I've been spending lately has been making new icons for the new features (and replacing some of the uglier ones).

[Windcatcher]

This is spectacular work and I have a question and the answer is probably something that is far off (development wise).

Mostly I'm wondering if there are plans to have things customizable that currently are limited by the SOE client. For example, you can change the AA's around but since the client wont let you earn AAExp until lvl 51, there is a limit to the customization of AA's. As well as other features that are coded to the client.
Also I'd be curious to the how...

I understand that right now your probably more concerned with getting the client as functional as Titanium.

I'll be posting the source, so customization will definitely be possible.

[Secrets]

I'll be posting the source, so customization will definitely be possible.

That's awesome news. Do you need an SVN repository for OpenZone & SimpleClient? You should look into releasing it under MIT or GPL on Google Code (code.google.com/p/) as they offer a whopping 2GB of data space I believe it was.

MIT is more open than GPL is, even though EQEmu uses GPL. The only thing about GPL that I dislike is the share-alike referendum, which a friend pointed out to me.

http://code.google.com/projecthosting/ has all the info you'd need. I prefer it over all other things for open source development, and EQEmu switched over from Sourceforge. Only caveat is you have to have a google account, which is free and takes ~3mins to register. The project creation literally takes 5 minutes to set up and have the first revision loaded. It also has guides on SVN if you don't have an SVN client. TortoiseSVN should run on 2000, which I think you were using to develop. Let me know if you need any help setting things up in PM; i'm really looking forward to this release :D

[Windcatcher]

Hosting has been a worry. I've been scrupulous in protecting my anonymity, and I'd like it to remain that way. I'm open to suggestions. I'd be willing to FTP it somewhere if someone wants to post it somewhere.

I've never been a real fan of overusing the GPL. I think it makes sense in certain situations (such as in developing something that will become an implementation standard), but for an application I tend to be an old-school "public domain" person, that is, I generally don't stick any license at all on something I write without a compelling reason. For the client, I think it's important that it not be GPL'ed since that would make it impossible to fight cheaters. I'm leaning toward LGPL'ing certain files, mostly those that deal with the XWF file format, to stave off "embrace and extend" tactics. Otherwise, I don't much care what people do with my code -- I already have a paying day job ;)

[GeorgeS]

Let me know if you need hosting on this emulator site. Rogean owns the site, but I have shared space there for my emu tools and misc. files, thus have direct access.


GeorgeS

[Lothrian]

I could probably host it for you as well

[Tharkun]

Correct me if I'm wrong, but with the source being available, would this mean the client could be run in a non-MS environment? (ie one's favorite unix/linux OS)

[Lothrian]

i think it could possibly be ported to linux but i doubt itll just run under linux as is

[Secrets]

Let me know if you need hosting on this emulator site. Rogean owns the site, but I have shared space there for my emu tools and misc. files, thus have direct access.


GeorgeS

I'd support GeorgeS here. He would be the person to talk to about hosting it. That way it's not under licensing, and you can remain anonymous.

[Windcatcher]

i think it could possibly be ported to linux but i doubt itll just run under linux as is

Your best bet would probably be to port it to Free Pascal + Lazarus. One of the things I had to do was upgrade ClearLogin to support 0.8.0 servers, and for kicks I made a Lazarus version of it in addition to the Delphi one. I'm not planning to attempt to port SC over, but it should be possible to have a single code base that works under Delphi 6 as well as FPC and Lazarus.

[Windcatcher]

Almost there...i've been redesigning the color subsystem to support separate ambient, diffuse, specular, and emissive colors. It's going to take too long to completely test everything, but I want to make sure that the important stuff is working. The hard stuff is all done, including upgrading OpenZone, but I need to fix my shaders. I also made a few game sounds so playing won't be completely silent. Once I get the shaders nailed down I just need to place some more sounds in the zones, re-export everything to XWA, and do some final testing (mainly zoning and checking mob locations).

[Windcatcher]

Progress update:

Shaders...pretty much perfected, just one more thing to verify.

Auto-attack: working, had to copy the generated map files from OpenZone to the server's maps folder.

Player heading: finally got the client correctly understanding the player's heading and correctly sending it to the server.

Merchant: fixed the problem with not being able to close the merchant window. It was because the confirmation opcode is 0000, so the cilent has to not wait for the ack in that case.

Selling items: fixed, had to flip all my nodrop flags in my DB since they're reversed in 0.8.0 vs. 0.5.5. Also had to add a flag to the client so it can tell the difference.

Further tests:

1. Zoning out of the tutorial zone.

2. Zoning between zones.

3. Checking my merchant placements in Lelembeth, because I changed them.

4. GM training.

5. Scribing, memorizing, and casting spells.

6. XP updates.

None of these will take very long, and I'm not going to test grouping because that's not really possible to do by myself.

[Windcatcher]

Here's a status update folks:

XPing ... Working

Zoning... Still a little buggy, sometimes it can't conect to the zone server. I might release that part as-is, assuming that some other devs here are interested in fixing it.

Spells ... Working

Merchant placement ... Good, but there's a problem with my collision avoidance all of a sudden. I don't know why yet.

There's one showstopper that I need to fix... The server is putting me at a bad Z coordinate when I zone in sometimes. It's related to the map file for the zone. I don't know if OpenZone is creating it improperly or if the server is parsing it improperly. I need to do some more investigating.

Texture animation rate is broken, should be an easy fix.

I'd also like to add better ambient lighting control to OpenZone, but it's a bottom priority. At a minimum, I need to add a placeholder in the XWF file format for it.

[Lothrian]

I wouldnt mind taking a crack at the zoning issue. I personally am thrilled at the progress and cant wait to try it out! Awesome work Windcatcher

[Windcatcher]

Did some more work on it today...

- Fixed the bad-Z showstopper, I needed to have OpenZone swap the X and Y coordinates when making .MAP files. While I was at it I added an option to just export the .MAP file without having to export the whole zone. There's also a similar option for just exporting sound info.

- Fixed the texture animation rate bug, this was also an Openzone bug.

- Fixed another OpenZone bug when exporting colors.

- Found a bug when destroying items ingame, I still need to fix it.

- Took a quick look at controlling ambient lighting. What I want to do is specify how much outdoor lighting affects individual objects so I can have outdoor areas respond to sunlight but not indoor areas. I'm not yet sure how best to handle it, but I'll think about it a little more tomorrow.

- The zoning bug is really annoying. Before I release this I want to take a couple of days looking at it to see if it's something easy to fix.

- Skill updates might be broken. I'm not sure yet, but I'm not seeing any notifications about skills improving when I do playtesting. I need to look at it, but hopefully it's something really simple like getting an opcode wrong or something. It was working when I was connecting to my 0.5.5 server, so it can't be all that hard to find.

[Windcatcher]

Getting there:

- Fixed the bug when destroying items, also fixed a bug with stackable items. It turned out that 0.8.0 looks at DB fields that 0.5.5 doesn't.

- I think for the lighting I'm going to allow a "sunresponse" number or something similar to be assigned to each object. Polygon-level control would be ideal, but that would be a real nightmare for the engine to handle. I might start taking a stab at it tomorrow.

[Lothrian]

Any new updates Windcatcher?

[Lothrian]

hey windcatcher are you still working on this at all? can we get a status update?

[DrWIggly]

I'd like to know to. How hard would it be to port to Lazerus?