Water Detection Patch
 

Here it is: EQEmu Water Detection beta, based on the 1070 code.

Zone patch: http://www.rama.demon.co.uk/zone.patch

utils/AZone patch:http://www.rama.demon.co.uk/azone.patch

Full Source Tree: http://www.rama.demon.co.uk/EQEmu-0....tection.tar.gz

Water maps for all S3D zones (16MB): http://www.rama.demon.co.uk/watermaps.tar.gz

It doesn't do a lot (anything) for Angelox's issue with bouncing mobs pathing through water in Dagnor's Cauldron,
which is what got me started down this road, but I think that is another issue, primarily mobs attempting
to path in the air above the water.

What these changes do are:

Amend azone to have an option to write out water maps (.wtr). These actually contain a copy of the BSP
tree from the WLD data in the S3D files with leaf nodes marked as to whether they are in water/lava.

Load these .wtr maps into the emulator.

Provide the ability to detect if a given co-ordinate is in water/lava.

Doesn't do BestZ calculation under water.

Add a new flag to the mob class indicating if it is in water or not. This is initialsed to false when
the mob is created set to true or false as appropriate in CalculatePosition2.

Turns flymode 1 on or off as mobs path into/out of water. This stops mobs falling
to the floor underwater. If you try Kedge/Powater before and after applying this patch, you will see
the difference.

I looked into also doing this as soon as the mob is spawned, rather than waiting for the waypoints to
kick in to avoid the initial 'fall to the floor', however I was unable to see an obvious way of doing
this due to the asynchronous way the spawns are qeueued up before being sent to the client.

I even tried setting all the unknown fields in the Spawn packet to 1s in the hope that there was a field
there which affected flymode, but didn't find anything.

I also altered the fishing code to use the water detection. The uses some constants relating to how
far in front and down from you the water has to be for you to be able to fish. These may need tweaking
a bit to match live. I noticed I left a debugging message in there which tells you the co-ords of the
end of your fishing line, which needs to come out.

You won't need to use azone if you download the water maps from the link above, just put the .wtr files
in your Maps directory.

If you want to use azone, to create a water map, supply the new -w parameter, e.g.

azone -w cauldron

Some S3D files which only have one type of special area, i.e. just water or just lava, don't tag the
region as to whether they are water or lava. In these cases, azone will default to tagging them as
water. The only exceptions I came across where:

arena (the firepots are flagged as lava)
lavastorm, fearplane (ring of fire), pofire, skyfire

In these cases, supply the -dl (default lava) flag to azone to mark these as lava, e.g.

azone -w -dl pofire

Comments:

The lava areas are included in the water maps, even though I don't know if there is any use for that.

The map could be optimised by pruning branches of the BSP tree that have no water/lava in them.

The walking of the BSP tree is currently done by recursion. This could be turned into a loop to avoid
some function call overhead.

If you don't have a .wtr map for a zone in your Maps directory, everything should just work as before.

This only works for zones in S3D format. If you are using a zone that has both S3D and EQG maps and
you are using the EQG version, don't use the .wtr map for that zone.

I did a search of the forums and couldn't find any information on where the BSP tree and special region
information is in the EQGs, hence no support.

You can find out whether you are in water or lava by using the #bestz command in-game.

If you try it, let me know your thoughts, bugs, suggestions.

I would add that Windcatcher's Openzone source and WLD documentation, along with the code that was already in Azone (FNW, Daeken_bb?) were the keys to getting this to work.

Damn you're good! thank you very much! I know what is wrong with Cauldon (just dawned on me), and can fix it ( I think) - I'll get back to you when I do.
First I need to implement your patches :)

Thank you!!!!!

awsome Derision, great work !

Merry Christmas to all!

Well, guess what I've been doing most the day?
This update all works very well - it was so nice to see Kedge Keep back to normal, just like the good'ol days. I never saw any problems with it, and it probably should be added to the source for further testing.
Dagnor's Cauldron; When I spawned Dagnors , I wanted the swimmers to swim across the water, just as I saw them on live. In order to keep a level swim, I used levitate in the water and stayed as close to the surface as possible. Well, this was not good: I had them too close to the surface, which resulted in what really is considered air, and ended in hopping.
-8 is about "sea-level" in Dagnors. this query;
will return most water z coords that need changing. I changed them -10, and now it's almost perfect. they swim across the lake, and you can see their names above the water, just as on live.
Only thing I saw was, when they first start the swim, they hop a few times - but I think this is a "lag" effect between server-client (sort of like when you first log in invisible with KoS mobs close by, they can see you for a few seconds, tell the server realizes you're really invisible).

For anyone trying to download the patches right now, my ISP has issues, so try later:

http://groups.google.com/group/demon...fa0ab3f9a6aeed

Bug
 

I found a bug. In Map.cpp, go to the end of the file (line 982) and change:

to:

Without this change, if a water map doesn't exist for a zone, InWater can/will return true and you will get mobs levitating on land.

I'll update the patches linked in the first post at some point to include this, but it won't be today.

I put the watermaps.tar.gz in the Rathe Forums downloads area - I Imagine eventually FNW or someone will get hold of all this, add it to the source and place the maps in the CVS.

Hey. Good work on this, I know a lot of people have been waiting for this one.

I want to get this put into the code base, but I dont want to commit it in its current form, as its a bit too messy. I had hoped that I would be able to find the time to clean it up myself, but it dosent seem to be happening. So, I am gunna post the major things and have hope that you can clean it up so we can get it in.

First of all, I dont really like the way it is integrated with the existing LOS code. At first, I thought it was extending the existing stuff to add water detection. Not until I read the apathing stuff did it become clear that this is a completely seperate system. So the #1 most important thing to me is to break all the code apart.

The changes to azone are really completely seperate except that they use some of the same library code. I would like to see a seperate tool built for it in order to make this clear. In order to not duplicate code, either just make a new target (awater) in the azone makefiles or just make a "library.cpp" and "library.c" file, and #include "../azone/blah.cpp" all the files into the appropriate extension library file (its ghetto, but it works). Split all your code over into awater.cpp and off we go with that one.

Then we get into zone, where we have the same issue, but things are a bit easier since we all link together. Instead of tapping off the existing Map object, it needs to be an entirely seperate object. WaterMap would be fine. Just follow the leader the way Map was done.

Now, on to actual code. The number one concern I have again is performance. All of the checks for water need to be conditionalied on at least two rules (no, I dont like using the presence of the water file as the toggle). Minimum is "water for fishing" and "water for pathing". I would love to see the same sort of price/performance tradeoffs which were present in the BestZ code made for the water code, where a server op can choose to only check at waypoints or on a slow timer or something as opposed to checking continuously as each mob moves. The position code just gets called way too often for big servers to use it.

For future reference (will go away when you split from Map anyhow), void* bad... avoid at all costs.

Minor but important code quality thing. When you are using constant value numbers to mean things, like your RegionType field, you should use an enum (but leave the region field in the ZBSP_Node struct as a long).

And thus we arrive at the final and big question. One concern I have here is that we are loading up an entire copy of the zone geometry (at least I assume the BSP tree closely resembles the geometry in complexity). I cant say that there is any other good way to accomplish it for sure, but I wanted to raise the topic for conversation. If there is any way to combine this with the LOS data to reduce the amount of crap people need to load into ram, it would be a good thing.

-FNW

I'll clean it up as you suggested, probably at the weekend.

The BSP tree is equivalent in function to the 'Nodes' quadtree in the normal map files, i.e. it subdivides the zone into boxes, however the water map files contain only the BSP tree with the leaf nodes marked if they are water/lava, there is no 'face' information in there, so you couldn't use it for LOS, or BestZ.


I'm sure a single map could be created that could be used for Water/LOS/BestZ, but it's not something I personally feel motivated to look into :)


I'll get back to you when I have had a chance to do the other cleanup work.

Why do it as a tree? Wouldn't it be much faster to hold the water/lava data as a 3d matrix that is quickly addressable by map position (each matrix entry representing a block of size x)? You should be able to entirely avoid the tree traversal that way. It should also reduce the size of your data (but just slightly).

Sorry if I misunderstand the problem.

This is how the EQ zone geometry is stored in the zone S3D files that come with the client. You sound like you know a lot more about 3D geometry than me, but there is an article over at Wikipedia explaining why BSP trees are used in 3D games (more for rendering and stuff):

http://en.wikipedia.org/wiki/Binary_space_partitioning

I just extracted the relevant bits of data in the same tree format it was organised in the zone files. I'm sure it could be optimised just for the purposes of water detection, but once I get something that 'works' for me, my interest kinda stops there :)

If I sounded smart, I had you fooled.

I don't think that BSP trees are the best data model for lava and water since we aren't trying to reduce the computation time of transforming from 3d space to 2d space. We just want to check an attribute of any specific point in 3d space. But I struggle with BSP trees, so maybe I'm just wrong. To me, it begs the question, "why did SOE use BSP trees for their water and lava?" Maybe it's a good way to go, I just think a 3d matrix is simpler and faster since you can check for lava or water without any traversals or transforms.

Maybe I should be quiet since, admittedly, I have 0 BSP experience other than that wiki you linked me (thanks).

Well, my thinking is that the client has to traverse the BSP tree when it renders the graphics, checks for collision detecion, Line of sight to mobs etc, so it is no extra overhead 'for the client' to check for water/lava, since it has already traversed the tree to the leaf node for those purposes.

Now, maybe/quite possibly the SOE *servers* have a different, more efficient mechanism for water detection, perhaps 3D Matrices as you suggest, but we can't know, (unless any ex-SOE programmer's who are reading this would like to add something :) )

That makes sense.

I PM'ed FNW details of the reworked patch for this along the lines he asked for, but I don't think he's been around for the last week, so I will post it here as well in case anyone
else wants to test the updated version.

----------------------------------------

I reworked the Water Detection patch. Diffs against the 1070 source are at:

http://www.rama.demon.co.uk/Reworked-water.patch

It affects these files

And adds these rules:

I thought about adding a test in zone.cpp so that if none of the rules where true, it wouldn't
try and load the water map, but I didn't do that.

Originally I only had the water detection in waypoints.cpp in CalcPosition2 and UpdateWaypoint, however
while I was reworking it, I also put it in the other places BestZ is calculated.

This caused an issue in the SendTo routine. What my code does is turn flymode 1 on when a mob is in water to
stop it sinking to the floor.

It appears the SendTo routine can be (always is) called before the spawn packet has been removed from the
queue and sent to the clients. In this case, the client ignores the flymode packet because the mob doesn't
exist for it yet.

This causes problems because all the water checks assume that if the mob has remained in water since the
last check, that the flymode 1 has already been put on the mob. This is so that unnecessary appearance packets aren't sent to the clients.

I therefore don't attempt to set flymode or set the mob's inWater flag in SendTo.

The other bug I noticed while reworking this is that the DeltaZ
in the spawn update packet is being ignored. I.e. if you watch the mobs pathing in water, they always
move horizontally and then jump up or down when they get to their waypoint (or when another position update packet is sent).

I don't know whether this is a side effect of flymode 1, or whether it's something else, e.g. maybe the offset
for DeltaZ in the spawn struct is not correct and it's something that is only noticeable in water. I
only have the 6.2 client so don't know if it works differently with Titanium. Whatever the problem is,
I couldn't find a way around it.

Do you think this patch will work on 1071? I saw it out and yours not there.

Should work fine, the only file that my patch touches that has been changed in 1071 is ruletypes.h, but my patch still applies fine and everything compiles (haven't actually fired up a server with 1071+WaterPatch, but it should work OK).

It's running TGC on with 1071 without any problems. I have the pathing support disabled, as bestz is too cpu intensive when you have 100 zones loaded, but fishing is working perfectly.

The only issue was azone wouldn't create maps of zones without water/lava, and without maps, fishing works anywhere in the zone. So, I just commented out the lava/water check in azone and recompiled to have it spit out those maps.

Oh, I guess I should also mention I also have this code running on my internal test with is based on 1071, with bestz fully enabled, and it works fine there.

Thanks for the feedback.

I spent 5 or 10 minutes fishing on TGC and didn't catch a thing, although it seems to detect water well enough :) (my fishing skill was low, and then my pole broke!)

Regarding BestZ, I appreciate your problem. My pathing correction tool is coming along quite well. I'm only working on developing it for outdoor zones at the moment (using LOIO as a test subject) as they are easier to fix semi-automatically (i.e. click a button to correct ground collisions or paths that are too high above the ground).

Maybe in a couple of weeks it will be in a state where I can use it produce a (nearly) complete set of corrected grids for a mostly outdoor zone (say LOIO, Frontier Mountains or Field Of Bone) that are free of hopping mobs without the need for the runtime BestZ correction. I've already proved to myself that I can do it for a couple of test grids, but there are still some glitches/bugs I need to work through.

Yeah, fishing sucks at the lower levels. You don't start seeing catches until your skill is around 20 or so. But, once you get to master you get a catch 50% of the time, and maxed skill guarantees something everytime. Though, I agree the water detection is working great!

You have me drooling about the pathing correction tool. Hopping prevention with no additional overhead is something I am very interested in! Could that also be used to get us valid grid points for fear and flee? If so, you just nailed one of the biggest problems in EQEmu. Maybe even the biggest.

Keep up the awesome work!

All I'm working on right now is the tool to correct the grid_entries so that mobs stay on the ground and paths between waypoints don't attempt to go under the ground. This should stop the hopping with no need for BestZ checks in the server.

As was discussed in another thread, FNW's Apathing tool takes these grid_entries along with spawn points, fear_hints etc, to build a zone wide pathing grid that could be used for fear, flee, etc.

Although it is related, it is not something I am actively looking at, although I plan to when I feel what I am currently working on is 'done', but that will probably be in a month or two.

The reworked patch has been comitted after a bit of polishing. Good work.

I hope that some brave soul will hunt down the water info in .ter files. I think we know enough about .ter to find out what material is on each surface, which might give us enough info to build our own bsp tree. I know I spent a long time messing around with the .ter file trying to understand its format. There is some large blob of data after the material data and before the geometry data. I highly suspect that this is (or at least contains) the BSP tree, but I wasnt able to discern any sort of structure from the data. If you look in azone, you will see that I finally got annoyed at it and hard coded a ton of offsets in there to skip over this blob... if somebody figured out the structure, it might help us with water as well as help us get normal maps from newer zones... who knows....

I gave it a shot tonight: http://i8.photobucket.com/albums/a2/...broodlands.jpg

The blue is water, the red stuff is lava, the off-white stuff is other stuff. Only tried it on broodlands so far...

I'm pretty sure that data block after the material list is probably info from that material list, just cause I can't imagine it being entirely a BSP tree when the materials looks structured like, one of the water planes as an example:

The things like fresnel bias and fresnal power and colors have to be stored somewhere in there.

...but it's of course just a theory of mine I can't prove it.

Brilliant code Derision!

Grabbed the latest Win32 compile (1073), put all the watermaps in the /maps folder and set the rules in the database. Zoned to POK and grabbed my fishing pole to test it out. I couldn't fish for landsharks any longer!! Actually had to be near water for fishing to work as it should.

Thanks for another sweet fix!

LoL! landsharks.
It really is nice- the land hopper and water fixes, just added so much more to the game (not to mention all the extra "playable" zones). I've been playing me a necro, got him to level 35 - Things have really changed since the first time I tried to play EqEmu.

Definitely Angelox.

The hopping mobs and water fixes make this emu a much, much better experience.

Derision;
I've been doing some work in Crystal Caverns.
Down at the bottom, where I used to camp stags and spiders, there were "velium crawlers" that lived in the water and were pretty active. In the area of the pond at the bottom of the waterfall- if you got too close to the pond side, and a crawler happend to roam close by, you'd have him wailing on you. This pond area seems to have dead spots that stick the roamer/crawlers, I also seem to not be able to place a waypont up close to the surface - all waypoints stick to the floor. I've made work arounds to all this, but would be nice if some of those roamers could swim closer to the surface. Also, they should be able to swim up and down the water fall, but that ends up in stuck npcs too. I'm wondering if anything can be done here.

Have you got all the watermap options set to true ?

I made a quick grid with the options set as above and the waypoints stayed where I was when I made them, and also the test mob I put on this grid was able to go up and down the waterfall:

This was with the 1080 code BTW, not that it should matter.

Yea, my rules were screwy, thanks. Hope anyone who copied my rules, knows to fix them.

Where can i get these files?
 

The link seems dead , where can i get patch + wtr + source please? :)