Water Detection Patch

[Derision]

Here it is: EQEmu Water Detection beta, based on the 1070 code.

Zone patch: http://www.rama.demon.co.uk/zone.patch

utils/AZone patch:http://www.rama.demon.co.uk/azone.patch

Full Source Tree: http://www.rama.demon.co.uk/EQEmu-0....tection.tar.gz

Water maps for all S3D zones (16MB): http://www.rama.demon.co.uk/watermaps.tar.gz

It doesn't do a lot (anything) for Angelox's issue with bouncing mobs pathing through water in Dagnor's Cauldron,
which is what got me started down this road, but I think that is another issue, primarily mobs attempting
to path in the air above the water.

What these changes do are:

Amend azone to have an option to write out water maps (.wtr). These actually contain a copy of the BSP
tree from the WLD data in the S3D files with leaf nodes marked as to whether they are in water/lava.

Load these .wtr maps into the emulator.

Provide the ability to detect if a given co-ordinate is in water/lava.

Doesn't do BestZ calculation under water.

Add a new flag to the mob class indicating if it is in water or not. This is initialsed to false when
the mob is created set to true or false as appropriate in CalculatePosition2.

Turns flymode 1 on or off as mobs path into/out of water. This stops mobs falling
to the floor underwater. If you try Kedge/Powater before and after applying this patch, you will see
the difference.

I looked into also doing this as soon as the mob is spawned, rather than waiting for the waypoints to
kick in to avoid the initial 'fall to the floor', however I was unable to see an obvious way of doing
this due to the asynchronous way the spawns are qeueued up before being sent to the client.

I even tried setting all the unknown fields in the Spawn packet to 1s in the hope that there was a field
there which affected flymode, but didn't find anything.

I also altered the fishing code to use the water detection. The uses some constants relating to how
far in front and down from you the water has to be for you to be able to fish. These may need tweaking
a bit to match live. I noticed I left a debugging message in there which tells you the co-ords of the
end of your fishing line, which needs to come out.

You won't need to use azone if you download the water maps from the link above, just put the .wtr files
in your Maps directory.

If you want to use azone, to create a water map, supply the new -w parameter, e.g.

azone -w cauldron

Some S3D files which only have one type of special area, i.e. just water or just lava, don't tag the
region as to whether they are water or lava. In these cases, azone will default to tagging them as
water. The only exceptions I came across where:

arena (the firepots are flagged as lava)
lavastorm, fearplane (ring of fire), pofire, skyfire

In these cases, supply the -dl (default lava) flag to azone to mark these as lava, e.g.

azone -w -dl pofire

Comments:

The lava areas are included in the water maps, even though I don't know if there is any use for that.

The map could be optimised by pruning branches of the BSP tree that have no water/lava in them.

The walking of the BSP tree is currently done by recursion. This could be turned into a loop to avoid
some function call overhead.

If you don't have a .wtr map for a zone in your Maps directory, everything should just work as before.

This only works for zones in S3D format. If you are using a zone that has both S3D and EQG maps and
you are using the EQG version, don't use the .wtr map for that zone.

I did a search of the forums and couldn't find any information on where the BSP tree and special region
information is in the EQGs, hence no support.

You can find out whether you are in water or lava by using the #bestz command in-game.

If you try it, let me know your thoughts, bugs, suggestions.

I would add that Windcatcher's Openzone source and WLD documentation, along with the code that was already in Azone (FNW, Daeken_bb?) were the keys to getting this to work.

[Angelox]

Damn you're good! thank you very much! I know what is wrong with Cauldon (just dawned on me), and can fix it ( I think) - I'll get back to you when I do.
First I need to implement your patches

Thank you!!!!!

[Cripp]

awsome Derision, great work !

Merry Christmas to all!

[Angelox]

Well, guess what I've been doing most the day?
This update all works very well - it was so nice to see Kedge Keep back to normal, just like the good'ol days. I never saw any problems with it, and it probably should be added to the source for further testing.
Dagnor's Cauldron; When I spawned Dagnors , I wanted the swimmers to swim across the water, just as I saw them on live. In order to keep a level swim, I used levitate in the water and stayed as close to the surface as possible. Well, this was not good: I had them too close to the surface, which resulted in what really is considered air, and ended in hopping.
-8 is about "sea-level" in Dagnors. this query;

SELECT * FROM grid_entries where zoneid=70 and z >=-50 and z<=-5;

will return most water z coords that need changing. I changed them -10, and now it's almost perfect. they swim across the lake, and you can see their names above the water, just as on live.

UPDATE grid_entries SET z=-10 where ( z >=-50 and z<=-5) and zoneid=70;

Only thing I saw was, when they first start the swim, they hop a few times - but I think this is a "lag" effect between server-client (sort of like when you first log in invisible with KoS mobs close by, they can see you for a few seconds, tell the server realizes you're really invisible).

[Derision]

For anyone trying to download the patches right now, my ISP has issues, so try later:

http://groups.google.com/group/demon...fa0ab3f9a6aeed

[Derision]

I found a bug. In Map.cpp, go to the end of the file (line 982) and change:

bool Map::InWater(float y, float x, float z) {
        if(BSP_Root) return(BSPReturnRegionType(1, y, x, z) == 1);
}

bool Map::InLava(float y, float x, float z) {
        if(BSP_Root) return(BSPReturnRegionType(1, y, x, z) == 2);
}

to:

bool Map::InWater(float y, float x, float z) {
        if(BSP_Root) return(BSPReturnRegionType(1, y, x, z) == 1);
        return false;
}

bool Map::InLava(float y, float x, float z) {
        if(BSP_Root) return(BSPReturnRegionType(1, y, x, z) == 2);
        return false;
}

Without this change, if a water map doesn't exist for a zone, InWater can/will return true and you will get mobs levitating on land.

I'll update the patches linked in the first post at some point to include this, but it won't be today.

[Angelox]

I put the watermaps.tar.gz in the Rathe Forums downloads area - I Imagine eventually FNW or someone will get hold of all this, add it to the source and place the maps in the CVS.

[fathernitwit]

Hey. Good work on this, I know a lot of people have been waiting for this one.

I want to get this put into the code base, but I dont want to commit it in its current form, as its a bit too messy. I had hoped that I would be able to find the time to clean it up myself, but it dosent seem to be happening. So, I am gunna post the major things and have hope that you can clean it up so we can get it in.

First of all, I dont really like the way it is integrated with the existing LOS code. At first, I thought it was extending the existing stuff to add water detection. Not until I read the apathing stuff did it become clear that this is a completely seperate system. So the #1 most important thing to me is to break all the code apart.

The changes to azone are really completely seperate except that they use some of the same library code. I would like to see a seperate tool built for it in order to make this clear. In order to not duplicate code, either just make a new target (awater) in the azone makefiles or just make a "library.cpp" and "library.c" file, and #include "../azone/blah.cpp" all the files into the appropriate extension library file (its ghetto, but it works). Split all your code over into awater.cpp and off we go with that one.

Then we get into zone, where we have the same issue, but things are a bit easier since we all link together. Instead of tapping off the existing Map object, it needs to be an entirely seperate object. WaterMap would be fine. Just follow the leader the way Map was done.

Now, on to actual code. The number one concern I have again is performance. All of the checks for water need to be conditionalied on at least two rules (no, I dont like using the presence of the water file as the toggle). Minimum is "water for fishing" and "water for pathing". I would love to see the same sort of price/performance tradeoffs which were present in the BestZ code made for the water code, where a server op can choose to only check at waypoints or on a slow timer or something as opposed to checking continuously as each mob moves. The position code just gets called way too often for big servers to use it.

For future reference (will go away when you split from Map anyhow), void* bad... avoid at all costs.

Minor but important code quality thing. When you are using constant value numbers to mean things, like your RegionType field, you should use an enum (but leave the region field in the ZBSP_Node struct as a long).

typedef enum {
RegionTypeNormal = 0,
RegionTypeWater = 1,
RegionTypeLava = 2
} WaterRegionType;

And thus we arrive at the final and big question. One concern I have here is that we are loading up an entire copy of the zone geometry (at least I assume the BSP tree closely resembles the geometry in complexity). I cant say that there is any other good way to accomplish it for sure, but I wanted to raise the topic for conversation. If there is any way to combine this with the LOS data to reduce the amount of crap people need to load into ram, it would be a good thing.

-FNW

[Derision]

Originally Posted by fathernitwit

So, I am gunna post the major things and have hope that you can clean it up so we can get it in.

I'll clean it up as you suggested, probably at the weekend.

The BSP tree is equivalent in function to the 'Nodes' quadtree in the normal map files, i.e. it subdivides the zone into boxes, however the water map files contain only the BSP tree with the leaf nodes marked if they are water/lava, there is no 'face' information in there, so you couldn't use it for LOS, or BestZ.


I'm sure a single map could be created that could be used for Water/LOS/BestZ, but it's not something I personally feel motivated to look into


I'll get back to you when I have had a chance to do the other cleanup work.

[narcberry]

Why do it as a tree? Wouldn't it be much faster to hold the water/lava data as a 3d matrix that is quickly addressable by map position (each matrix entry representing a block of size x)? You should be able to entirely avoid the tree traversal that way. It should also reduce the size of your data (but just slightly).

Sorry if I misunderstand the problem.

[Derision]

Originally Posted by narcberry

Why do it as a tree? Wouldn't it be much faster to hold the water/lava data as a 3d matrix that is quickly addressable by map position (each matrix entry representing a block of size x)? You should be able to entirely avoid the tree traversal that way. It should also reduce the size of your data (but just slightly).

Sorry if I misunderstand the problem.

This is how the EQ zone geometry is stored in the zone S3D files that come with the client. You sound like you know a lot more about 3D geometry than me, but there is an article over at Wikipedia explaining why BSP trees are used in 3D games (more for rendering and stuff):

http://en.wikipedia.org/wiki/Binary_space_partitioning

I just extracted the relevant bits of data in the same tree format it was organised in the zone files. I'm sure it could be optimised just for the purposes of water detection, but once I get something that 'works' for me, my interest kinda stops there

[narcberry]

If I sounded smart, I had you fooled.

I don't think that BSP trees are the best data model for lava and water since we aren't trying to reduce the computation time of transforming from 3d space to 2d space. We just want to check an attribute of any specific point in 3d space. But I struggle with BSP trees, so maybe I'm just wrong. To me, it begs the question, "why did SOE use BSP trees for their water and lava?" Maybe it's a good way to go, I just think a 3d matrix is simpler and faster since you can check for lava or water without any traversals or transforms.

Maybe I should be quiet since, admittedly, I have 0 BSP experience other than that wiki you linked me (thanks).

[Derision]

Originally Posted by narcberry

I don't think that BSP trees are the best data model for lava and water ...

Well, my thinking is that the client has to traverse the BSP tree when it renders the graphics, checks for collision detecion, Line of sight to mobs etc, so it is no extra overhead 'for the client' to check for water/lava, since it has already traversed the tree to the leaf node for those purposes.

Now, maybe/quite possibly the SOE *servers* have a different, more efficient mechanism for water detection, perhaps 3D Matrices as you suggest, but we can't know, (unless any ex-SOE programmer's who are reading this would like to add something )

[narcberry]

Originally Posted by Derision

Well, my thinking is that the client has to traverse the BSP tree when it renders the graphics, checks for collision detecion, Line of sight to mobs etc, so it is no extra overhead 'for the client' to check for water/lava, since it has already traversed the tree to the leaf node for those purposes.

That makes sense.

[Derision]

I PM'ed FNW details of the reworked patch for this along the lines he asked for, but I don't think he's been around for the last week, so I will post it here as well in case anyone
else wants to test the updated version.

----------------------------------------

I reworked the Water Detection patch. Diffs against the 1070 source are at:

http://www.rama.demon.co.uk/Reworked-water.patch

It affects these files

entwisd@rama ~/EQEmu-0.7.0-1070 $ patch -p0 < /tmp/Reworked-water.patch
patching file ./common/ruletypes.h
patching file ./utils/azone/Makefile
patching file ./utils/azone/awater.cpp
patching file ./utils/azone/awater.h
patching file ./utils/azone/wld.c
patching file ./utils/azone/wld.h
patching file ./zone/command.cpp
patching file ./zone/forage.cpp
patching file ./zone/makefile.common
patching file ./zone/mob.cpp
patching file ./zone/mob.h
patching file ./zone/watermap.cpp
patching file ./zone/watermap.h
patching file ./zone/waypoints.cpp
patching file ./zone/zone.cpp
patching file ./zone/zone.h

And adds these rules:

+RULE_BOOL ( Watermap, CheckWaypointsInWaterWhenLoading, false ) // Does not apply BestZ as waypoints are loaded if they are in water
+RULE_BOOL ( Watermap, CheckForWaterAtWaypoints, false) 		// Check if a mob has moved into/out of water when at waypoints and sets flymode
+RULE_BOOL ( Watermap, CheckForWaterWhenMoving, false)		// Checks if a mob has moved into/out of water each time it's loc is recalculated
+RULE_BOOL ( Watermap, CheckForWaterOnSendTo, false)		// Checks if a mob has moved into/out of water on SendTo
+RULE_BOOL ( Watermap, CheckForWaterWhenFishing, false)		// Only lets a player fish near water (if a water map exists for the zone)
+RULE_REAL ( Watermap, FishingRodLength, 30)			// How far in front of player water must be for fishing to work
+RULE_REAL ( Watermap, FishingLineLength, 40)			// If water is more than this far below the player, it is considered too far to fish

I thought about adding a test in zone.cpp so that if none of the rules where true, it wouldn't
try and load the water map, but I didn't do that.

Originally I only had the water detection in waypoints.cpp in CalcPosition2 and UpdateWaypoint, however
while I was reworking it, I also put it in the other places BestZ is calculated.

This caused an issue in the SendTo routine. What my code does is turn flymode 1 on when a mob is in water to
stop it sinking to the floor.

It appears the SendTo routine can be (always is) called before the spawn packet has been removed from the
queue and sent to the clients. In this case, the client ignores the flymode packet because the mob doesn't
exist for it yet.

This causes problems because all the water checks assume that if the mob has remained in water since the
last check, that the flymode 1 has already been put on the mob. This is so that unnecessary appearance packets aren't sent to the clients.

I therefore don't attempt to set flymode or set the mob's inWater flag in SendTo.

The other bug I noticed while reworking this is that the DeltaZ
in the spawn update packet is being ignored. I.e. if you watch the mobs pathing in water, they always
move horizontally and then jump up or down when they get to their waypoint (or when another position update packet is sent).

I don't know whether this is a side effect of flymode 1, or whether it's something else, e.g. maybe the offset
for DeltaZ in the spawn struct is not correct and it's something that is only noticeable in water. I
only have the 6.2 client so don't know if it works differently with Titanium. Whatever the problem is,
I couldn't find a way around it.