Question about collecting

[provocating]

I am just curious on doing a packet collect of a zone if you stop, or accidentally zone out, or even have to stop because it is such a complicated zone then why can you not piece together captures ? It would seem since you are generating an SQL file you could do a compare with a program and see where the differences are ?

I am just curious because I am currently doing a zone and notice that I have some spawns that a previous dump someone did, were missing.

[Derision]

Well, you can piece captures together manually if you want, e.g. if you have a named/rare spawn that was in your new capture that wasn't in a previous one, you can just take the INSERTs for that mob and source it in.

Of course you would need to change the id column to be one after the last insert from the previous collect.

Also, there is no automated way oif knowing which mob from the previous collect was the placeholder for the rare mob, unless it doesn't roam, in which case you can just look for a mob in the previous collect at the same location.

If it does roam, then you would have to look at the grid entries to figure out which the placeholder was. Of course if you are familiar with a particular zone on Live, then this would be a lot easier.

[provocating]

Well my log for westfreeport was 10 times large than the one here ...

http://www.peqtgc.com/logs/

Do you want to look at it before I post it somewhere ?

[provocating]

Well I uploaded it, guess I should had put my name on the filename but I did not.

http://www.peqtgc.com/logs/file.php?fm_path=/classic%20-%20overhauled/freeportwest.zip

If someone could check it to make sure I did it right. I spent about 3-4 hours in freeportwest trying to make sure I opened every vendor and combed every inch of the zone, using the automap in EQ to make sure I touched all areas. If this is a way i can contribute then this is something I can and do not mind doing.

[Derision]

I just sourced in that log and can verify it is a good collect. You have a few more spawns than the previous collect (I think there are some Christmas mobs up at the moment), about 50 more merchants, and several more ground spawns (possibly seasonal, maybe added since the original collect, etc.), plus loads more pathing data.

So, good job :) It's really up to you if you want to spend time collecting. I'm, sure the collect will come in useful, either to PEQ when they reach that expansion in the progression, or to other server ops who want to populate those zones sooner and not wait for PEQ.

[provocating]

It's really up to you if you want to spend time collecting.

I want to do anything to be useful instead of being a leech and using the product without chipping in, so yes I am going to continue this. My only issue is my guy hit level 65 and I never leveled him further. So I will collect every zone I can collect. I am going to start with Freeport and the adjacent zones, then spider out from there.

I just really wanted to know if I did it right, the last thing I want to do is spend hours and not get a good extraction. I am saving the pcap files to my server in case we need them in the future. Doing East Freeport as we speak.

[provocating]

Oh noooes. I was about 30 minutes done from a killer feed from East Freeport, wireshark crashed.

[provocating]

Okay East Freeport done, another large one, around 10mb.

Derision you seemed to have no issues getting the SQL into your Database. How did you know exactly what rows to delete so easily ?

[provocating]

I just checked before I imported the new West Freeport in. The original West Freeport had 378 spawn points, this new one has 433 spawn points. I just zoned into West Freeport and I repopped the zone, damn I am impressed. I actually got the group of guards pathing correctly. It is worth the effort just to see them pathing like they should. Whoever wrote the extractor, give them a raise.

My database went up 10mb in size too. I will do adjacent zones tomorrow.

UPDATE

I just checked my East Freeport collect after sourcing it in. I have a bunch of Tyker Renlin's and Plnorrick Spinecracker's apawning at the docks and pathing. Not sure what happened there. Also the boats are patching into the rocks, which that could be explained by me not knowing that I was collecting information about the boats, I can sit around the water next time. Any help on this would be welcome, not sure what I did wrong. Could this be caused by me passing by the same area twice or something ? I do not know enough about what goes on in the background to make an assumption on what I did wrong on the packet collect. I definitely took my time and tried to be thorough. West Freeport looks great so far, other than there are multiple spawns of like the Holiday spawns.

[Derision]

According to Allakazam, Tyker Renlin spawns and then despawns after 4 minutes, so getting multiple spawns is to be expected. I imagine the other NPC must similarly respawn frequently.

The problem with collecting pathing data is that Live sends updates less frequently the further away you are from a mob, so you can get fairly good pathing for the mobs that are nearby, but not the ones further away.

Problem is, if you sit near the docks to get good pathing for the boats, then the mobs on the other side of the zone will have poor pathing. There is no easy answer to this. I would just treat the collected pathing data as a 'rough guide' to be fixed by hand later.

Regarding me getting the West Freeport data sourced in quickly, that just happens to be the zone I used to test the collector after Live patches, so I had already nuked the PEQ data for that zone and so I just needed to source in your collect without worrying about having to delete rows first, as all my data from that zone was already numbered by the collector, and so the auto-generated DELETEs at the top of the SQL deleted everything.

[provocating]

You are awesome, that explains it.


Problem is, if you sit near the docks to get good pathing for the boats, then the mobs on the other side of the zone will have poor pathing. There is no easy answer to this. I would just treat the collected pathing data as a 'rough guide' to be fixed by hand later.

But if you also spent time on the other side of the zone would you not also have good pathing, does it somehow overwrite the other ? Not sure how it is coded.

[Chanus]

You are awesome, that explains it.



But if you also spent time on the other side of the zone would you not also have good pathing, does it somehow overwrite the other ? Not sure how it is coded.

I think he's just saying you can't get it all at once, you'd need to do it in pieces to get everything accurately... like the Heisenberg Pathing Uncertainty Principle :P

[Derision]

But if you also spent time on the other side of the zone would you not also have good pathing, does it somehow overwrite the other ? Not sure how it is coded.
Say you are near mob A, which is walking a square route, say around the outside of a building. While you are near it, you get good pathing data.

Now if you move to mob B on the other side of the zone, the server is still sending you movement data for mob A, just not as frequently. What will likely happen now is you get a movement update for mob A when it is on one side of the building (let's call that grid_entry n), and the in the next update (n + 1) it may have turned the corner and moved some distance.

When the SQL is sourced in and you enter the zone in EQEmu, when the mob gets to waypoint n and sets of toward waypoint n+1, it will walk through the walls of our hypothetical building, because the chances of us getting a movement update in the capture at the precise time it was at the corner of the building and making a 90 degree turn are not very high.

Again, this can be fixed by manually editing the database later, by deleting the grid_entries other than the ones that were captured when you were nearby the mob and got frequent movement updates, which would have included direction changes.

Manual editing will also be required if you are in the zone for a long time. If you are in a zone for hours, you will have captured the mob making many circuits of its patrol route, so any waypoints beyond that required to complete one circuit are redundant and should be deleted.

[provocating]

That makes sense, thanks again for replying Derision.