Go Back   EQEmulator Home > EQEmulator Forums > Support > Support::Packetcollector

Support::Packetcollector Any PacketCollector related problems or questions should be posted here.

Reply
 
Thread Tools Display Modes
  #1  
Old 05-22-2010, 01:49 PM
Derision
Developer
 
Join Date: Feb 2004
Location: UK
Posts: 1,540
Default EQExtractor2

I've been tooling around with Visual C# 2010 the past couple of days and one thing led to another and I wrote an EQ Packet decoder and extended it to dump out SQL statements for Doors and NPC Spawns.

I collected data for the revamped highholdpass and freeport west zones and thought I'd post the SQL for anyone interested:

<Link removed. This data is now incorporated into the PEQ database>

Note that this is just doors and static spawns, no grids or anything, and the mobs will spawn wherever they happened to be when I collected the data. It's all basic stuff, but I thought I'd throw it out there.

Last edited by Derision; 06-12-2010 at 12:45 PM..
Reply With Quote
  #2  
Old 05-22-2010, 03:32 PM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 2,071
Default

Although this seemed to work fine, in West Freeport I could really notice that some NPC's were cut off beyond a certain radius. NPC's from out by commonlands and the temple were non-existent, as well as some other NPC's in the outer perimeter.

Are you in the middle of developing this packet collector further hence just the SQL files?

Thanks for this Derision.
Reply With Quote
  #3  
Old 05-22-2010, 03:55 PM
Derision
Developer
 
Join Date: Feb 2004
Location: UK
Posts: 1,540
Default

Quote:
Originally Posted by Akkadius View Post
Although this seemed to work fine, in West Freeport I could really notice that some NPC's were cut off beyond a certain radius. NPC's from out by commonlands
and the temple were non-existent, as well as some other NPC's in the outer perimeter.
It would help a lot if you could give me a #loc where you are standing, so I can go on live and compare the mobs I collected with what are on live and see what are missing.

It maybe that Live only sends mobs in a certain radius of the player, so I need to run around the zone to get them all.

Quote:
Are you in the middle of developing this packet collector further hence just the SQL files?
This is my first C# program of any size, and certainly isn't anywhere near release quality (hard coded file names, very fragile etc.). I just thought it was cool to see spawns in those
two revamped zones which is why I posted the .SQL for them.
Reply With Quote
  #4  
Old 05-22-2010, 04:21 PM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 2,071
Default

Quote:
Originally Posted by Derision View Post
It would help a lot if you could give me a #loc where you are standing, so I can go on live and compare the mobs I collected with what are on live and see what are missing.

It maybe that Live only sends mobs in a certain radius of the player, so I need to run around the zone to get them all.



This is my first C# program of any size, and certainly isn't anywhere near release quality (hard coded file names, very fragile etc.). I just thought it was cool to see spawns in those
two revamped zones which is why I posted the .SQL for them.
Aye I was just wondering where you were on this because I could have alot of fun with it.

And yes, I am pretty positive for lag reasons there is a certain radius that you cache info wise. But from what it looked like in freeport is that you packet collected from the center. Basically every mob outside the same radius is missing. I would just run your query to check it out, I pulled it up on a MQ2 map read to see all the mob positioning. And yes, I only use MQ for map purposes.
Reply With Quote
  #5  
Old 05-22-2010, 07:38 PM
joligario's Avatar
joligario
Developer
 
Join Date: Mar 2003
Posts: 1,490
Default

Yeah, on live they implemented an anti-MQ2Map feature a while back where they now only send data on NPCs within a certain radius.
Reply With Quote
  #6  
Old 05-22-2010, 08:49 PM
trevius's Avatar
trevius
Developer
 
Join Date: Aug 2006
Location: USA
Posts: 5,946
Default

Well, that is only halfway true. It does help cut down the bandwidth usage and the usefulness of ShowEQ and MQ2 Maps. But, they do still send zone-wide spawns, they just don't send position updates after that point, and maybe not even new spawns, I am not sure. But, if you zone into Blightfire for example, you will get spawns for the entire zone on ShowEQ, so I am pretty sure it still sends all of them on the initial zone in.
__________________
Trevazar/Trevius Owner of: Storm Haven
Everquest Emulator FAQ (Frequently Asked Questions) - Read It!
Reply With Quote
  #7  
Old 05-23-2010, 12:16 PM
Derision
Developer
 
Join Date: Feb 2004
Location: UK
Posts: 1,540
Default

I had a look at freeportwest with MQ2Map and also moved around the zone and I didn't see any obvious missing spawns.

As Trevius says, Live seems to send all the spawns when you zone in, then for mobs within a certain radius, you get rapid position updates for them (with deltas),
and for mobs outside that radius, you get less frequent updates.

I've extended my collector to create grids. Here is an updated .SQL for freeportwest with grids:

<Link removed. This data is now incorporated into the PEQ database>

You will see that for mobs inside the city, near to where I was, e.g the patrolling knights and guards, there are lots of grid entries, because the position updates come every couple
of seconds, whereas for the mobs outside the gates (snakes, bats and the like), there are considerably fewer updates.

Automatic grids created from packet collects are never going to be much good for that reason, although it does give the world builder a starting point I suppose.

Anyway, I guess the next step is for me to get the tool to the point where it is ready for release as an executable at least, so other people can get to play with it.
Probably have time to do that next weekend.

I should add that while in theory it should be possible to have it record a live EQ stream as you are playing, the way I have it setup is to use capture files collected with Wireshark.
So you go to char select, start Wireshark capturing, zone in with a character, come back 20 minutes later stop the capture, save the captured packets and run the extractor
on the .pcap file.

Last edited by Derision; 06-12-2010 at 12:46 PM..
Reply With Quote
  #8  
Old 05-24-2010, 01:29 AM
robinreg
Sarnak
 
Join Date: Feb 2010
Location: California
Posts: 88
Default

This version of highpasshold would be good to replace the old highpass for the SoF/SoD that currently has problems with it. I tried replacing it with the new version in PEQ database. Only problem I am having is the zone point to zoning out of HighPassHold. I can zone in from Kithicor, Keep and East Karana but I can not get the zone out to work.
Reply With Quote
  #9  
Old 05-28-2010, 10:55 PM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 2,071
Default

Derision, would you please please please packet collect the doors/objects for Oldhighpass? Could that be doable? Or are you getting closer to a working version of this for others?
Reply With Quote
  #10  
Old 05-29-2010, 03:22 AM
Derision
Developer
 
Join Date: Feb 2004
Location: UK
Posts: 1,540
Default

Quote:
Originally Posted by Akkadius View Post
Derision, would you please please please packet collect the doors/objects for Oldhighpass? Could that be doable? Or are you getting closer to a working version of this for others?
Is oldhighpass accessable on Live ? It seems like it should be part of Seeds Of Destruction, but I don't see it listed here: http://everquest.allakhazam.com/db/z...of+Destruction

Anyway, assuming it is a SoD zone and it has pre-req tasks for entrance, I don't have any way of getting in there.

I'll try and put a version of my tool out some time this weekend.
Reply With Quote
  #11  
Old 05-29-2010, 04:03 AM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 2,071
Default

Quote:
Originally Posted by Derision View Post
Is oldhighpass accessable on Live ? It seems like it should be part of Seeds Of Destruction, but I don't see it listed here: http://everquest.allakhazam.com/db/z...of+Destruction

Anyway, assuming it is a SoD zone and it has pre-req tasks for entrance, I don't have any way of getting in there.

I'll try and put a version of my tool out some time this weekend.
I had a feeling it wasn't either, I was about to check myself 8 /
Reply With Quote
  #12  
Old 05-29-2010, 02:25 PM
Derision
Developer
 
Join Date: Feb 2004
Location: UK
Posts: 1,540
Default

I've uploaded the executable here: http://code.google.com/p/projecteqem...2.rar&can=2&q=

Prereqs: .NET Framework 4, a Live EQ account and Wireshark to capture the .pcap files that this program processes.

When capturing with Wireshark, ALWAYS start the capture with your character at char select, then zone-in once the capture has started.

The program will only process data for a single zone, so if you move to another zone, only the data for the first zone you entered after starting the capture will be processed.

This is still very much a work-in-progress, but feel free to report any bugs or suggestions here.
Reply With Quote
  #13  
Old 05-29-2010, 08:03 PM
cavedude's Avatar
cavedude
The PEQ Dude
 
Join Date: Apr 2003
Location: -
Posts: 1,988
Default

Tried it out on moors, works perfectly! You can see the end results in the next PEQ database
Reply With Quote
  #14  
Old 05-29-2010, 09:07 PM
trevius's Avatar
trevius
Developer
 
Join Date: Aug 2006
Location: USA
Posts: 5,946
Default

Quote:
Originally Posted by cavedude View Post
Tried it out on moors, works perfectly! You can see the end results in the next PEQ database
Haha, I tried it out on the same zone and yeah, it worked pretty awesomely. I have a few pcaps saved from random collects here and there and figure I might as well use those through this tool first, then will probably start collecting more zones later. Excellent work as always, Derision I can't wait to play with this a bit more. It will definitely save me a ton of time on doors and ground spawns at the very least.

Some other info that might be good to pull would be zone points and maybe even newzone for zone settings, since most of the new zones are just copy/paste of a single example in the zone table. Now that the zone table can have multiple entries for the same zone, I think we could just add an id field as the primary key.
__________________
Trevazar/Trevius Owner of: Storm Haven
Everquest Emulator FAQ (Frequently Asked Questions) - Read It!
Reply With Quote
  #15  
Old 05-30-2010, 12:42 AM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 2,071
Default

Quote:
Originally Posted by Derision View Post
I've uploaded the executable here: http://code.google.com/p/projecteqem...2.rar&can=2&q=

Prereqs: .NET Framework 4, a Live EQ account and Wireshark to capture the .pcap files that this program processes.

When capturing with Wireshark, ALWAYS start the capture with your character at char select, then zone-in once the capture has started.

The program will only process data for a single zone, so if you move to another zone, only the data for the first zone you entered after starting the capture will be processed.

This is still very much a work-in-progress, but feel free to report any bugs or suggestions here.
There are 655 npc's in Dragonscale Hills. Simply amazing Derision. Wow.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

   

All times are GMT -4. The time now is 02:59 AM.


 

Everquest is a registered trademark of Daybreak Game Company LLC.
EQEmulator is not associated or affiliated in any way with Daybreak Game Company LLC.
Except where otherwise noted, this site is licensed under a Creative Commons License.
       
Powered by vBulletin®, Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Template by Bluepearl Design and vBulletin Templates - Ver3.3