EQEmulator Forums

EQEmulator Forums (https://www.eqemulator.org/forums/index.php)
-   Development::Database/World Building (https://www.eqemulator.org/forums/forumdisplay.php?f=596)
-   -   Live Data Collection (Need Community Help) (https://www.eqemulator.org/forums/showthread.php?t=35801)

Akkadius 09-30-2012 11:24 AM

Quote:

Originally Posted by blackdragonsdg (Post 212964)
Hey Akkadius your website rejected the log I attempted to upload tonight. It said it was not a log file but it is a legit log file. I tried to upload the file a second time thinking maybe it was just a fluke but it rejected it again. Any ideas?

It's going to look for the eqlog in the name as well as .txt

blackdragonsdg 10-01-2012 02:53 AM

Quote:

Originally Posted by Akkadius (Post 212971)
It's going to look for the eqlog in the name as well as .txt

Well it wasn't the name that caused the problem. Just a guess but I think 100% packet loss around hop 22 may have had something to do with my upload not working.

Akkadius 10-01-2012 02:40 PM

Quote:

Originally Posted by blackdragonsdg (Post 213007)
Well it wasn't the name that caused the problem. Just a guess but I think 100% packet loss around hop 22 may have had something to do with my upload not working.

Try it again, I changed some max upload size settings on the webserver end.

How large was the file that you were trying to upload?

blackdragonsdg 10-01-2012 02:57 PM

Quote:

Originally Posted by Akkadius (Post 213019)
Try it again, I changed some max upload size settings on the webserver end.

How large was the file that you were trying to upload?

It uploaded successfully today and the file size 2.08MB.

Akkadius 10-01-2012 11:43 PM

Thanks for those who have uploaded logs, but so far this is looking pretty pathetic.

We really need to get a real density of logs before any parsing systems can really be developed around this.

If you know someone with a mother load of logs spanning lots of content please contact them and see if they are willing to contribute to this.

pfyon 10-02-2012 10:29 PM

Haha I was just thinking about this a couple of weeks ago. Glad someone got around to writing a parser before me :)

I found 80 from a variety of characters + another 26 that I renamed as part of archiving back from 2002.

Akkadius 10-02-2012 10:45 PM

Quote:

Originally Posted by pfyon (Post 213065)
Haha I was just thinking about this a couple of weeks ago. Glad someone got around to writing a parser before me :)

I found 80 from a variety of characters + another 26 that I renamed as part of archiving back from 2002.

Oh wow, logs from 03/04. Now we're talking progress.

Once again if I can encourage anyone and everyone to try and get them to upload historic character log files this helps the project quite a bit.

If anyone has questions please let me know.

Thanks,
Akkadius

demonstar55 10-02-2012 10:50 PM

Have you thought about posting this on like the EQ sub-reddit? I'm sure there would be people at least interested in submitting some and there is a larger chance they have logs.

pfyon 10-02-2012 11:35 PM

Any chance you can release some aggregated data once you're done with this project? I'd like to see how older mobs compare (hp/damage wise) to what we have in the peq DB.

Akkadius 10-03-2012 12:15 AM

Quote:

Originally Posted by pfyon (Post 213069)
Any chance you can release some aggregated data once you're done with this project? I'd like to see how older mobs compare (hp/damage wise) to what we have in the peq DB.

Yeah I definitely will, I just need to collect a decent amount of logs before I start putting together the parser.

In the middle of working on some other rather large PEQ parsing projects to get loot, merchantlist, tradeskills from all expansions.

In the meantime I just need to start building the private repository so I have something to work with when the script is being created to parse through all of the text files.

After that I plan on having a web page that people can go to and see a hitlist of what has been collected and what hasn't

Maze_EQ 10-03-2012 12:18 PM

Question....Is test server data useful? I've got access to a multitude of accounts that can basically box everything up to VoA stuff.

pfyon 10-03-2012 09:37 PM

Finished my first 80 using the web interface, found another 46 (in addition to the archived ones).

After sitting in front of my computer for an hour doing those, I decided I'd script it. If you have a linux box and lots of logs, try the following script:
Code:

for x in eqlog*.txt; do echo "Starting $x"; curl -F FileName=@$x -F EQEmuForumName=CHANGEME http://eoc.akkadius.com/eoc/EQEmuDataCollect/LogCollector.php &> /dev/null; done
Change or remove the '-F EQEmuForumName=CHANGEME' to your forum name if you want it to be included.

Akkadius 10-03-2012 10:08 PM

That's quite the collection you have there and that definitely will help.

I've updated the form to include how many logs we've collected so far and the size:

http://eoc.akkadius.com/eoc/EQEmuDat...gCollector.php

pfyon 10-03-2012 11:39 PM

How are you storing the logs? Are you using the original filename? I have multiple logs which share the same filename (I would copy them elsewhere and/or rename them to make parsing of more recent stuff easier) which, if you keep them as the same filename, could get overwritten.

Akkadius 10-03-2012 11:55 PM

Quote:

Originally Posted by pfyon (Post 213095)
How are you storing the logs? Are you using the original filename? I have multiple logs which share the same filename (I would copy them elsewhere and/or rename them to make parsing of more recent stuff easier) which, if you keep them as the same filename, could get overwritten.

No, I thought that through beforehand. They never get saved as the original filename.


All times are GMT -4. The time now is 05:47 PM.

Powered by vBulletin®, Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.