New Years resolutions anyone?

[Angelox]

Very nice , and needed!
Although you got much more than I ever found, I attached my "data.php" which I've been building on , I already added a bunch of yours, but it also has corrections and a few more "unknowns" added;
http://www.nahunta.org/~angelox/files/data.php.txt

[Aerewen]

Part of the problem with the lack of a public database is the fact that we don't have a common numbering system for anyone to modify from...

think about it... someone gives you a SQL file that adds say 200 doors to your database that populate pok, abyssmal sea, and say uh shadowhaven as an example... those doors will auto increment their id field so the doors that might be numbered 200-400 on your database might be 500-700 on someone else's

now yeah i know doors dont really matter because the id means nothing really, but think about the npc_types table, the spawngroups spawn2 spawnentry etc etc

we have no default numbering system for those ID's that *DO* matter...

a while back the concept of zonenumber * 1000 was applies to the npc_types table... but it's not consistent. some older zones still have npc's with numbers out of that scope. and the other problem is that we cant do a mass modify script using names extracted from spawn lists because there are a ton of different rat, bat, orc, goblin, giant, etc etc mobs in different zones and i can only hope that they all have different npc_type values in the existing database or we're a lot worse off than i thought we were lol

If we were to take the time to mod the database to a default numbering scheme... then we could start adding to a larger database because we would know that every door, spawngroup, spawn entry, etc etc in the plane of knowledge has an id value between x and y which means that if I make a perfect copy of the live servers' POK i can make 2 SQL files to distribute it.

one does DELETE FROM `blah` WHERE `id` >= 202000 AND `id` < 203000 etc etc

and the other adds my new stuff

reboot and tadaa. but the way it is now... i had to create a new table which was a copy of the npc_types table, then delete 3 rows from it to fit the insert query for the newest pok i found to start working from. then after the inster was done, add those rows back in to make it work with the latest db version, export it all to sql, delete all the npc types entries for pok, delete all the spawn groups etc etc etc, then reinsert my sql file into npc types with all the new values for the merchantlists etc.

difference? took me about 3 hours to source it in, VS about 5 minutes if we had a global setup for the database that everyone used.

im a php/mysql programmer so im actually working on a php system that will go through and revamp my entire database using a numbering scheme of 1000*zoneid on auto increment.

so far the past couple attempts have screwed up loot, merchants, pets, spells etc... so it's still a work in progress, but once i have it finished i will release the new database, not the utility.

if i put out the utility then everyone who has their own custom database work will just use it to mod their db and continue to mooch our releases without any contributions of their own. but by releasing the database only we force them to conform if they want to use our db updates :p

[CrabClaw]

The above example reminds me of what they did when Total Annihilation (a little RTS game with robots) modding was popular. The database was open ended on IDs but to avoid overwriting each other's work they agreed upon ranges of data IDs for each mod out there. So the basic live-like data went from 1 to X and then in blocks of 1000's people reserved space for custom content.

But, over my years watching and trying to contribute what I can to this project, I can say it's like having a overloaded garage full of content with no one really wanting to dig and clean it. But yes there is redundancy in the npc_types database there are 14 same level Snow Dervishes, and 17 'a stonegazer cockatrice''s, when there should only be two stonegazer cockatrices (a level 33-35-ish for the OT, and a level 37-40-ish for the DL and TT).

It is doable bite by byte though.

*looks at his garage*

*hides*

[eq4me]

The big problem is that some of the tables that need to be loaded into shared memory are memory hogs. Right now you just cant space IDs out for further addons without wasting huge ammounts of Memory.

More discussion is here:
http://www.eqemulator.net/forums/sho...0&page=1&pp=15
and here:
http://www.eqemulator.net/forums/showthread.php?t=21466

Angelox: Since PEQ picked up tremendous speed in the last few weeks did you consider joining up with em? I am pretty sure the rigid policies of the last Maintainer are somewhat loosened and the guys from PEQ might be happy to implement many of your additions and corrections. They may stop at purely handspawned zones though. Provided that PEQ Database releases will be more frequent i am sure this would prevent alot of double work as both you and the PEQ Team are prone to fix the same bugs, just look the bug submissions and resolves on the PEQ board ...

Well, back to lurking for now.

[Angelox]

Originally Posted by eq4me

The big problem is that some of the tables that need to be loaded into shared memory are memory hogs. Right now you just cant space IDs out for further addons without wasting huge ammounts of Memory.

More discussion is here:
http://www.eqemulator.net/forums/sho...0&page=1&pp=15
and here:
http://www.eqemulator.net/forums/showthread.php?t=21466

Angelox: Since PEQ picked up tremendous speed in the last few weeks did you consider joining up with em? I am pretty sure the rigid policies of the last Maintainer are somewhat loosened and the guys from PEQ might be happy to implement many of your additions and corrections. They may stop at purely handspawned zones though. Provided that PEQ Database releases will be more frequent i am sure this would prevent alot of double work as both you and the PEQ Team are prone to fix the same bugs, just look the bug submissions and resolves on the PEQ board ...

Well, back to lurking for now.

Let me re-phrase what I mean, and sum it all up; there is no public database; PEQ database belongs to PEQ server, and the drive is towards the success of the PEQ server, just as is with all the servers at EqEmu. You place a server up, it becomes competitiion, You see, we are not working for each other, we are competing with each other. No hard feelings, it's just the way things works out.
We need to make a database to where the priority is to the database, and not any specific server. Something that is totally accessable by anyone who qualifies, and totatally viewable/downloadble by anyone, anytime, and *no strings attached*.
But this is something I tried for when I first started, and i started posting anything I did with hopes as to start something, it worked somewhat - but there's more to it than that I guess. I suggested we combine what we all have now ( the three database), and they can be combined and optimized, use it to start a new database with a new year. If something is not right, it can be fixed or corrected, but at least we started with everyones work. and it is "public domain".

[cavedude]

PEQ is not about the server at all. I don't care how popular it is or who likes it or not. I don't deal with players well, and luckily WildcardX is kind enough to do so for me.

The only goal of PEQ is database development, the server is just a testbed for any changes we make. IMHO, running a centralized server for players to login into and catch bugs is the best way to develop a database.

Anybody who wishes to help us out are more than welcome to do so. We can find a job for anybody regardless of knowledge/skill level.

[Zengez]

I agree with Cavedude, which is why i wanted to start something just like it, but the drawback I knew of to PEQ (haven't really been keeping up a lot lately) was that there were a very very select few in charge of doing any database work, which was good in that you don't have newbins messing everything up, but if those people drop the project/have RL stuff/ get stuck on a project/whatever then the project stalls, and we all look on with frustrated hopes of getting another expansion poped and some of us thinking "Man, I could be helping get stuff done if I were able..." which is exactly why I wanted a truly open server to spawn/test/update/etc... the problem being again the newbins going "Hey! lets make all the giant bats in north Qeynos into mini Cazic Thules!" which then messes up all the hard work we're trying to accomplish.

Solutions? I don't know... perhaps a mass recruition of database editors into a world using the newest databases available to start work... perhaps the best and the brightest move on to PEQ, kind of an 'application' to join PEQ team, thus allowing a larger pool of talented and able people to work on the real database, avoiding the newbins and more importantly getting people in to work on the real database structure that people can use...

As to Angelox's worries about competition, I definitely see where he's coming from, but I don't know if you're going to have a lot of choice if you want to do anything more than just have people post updates and snippets of code every time they finish a half dozen spawns for 'somebody out there' to include into a database that they might then share... /shrug just my thoughts.

And now to eat, hopefully some of that made sense.

EDIT::

Oh yea, as the actual main part I meant to write, The idea of coming up with a cohesive and mandated ID structuring system would be a great idea and a good way to give order to the chaos so that some of us that don't feel like diving into the database for hours at a stretch could do some work occasionally without getting lost in the myriad of mapping ids and tables...

Also, some kind of structure between the loottable id, npc id, and the spawngroup id would be immensely helpful too, but thats for another tangent.

[John Adams]

Originally Posted by eq4me

They may stop at purely handspawned zones though.

For the love of God and all that's Holy... this is probably the 5th time I've asked this with no acknowledgement coming back from anyone who might know... eq4me's statement eludes to the fact there might be a way to NOT have to hand-spawn zones. If this is true, please oh PLEASE let the rest of us know. We could do so much more [accurate] work if we still had this capability.

I understand the builders/extractors/etc are not kept up with whatever encryption Live has week after week. But if there is some other way, I'm begging... please let us know.

As an aside, I have SEQ running (and am totally amazed by it, btw) and grabbing collects from any zone I can just to see what it looks like. Most of it is garbled, of course. I cannot read this myself, without knowledge. I'd love to take my new collects and stuff them into a database, even if it were by hand INSERTS of the raw data. I just need to be able to read the packets. PF2Text doesn't seem to be anywhere anymore (at least I cannot find it). Is there any info, at all? SEQ lacks a users guide or walkthrough, so it's nearly impossible to figure out except by trial and error - which is what I've been doing the last week.

Yes, I am stomping my feet. And begging. Please, if there's still a way to benefit from collects, tell us. I'll start now on any zone I have access to.

(btw, this was not @eq4me, his comment just reminded me how ignored this question has gone)

[eq4me]

Yes, I wanted to hint to this post:

http://www.eqemulator.net/forums/sho...769#post126769

The big question is if someone other than FNW is able to update eqextractor.

[Angelox]

Originally Posted by eq4me

Yes, I wanted to hint to this post:

http://www.eqemulator.net/forums/sho...769#post126769

The big question is if someone other than FNW is able to update eqextractor.

I will state that it would be crazy IMO to not use collected data to initially spawn a zone. If you ask me, this is the primary reason that PEQs database still exists when so many other (hand spawned) ones do not... people making shit up from memory just dosent work, especially on a large scale, so many intricacies that make the world emmersive are lost. PEQ exists on the foundation of strict controls on modifications, such that somebody who thinks they remember something a certain way does not screw up the database without doing their homework first. This was a very significant problem for the older database

You know, I could say a lot here, but I really don't want to start anything
But I will say, that when I first started here, it also was said that merging databases was "very hard", almost impossible", and a "waste of time".
I will also like to say that the database I have been working on for all of you, has "hand spawned " and "merged" zones (you would never know unless I told you).
I am very gratefull to a lot of the Devs and programmers in this forum (for all their help), so I will continue to "bite my tongue" , and delete the rest of what I wanted to say.

[Zengez]

Just thought I'd mention it, Tried out the packet collector, and found that I could get it to log my zoning just fine, but any actual information in the zone was not logged at all. Tried all the devices and whatnot with no luck.

EDIT::

Apparently the packets don't actually appear in the file until it's done building which is when I zone out... which makes sense now that I think about it, so scratch this, let the collections begin;

PS: Is there a list somewhere of zones needing collecting?

[John Adams]

Originally Posted by Zengez

PS: Is there a list somewhere of zones needing collecting?

Yes, everything from PoP release to DoDH and beyond, if you're bored.

I'm doing the same with SEQ, and getting some cool info in just the text dumps (still not the golden data I am after, but still fun to play with). Let's see if we cannot bring the packet God out of hiding.

[cavedude]

PEQ actually has most zones our client supports spawned. What we are missing are mainly nightmareb, and most LDoN zones. The rest will be released once we have tweaked things a bit more.