Big Picture Question

Damilis · #1 09-09-2005, 04:06 AM

Okay, hopefully i wont get flamed outta town for this possibly dumb question:

I understand that major code changes happen from version to version, and that requires releaseing new patches, etc. But why will a core db from, say 5.5, be incompatable with, say 0.6.2 ? Is the the db structure, aka collumns and relationships that changes or is it something else?

I am curious because the closer we (by we i really mean the Devs!) get to getting a 'perfected' emu server, then , usually, the smaller the changes are from ver to ver and am wondering if its still to soon to start thinking about a standardized DB format, reguardless of content. Ideally, with identical db formats, one could simply switch from "0.6.2-traditionalEQ" db to "0.6.2-WickedCrazyCustom" db by way of updating an .ini and restarting server.

Additionally, a standard would ideally allow for a more stable software package.

Yes i remember this comming up a few years back, but like i said above, maybe now is the time, or maybe really soon.

Pros/Cons... what do yall think?

sdabbs65 · #2 09-12-2005, 04:44 AM

Quote:

Originally Posted by Damilis

Okay, hopefully i wont get flamed outta town for this possibly dumb question:

I understand that major code changes happen from version to version, and that requires releaseing new patches, etc. But why will a core db from, say 5.5, be incompatable with, say 0.6.2 ? Is the the db structure, aka collumns and relationships that changes or is it something else?

I am curious because the closer we (by we i really mean the Devs!) get to getting a 'perfected' emu server, then , usually, the smaller the changes are from ver to ver and am wondering if its still to soon to start thinking about a standardized DB format, reguardless of content. Ideally, with identical db formats, one could simply switch from "0.6.2-traditionalEQ" db to "0.6.2-WickedCrazyCustom" db by way of updating an .ini and restarting server.

Additionally, a standard would ideally allow for a more stable software package.

Yes i remember this comming up a few years back, but like i said above, maybe now is the time, or maybe really soon.

Pros/Cons... what do yall think?

My Guess is the db structures, have new Fields in them...
ie: augments; were never in the older releases.. thats why it changes all the time...to keep up with the jones's.

mystic414 · #3 09-12-2005, 08:43 AM

Yeah, database tables change frequently as we add support for new features or re-arrange things to make more sense.

Damilis · #4 09-13-2005, 12:33 AM

Well what i was really talking about was a standardization format.

Example:
The first 30 columns in the items table is required to be the same for any eqemu db from here on out (except for major binary revisions of course) and any extra data that an individual server might add is required to be in column 31+.

That way, you can run a query on your own servers items table only returning the first 30 columns, then do a dump on the results and be able to post a viable, working database to contribute to the eqemu cause... althewhile keeping any inhouse 'propritary' info to yourself.

Ideally, one can use these querys as a sort of data-adapter between a customized server and a standardized database.

Additionally, by having a standardized db schema, then it *should* be easier to detect the format of a db and determine compatability with a .exe

what do yall think? is it worth the effort?

Cisyouc · #5 09-13-2005, 02:43 PM

3 problems I see with that.

1. If a server has custom columns in their items table, chances are, their items are unique and aren't really modeling after anything in particular (like EQLive), so why would they contribute? Why not just play on their server?

2. If such a server did contribute items, chances are the columns 31+ they are dropping contain necessary information to make that item work as intended, otherwise, they wouldn't have the column. So such an contribution may not really be worth it anyway because it is not how it was originally designed to be.

3. All servers of the same version share the same database schema unless modified. If a change in code has required the schema to be changed for the server, chances are, the server isn't going to be contributing database information (specifically, items).

Damilis · #6 09-15-2005, 11:25 PM

I totally agree with everything you said Cisyouc, however i think i didnt phrase my post correctly. I was merely talking about the standardization from a devs/db creator point of view. Example: EQServer x is running 6.0 db and when 6.2 comes out they have to do quite a bit of work to change the db over to the new schema. Why couldn't it be more standardized to ease the process?
Maybe a modular style array of .sql's....
eh well, its early/late and i need to gather my thoughts before posing more ideas. Thanks for listening.

Cisyouc · #7 10-08-2005, 09:11 AM

Quote:

Originally Posted by Damilis

I totally agree with everything you said Cisyouc, however i think i didnt phrase my post correctly. I was merely talking about the standardization from a devs/db creator point of view. Example: EQServer x is running 6.0 db and when 6.2 comes out they have to do quite a bit of work to change the db over to the new schema. Why couldn't it be more standardized to ease the process?
Maybe a modular style array of .sql's....
eh well, its early/late and i need to gather my thoughts before posing more ideas. Thanks for listening.

The changelog will tell you what you need to do to update your database. If the playerprofile needs to be changed, they will tell you if an updater for your character_ table is included. More standardized than this will probably be more work for the devs than we need.

monalin crusader · #8 10-10-2005, 02:46 AM

Ok well... some reasons why we dont have a standard for the DB's. I'm not really a dev but I do understand the db and versions ect. Like mentioned above there is content that changes, and there is absolutely nothing u can do about that. People realize that the way they had a table organized in an older version limits its ability. For example, the old grid system is somewhat limited, now we use two tables for grids instead of one. And there is no max number of waypoints, and theres no wasted space in the DB and there are some other reasons why the Dev's changed the grid system. But for the most part its usually items or the npc tables that need modifications. Without changeing the tables how can we improve the Emulator?

Although having a standardized db seems like the ideal situation its really not... and to be honest changing the db is as simple as typeing 'source db.sql;'. You might argue that even though sourcing takes very little time, the time it takes for a dev to redesign and impliment a table modifications is time consumeing. Well thats what we're for, or they are atleast, anything that improves the Emulator is time well spent no matter how much time is invested. If your going to complain about the db changing from version to version, just stay the same version. What is a version anyway? Isnt it a modification of what already existed?

Damilis · #9 10-10-2005, 12:38 PM

Well I surely hope nothing I have said has been interpreted as a complaint because I just recently finished reading all the 0.6.2 source code and, man, talk about ingenious.
As for the DB schema changes and why they happen, no explanation needed. As a software developer, I fully understand the evolution of software from planning stages, thru alpha and beta, to release. But, and this is what I have been talking about, as a project reaches closer and closer to release v1.0, aka exits beta stage, the size and quantity of changed drops exponentially. Any form of standardization has to be implemented in the very beginning (which by nature of the EQEmu project and with all reverse engineering endeavors, is impossible) or near the end of beta. This is where I see the State of EQEmu currently.
I have been following the evolution of this project pretty closely for several years now (Since Dec '02) and I have seen the pain and anguish caused by major code and db changes. Wonderful tools laid to waste by the change, entire DBs of custom content now rendered nearly useless, etc, etc. And such is the way of change, not just with EQEmu but with life in general. </Philosophy> As a result, however, I have noticed an increase in reluctance to create new tools/content, especially with the ever present fear of antiquation looming overhead.
Since the devs are soooooo close to having a real release version, I was just 'sticking the feelers out' to see if others think its time to start tossing around standardizations in db format for the purposes of minimizing the impact of change on a server with a custom db and smoothing out data exchange between people. I think that the Emu can benefit greatly from this, and as my knowledge of the deep workings of the Emu grows, so does my desire to spearhead this project.
I do understand that major changes are still on the horizon, I am not that naive, but I had a Chief (yes I am a 10 year squid/bubblehead) who once said: "Revolutionary changes demand an evolution in a Standard." Granted the use was drastically different, he was complaining about Reactor Plant Startup Procedures, but I see it as applicable here.

BUT, if no one gives a squat, ill just standardize myself and shaddap

Thanks for reading anyways.

**fathernitwit** · #10 10-10-2005, 02:51 PM

emu is far from leaving its state of turm oil, I wouldent expect to ever see it stop. The database is far from perfect, and never will be. Theres hundreds of unimplemented features and rewrites of existing features to be done. When the DB changes, it changes for a reason, and a standard is not going to facilitate the change. Even if a 'stable' release was tagged, it would last about a month before one of the devs put some DB changes in, which would not be in the stable branch because its 'standardized', so they go into the dev branch, and the dev branch would become what it is today... the only thing people are using because thats where all the cool shit is, and it cant be merged because it has 'db changes which break the standard'... the story goes on...

since I got here, database updates have been well documented, and scripts are provided to do anything that is more complicated.

Tools are not broken because of database changes (well, they are, but...), they remain broken because the people who release them do not release the source code for them, so they cannnot be updated by the community, or they are written so terribly or in such an obscure language that nobody else can compile them.

Databases of custom content are lost because the people who develop them follow no form or discipline in their records, and as such they cannot be integrated into future databases. Or, they approximate enough stuff to make it undesireable to use them.

Damilis · #11 10-10-2005, 08:59 PM

Hmm... Okay, I understand all that, but it seems that no one is getting quite what I am saying... probably because I am not expressing myself correctly. Since I cant seem to do that, I am just gonna let this drop. Thanks for your comments a great insight though!

sdabbs65 · #12 10-13-2005, 02:31 AM

You can't stop progress if we did we would still be living in caves and banging chicks over the top of there head with clubs.
Version 1.0 of man was sweet back then...

as for the DB... I agree it will never be standard thats why we need people to share stuff like a working EQExtractor and a Working EQBuilder.
but like FNW said ....people stoped developing them and also stopped shareing.
so...we are left with scraps to peace togher.. My database is around 600 megs
just for the spawns... and is still missing a lot of new zones/tradeskill recipies/etc/etc that will never be completed.
but we get what we can working and build it a little more every day.

Dr Zauis · #13 10-13-2005, 05:20 AM

ou can't stop progress if we did we would still be living in caves and banging chicks over the top of there head with clubs.
Version 1.0 of man was sweet back then...

Yep..Progress Sucks!

sdabbs65 · #14 10-18-2005, 06:03 AM

Quote:

Originally Posted by Dr Zauis

ou can't stop progress if we did we would still be living in caves and banging chicks over the top of there head with clubs.
Version 1.0 of man was sweet back then...

Yep..Progress Sucks!

Im starting to relate.

Cornelius: If you are caught by the gorillas, you must remember one thing.
John Brent: What's that?
Cornelius: Never to speak!
John Brent: What the hell would I have to say to a gorilla?