Bogus

[classiceq]

So here is what happened....

One of my kids decided to 'learn' mysql and ended up deleting my eqemu database. Fortunately I had a backup but it doesn't appear to be restoring correctly. The server starts up ok with no errors.

However....here is the important stuff...

[Debug] Starting Log: logs/eqemu_debug_world_3548.log
[Debug] [WORLD__INIT] Loading server configuration..
[Debug] [WORLD__INIT] Log settings loaded from log.ini
[Debug] [WORLD__INIT] CURRENT_WORLD_VERSION:EQEMu 0.7.0
[Debug] [WORLD__INIT] Connecting to MySQL...
[Status] Starting Log: logs/eqemu_world_3548.log
[Status] Using database 'peq' at localhost:3306
[Debug] [WORLD__INIT] Starting HTTP world service...
[Debug] [WORLD__HTTP] Loaded mime types from mime.types
[Debug] [WORLD__INIT] Loading variables..
[Debug] [WORLD__INIT] Loading zones..
[Debug] [WORLD__INIT] Clearing groups..
[Debug] [WORLD__INIT] Clearing temporary merchant lists..
[Debug] [WORLD__INIT] Loading items..
[Status] EMuShareMem loaded
[Status] Loading items from database: count=62255
[Debug] [WORLD__INIT] Loading guilds..
[Debug] [RULES__CHANGE] Loading rule set 'default' (0)
[Debug] [RULES__CHANGE] Set rule GM:MinStatusToZoneAnywhere to value 250
[Debug] [RULES__CHANGE] Set rule Character:MaxLevel to value 70
[Debug] [RULES__CHANGE] Set rule Character:LeaveCorpses to value true
[Debug] [RULES__CHANGE] Set rule Character:LeaveNakedCorpses to value true
[Debug] [RULES__CHANGE] Set rule Character:ExpMultiplier to value 2.5000000000000
[Debug] [RULES__CHANGE] Set rule Character:DeathExpLossLevel to value 10
[Debug] [RULES__CHANGE] Set rule Character:CorpseDecayTimeMS to value 86400000
[Debug] [RULES__CHANGE] Set rule Character:AutosaveIntervalS to value 300
[Debug] [RULES__CHANGE] Set rule Character:HPRegenMultiplier to value 100
[Debug] [RULES__CHANGE] Set rule Character:ManaRegenMultiplier to value 100
[Debug] [RULES__CHANGE] Set rule Character:EnduranceRegenMultiplier to value 100
[Debug] [RULES__CHANGE] Set rule Guild:MaxMembers to value 2048
[Debug] [RULES__CHANGE] Set rule Skills:MaxTrainTradeskills to value 21
[Debug] [RULES__CHANGE] Set rule Pets:AttackCommandRange to value 150.0000000000000
[Debug] [RULES__CHANGE] Set rule World:ZoneAutobootTimeoutMS to value 120000
[Debug] [RULES__CHANGE] Set rule World:ClientKeepaliveTimeoutMS to value 95000
[Debug] [RULES__CHANGE] Set rule Aggro:PetSpellAggroMod to value 10
[Debug] [RULES__CHANGE] Set rule Aggro:SongAggroMod to value 33
[Debug] [RULES__CHANGE] Set rule Aggro:SpellAggroMod to value 100
[Debug] [RULES__CHANGE] Set rule Combat:BaseCritChance to value 0.0000000000000
[Debug] [RULES__CHANGE] Set rule Combat:WarBerBaseCritChance to value 0.0300000000000
[Debug] [RULES__CHANGE] Set rule Combat:BerserkBaseCritChance to value 0.0600000000000
[Debug] [RULES__CHANGE] Set rule Combat:NPCBashKickLevel to value 6
[Debug] [RULES__CHANGE] Set rule Character:ConsumptionMultiplier to value 200
[Debug] [RULES__CHANGE] Set rule Spells:AutoResistDiff to value 15
[Debug] [RULES__CHANGE] Set rule Spells:ResistChance to value 2.0000000000000
[Debug] [RULES__CHANGE] Set rule Spells:ResistMod to value 0.4000000000000
[Debug] [RULES__CHANGE] Set rule Spells:PartialHitChance to value 0.7000000000000
[Debug] [RULES__CHANGE] Set rule Combat:ClientBaseCritChance to value 0.0000000000000
[Debug] [RULES__CHANGE] Set rule Zone:NPCGlobalPositionUpdateInterval to value 60000
[Debug] [RULES__CHANGE] Set rule NPC:MinorNPCCorpseDecayTimeMS to value 600000
[Debug] [RULES__CHANGE] Set rule NPC:MajorNPCCorpseDecayTimeMS to value 1200000
[Debug] [RULES__CHANGE] Set rule Zone:GraveyardTimeMS to value 1200000
[Debug] [RULES__CHANGE] Set rule Zone:EnableShadowrest to value true
[Debug] [RULES__CHANGE] Set rule Map:FixPathingZWhenLoading to value true
[Debug] [RULES__CHANGE] Set rule Map:FixPathingZAtWaypoints to value true
[Debug] [RULES__CHANGE] Set rule Map:FixPathingZWhenMoving to value true
[Debug] [RULES__CHANGE] Set rule Map:FixPathingZOnSendTo to value false
[Debug] [RULES__CHANGE] Set rule Zone:ClientLinkdeadMS to value 180000
[Debug] [RULES__CHANGE] Set rule NPC:UseItemBonusesForNonPets to value true
[Debug] [RULES__CHANGE] Set rule Map:FixPathingZMaxDeltaSendTo to value 20.0000000000000
[Debug] [RULES__CHANGE] Set rule Map:FixPathingZMaxDeltaLoading to value 20.0000000000000
[Debug] [RULES__CHANGE] Set rule Map:FixPathingZMaxDeltaMoving to value 20.0000000000000
[Debug] [RULES__CHANGE] Set rule Map:FixPathingZMaxDeltaWaypoint to value 20.0000000000000
[Debug] [RULES__CHANGE] Set rule Character:HealOnLevel to value false
[Debug] [RULES__CHANGE] Set rule Character:FeignKillsPet to value false
[Debug] [RULES__CHANGE] Set rule Character:ItemManaRegenCap to value 15
[Debug] [RULES__CHANGE] Set rule Character:ItemHealthRegenCap to value 15
[Debug] [RULES__CHANGE] Set rule Combat:UseIntervalAC to value true
[Debug] [RULES__CHANGE] Set rule Combat:PetAttackMagicLevel to value 30
[Debug] [RULES__CHANGE] Set rule NPC:SayPauseTimeInSec to value 10
[Debug] [RULES__CHANGE] Set rule NPC:OOCRegen to value 0
[Debug] [RULES__CHANGE] Set rule Aggro:SmartAggroList to value true
[Debug] [RULES__CHANGE] Set rule Aggro:SittingAggroMod to value 35
[Debug] [RULES__CHANGE] Set rule Aggro:MeleeRangeAggroMod to value 20
[Debug] [RULES__CHANGE] Set rule Aggro:CurrentTargetAggroMod to value 0
[Debug] [RULES__CHANGE] Set rule Aggro:CriticallyWoundedAggroMod to value 100
[Debug] [RULES__CHANGE] Set rule Aggro:SlowAggroMod to value 450
[Debug] [RULES__CHANGE] Set rule Aggro:IncapacitateAggroMod to value 500
[Debug] [RULES__CHANGE] Set rule Aggro:MovementImpairAggroMod to value 175
[Debug] [RULES__CHANGE] Set rule NPC:BuffFriends to value true
[Debug] [RULES__CHANGE] Set rule Character:DeathItemLossLevel to value 10
[Debug] [RULES__CHANGE] Set rule Watermap:CheckWaypointsInWaterWhenLoading to value true
[Debug] [RULES__CHANGE] Set rule Watermap:CheckForWaterAtWaypoints to value true
[Debug] [RULES__CHANGE] Set rule Watermap:CheckForWaterWhenMoving to value true
[Debug] [RULES__CHANGE] Set rule Watermap:CheckForWaterOnSendTo to value false
[Debug] [RULES__CHANGE] Set rule Watermap:CheckForWaterWhenFishing to value true
[Debug] [RULES__CHANGE] Set rule Watermap:FishingRodLength to value 30.0000000000000
[Debug] [RULES__CHANGE] Set rule Watermap:FishingLineLength to value 40.0000000000000
[Debug] [WORLD__INIT] Loaded default rule set 'default'
[Debug] [WORLD__INIT] Loading EQ time of day..
[Debug] [WORLD__INIT] Loading launcher list..
[Debug] [WORLD__INIT] Reboot zone modes ON
[Debug] [WORLD__INIT] Deleted 0 stale player corpses from database
[Debug] [WORLD__INIT] Deleted 0 stale player backups from database
[Debug] [WORLD__INIT] Zone (TCP) listener started.
[Debug] [WORLD__INIT] Client (UDP) listener started.
[Debug] [NET__IDENTIFY] Registered patch 6.2
[Debug] [NET__IDENTIFY] Registered patch Titanium
[Debug] [NET__IDENTIFY] Registered patch Anniversary
[Debug] [NET__IDENTIFY] Registered patch Live
[Debug] [WORLD__LS] Connecting to login server...
[Debug] [WORLD__LS] Setting World to MiniLogin Server type
[Debug] [WORLD__LS] Connected to Loginserver: 192.168.250.174:5999

Everything looks good to this point.... Now let's log in...This is where it gets interesting....

After I make a client connection to an existing (restored) account this is what happens.

[Debug] [WORLD__CLIENT] New connection from 71.40.13.254:1476
[Debug] [NET__IDENT_TRACE] 71.40.13.254:1476: First opcode 0x4dd0 did not match expected 0x2792
[Debug] [NET__IDENT_TRACE] 71.40.13.254:1476: Tried patch 6.2_world, and it did not match.
[Debug] [NET__IDENT_TRACE] 71.40.13.254:1476: First opcode 0x4dd0 did not match expected 0x2ec9
[Debug] [NET__IDENT_TRACE] 71.40.13.254:1476: Tried patch 6.2_zone, and it did not match.
[Debug] [NET__IDENT_TRACE] 71.40.13.254:1476: First opcode matched 0x4dd0 and length matched 464
[Debug] [NET__IDENTIFY] Identified stream 71.40.13.254:1476 with signature Titanium_world
[Debug] [WORLD__CLIENT] New client from 71.40.13.254:1476
[Debug] [WORLD__CLIENT] rovaldi: Logged in. Mode=(CharSel)
[Debug] [WORLD__CLIENT] rovaldi: MiniLogin Account #13
Got a bogus character (Achilles) Ignoring!!!
PP length =0 but PP should be 19568
Got a bogus character (Architect) Ignoring!!!
PP length =0 but PP should be 19568
Got a bogus character (Gozer) Ignoring!!!
PP length =0 but PP should be 19568
Got a bogus character (Razzle) Ignoring!!!
PP length =0 but PP should be 19568
Got a bogus character (Rian) Ignoring!!!
PP length =0 but PP should be 19568
Got a bogus character (Rovaldi) Ignoring!!!
PP length =0 but PP should be 19568

I restored everything pertaining to accounts including but not limited to character_, inventory, faction etc.

I searched the forums for bogus character and everyone uses the words bogus and characters in their regular posts so that was of no help. The articles that did turn up with some relevance were pre-0.7.0.

I also checked to verify the association between the account id and the characters and they are correct. I can't help but think I am missing either a table or field in a table.

Any ideas?

Thanks in advance.

- Razzle

[Theeper]

It looks like the profile blobs in your character_ table are empty.

[AndMetal]

It looks like the profile blobs in your character_ table are empty.

I concur. The profile blob (http://www.eqemulator.net/wiki/wikka.php?wakka=EQEmuDBSchemacharacterprofile) contains exactly 19568 characters according to the source code, which matches what the error is throwing. In addition, it says the length is 0, so it seems like there's just absolutely nothing there.

How did you backup the DB? To a .sql file?

[classiceq]

I have verified they are in fact empty. Due to the nature of the blob is there a way to fix it or am I out of luck? I tried a couple different utilities to access the blob. Namely Kayotro's sql editor. If I opened the account it would show the characters associated with the account followed by errors as you would expect.

The database was backed up to a .sql file.

Thanks.

- Razzle

[cavedude]

I'm sorry that their learning process resulted in the destruction of your database. If they're stilling willing to learn (and you're still willing to let them, hehe) I'd recommend creating a new username for them under Windows, and restricting their db access to their own work, keeping yours safe.

On to the matter at hand. How exactly did you backup your database? mysqldump or one of those gui programs? mysqldump is the most secure way to go, and will dump all data, including the nasty BLOB stuff. A full dump is easily done at a traditional command prompt using:

mysqldump -u username -p --extended-insert dbname > dbname.sql

That'll dump the whole database to your current dir. The username/password is optional and may not be required, but i do recommend the extended-insert, as it makes the sourcing process much faster.

As for getting your data back. Are the blobs in character_backup intact? If so, you're in luck. If not, then I'm afraid your toons are lost for good, unless you have another backup lying around.

[classiceq]

After finishing with Administative Punishment....I built him something on a seperate machine to learn on.

Unfortunately from the sounds of it I am a day to late. I used a gui program to back up the data.

Oddly enough it is the only thing I use it for. Not sure why actually. Everything else is done via command line.

I checked all my backups doing a few test restores along the way and none of them have any data in the blob.

Lesson learned.

Thanks for the info guys. I appreciate you taking the time to help out.

- Razzle

[cavedude]

Well, it's not a total loss you still have your inventories intact. Items are the real pain to restore, imho. What I'd do in your case is mark down the current IDs of all your toons and then wipe the character_ table. Then, re-create the toons in-game. Go back to the db, and change the toon's IDs so they match their old values. The next time you log in, all of your items will be there as you left them. From there, you just need to use GM commands to get your level, AAs, skills, and spell scrolls back. If you have a small private server, that won't be too bad. After you're done, immediately make a backup with mysqldump and check to make sure that the BLOBs are intact in it.

[classiceq]

Thanks CD. That is exactly what I started doing. It is tedious but will serve its purpose.

From the looks of it I have about 60 toons to create maybe less. I will only create those that have logged in for the past 30 days.

Should make for a interesting weekend. Can't ride because of the rain so I might as well make it an EQEMU weekend.

- Razzle

[AndMetal]

I'm not sure if this will actually work, but you might want to try inserting data without any values into the blobs (all binary 0's). I would imagine the easiest way to do this would be the following:


UPDATE character_ SET profile = RPAD('',19568,CHAR(0)) WHERE LENGTH(profile) != 19568


To clarify, that's 2 apostrophies inside the RPAD. This should create a blob 19568 characters long with all zeroed information wherever the blob isn't the appropriate length. However, I'm not sure if the server will be able to do anything with it, other than maybe inserting the character name, because it is stored in the name field also.

[classiceq]

One problem I did find is the race information is stored in the blob. With out that info I can't do a mass creation of toons.

I will try to insert the data into the blob and see what happens.

- Razzle

[classiceq]

I used the sql command to modify the blob and it completes without error but doesn't make any changes that I can see. The blob is still empty.

- Razzle

[AndMetal]

Even though it doesn't show anything, there should still be information there (the ASCII character 0 is NULL). This is the easiest way to make sure:

SELECT LENGTH(profile) FROM character

If you want to, you can also add a WHERE or LIMIT clause to limit the results to specific records (good for a larger database), but it should return a length of 19568. If not, it might require adding some info to the first few bytes of data (the checksum field):

UPDATE character_ SET profile = RPAD('ABCD',19568,CHAR(0)) WHERE LENGTH(profile) != 19568

To be honest though, I think the better way to do it would be either to do what cavedude recommended & recreate the characters, etc. However, you could also just inject the known info (race, diety, etc) using other SQL commands (http://www.eqemulator.net/forums/showthread.php?t=24386#post143261).

Either way, good luck with the project :-)

[nosfentora]

I just used Navicat 8 to dump the character_ table to an .sql file.
upon importing with mysql, and with navicat i am getting the same error:


'ERROR 1054 (42S22): Unknown column '0x0A04E78E4E6F7366656E746F72000000000000000000000 000000000000000' in 'field list'
Any thoughts?

Also - what is the purpose of the character_backup table? i was able to import that table back into the db without a problem.
is there a way to re-load the character info to the character_ table from character_backup?

[nosfentora]

I've tried the in-game #charbackup and it doesn't seem to work.
i tried creating a new character (not the same name as my old one) and then ran the #charbackup restore 125 2059 (the charid and backup id)
it said character restored but the character_ table didn't list anything new.

i also tried creating a new char with the same name (race, etc) and ran the #charbackup restore 125 2059 and it said character restored, but again, the db table was unchanged.

does #charbackup restore work?

[nosfentora]

it seems like the info getting dumped for the extprofile field is getting corrupted somewhere along the line.

i added the profile info back, but am unable to add the extprofile info (see the error above).