Zones Restarting and deleting corpses!

[cubber]

Trevius Edit: I moved these posts from the SoF Development thread into here. It was getting the other thread off-topic.

I noticed in the last changelog you reverted something because of issues with titanium. Was this issue causing zones to crash and restart? Over the last 2 rev's I have had random zone crashes, and even had one where I crashed after I died and my corpse poofed! I had to revert to backup from the previous day.

I just compiled the rev 346 as of today and have not had a chance to test.

[Andrew80k]

Originally Posted by cubber

I noticed in the last changelog you reverted something because of issues with titanium. Was this issue causing zones to crash and restart? Over the last 2 rev's I have had random zone crashes, and even had one where I crashed after I died and my corpse poofed! I had to revert to backup from the previous day.

I just compiled the rev 346 as of today and have not had a chance to test.

These crashes are often the result of database errors. Check your logs to see if there is anything in them to cause this.

[cubber]

I did go through the logs and only saw messages that stated that the dynamic zone I was in restarted.

[cubber]

Please look at this thread I started in the linux forums:

http://www.eqemulator.net/forums/showthread.php?t=27523

it explains more about the issue I am having. It is 100% reproduceable.

The problem has to do with death, if a character or bot dies it crashes the zone and deletes the characters corpse. When the character dies it almost looks like the titanium client is trying to use the newer death window meathod because a text message pops up in the center of my screen telling me what NPC killed me, but I get no confirmation box popup. Then the screen just lags and finally boots me out.

pulling down svn rev 349 now, gonna build and test.

[trevius]

I don't normally like to write things off unless I am 100% sure, but since your problem is caused by a bot dying, then I think it is probably a bot related issue. I haven't seen anything like this or heard any reports of anything like this on my server, but I also don't run bots. There have been alot of bot changes too lately, so that is just as likely to be the cause, if not more.

Maybe keep rolling back revisions 1 by 1 until you get to the first one that doesn't cause that crash. Then you know it was the one right after that that started it.

[cubber]

I will do some more troubleshooting over the next few days. This issue does not just happen when a bot dies as I have stated before and in the screenshot. If I die with a normal PC character with no bots up it happens and the corpse vanishes from the world.


Gonna pulling down the latest clean sources first and building without bots. I will report back.

[cubber]

Ok with clean svn sources rev 350 no bot support included the crash still happens when a character dies. However now I don't even see the character's corpse, the zone just drops and the character can still run around the zone until I /q out. But I cannot interact with anything.

here is the output of eqemu_debug_world.log when it happens:

15619 [02.20. - 08:34:16] [WORLD__CLIENT] New connection from 74.41.185.190:33336
15619 [02.20. - 08:34:16] [NET__IDENT_TRACE] 74.41.185.190:33336: First opcode 0x4dd0 did not match expected 0x2792
15619 [02.20. - 08:34:16] [NET__IDENT_TRACE] 74.41.185.190:33336: Tried patch 6.2_world, and it did not match.
15619 [02.20. - 08:34:16] [NET__IDENT_TRACE] 74.41.185.190:33336: First opcode 0x4dd0 did not match expected 0x2ec9
15619 [02.20. - 08:34:16] [NET__IDENT_TRACE] 74.41.185.190:33336: Tried patch 6.2_zone, and it did not match.
15619 [02.20. - 08:34:16] [NET__IDENT_TRACE] 74.41.185.190:33336: First opcode matched 0x4dd0 and length matched 464
15619 [02.20. - 08:34:16] [NET__IDENTIFY] Identified stream 74.41.185.190:33336 with signature Titanium_world
15619 [02.20. - 08:34:16] [WORLD__CLIENT] Checking inbound connection 74.41.185.190 against BannedIPs table
15619 [02.20. - 08:34:16] [WORLD__CLIENT] Connection 74.41.185.190 PASSED banned IPs check.  Processing connection.
15619 [02.20. - 08:34:16] [WORLD__CLIENT] myaccount: Logged in. Mode=(CharSel)
15619 [02.20. - 08:34:16] [WORLD__CLIENT] myaccount: MiniLogin Account #1
15619 [02.20. - 08:35:06] [WORLD__CLIENT] myaccount: Attempting autobootup of tutorialb (189)
15619 [02.20. - 08:35:07] [WORLD__ZONE] [11] Setting to 'tutorialb' (189)
15619 [02.20. - 08:35:07] [WORLD__CLIENT] myaccount: Entering zone tutorialb (189)
15619 [02.20. - 08:35:07] [WORLD__ZONE] [11] [tutorialb] Broadcasting a world time update
15619 [02.20. - 08:35:07] [WORLD__ZONE] [11] [tutorialb] Setting to 'tutorialb' (189)
15619 [02.20. - 08:35:07] [WORLD__CLIENT] myaccount: Sending client to zone tutorialb (189) at purpletoad.no-ip.org:7010
15619 [02.20. - 08:35:07] [WORLD__CLIENT] myaccount: Client disconnected (not active in process)
15619 [02.20. - 08:38:27] [WORLD__ZONELIST] Removing zoneserver #11 at :7010
15619 [02.20. - 08:38:27] [WORLD__ZONELIST] Hold Zones mode is ON - rebooting lost zone
15619 [02.20. - 08:38:27] [WORLD__LAUNCH] zone: dynamic_10 reported state STOPPED (1 starts)
15619 [02.20. - 08:38:39] [WORLD__LAUNCH] zone: dynamic_10 reported state STARTED (2 starts)
15619 [02.20. - 08:38:41] [WORLD__ZONE] New TCP connection from 127.0.0.1:50310
15619 [02.20. - 08:38:41] [WORLD__CONSOLE] New zoneserver #14 from 127.0.0.1:50310
15619 [02.20. - 08:38:41] [WORLD__ZONE] [14] Zone started with name dynamic_10 by launcher zone
15619 [02.20. - 08:38:41] [WORLD__ZONE] [14] Auto zone port configuration.  Telling zone to use port 7013

Note I am booting 10 dynamic zones and 3 statics (pok, pot, bazaar) the statics are using dynamic ports.

edit: Just tested again and I did actually die the second time and see my corpse hit the floor, got the message saying who killed me and the zone crashed. So I guess it can crash before or after your body hits the floor. Note that if I actually see my corpse when the zone crashes I cannot /q out I need to kill the process or wait for it to close it self out to login.

[cubber]

Ok I think I found my problem. I was able to die properly and actually zone after dying, then I was able to retrieve my corpse!

Seems the emulator does not like to be compiled with the following CFLAGS changes to each makefile:

-fomit-frame-pointer -fforce-addr

I was using these in my make files along with -march=athlon-xp

Once I removed the -fomit-frame-pointer and -fforce-addr and recompiled with out bots enabled all worked properlly. Gonna recompile now with bots enabled and see what happens.

Should I just be changing the march= in the make files or are there other safe optimizations I can use?

I have tried adding -O2 to each one or changing the ones that are -O to -O2 with out issues.

[cubber]

This still happens with bots enabled when a bot dies. But not when a PC dies. The only thing in my make files is -march=athlon-xp where there is a -march=i686 in the makefile. I have been using this since before 0.7.0 with no issues. Will like to this on the bot thread.

[cubber]

More testing more notes:

Seems to be a lot more stable now but here is what I have found so far.

PC death = no crash

Bot death = sometimes crash, sometimes they will crash the zone other times they wont. I have been just killing a bot one at a time, sometimes the first death does it sometimes it takes 2 or 3 bot deaths to crash the zone. I summon them one at a time and just send them at Grand Master Glox in East Cabilis then FD until they are dead. If I can get stand up and still do things like con npc's, use merchants, cast spells etc then I am good. But if I can't do anything then the zone crashed and I gotta kill the game.

[KingMort]

oof sounds like you have a binary issue bud..

[cubber]

This issue is fixed in Rev 355, seems it was an issue related to the bot code.