EQEmulator Forums

EQEmulator Forums (http://www.eqemulator.org/forums/index.php)
-   General::News (http://www.eqemulator.org/forums/forumdisplay.php?f=594)
-   -   Zone MMF Files (http://www.eqemulator.org/forums/showthread.php?t=40805)

Uleat 07-28-2016 10:57 PM

Zone MMF Files
 
A memory-mapped file feature has been added to zone maps.


This feature should alleviate some wasted memory in the map loading process and quicken zone boot-up times by a percentage.
(Actual benefit will vary depending hardware, zone size, etc...)


Please consider using this option as experimental for the time being.
[continued...]

Uleat 07-28-2016 10:57 PM

[continuation...]

This feature essentially saves the post-load state of the zone map..

..meaning, that once a zone map file has been loaded and the complex transformation has occurred, the in-memory object is saved
to a memory-mapped file for future loading.


This is not typical of a 'standard' server..but, in testing, I realized a huge difference in map loading time for my test configuration (in debug build.)

- pre-mmf conversion map load time of moors: ~85 seconds
- post-mmf conversion map load time of moors: ~0.2 seconds (uncompressed format)

Yes, that is not a typo. that's a decrease of ~84.2 seconds.

This is an exaggerated example and is not typical (or even close) to the realized performance increase.
(Obviously, I found a bottleneck in my system..but, its alleviation should help some with most peoples' servers.)


I decided to implement compression into the file storage because I noticed that some of the files were extremely large (77+ megs for moors,)
and this is where, I suspect, most people will encounter problems...

Using compression, the additional file storage appears to be approximately 3.19 gigs total for all files in the maps directory (up 2.67 gigs.)

Non-compressed (raw) files took that requirement up to 8.69 gigs of storage (up 5.5 gigs over compressed MMF usage.)

I think the slight zone boot-up overhead of compressing the MMF is well worth its use.


Problems with the zlib1.dll can summarized to this: "It will crash!"

The server code is not the problem..but, rather the way that the zlib dll is configured when it is built.


These particular zone map conversions were consistent crashes with my system: abysmal, acrylia and moors

There is no guarantee that testing only these three maps will ensure safety with other conversions..but, it's a start.

To test, open a command prompt window and change to your server directory.

Then, use these three commands:
Code:

zone convert_map abysmal.map
zone convert_map acrylia.map
zone convert_map moors.map

These commands will start zone.exe in a special 'map conversion' mode and will produce a mmf file of the zone map - if successful.

If you get a 'zone.exe has stopped' type pop-window, hit continue and the stack should dump to the console window. (Linux systems will probably
complain as well..but, I'm not sure how it responds to issues like this.)

Look just above the 'SaveMMF' call for a zlib crc message.

If the call is to 'crc32_combine64' and you have a 32-bit server build, this is your problem. You will need to compile your own version of zlib
with source code from here: http://www.zlib.net/ (download the source code, not the compiled DLL.)

This solution should solve the problem with the conversion crash. If you have other problems not described in this thread, please post them in a new one.


Please see the changelog for more information.


EDIT:

The batch conversion script will process all map files in the 'maps' sub-directory.

It took over two hours on my system..but, performance will vary according to your system's specs.


I have not tested running the conversion script on a running server..but, it may be possible since there are checks in-place
to work around 'partial' files.

I'm thinking that either the conversion's zone instance will fail conversion or the booted zone's save process will fail..but, a crash
shouldn't ensue nor should a mmf not be created by the 'winning' process.

Again, untested theory.


EDIT2:

I should iterate that this feature is really designed for low-spec hardware/dynamic zone servers.

Due to the nature of those criteria, this feature should really help client load times on those servers..

..but, that's not to say that higher-end servers wouldn't benefit from this feature.

It is a forced recompile option..so, its use is left up to the individual admins to enable it on their servers.

DanCanDo 07-29-2016 01:10 AM

Just compiled and ran the conversion script. (mine only took 40 minutes)(chuckle)
On Win 7(64) box, everything went smooth. No errors, bugs or insects.
Zoning to Moors took 19-22 seconds.

On Linux box, (Debian 7.8) however, didn't get past compiling, with these errors:

Code:

/home/eqemu/source/zone/raycast_mesh.cpp: In member function ‘void RAYCAST_MESH::MyRaycastMesh::serialize(std::vector<char>&)’:
/home/eqemu/source/zone/raycast_mesh.cpp:1182:42: error: cast from ‘RAYCAST_MESH::NodeAABB*’ to ‘RmUint32 {aka unsigned int}’ loses precision [-fpermissive]
/home/eqemu/source/zone/raycast_mesh.cpp:1182:60: error: cast from ‘RAYCAST_MESH::NodeAABB*’ to ‘RmUint32 {aka unsigned int}’ loses precision [-fpermissive]
/home/eqemu/source/zone/raycast_mesh.cpp:1188:42: error: cast from ‘RAYCAST_MESH::NodeAABB*’ to ‘RmUint32 {aka unsigned int}’ loses precision [-fpermissive]
/home/eqemu/source/zone/raycast_mesh.cpp:1188:61: error: cast from ‘RAYCAST_MESH::NodeAABB*’ to ‘RmUint32 {aka unsigned int}’ loses precision [-fpermissive]
make[2]: *** [zone/CMakeFiles/zone.dir/raycast_mesh.cpp.o] Error 1
make[1]: *** [zone/CMakeFiles/zone.dir/all] Error 2
make: *** [all] Error 2


DanCanDo 07-29-2016 02:11 AM

Quote:

Originally Posted by Uleat (Post 250316)

I have not tested running the conversion script on a running server..but, it may be possible since there are checks in-place
to work around 'partial' files.

I'm thinking that either the conversion's zone instance will fail conversion or the booted zone's save process will fail..but, a crash
shouldn't ensue nor should a mmf not be created by the 'winning' process.

Again, untested theory.

I took the liberty of testing the conversion script on a running server.(win 7)
I logged one toon in and had it running around zoning while the conversion was
going on.
I kept an eye on the progress of the conversion and timed it out to zone in to
a map that was being processed. It all went well without any hitches.

Uleat 07-29-2016 04:57 PM

Quote:

Originally Posted by DanCanDo
Zoning to Moors took 19-22 seconds.

Did you happen to get a 'control' time so that a comparison can be made? :)


I'll take a look at that linux conversion error. Shouldn't be too hard to find the proper casting method.

Thanks!

DanCanDo 07-29-2016 06:10 PM

Quote:

Originally Posted by Uleat (Post 250320)
Did you happen to get a 'control' time so that a comparison can be made? :)

Sorry, you'll have to give me a little guidance on that one. As in Huh? (chuckle)

Uleat 07-29-2016 06:16 PM

Sorry :)

A zoning time comparison using the old versus new methods.

DanCanDo 07-29-2016 06:20 PM

Oh, actually yes, I was paying attention to the old vs new. I based this on zoning to moors
only. Before it was 25-33 seconds on average, sometimes higher, but never below 25
After conversion it was a steady 19-22 secs

Uleat 07-29-2016 06:29 PM

Nice! Thanks for that info!


I pushed a (hopeful) fix for that 64-bit compile issue.

Let me know if that does not work.

DanCanDo 07-29-2016 06:48 PM

I just pulled that fix and compiled on the linux box again. No compile errors this time :)
I'll get the server booted up and get the map conversion done, then test it all out, to
see how the linux server takes to it all.

DanCanDo 07-29-2016 06:53 PM

Oh, ok, it rapidly gave me "conversion failed" on all of them, so I didn't get too far there.
Didn't give me any visible errors in window, so I will see if I can some kind of print.

Uleat 07-29-2016 07:37 PM

Not sure what linux should be reporting with a dll failure.

I'll look around and see if I can find a way to trace it to the dll.


EDIT:

The code that was 'fixed' takes the offset of the base pointer and subtracts it from the pointer reference, then divides by the size of the
object to get an 'index' that can be used to deference the base pointer during the load process.

If that index is computed to be '< 0' or '>= mNodeCount' (which should not occur) .. you should be getting an exception error during the LoadMMF call.

Since this is happening during the conversion/save process, it most likely is dll-related..but, will need more info before confirming that.

DanCanDo 07-29-2016 08:08 PM

Is there a way I can get that script to spit out some kind of log ?
I'm just running it with perl command from cmd window in linux, but
it won't give me any kind of feedback other than failed

Uleat 07-29-2016 08:12 PM

The 'status' message actually comes from zone.exe and is passed to the script.

Try invoking it using the manual format and see what happens.

Code:

./zone convert_map abysmal.map
(or whatever the linux command line requires..)

DanCanDo 07-29-2016 08:23 PM

Sorry, all I got was - abysmal.map conversion failed

Uleat 07-29-2016 08:27 PM

I don't think the logging system is fully enabled when using the command line argument.

If you can, try starting the server and entering a zone. That should trigger the automated conversion process and there are more in-depth
messages that should be processed on a failed conversion.


EDIT: *Processed to the zone logs..

DanCanDo 07-29-2016 09:06 PM

I maxed out the logsys for zone server. I logged a toon in and zoned around.
Still nothing abnormal showing up and no entries in logs regarding conversion.

DanCanDo 07-29-2016 09:14 PM

I am not sure if this is relevant, after zoning, something I got in my gm client window,
but also the last line entered in log - [Zone Server] Got 0x0155 from world

Uleat 07-29-2016 09:20 PM

If your linux server did not have any existing mmf files, you should have gotten at least one load failure message:
https://github.com/EQEmu/Server/blob...e/map.cpp#L997

And this keeps the auto-conversion process from performing the action if an mmf already exists:
https://github.com/EQEmu/Server/blob.../map.cpp#L1089

The only real way to not have any logs messages regarding that would be if you already had an mmf file(s) in the maps folder.

Uleat 07-29-2016 09:24 PM

If that was a servertalk packet, it should be irrelevant: https://github.com/EQEmu/Server/blob...vertalk.h#L116

DanCanDo 07-29-2016 09:25 PM

Ya, one of the first things I checked after not getting anything in logs was to see if any
conversion existed in the map folder, particularly with Nexus, where I zoned, but there is
no mmf files at all. I'm going to try recompiling again and see if it makes a difference.

DanCanDo 07-29-2016 10:01 PM

Well, I tried, still the same. Can't figure out why I am not getting any errors.
I always thought of linux as more cooperative than windows (chcuckle)
I wish someone else with a linux box could try this out and see if they get
what I am getting. Zoning time is not really a concern of mine, but would be
nice to get it all smoothed out for anyone wanting to use it.

Uleat 07-29-2016 10:02 PM

I added a 'discovery' message in the Map::Load() function for when a mmf exists.

At this point, every zone boot-up should report something about the state of its mmf.

(Still not sure why you're not getting any messages for the failure...)


EDIT: I still think it's related to the dll..but, it should either be logging or crashing...

DanCanDo 07-29-2016 10:41 PM

I grabbed that last update and recompiled. Again, this is all I got :
Code:

[07-29-2016 :: 20:32:48] [Zone Server] Got 0x0023 from world:
[07-29-2016 :: 20:32:48] [Zone Server] Got 0x200e from world:
[07-29-2016 :: 20:32:48] [Zone Server] Process Received Message SyncWorldTime
[07-29-2016 :: 20:32:48] [Zone Server] Time Broadcast Packet: EQTime [02:32 pm]
[07-29-2016 :: 20:32:49] [Zone Server] Got 0x0027 from world:
[07-29-2016 :: 20:32:50] [Quests] Useless use of a variable in void context at quests/nexus/player.pl line 7.
[07-29-2016 :: 20:32:51] [Zone Server] Got 0x0155 from world:

I tried doing the manual ./zone convert_map abysmal.map and got same, previous result simply
stating conversion failed. No errors, crashes or anything. I'm stumped totally.

Uleat 07-29-2016 11:31 PM

Do you have your logsys_category 'zone_server' 'file' entry set to at least 1?

DanCanDo 07-29-2016 11:33 PM

Oh, I started with 1, then went to 3 and then to 5. Thats why I am confused.
Whether set to 1, 3 or 5 nothing changes much.
Edit: Just to add, that's restarting server each time.

DanCanDo 07-29-2016 11:47 PM

But on an off note, something ironicly funny, zoning to moors - 16 seconds (chuckle)
This is using the same windows client, but logging on to linux server without the mmf.

Uleat 07-30-2016 12:03 AM

This is what I get for a non-existing mmf:
Quote:

[07-29-2016 :: 23:25:28] [Status] Init Finished: ZoneID = 395, Time Offset = 0
[07-29-2016 :: 23:25:28] [Zone Server] Failed to load Map MMF file: 'Maps/moors.mmf' - could not open file
[07-29-2016 :: 23:25:56] [Error] Path File Maps/moors.path not found.
[07-29-2016 :: 23:25:56] [Normal] ---- Zone server moors, listening on port:7001 ----
..and for a mmf:
Quote:

[07-29-2016 :: 23:24:44] [Status] Init Finished: ZoneID = 279, Time Offset = 0
[07-29-2016 :: 23:24:44] [Zone Server] Zone map mmf found - using it in lieu of 'Maps/abysmal.map'
[07-29-2016 :: 23:24:44] [Error] Path File Maps/abysmal.path not found.
[07-29-2016 :: 23:24:44] [Normal] ---- Zone server abysmal, listening on port:7002 ----

DanCanDo 07-30-2016 12:06 AM

I'm going to duplicate that on my windows box. Maybe something is wrong with the whole
logging system on my linux box, it's weird.

POST EDIT: Not sure exactly whats going on, but looks like I have to spend a few hours
figuring out why the logs are not dishing out in either windows or linux. I'm assuming it's
something to do with the DB and logsys tables. This time, in windows, I pulled a totally
fresh source from git and set the logging at level 3 in the cmakelist for all of it.
In the DB logsys category table, I set a lot of it to level 3. So back to peq school for
better grades (chuckle)

DanCanDo 07-30-2016 03:21 AM

Ok, Uleat, I got the logsys prob sorted out, but I got the same thing you get. Nothing extra and the
conversion still won't work in the linux server. No probs at all with windows. Just devious Debian. :)
Just to clarify, the database being used on win 7 box and linux box are exact copies of same DB.

Code:

[07-30-2016 :: 01:14:04] [Status] Init Finished: ZoneID = 152, Time Offset = 0
[07-30-2016 :: 01:14:04] [Zone Server] Failed to load Map MMF file: 'Maps/nexus.mmf' - could not open file
[07-30-2016 :: 01:14:04] [Error] Path File Maps/nexus.path not found.
[07-30-2016 :: 01:14:04] [Normal] ---- Zone server nexus, listening on port:7004 ----
[07-30-2016 :: 01:14:04] [Status] Zone Bootup: nexus (152: 0)


DanCanDo 07-30-2016 04:15 AM

BINGO! I "think" I found the reason the mmf system is not working in linux. Something that is so obvious,
I was overlooking it until now (chuckle)
Code:

[07-30-2016 :: 01:53:21] [Zone Server] Failed to load Map MMF file: 'Maps/abysmal.mmf' - could not open file
If you notice, it's looking for Maps, instead of maps, linux being case sensitive.
EDIT: I forced the config to look for Maps and changed the folder name to Maps. The conversion is working in linux
now. :) BUT to get the full automated conversion, I had to change it in the script as well.

Uleat 07-30-2016 04:57 PM

I know that linux is case-sensitive..but, I used existing code as a reference.

I'm really surprised that linux doesn't complain about 'Maps' when running normally..since, it grabs the information from the same place.


I'll talk it over with some of the linux-based devs and see what the current standard is and see what we can do.


Thanks for verifying that!

DanCanDo 07-30-2016 05:58 PM

This was something I encountered before, when first setting up linux.
It does run normally as long as the config points to the same name as
the folder (whether upper or lower case), as long as they are both the
same.
I did notice, if someone downloads the full source from Git, if they use
the eqemu_config(full) file included in the utils/defaults folder, by default,
under directories it points to Maps, uppercase.

wirepuller134 07-31-2016 01:16 AM

We let it do the conversion on zones while zoning to see how it did. It worked well. Then we stopped the server and converted all of them. No errors. Zoning into the moors before was 1 min 6 seconds. now 16 seconds. We are running Windows 7 64, on a busy system. The main nic is connecting at 10 BaseT (this is a server that is going on 12 years old, so it's on our list to replace) so is a factor with the server streaming to our televisions as well. But I can say there was a noticeable improvement in zoning times. Thanks Uleat, this is going to be big step forward.

DanCanDo 07-31-2016 10:43 PM

Thats what I am getting now on linux server box is 16 seconds in to moors.
12 year old server ? I love it when people keep things like that up and running.
It must be one of those old "solid" boxes if your running the server, as well as
streaming to tv's (chuckle)

Uleat 08-03-2016 10:55 PM

In addition to this feature, another recent update associated with sending spawned entities to the client has help reduce client load times - in certain situations.

I have timing code in place to analyze this process and will continue to pursue effective changes that improve client load time performance.


Reviewing the process, however, most of the remaining hangups appear to be client related..or at least in the handling of certain packets from the client.

DanCanDo 08-04-2016 02:15 AM

Something I did notice with this one client/puter, sometimes when zoning, even in to
a simple place, like Nexus, It is a little slow drawing the models of npc's. I've always
been convinced it's totally a client side issue. I haven't dug in to that issue much at
all, been too busy focusing on the server.(chuckle)


All times are GMT -4. The time now is 11:46 AM.

Powered by vBulletin®, Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.