Zone MMF Files
A memory-mapped file feature has been added to zone maps.
This feature should alleviate some wasted memory in the map loading process and quicken zone boot-up times by a percentage.
(Actual benefit will vary depending hardware, zone size, etc...)
Please consider using this option as experimental for the time being.
This feature essentially saves the post-load state of the zone map..
..meaning, that once a zone map file has been loaded and the complex transformation has occurred, the in-memory object is saved
to a memory-mapped file for future loading.
This is not typical of a 'standard' server..but, in testing, I realized a huge difference in map loading time for my test configuration (in debug build.)
- pre-mmf conversion map load time of moors: ~85 seconds
- post-mmf conversion map load time of moors: ~0.2 seconds (uncompressed format)
Yes, that is not a typo. that's a decrease of ~84.2 seconds.
This is an exaggerated example and is not typical (or even close) to the realized performance increase.
(Obviously, I found a bottleneck in my system..but, its alleviation should help some with most peoples' servers.)
I decided to implement compression into the file storage because I noticed that some of the files were extremely large (77+ megs for moors,)
and this is where, I suspect, most people will encounter problems...
Using compression, the additional file storage appears to be approximately 3.19 gigs total for all files in the maps directory (up 2.67 gigs.)
Non-compressed (raw) files took that requirement up to 8.69 gigs of storage (up 5.5 gigs over compressed MMF usage.)
I think the slight zone boot-up overhead of compressing the MMF is well worth its use.
Problems with the zlib1.dll can summarized to this: "It will crash!"
The server code is not the problem..but, rather the way that the zlib dll is configured when it is built.
These particular zone map conversions were consistent crashes with my system: abysmal, acrylia and moors
There is no guarantee that testing only these three maps will ensure safety with other conversions..but, it's a start.
To test, open a command prompt window and change to your server directory.
Then, use these three commands:
If you get a 'zone.exe has stopped' type pop-window, hit continue and the stack should dump to the console window. (Linux systems will probably
complain as well..but, I'm not sure how it responds to issues like this.)
Look just above the 'SaveMMF' call for a zlib crc message.
If the call is to 'crc32_combine64' and you have a 32-bit server build, this is your problem. You will need to compile your own version of zlib
with source code from here: http://www.zlib.net/ (download the source code, not the compiled DLL.)
This solution should solve the problem with the conversion crash. If you have other problems not described in this thread, please post them in a new one.
Please see the changelog for more information.
The batch conversion script will process all map files in the 'maps' sub-directory.
It took over two hours on my system..but, performance will vary according to your system's specs.
I have not tested running the conversion script on a running server..but, it may be possible since there are checks in-place
to work around 'partial' files.
I'm thinking that either the conversion's zone instance will fail conversion or the booted zone's save process will fail..but, a crash
shouldn't ensue nor should a mmf not be created by the 'winning' process.
Again, untested theory.
I should iterate that this feature is really designed for low-spec hardware/dynamic zone servers.
Due to the nature of those criteria, this feature should really help client load times on those servers..
..but, that's not to say that higher-end servers wouldn't benefit from this feature.
It is a forced recompile option..so, its use is left up to the individual admins to enable it on their servers.
Just compiled and ran the conversion script. (mine only took 40 minutes)(chuckle)
On Win 7(64) box, everything went smooth. No errors, bugs or insects.
Zoning to Moors took 19-22 seconds.
On Linux box, (Debian 7.8) however, didn't get past compiling, with these errors:
I logged one toon in and had it running around zoning while the conversion was
I kept an eye on the progress of the conversion and timed it out to zone in to
a map that was being processed. It all went well without any hitches.
I'll take a look at that linux conversion error. Shouldn't be too hard to find the proper casting method.
A zoning time comparison using the old versus new methods.
Oh, actually yes, I was paying attention to the old vs new. I based this on zoning to moors
only. Before it was 25-33 seconds on average, sometimes higher, but never below 25
After conversion it was a steady 19-22 secs
Nice! Thanks for that info!
I pushed a (hopeful) fix for that 64-bit compile issue.
Let me know if that does not work.
I just pulled that fix and compiled on the linux box again. No compile errors this time :)
I'll get the server booted up and get the map conversion done, then test it all out, to
see how the linux server takes to it all.
Oh, ok, it rapidly gave me "conversion failed" on all of them, so I didn't get too far there.
Didn't give me any visible errors in window, so I will see if I can some kind of print.
Not sure what linux should be reporting with a dll failure.
I'll look around and see if I can find a way to trace it to the dll.
The code that was 'fixed' takes the offset of the base pointer and subtracts it from the pointer reference, then divides by the size of the
object to get an 'index' that can be used to deference the base pointer during the load process.
If that index is computed to be '< 0' or '>= mNodeCount' (which should not occur) .. you should be getting an exception error during the LoadMMF call.
Since this is happening during the conversion/save process, it most likely is dll-related..but, will need more info before confirming that.
Is there a way I can get that script to spit out some kind of log ?
I'm just running it with perl command from cmd window in linux, but
it won't give me any kind of feedback other than failed
The 'status' message actually comes from zone.exe and is passed to the script.
Try invoking it using the manual format and see what happens.
Sorry, all I got was - abysmal.map conversion failed
|All times are GMT -4. The time now is 01:02 AM.|
Powered by vBulletin®, Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.