Thread: Zone MMF Files
View Single Post
  #2  
Old 07-28-2016, 10:57 PM
Uleat's Avatar
Uleat
Developer
 
Join Date: Apr 2012
Location: North Carolina
Posts: 2,249
Default

[continuation...]

This feature essentially saves the post-load state of the zone map..

..meaning, that once a zone map file has been loaded and the complex transformation has occurred, the in-memory object is saved
to a memory-mapped file for future loading.


This is not typical of a 'standard' server..but, in testing, I realized a huge difference in map loading time for my test configuration (in debug build.)

- pre-mmf conversion map load time of moors: ~85 seconds
- post-mmf conversion map load time of moors: ~0.2 seconds (uncompressed format)

Yes, that is not a typo. that's a decrease of ~84.2 seconds.

This is an exaggerated example and is not typical (or even close) to the realized performance increase.
(Obviously, I found a bottleneck in my system..but, its alleviation should help some with most peoples' servers.)


I decided to implement compression into the file storage because I noticed that some of the files were extremely large (77+ megs for moors,)
and this is where, I suspect, most people will encounter problems...

Using compression, the additional file storage appears to be approximately 3.19 gigs total for all files in the maps directory (up 2.67 gigs.)

Non-compressed (raw) files took that requirement up to 8.69 gigs of storage (up 5.5 gigs over compressed MMF usage.)

I think the slight zone boot-up overhead of compressing the MMF is well worth its use.


Problems with the zlib1.dll can summarized to this: "It will crash!"

The server code is not the problem..but, rather the way that the zlib dll is configured when it is built.


These particular zone map conversions were consistent crashes with my system: abysmal, acrylia and moors

There is no guarantee that testing only these three maps will ensure safety with other conversions..but, it's a start.

To test, open a command prompt window and change to your server directory.

Then, use these three commands:
Code:
zone convert_map abysmal.map
zone convert_map acrylia.map
zone convert_map moors.map
These commands will start zone.exe in a special 'map conversion' mode and will produce a mmf file of the zone map - if successful.

If you get a 'zone.exe has stopped' type pop-window, hit continue and the stack should dump to the console window. (Linux systems will probably
complain as well..but, I'm not sure how it responds to issues like this.)

Look just above the 'SaveMMF' call for a zlib crc message.

If the call is to 'crc32_combine64' and you have a 32-bit server build, this is your problem. You will need to compile your own version of zlib
with source code from here: http://www.zlib.net/ (download the source code, not the compiled DLL.)

This solution should solve the problem with the conversion crash. If you have other problems not described in this thread, please post them in a new one.


Please see the changelog for more information.


EDIT:

The batch conversion script will process all map files in the 'maps' sub-directory.

It took over two hours on my system..but, performance will vary according to your system's specs.


I have not tested running the conversion script on a running server..but, it may be possible since there are checks in-place
to work around 'partial' files.

I'm thinking that either the conversion's zone instance will fail conversion or the booted zone's save process will fail..but, a crash
shouldn't ensue nor should a mmf not be created by the 'winning' process.

Again, untested theory.


EDIT2:

I should iterate that this feature is really designed for low-spec hardware/dynamic zone servers.

Due to the nature of those criteria, this feature should really help client load times on those servers..

..but, that's not to say that higher-end servers wouldn't benefit from this feature.

It is a forced recompile option..so, its use is left up to the individual admins to enable it on their servers.
__________________
Uleat of Bertoxxulous

Compilin' Dirty

Last edited by Uleat; 07-29-2016 at 06:41 PM..
Reply With Quote