Go Back   EQEmulator Home > EQEmulator Forums > General > General::News

General::News EQemu news posts.

Reply
 
Thread Tools Display Modes
  #1  
Old 01-13-2018, 07:48 PM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 1,895
Default Dev Blog: New Server Config Loader (Json)



Thanks to the work of Shin Noir (Xackery) from RebuildEQ, we've converted our server config loader from XML to JSON.

This is now live, whenever new binaries are used, world.exe will automatically pull down a config converter before attempting to load the config. This looks something like this:

https://i.imgur.com/yobyMNg.png

World will use the new eqemu_config.json effective immediately. All other utilities that were built around loading config variables from XML will have to be adjusted or updated to use the JSON format.

A backup of the old XML config will be located in serverdir/backups/
  • The Perl MySQL plugin has been updated already: see here
  • Linux Installer now uses the new .json format
  • Windows Installer now uses the new .json format
  • DiscordEQ OOC chat relay built by Shin should have a new release ready for this change: see here

If you find any tools or utilities tied to the old XML format that needs to be changed or adjusted, let us know in this thread and we will make sure that things get adjusted appropriately.
__________________

Last edited by Akkadius; 01-13-2018 at 08:02 PM..
Reply With Quote
  #2  
Old 01-13-2018, 08:52 PM
The_Beast's Avatar
The_Beast
Discordant
 
Join Date: May 2016
Location: Under a rock
Posts: 290
Default

Sorry for an annoying question, lol. I was just curious about the future. If one was using old binaries, (for example, compiled last week),
is anything planned for the future that would interfere with older binaries patching into the public eqemu login server ?
This whole .json thing is something I need to learn about, so just checking for future reference
Reply With Quote
  #3  
Old 01-14-2018, 05:31 AM
The_Beast's Avatar
The_Beast
Discordant
 
Join Date: May 2016
Location: Under a rock
Posts: 290
Default

BUG ??
When I ran new binaries, it got this far and just hung there. But in my server directory, I noticed it kept deleting the
eqemu_server.pl file and replacing it. Just kept doing that continuously, (looped) It was just a test box anyways.
I also tried just running the script by itself (without binaries) and got the same thing.
Code:
Pulling down EQEmu Server Maintenance Script (eqemu_server.pl)...
[Update] Script has been updated, updating...
[Install] Installing :: eqemu_server.pl
[Update] No script update necessary...
EDIT - The "new" script works fine with old binaries, but I also noticed, from trying to run it before, it never did
update anything related to json conversion. (no config or anything). I tried both my own compiled new binaries
and the precompiled downloaded ones. Couldn't get past the "hanging" with any new binaries.
Reply With Quote
  #4  
Old 01-14-2018, 09:11 PM
The_Beast's Avatar
The_Beast
Discordant
 
Join Date: May 2016
Location: Under a rock
Posts: 290
Default

I grabbed the "fixed" eqemu_server.pl from gitpull today and works like a charm. Thank You
Reply With Quote
  #5  
Old 01-14-2018, 09:59 PM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 1,895
Default

Quote:
Originally Posted by The_Beast View Post
I grabbed the "fixed" eqemu_server.pl from gitpull today and works like a charm. Thank You
Glad you got it sorted - however if that is happening to other folks I will need to iron out whatever was causing it

No worries
__________________
Reply With Quote
  #6  
Old 01-15-2018, 02:59 AM
The_Beast's Avatar
The_Beast
Discordant
 
Join Date: May 2016
Location: Under a rock
Posts: 290
Default

Smee again :P
Seems to have downloaded, yet, a newer script, but it's spitting out errors. It doesn't do the json conversion for xml
and oddly enough, it's setting a db version for bots, which I don't even have enabled, lol It set the version at 0 in DB.
This is all on a test box that I'm using to help work out bugs, so I can freely wipe the server and redo anytime.

Code:
[Update] Script has been updated, updating...
[Install] Installing :: eqemu_server.pl
[Update] No script update necessary...
cannot open file  at eqemu_server.pl line 326. <---
[Update] Bots database up to Date: Continuing World Bootup...
Invalid command 'start_from_world'
[Install] Done
cannot open file  at eqemu_server.pl line 326. <---
[Update] Bots database up to Date: Continuing World Bootup...

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> EQEmu Server Main Menu >>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

 [database]     Enter database management menu
 [assets]       Manage server assets
 [new_server]   New folder EQEmu/PEQ install - Assumes MySQL/Perl installed
 [setup_bots]   Enables bots on server - builds code and database requirements

 exit

Enter a command #> database

>>> Database Menu

 [backup_database]              Back up database to backups/ directory
 [backup_player_tables]         Back up player tables to backups/ directory
 [backup_database_compressed]   Back up database compressed to backups/ director
y

 [check_db_updates]             Checks for database updates manually
 [check_bot_db_updates]         Checks for bot database updates

 [aa_tables]                    Downloads and installs clean slate AA data from
PEQ
 [remove_duplicate_rules]       Removes duplicate rules from rule_values table
 [drop_bots_db_schema]          Removes bot database schema

> main - go back to main menu
Enter a command #> check_db_updates
[Database] Retrieving latest database manifest...
[Download] Saved: (db_update/db_update_manifest.txt) from https://raw.githubuser
content.com/EQEmu/Server/master/utils/sql/db_update_manifest.txt
[Database] Reading manifest...

[Database] Retrieving latest database manifest...
[Download] Saved: (db_update/db_update_manifest.txt) from https://raw.githubuser
content.com/EQEmu/Server/master/utils/sql/db_update_manifest.txt

[Database] No updates need to be run...
[Database] Setting Database to Binary Version ((xmltojson.exe) from https://raw.
githubusercontent.com/EQEmu/Server/master/utils/xmltojson/xmltojson-windows-x86.
exe
Converting eqemu_config.xml to eqemu_config.json
Server element not found <----
Binary Database Version) if not already...

ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the ma
nual that corresponds to your MySQL server version for the right syntax to use n
ear 'from https://raw.githubusercontent.com/EQEmu/Server/master/utils/xmltojson/
xmlto' at line 1
Reply With Quote
  #7  
Old 01-15-2018, 03:09 AM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 1,895
Default

I just ran a conversion using the latest script and latest binaries and it converted just fine, not sure how you're running into that issue.

Are you running your server binaries within the context of the server directory?

Are you doing something different?
__________________
Reply With Quote
  #8  
Old 01-15-2018, 03:28 AM
Darkscis
Sarnak
 
Join Date: Mar 2015
Posts: 60
Default

I can confirm I also just ran the latest script on a virtual machine that hasn't even been turned on for 6 months. It converted the .xml to .json without issue.
Reply With Quote
  #9  
Old 01-15-2018, 03:49 AM
Shin Noir's Avatar
Shin Noir
Legendary Member
 
Join Date: Apr 2002
Location: Seattle, WA
Posts: 471
Default

Quote:
Originally Posted by The_Beast View Post
Smee again :P
Seems to have downloaded, yet, a newer script, but it's spitting out errors. It doesn't do the json conversion for xml
and oddly enough, it's setting a db version for bots, which I don't even have enabled, lol It set the version at 0 in DB.
This is all on a test box that I'm using to help work out bugs, so I can freely wipe the server and redo anytime.
Look at your eqemu_config file, and check the top line.
It is saying it can't find the server element..
See if server is there. Is it by chance not all lowercase ? e.g. Server?
__________________

~Shin Noir
Rebuild EQ.com
Reply With Quote
  #10  
Old 01-15-2018, 04:12 AM
The_Beast's Avatar
The_Beast
Discordant
 
Join Date: May 2016
Location: Under a rock
Posts: 290
Default

In reference my other post (see "works like a charm"), the whole server was working perfect without issues. Then tonight, all I did was
run the eqemu_server.pl script (in the server directory), and wanted to check_db_updates. But when I ran the script, it pulled an updated
version and got those errors shown. World won't even run now, so have to find out why. But as I said, I did nothing except run the script
for db updates, like I've been doing for a long time now.

Just to add: I've been trying to get it to convert the config over to json.(again). I have 3 different configs setup on this one box, for test.
One for local, one for lan, and one for public. I was simply trying to get them all (one by one) converted over to json. But deleting the json
config and putting one of those old configs in, it won't convert them now.

Quote:
Originally Posted by Akkadius View Post
whenever new binaries are used, world.exe will automatically pull down a config converter before attempting to load the config.
This is the reason I "assumed", if I deleted the json config, replace it with another older xml config, it would
convert it over when world was run. Guess I assumed wrong, lol
Reply With Quote
  #11  
Old 01-15-2018, 04:58 AM
The_Beast's Avatar
The_Beast
Discordant
 
Join Date: May 2016
Location: Under a rock
Posts: 290
Default

Here, just to simplify what I am talking about. I put new binaries in the folder. There is an eqemu_config.xml file in there.
I went into folder with a cmd window and run world and see what happens. Even though it "implies" that it is converting
the xml over to json, it doesn't do it. This is all that happens :
Code:
D:\eqemutest>world
Pulling down EQEmu Server Maintenance Script (eqemu_server.pl)...
[Download] Saved: (xmltojson.exe) from https://raw.githubusercontent.com/EQEmu/S
erver/master/utils/xmltojson/xmltojson-windows-x86.exe
Converting eqemu_config.xml to eqemu_config.json
Server element not found
cannot open file  at eqemu_server.pl line 326.
[World Server] Loading server configuration..
Error from reader: * Line 1, Column 1
  Syntax error: value, object or array expected.
[World Server] Loading server configuration failed.
For the moment, I can't figure it out yet. My coffee is getting cold, lol
Reply With Quote
  #12  
Old 01-15-2018, 05:14 AM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 1,895
Default

The_Beast - see what Shin posted above

Looks like your XML is malformed or non-standard

If you can paste it to us via pastebin or otherwords with sensitive info redacted we could help point you in a direction
__________________
Reply With Quote
  #13  
Old 01-15-2018, 05:25 AM
The_Beast's Avatar
The_Beast
Discordant
 
Join Date: May 2016
Location: Under a rock
Posts: 290
Default

Hahaha, this funny. First of all Shin, sorry, when you said check top line, I thought you meant the json config. But just found out
if your xml config has "special characters" in the server name, the conversion won't work. I removed them and problem solved.
Reply With Quote
  #14  
Old 01-15-2018, 05:25 AM
Akkadius's Avatar
Akkadius
Administrator
 
Join Date: Feb 2009
Location: MN
Posts: 1,895
Default

Quote:
Originally Posted by The_Beast View Post
Hahaha, this funny. First of all Shin, sorry, when you said check top line, I thought you meant the json config. But just found out
if your xml config has "special characters" in the server name, the conversion won't work. I removed them and problem solved.
Glad you got it sorted!
__________________
Reply With Quote
  #15  
Old 01-20-2018, 01:31 PM
Vexyl
Hill Giant
 
Join Date: Oct 2009
Location: U.S.A.
Posts: 181
Default

Interesting, but why not just use Lua for config since the server is already being linked against it?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is On

Forum Jump


All times are GMT -4. The time now is 01:47 AM.


 

Everquest is a registered trademark of Daybreak Game Company LLC.
EQEmulator is not associated or affiliated in any way with Daybreak Game Company LLC.
Except where otherwise noted, this site is licensed under a Creative Commons License.
       
Powered by vBulletin®, Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Template by Bluepearl Design and vBulletin Templates - Ver3.3