Akkadius
01-04-2018, 11:13 PM
Greetings Norrathians!
This blog update is to make everyone aware of the major changes that have happened over the past year.
We've made massive progress on so many fronts this last year, many milestones that would otherwise be hard to know about unless you've been paying very close attention to commits and changelog messages, so here we go!
https://i.imgur.com/0DlHdH4.png
Netcode Rewrite (KLS)
With the recent MASSIVE netcode overhaul work of KLS and assist of the implementation of myself, we've been able to break through so many stability/scaling issues of the past. EZ for example has been able to sustain 300-400 players in a combat zone without any issue
The emulator application level netcode usage was put under a microscope and we found excessive packet send issues, identified and implemented countermeasures to insure a smooth gameplay experience
Animation packet reductions (https://github.com/EQEmu/Server/commit/62e4169e504d47d285ababeef35d726e1c06e9f2)
Massive reductions in unnecessary network traffic especially during high spam combat fights (https://github.com/EQEmu/Server/commit/127f51e7587b0d354f0f326f8661a640baf313e2)
Mob position update spam has been heavily reduced by doing more updates around the player than spamming updates from NPC's across the zone (https://github.com/EQEmu/Server/commit/ec00daa5be0ffbfc61623abc5bc1235f95f7da4d)
Lots of packets such as /say /emote, cast, animations, particles, damage/spell/song/critical messages were all spending packets in distances that could cause scaling issues with intense combat/burn situations - set ranges have been implemented to match what is observed on Live (https://github.com/EQEmu/Server/commit/ab372b0f6b6a36173c725f328c8ece14b07e)
Many more changes
Implementations
[mackal] Auras have been implemented
[mackal] Food and Water consumption algorithms have been fixed to match close to live
[mackal] Emulator Combat engine has been massively overhauled to match Live like formulas to be 98% accurate with Live changelog (https://github.com/EQEmu/Server/search?q=performance&type=Commits&utf8=%E2%9C%93)
[mackal] Extended target functionality
Server Installers
[Akkadius/N0ctrnl] Linux and Windows installers were introduced in late 2016 to streamline the EQEmu Server installation processes
Linux Installs to date: 959
Windows Installs to date: 3,489
Mob Z Axis Breakthrough
[Akkadius/Noudess/KLS] - Mob Z axis correction has had a major breakthrough in 2017 - we cracked race/size offsets for models - this helps bigly with
Corpses going underground
NPC's hopping during a pull
Pet's hopping during combat
NPC's hopping while pathing
NPC's going half into the ground while pathing
NPC's falling through the ground in general
While these problems can still happen depending on the zone and situation, the chances of them occurring have been vastly decreased
Server Side Performance Enhancements (Akkadius)
I've personally taken this issue down to the core, because we're at a mature point in our project where we can focus on shaving the fat off of some of our quickly implemented code that has created many performance issues.
We've been able to continually measure PEQ and EZ zone processes running hot with players in them, profile the code stack and find problematic code paths and measure over time. We've come a long way.
It allows servers to run better, faster, with less cpu cycle consumption to create the best gaming experience possible!
PEQ with 840 players online (chart by netdata) at a steady low cpu util
<img src="https://i.imgur.com/l9Iv6eB.png" style="height:300px;width:auto">
All Log calls in the source were switched to a C macro to do integer checking before trying to process strings, these keeps hot path logging strings from being pushed/popped off the code stack and made a huge difference on server runtime performance (https://github.com/EQEmu/Server/commit/7aa1d243b0483ad9041537aada44f923bf923390)
Performance adjustment to SendPosition update logic (https://github.com/EQEmu/Server/commit/be0374d1975c92841f8d2c1847a24cdef36bb8f9)
Fixed a low level constant quest file exist check during some routines (https://github.com/EQEmu/Server/commit/9d0308c9bf844fc0f9579d540f24cfaa7de19c69)
Slowed down zone main process loop FPS to 60 when booted, sleep zone processes slowed down heavily as well (https://github.com/EQEmu/Server/commit/af9a34e4e770351d393152ff64640929238826c9)
Reworked how client to NPC aggro checks are made (https://github.com/EQEmu/Server/commit/4b6ce1c19edae8a4db9d473ffa1790900858da04)
Fixed a large overhead issue where every single NPC in a zone was checking to depop themselves as a swarm pet every 3ms regardless of being a swarm pet or not. Swarm pets now check to depop only when their timer is up (https://github.com/EQEmu/Server/commit/bf45a5a44f1230fab180c9012dfb9bc0fd8489cb)
Removed a timer where clients would constantly calculate light amount on equipment every 600ms, instead clients will update light when changing equipment or entering a zone (https://github.com/EQEmu/Server/commit/47691c2102a0933c458b040b5c972355e158ecec)
Disabled enraged timer checks for NPC's that do not actually have enrage as a special attack (https://github.com/EQEmu/Server/commit/f931ef7bcb70d2134d71cf86c804bc6301001fd2)
Sped up saylink retrieval x1000 helpful for dialogues, plugins with many saylinks via adding a string index to the saylink table (https://github.com/EQEmu/Server/commit/357be65a69e05121210ac0a2daa89c4a90d69ad4)
Improved constant NPC AI Check close casting code for beneficial spells
So many other changes
Please let me know if I missed something worth mentioning here, the idea is to highlight what we have been doing as a whole - cause not everyone is up to speed all the time....
To keep from making this post too long - there's tons in the Github commits (https://github.com/EQEmu/Server/commits/master) and EQEmu changelog (https://raw.githubusercontent.com/EQEmu/Server/master/changelog.txt) that may not be seen here, and many people are contributing to bug fixes and slight feature changes to the project everyday!
Thanks to everyone who brings this project together
https://i.imgur.com/VFFJn7r.png
Documentation
We have a lot of work to do in this department and is one of my highest priorities, details posted here (http://www.eqemulator.org/forums/showthread.php?t=41656)
Navmesh
KLS has spent a lot of time working on a Navmesh system for our code and it has come a long way, from everything to a navmesh tile generator to in game logic to follow those tiles
Navmesh is basically waypoint pathfinding, using a shortest path algorithm to determine what tiles/path to use between it and its target
The Navmesh tool KLS built generates 95% of the tiles automatically, you may need to create manual connections in zones where necessary
You can see an example of this below
ujtqipXAP1E
EQEmu Server Config Format
Thanks to Shin Noir - the server standard config will soon be .json as soon as I can merge it into master and have time to resolve any issues
Future Loginserver release will also use .json
Another thought is to also define paths to resources so the main server folder can become decluttered of resources/files
Expedition System and Shared Tasks
The groundwork for this system has been started again - but progress has been halted for now, but this is an immediate future priority
http://peqtgc.com/forums/styles/ProjectEQ.png
Our brother project PEQ has made some considerable strides as well
New website and forums: http://peqtgc.com/forums/index.php?articles/
Released Omens of War expansion
Dragons of Norrath pre-work
Continual work on old content continuity
Ever growing population
Streamlined 'Getting Started Process'
The team has been making solid progress and if you're interested in helping out - check out Discord on the left and the #public channel and jump in!
Project tracker: https://trello.com/b/fDpukzMU/projecteq
Systems Pending Development: Expeditions DZ system - Shared Tasks
This blog update is to make everyone aware of the major changes that have happened over the past year.
We've made massive progress on so many fronts this last year, many milestones that would otherwise be hard to know about unless you've been paying very close attention to commits and changelog messages, so here we go!
https://i.imgur.com/0DlHdH4.png
Netcode Rewrite (KLS)
With the recent MASSIVE netcode overhaul work of KLS and assist of the implementation of myself, we've been able to break through so many stability/scaling issues of the past. EZ for example has been able to sustain 300-400 players in a combat zone without any issue
The emulator application level netcode usage was put under a microscope and we found excessive packet send issues, identified and implemented countermeasures to insure a smooth gameplay experience
Animation packet reductions (https://github.com/EQEmu/Server/commit/62e4169e504d47d285ababeef35d726e1c06e9f2)
Massive reductions in unnecessary network traffic especially during high spam combat fights (https://github.com/EQEmu/Server/commit/127f51e7587b0d354f0f326f8661a640baf313e2)
Mob position update spam has been heavily reduced by doing more updates around the player than spamming updates from NPC's across the zone (https://github.com/EQEmu/Server/commit/ec00daa5be0ffbfc61623abc5bc1235f95f7da4d)
Lots of packets such as /say /emote, cast, animations, particles, damage/spell/song/critical messages were all spending packets in distances that could cause scaling issues with intense combat/burn situations - set ranges have been implemented to match what is observed on Live (https://github.com/EQEmu/Server/commit/ab372b0f6b6a36173c725f328c8ece14b07e)
Many more changes
Implementations
[mackal] Auras have been implemented
[mackal] Food and Water consumption algorithms have been fixed to match close to live
[mackal] Emulator Combat engine has been massively overhauled to match Live like formulas to be 98% accurate with Live changelog (https://github.com/EQEmu/Server/search?q=performance&type=Commits&utf8=%E2%9C%93)
[mackal] Extended target functionality
Server Installers
[Akkadius/N0ctrnl] Linux and Windows installers were introduced in late 2016 to streamline the EQEmu Server installation processes
Linux Installs to date: 959
Windows Installs to date: 3,489
Mob Z Axis Breakthrough
[Akkadius/Noudess/KLS] - Mob Z axis correction has had a major breakthrough in 2017 - we cracked race/size offsets for models - this helps bigly with
Corpses going underground
NPC's hopping during a pull
Pet's hopping during combat
NPC's hopping while pathing
NPC's going half into the ground while pathing
NPC's falling through the ground in general
While these problems can still happen depending on the zone and situation, the chances of them occurring have been vastly decreased
Server Side Performance Enhancements (Akkadius)
I've personally taken this issue down to the core, because we're at a mature point in our project where we can focus on shaving the fat off of some of our quickly implemented code that has created many performance issues.
We've been able to continually measure PEQ and EZ zone processes running hot with players in them, profile the code stack and find problematic code paths and measure over time. We've come a long way.
It allows servers to run better, faster, with less cpu cycle consumption to create the best gaming experience possible!
PEQ with 840 players online (chart by netdata) at a steady low cpu util
<img src="https://i.imgur.com/l9Iv6eB.png" style="height:300px;width:auto">
All Log calls in the source were switched to a C macro to do integer checking before trying to process strings, these keeps hot path logging strings from being pushed/popped off the code stack and made a huge difference on server runtime performance (https://github.com/EQEmu/Server/commit/7aa1d243b0483ad9041537aada44f923bf923390)
Performance adjustment to SendPosition update logic (https://github.com/EQEmu/Server/commit/be0374d1975c92841f8d2c1847a24cdef36bb8f9)
Fixed a low level constant quest file exist check during some routines (https://github.com/EQEmu/Server/commit/9d0308c9bf844fc0f9579d540f24cfaa7de19c69)
Slowed down zone main process loop FPS to 60 when booted, sleep zone processes slowed down heavily as well (https://github.com/EQEmu/Server/commit/af9a34e4e770351d393152ff64640929238826c9)
Reworked how client to NPC aggro checks are made (https://github.com/EQEmu/Server/commit/4b6ce1c19edae8a4db9d473ffa1790900858da04)
Fixed a large overhead issue where every single NPC in a zone was checking to depop themselves as a swarm pet every 3ms regardless of being a swarm pet or not. Swarm pets now check to depop only when their timer is up (https://github.com/EQEmu/Server/commit/bf45a5a44f1230fab180c9012dfb9bc0fd8489cb)
Removed a timer where clients would constantly calculate light amount on equipment every 600ms, instead clients will update light when changing equipment or entering a zone (https://github.com/EQEmu/Server/commit/47691c2102a0933c458b040b5c972355e158ecec)
Disabled enraged timer checks for NPC's that do not actually have enrage as a special attack (https://github.com/EQEmu/Server/commit/f931ef7bcb70d2134d71cf86c804bc6301001fd2)
Sped up saylink retrieval x1000 helpful for dialogues, plugins with many saylinks via adding a string index to the saylink table (https://github.com/EQEmu/Server/commit/357be65a69e05121210ac0a2daa89c4a90d69ad4)
Improved constant NPC AI Check close casting code for beneficial spells
So many other changes
Please let me know if I missed something worth mentioning here, the idea is to highlight what we have been doing as a whole - cause not everyone is up to speed all the time....
To keep from making this post too long - there's tons in the Github commits (https://github.com/EQEmu/Server/commits/master) and EQEmu changelog (https://raw.githubusercontent.com/EQEmu/Server/master/changelog.txt) that may not be seen here, and many people are contributing to bug fixes and slight feature changes to the project everyday!
Thanks to everyone who brings this project together
https://i.imgur.com/VFFJn7r.png
Documentation
We have a lot of work to do in this department and is one of my highest priorities, details posted here (http://www.eqemulator.org/forums/showthread.php?t=41656)
Navmesh
KLS has spent a lot of time working on a Navmesh system for our code and it has come a long way, from everything to a navmesh tile generator to in game logic to follow those tiles
Navmesh is basically waypoint pathfinding, using a shortest path algorithm to determine what tiles/path to use between it and its target
The Navmesh tool KLS built generates 95% of the tiles automatically, you may need to create manual connections in zones where necessary
You can see an example of this below
ujtqipXAP1E
EQEmu Server Config Format
Thanks to Shin Noir - the server standard config will soon be .json as soon as I can merge it into master and have time to resolve any issues
Future Loginserver release will also use .json
Another thought is to also define paths to resources so the main server folder can become decluttered of resources/files
Expedition System and Shared Tasks
The groundwork for this system has been started again - but progress has been halted for now, but this is an immediate future priority
http://peqtgc.com/forums/styles/ProjectEQ.png
Our brother project PEQ has made some considerable strides as well
New website and forums: http://peqtgc.com/forums/index.php?articles/
Released Omens of War expansion
Dragons of Norrath pre-work
Continual work on old content continuity
Ever growing population
Streamlined 'Getting Started Process'
The team has been making solid progress and if you're interested in helping out - check out Discord on the left and the #public channel and jump in!
Project tracker: https://trello.com/b/fDpukzMU/projecteq
Systems Pending Development: Expeditions DZ system - Shared Tasks