Memory Leaks and Lags

This is a bit technical, but some people may find it interesting!

I'm aware that the server running Avagoria main world keeps getting slower and slower, until it's impossible to even log in.  This is, it turns out, down to an ever increasing consumption of memory by the world server process.  Once it exhausts physical memory, it starts swapping out to disk, and this slows things down considerably.  Builders behaved differently - once it ran out of memory, it crashed and restarted.  I'm not sure which is better...

It's common knowledge that the server is written in PHP, specifically using ReactPHP, an event-driven framework.  This means that the one PHP process that runs the world stays running continuously. This is in contrast to PHP's more common use on webservers, where every access to a web page runs a new process that serves up the page and then terminates.

The consequence of this is that anything that allocates memory, and doesn't release it, slowly increases the total used, which is the symptom we're seeing. It's taken me some time to track down the reason, and it turns out to be a known issue with PHP.

A little background, first. The AltAway server is very modularised.  This is a consequence of how it was written originally.  There is a core code that runs a ReactPHP "loop", which responds to incoming messages from clients, and runs various periodic server-controlled events, such as the spawners.  Each of these pieces of code was written individually, and as such, is located in individual files. When a message comes in from a client, for instance #103 to move between locales, the relevant bit of code is loaded from disc and executed.  This used the PHP "include" method.  It turns out, however, that PHP "remembers" every included file, so every time a message was processed, it increase the memory usage!

In an attempt to stop this, I made the server read the individual files into an array, then I can just "eval" the individual element of the array when needed.  This reduced disk accesses, which will have sped things up slightly, but didn't help with the memory issues; it turns out that "eval" does an in-memory equivalent of writing a file and then including it, with the same consequential consumption of memory every time it happens.. Indeed, it's slightly worse as I'm storing an extra copy in the array beforehand!

My current solution is instead to read in each file, enclose it in a class definition, "eval" that, instantiate an example of that class, and store that in the array.  Then each time we get a message that means we need to run that code, just call the class using it's example in the array.  We still use up memory on the initial eval and stored class object, but that's only once per function, not once every call.  This is currently under test on the builders world, and has definitely solved the memory consumption issue.  Once I've got some other changes finalised, I'll be upgrading the main world server.

The eventual solution, at least for the message processing, will be to have the message handling code in libraries that are included at startup, rather than as individual scripts loaded up later via eval.  Although this will involve a restructuring of the running code, this isn't as huge an issue as it sounds, as the files that the server runs are actually parsed out of somewhat abstracted source code files, so I just need to update the code that transforms those into the server files.  This will be a project for another time, though, but it will again speed things up, especially as everything will then be visible to optimisation and acceleration code.

There's a lot of tidying up and refactoring of the server source coded needed... this is a consequence of several years development and enhancement of a bit of code that was thrown together as a quick "proof of concept" for use before it was re-written in a more suitable language, but that never happened, and the original code has instead just grown and grown.