Add async versions of functions

[not-an-aardvark]

So I realized that whenever we load data that we don't already have cached, we call require on a new file, which does synchronous filesystem IO and blocks the event loop. This prevents anything else from happening (e.g. requests to the server) while that's going on.

We should probably add versions of the functions that load things asynchronously, and use those instead.

[AlMcKinlay]

So, do you have an example of this? I'm not sure I'm 100% sure what you are talking about.

[not-an-aardvark]

Suppose I want to add parsed data to a pokemon in the database (which the porybox server has to do quite a lot, often multiple times per request). In order to retrieve the data, pkparse needs to load the following files:

• 1 file with data for the species
• 4 files with data for the moves
• 1 file with data for the ability
• 1 file with data for the region
• 1 file with data for the nature
• 1 file with data for the item

All of these file accesses use require, which is synchronous. So the server is frozen and literally can't do anything else while pkparse loads 9 files from the filesystem (or for a single box page, 270 files). This is a serious performance issue.

It's mitigated slightly by the fact that require caches things, so if the server is up for long enough it will eventually hold the files it needs in its cache. But we have a lot of separate data files, so this takes awhile.

[AlMcKinlay]

Ah yes, that makes sense.

1 thing we could to that would be simple (but would increase ram usage in the short term) would be using require all to get node to cache all the files. It's a bit of a hack, but it would at least remove the issue partially right now, with very little change.

[Raia]

@kamronbatman Assigning you here, as for the actual implementation details I'll let you work that out with @not-an-aardvark ✌️