Runtime bottleneck

[WilliamRagstad]

A single WXRuntime thread holds a context with HashMap<WXModulePath, deno_core::JsRuntime>.
When a request is received by the web server, it sends a channel message to this thread here:

webx/src/engine/runtime.rs

Lines 698 to 701 in 8d00005

	pub async fn run(&mut self) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
	self.recompile();
	loop {
	if let Ok(msg) = self.messages.recv() {

This quickly becomes a synchronization bottleneck when many clients' request require JavaScript code execution in a JsRuntime.
The current solution is a temporary "working" method of having multiple worker threads "share" a mutable reference to a JavaScript runtime via channels. (See thread in #2)

To optimize this and ensure clients don't interrupt, block, or interfere with any other client request execution, create a new JsRuntime for each incoming request. Yes, this is a huge cost and will become one of the main throttles for WebX technologies, but it is a sacrifice for ensuring correctness and safety.

[WilliamRagstad]

Solution Suggestion

Use a thread-safe JsDispatcher struct holding a dynamic re-sizable vector of initialized and ready JsRuntime instances.
This method might work due to the nature of WebX's functional and declarative paradigm, allowing easy parallelization and fast APIs via Rust native FFI mappings to JavaScript. Also, global variables in WebX will all end up behind a Mutex, where mutable access locks and will always block until ready, preserving the memory safety ideology of Rust.
One of the key advantages of WebX is its ability to leverage Tokio for efficient multi-threading, enabling it to potentially outperform other web server frameworks in the JavaScript ecosystem. This is particularly beneficial as these frameworks are often limited by the single-threaded nature of JavaScript.

The JsDispatcher will act as a close-to-constant lookup table, holding initialized JsRuntime instances ready to run in the current thread owned by Tokio. If there is high demand and too few JsRuntimes, the dispatcher automatically creates and initializes more. If the demand drops, lowering the number of occupied runtimes, it automatically removes some after some time to reduce its memory usage, profile and footprint.

Each async client handler task in the web server will get it's own reference to the JsDispatcher, to use if necessary when executing JavaScript user-code in its own thread.