Top level await for Node.js process

[ccmcintyre-kingsds]

Node.js supports await/async behaviour even in the top-level scope, either through the --harmony-top-level-await flag prior to Node.js 14.8, or for ES modules since Node.js 14.9. Qualifying as an ES module for the purpose of this feature is in principle achievable using .mjs file extensions, declaring "type": "module" in the relevant package.json file, or by passing an --input-type=module flag to the node process when eval-ing script strings. This lattermost option sounds most relevant to Bifrost, but evidence suggests that none of these flags are meaningful when passed to to the Node.js process initialization (via subprocess.Popen in python, at py_nodejs.py), which is setting up a process which will be called via streaming pipes and populated via shared memory, rather than passing scripts to be directly eval-ed by a new Node.js process.

Top-level await is a desirable feature to support for Bifrost broadly, if possible. When investigating solutions, please consider both the above caveats relating to the relevant flags, and also consider how solutions might impact or be impacted by the following behaviour of Bifrost's current script evaluation code, taken from main.js:

//evaluate some script in the given context.
//Note that we await in order to allow for scripts which end in a promise
//to fully complete
async evaluate(script){
    await vm.runInContext(script, this.context);
}

Whatever solution is implemented should be able to support the async initalization of dcp-client using top-level await.

[wesgarland]

Remember also that ES6 modules are by definition strict-mode code, which is generally a good thing, but has observable differences from non-strict code.

[wesgarland]

Also - if you are executing the code programatically, it's trivially easy to wrap an async iife around it if that gets you what you want. And you can do async dcp-client init without top-level await; you can use regular promise syntax instead. dcp-client.init().then(run my code);

[ccmcintyre-kingsds]

@wesgarland

Remember also that ES6 modules are by definition strict-mode code, which is generally a good thing, but has observable differences from non-strict code.

Thank you for the reminder; that will be good to keep in mind, and we probably should give some broader thought to how we treat and enforce (or don't enforce) strict mode in Bifrost.

Also - if you are executing the code programatically, it's trivially easy to wrap an async iife around it if that gets you what you want. And you can do async dcp-client init without top-level await; you can use regular promise syntax instead. dcp-client.init().then(run my code);

Fair, and nothing prevents us from doing so with dcp-client at this time. The top-level await feature still seems like something we want to support in general for people using Bifrost's everyday Python+Node.js interop, but it's certainly not blocking functionality for distributing workloads, if we need to push this feature down the road.