Add: "include" keyword, allowing to include files without external tools

[glx22]

Many GRF workflows use tools to combine multiple files into a single NML (C preprocessor, python, ...).

So I though it would be nice to be able to do that natively in NML. It's done during parsing by switching lexer data.

I also added optional <ID> = <expression> as include parameters, allowing to replace some parts inside the included file during parsing (very basic equivalent of #define from C preprocessor).

Include syntax is include(<filename>[, <ID> = <expression>[, ...]]);.

Added identifier concatenation support using <identifier>##<identifier>.

--nml output for 042_include.nml:

param[0] = 0;

param[0] = (param[0] + 1);

const _NAMExy = 1;
param[0] = (param[0] + _NAMExy);


param[0] = (param[0] + 2);

const _testxy = 2;
param[0] = (param[0] + _testxy);


param[0] = (param[0] + 3);

const _test2xy = 3;
param[0] = (param[0] + _test2xy);

[andythenorth]

Tested simplest case with FIRS, works as expected.

andythenorth/firs@0beca16

[andythenorth]

Tested locally as working with params.

include("src/polar_fox/cargo_classes/cargo_class_constants.nml", cabbage=2, ham=1);

[zephyris]

Is there anything us NewGRF authors can do to help test this? It would be nice to get it out of draft status!

[frosch123]

The character to concat identifiers is .
Classic preprocessors would use ##

Do we block pontential future "member access" by using "." for concatenation?

[glx22]

Well it's still very hacky, and needs more work.

[Pixel-Tony]

I wholeheartedly approve of the idea, because more code reusability is always welcomed, but I stand against token concatenation and identifier mutation capabilities being included.

One of my main arguments against it is, if user chooses to use this feature to create templated names a. e. purchase spriteset names with common prefix, then he will still need to write (and remember) full prefix in the place of its usage.

I am currently developing a language server for NML and ability to join identifiers makes a lot of things ambiguous and error-prone, just like complex defines in C do. IMHO, we should fix boilerplate-y sections instead of providing generic macros support.

[Pixel-Tony]

Also, we should consider incomplete code inside included file: is the intended behavior to allow users to use includes inside other AST nodes or parse file separately and include received statements to the resulting unit at the top level?

[andythenorth]

I am still strongly in favour of this, but I'm unsure how to help move it forward.

I am not convinced that we need parameterised includes. Simplicity is the goal for an include feature IMO.

That said, I wonder if we could target, not a file, but a script with params, returning nml as a string?
E.g. include("src/foo.py cabbage=2 ham=1")

Then the author could template the output of foo.py using whatever means they choose, whilst keeping it simple for other cases to use static nml via include("src/bar.nml")