Replace toml configuration with CLI + env args

[Mirko-von-Leipzig]

Originally suggested in #647, paraphrasing motivation here.

toml files are nice for low-change, manual configuration as it gets out of the way for conventional use. It is however poorly suited for scripting and devops workflows:

1. Always an extra init step, and an extra file to manage.
2. The file is difficult to edit via scripts. This requires additional tooling to edit - basic grep/sed isn't good enough as you need a section parser (e.g. there are multiple endpoint = xxx lines under different sections.

I propose shifting to a fully CLI/clap based approach. We could combine this with toml files by having cli args take precedence but in my experience this becomes a pain to maintain. The "optimal" naming scheme is also often different between the two so things drift.

There are situations where having file based configuration is great. One example is updating configs across multiple instances to identical settings - ideally you just copy the file around. Or users that don't want to specify a long list of non-default cli args (though the latter could be done via alias). For this usage I propose environment args. clap supports these out of the box, and does the correct thing. File configuration then becomes a .env file.

An additional bonus is that many deployment tools support env var based flows, e.g. systemd service files let you specify env variables, or even .env files. Similarly in docker.

---

This is a rather large breaking change; so I'd like to get consensus on this before proceeding.

[SantiagoPittella]

We can use dotenvy for the .env to work on Windows machines too.

[bobbinth]

I think this works on Linux-based systems, but not on Windows. Not super relevant for the node, but for the client, we may want to use something like dotenvy to:

• Make things work on Windows and on Linux the same way.
• The user doesn't need to do source .env as it may be a source of potential bugs.

[Mirko-von-Leipzig]

The client doesn't really have many config options, does it? Apart from miden init ... which should store the config in the database maybe.

[Mirko-von-Leipzig]

I think this works on Linux-based systems, but not on Windows

Brief search confirms this :/ Always Windows being a pain.

[SantiagoPittella]

So, I think that in this particular case the miden-proving-service is more similar to the node, right? In that case I will remove dotenvy.

[bobbinth]

So, I think that in this particular case the miden-proving-service is more similar to the node, right? In that case I will remove dotenvy.

Yes - that's how i think about it too.

[Mirko-von-Leipzig]

@bobbinth one possible issue is with the genesis block configuration. I imagine that would have to remain a toml file?

We can add a default so that we don't have to keep an actual toml file around, but I don't know one would configure the genesis block with arbitrary data?

[bobbinth]

Yeah, I think it might have to remain a TOML file. Right now, it is pretty basic (and probably not super useful), but I was thinking in the future it would contain information about the canonical bridge - though, the details of this are not well defined yet.

[Mirko-von-Leipzig]

I suppose that begs the question - does this need to be configurable at all? If its only the bridge then that would be quite limited in configuration I imagine?

e.g. right now we could have this configured as just token, amount and decimal points which is cli friendly. Supporting an arbitrary number of faucet accounts is not very cli friendly, though technically possible.

[bobbinth]

I suppose that begs the question - does this need to be configurable at all? If its only the bridge then that would be quite limited in configuration I imagine?

I think it would need to be configurable at least to support different settings for mainnet vs. testnet vs. devnet. But difficult to tell right now how many actual settings this would require (e.g., would it be just address of Ethereum contract? or maybe we'll need to have some AggLayer-specific settings as well).

e.g. right now we could have this configured as just token, amount and decimal points which is cli friendly. Supporting an arbitrary number of faucet accounts is not very cli friendly, though technically possible.

Agreed - though I'm also not sure if we'd need to pre-create some number of accounts at genesis. For example, we may want to create specially-configured faucets for USDC and USDT which would somehow be tied to the bridge so that when people bridge these assets from Ethereum, they appear as being issued by these faucets. Though, this all depends on the actual bridge design which is still TBD.

[Mirko-von-Leipzig]

Implemented in #732 and #749