-
Notifications
You must be signed in to change notification settings - Fork 68
Description
The flagd init code is not ideal.
- the
FlagdProvider'sinitializefunction is called, which callsflagResolver.init()... the method then blocks waiting forsyncResources.waitForInitialization(...) syncResources.waitForInitialization(...)always runs until the deadline timeout; it would be ideal if it simply immediately returned if an error occurs, and only timed-out if nothing happens- the error returned by
initializedoes not return the exception thrown by the stream
Fixing the above should improve the DX of the provider, and ease troubleshooting.
More background from a previous PR is below:
@leakonvalinka ... I kept seeing intermittent test failures with this PR (this was the biggest example. At first, I thought it was a race-condition in the test itself, but it was not). I have removed this now, because I found the root cause:
The main issue was here. It was not any logical error with the code you added, but your changes surfaced an existing issue. The race condition works like this:
- the
FlagdProvider'sinitializefunction is called, which callsflagResolver.init()... the method then blocks waiting forsyncResources.waitForInitialization(...) - normally, in error situations, this method then times out (another thread runs error handlers)
- however, in our situation, this method doesn't just time out; due to our changes to shut down when FATAL errors are seen, the method starting throwing
IllegalStateException(see the condition at the end of the method) - since this is not a FATAL error, this was thrown by init, and depending on timing, would "override" the FATAL with a non-fatal error (the
IllegalStateException)
My fix to throw the Fatal error depending on the last event is a bandaid fix, because this PR has already been open for a while.
To really fix the issue, we should improve the way the waiting in the init works, and actually capture the error during init, and not wait for the full timeout every time. I will open a new issue for this.
Originally posted by @toddbaert in #1624 (comment)