CICD bot should show all details of the `prod` plan in the preview

[plaflamme]

Currently, the Prod Plan Preview step summary in GitHub shows the diff against prod and lists the indirectly affected models. On the other hand, the PR Environment Synced step summary shows the categorization of directly modified models, but only for the change to the virtual environment.

It's currently not possible to know what the categorization for indirectly modified models are for the eventual prod plan. That is, we can only see what models are affected, but we cannot (clearly) see that they are indirectly modified non-breaking, breaking or forward-only. Furthermore, the "backfill" list doesn't show the dates that need to be backfilled or whether a model is a view and its view will be recreated (like the cli output shows).

This leads to the situation where it's hard to say for sure what will actually happen when a PR is /deployed. Ideally, the Prod Plan Preview would show everything explicitly, that is, list all affected model, the categorization of the change for each of them and the list of all backfilled models and the corresponding dates (or that a view will be re-created). Without this, it's difficult to know exactly what will be deployed since there might be many modifications to the virtual environment that the latest PR Environment Synced doesn't show.

[erindru]

Hey @plaflamme are the screenshots in my PR more like what you were thinking?

[plaflamme]

@erindru Yes, that's great. One other thing to note is that, sometimes, when the list of models to "backfill" is long, it gets truncated. This is problematic since it makes it difficult to know if there's any non-desirable consequences in the list that might have been missed elsewhere. So ideally, the list would not get truncated under any circumstance.

Ultimately though, what is truly important is to be able to see what the prod plan will do to downstream models, not so much what is being changed directly (since the "Files changed" tab from the GH PR already more or less of shows this). So if any kind of truncation is necessary (say to limit the amount of data due to GH constraints) I would recommend skipping the diffs and keep the list of prod "backfill"s.

[erindru]

Thanks for the feedback, you're right it's currently based around the "happy" case where there aren't 200 models to backfill. I need to investigate the larger case more and figure out how to reproduce it locally so I can ensure the output is still usable.

There is a limit to how much text can be jammed into a GitHub Check, it allows 65535 bytes but of course this can result in less characters if some are multi-byte utf8 characters. So even with trimming down the output we might still hit this limit, in which case the existing logic to print the remainder to the console will kick in as a last resort

[erindru]

So something like this? #4915

[plaflamme]

@erindru yes, that's great, thanks!

[erindru]

These changes have been released in v0.200.0