feat!: add ignore destructive support

[eakmanrq]

BREAKING: columns_to_types has been renamed to target_columns_to_types for engine adapter methods.

This PR adds support for "ignoring" a destructive change using the on_destructive_change property. As documentation says, this property should not be used in most cases and is really just being added for dbt support. It could be useful though for an "escape-hatch" situation.

The concept of ignoring a destructive change is pushed down to the SchemaDiffer class level because the SchemaDiffer needs to be aware of this property is being used due to positional changes. If the engine supports positional changes and the SchemaDiffer wasn't aware then it could generate alter commands that involve changing columns assuming a column order that wouldn't really happen.

Prior to this PR, the engine adapter class assumed that the Query or DataFrame had the same columns as columns_to_types. columns_to_types has been renamed to target_columns_to_types and now represents the target table columns and can now drift out of sync of the "source" query or DataFrame. To account for this, engine adapter methods which take a QueryOrDF now also take a source_columns which tells the adapter what columns are in that source. If needed, it will pad out the columns in the source to match the target.

Note: I am intentionally not changing dbt adapter implementation since I plan on updating it once on_additive_change support is added in follow up PR. That will isolate the breaking changes related to on_schema_change support to a single PR and it is just breaking for dbt adapter.

[izeigerman]

Why would it result in data loss?

[izeigerman]

Also how can it result in error?

[eakmanrq]

Also how can it result in error?

If you made a data type change to a new type that isn't coerceable. The next time it runs it would error trying to insert the rows (since the underlying table wasn't actually changed).

Why would it result in data loss?

Not certain yet. Putting this warning for now as we do more testing with this fully implemented in the dbt adapter. After a bit if we don't identify any issues I can remove this part of the warning. For now though I would rather overly caution native users about this feature.

[izeigerman]

wasn't aware then it could generate alter commands that involve changing columns assuming a column order that wouldn't really happen.

I'm somewhat confused. Does changing the position result in a destructive change?

[eakmanrq]

I'm somewhat confused. Does changing the position result in a destructive change?

No it doesn't. The SchemaDiffer though requires that it can properly track the future state of the changes it is generating in order to properly understand where columns should be inserted. So if it thought columns would be dropped when they would actually still exist then it might put columns in expected positions as a result.

[eakmanrq]

Discussed internally and we don't see a reason why exp.DerivedTable was expected before and that code is from version 0.0.2 so assuming it is no longer needed.