Skip to content

fixed structfield comparison when dataType is array#30

Open
dfarren wants to merge 1 commit intoMrPowers:mainfrom
dfarren:structfields_with_arrays_comparison_fix
Open

fixed structfield comparison when dataType is array#30
dfarren wants to merge 1 commit intoMrPowers:mainfrom
dfarren:structfields_with_arrays_comparison_fix

Conversation

@dfarren
Copy link

@dfarren dfarren commented Jun 22, 2021

The [newly created] test below fails. This because are_structfields_equal doesn't check for the case when the dataType is an array. If the dataType is array, then the nullability shouldn't matter when argument ignore_nullability=True.

def it_can_perform_nullability_insensitive_comparisons_with_arrays():
    sf1 = StructField("hi", ArrayType(IntegerType(), True), False)
    sf2 = StructField("hi", ArrayType(IntegerType(), False), True)
    assert are_structfields_equal(sf1, sf2, ignore_nullability=True) == True

This PR fixes this by adding check_type_equal_ignore_nullable(sf1, sf2) in the are_structfields_equal function.

@MrPowers
Copy link
Owner

Does this work for nested array columns?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants