‼️ Make needs_fields and add_field default to nullable and no default

[chrisjsewell]

Currently, for all of:

needs_extra_options = [{"name": "field1"}]
add_extra_option("field1")
needs_fields = {"field1": {}}
add_field("field1")

These create a field with:

• a string type
• a non-nullable value
• a default value of ""

whereas for all of:

needs_extra_options = [{"name": "field1", "schema": {"type": "string"}}]
add_extra_option("field1", schema={"type": "string"})
needs_fields = {"field1": {"schema": {"type": "string"}}}
add_field("field1", schema={"type": "string"})

These create a field with:

• a string type
• a nullable value
• no default value (i.e. the default is null)

The former is not intuitive, and derives from wanting to be back-compatible, when we moved from untyped extra options.

---

This PR does not change the behaviour of needs_extra_options / add_extra_option, which are anyhow deprecated,
but for:

needs_fields = {"field1": {}}
add_field("field1")

These now create a field with:

• a string type
• a nullable value
• no default value (i.e. the default is null)

---

In order to aide users in transitioning from needs_extra_options to needs_fields,
we want to clearly indicate that the behaviour is different.

i.e. if they want exactly the same behaviour they should use:

needs_extra_options = [{"name": "field1"}]
needs_fields = {"field1": {"nullable": False, "default": ""}}

Therefore, if none of schema, nullable or default is added to an item in needs_fields (or add_field),
then we emit a warning:

Field 'my_field' (from add_field) has no 'schema', 'nullable' or 'default' defined,
which defaults to a string schema with nullable=True and no default.
To aide with backward compatibility please define at least one. [needs.config]

i.e. this would emit a warning:

needs_fields = {"field1": {}}

but these would not:

needs_fields = {"field1": {"nullable": True}}
needs_fields = {"field1": {"default": "abc"}}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.80%. Comparing base (4e10030) to head (ec79a3f).
⚠️ Report is 239 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1645      +/-   ##
==========================================
+ Coverage   86.87%   88.80%   +1.92%     
==========================================
  Files          56       70      +14     
  Lines        6532     9991    +3459     
==========================================
+ Hits         5675     8873    +3198     
- Misses        857     1118     +261     

Flag	Coverage Δ
pytests	88.80% <100.00%> (+1.92%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ubmarco]

LGTM. I think after merge and before release we need documentation pieces that explain the new behavior of needs_fields in contrast to the old world.