♻️ Store unset extra options as `Undefined` and remove from `needs.json`

[chrisjsewell]

As discussed in more detail at https://github.com/useblocks/sphinx-needs/issues/1082#issuecomment-1948796243, at present every stored need (dict) item in a project is required to have every key present. In particular this is required so that filter strings (which are evaluated as Python code) can be evaluated for all needs; for example "xxx == 1 " would raise a NameError if xxx was not a key of the need.

As discussed also in https://github.com/useblocks/sphinx-needs/issues/1122#issue-2147120566, this has meant that currently all fields added to NEEDS_CONFIG.extra_options, are added to every need item, with a default of "". Storing as a "" makes it difficult to tell if this was specifically set by the user or not, and consequently whether it can be removed from the needs.json

By contrast, jinja2 has the concept of https://jinja.palletsprojects.com/en/3.0.x/api/#undefined-types, by default assigned to any variable not provided as part of the context.

This PR changes the default to an Undefined instance and, when writing to a needs.json, removes and keys with this value. As can be seen from the tests, for a standard need item, this removes 18 fields.

This solution should be "minmally breaking", in that mainly Undefined acts the same as "". Although there may some rare exceptions, for example, the calc_sum filter function expected float(value) to raise a ValueError, but it now raises an UndefinedError.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 89.18919% with 8 lines in your changes missing coverage. Please review.

Project coverage is 85.90%. Comparing base (cf5598f) to head (2d3e785). Report is 37 commits behind head on master.

Files	Patch %	Lines
sphinx_needs/api/need.py	85.71%	7 Missing :warning:
sphinx_needs/data.py	94.73%	1 Missing :warning:

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1125   +/-   ##
=======================================
  Coverage   85.90%   85.90%           
=======================================
  Files          56       56           
  Lines        6511     6557   +46     
=======================================
+ Hits         5593     5633   +40     
- Misses        918      924    +6     

Flag	Coverage Δ
pytests	85.90% <89.18%> (+<0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[chrisjsewell]

superceded by #1232