fix: fix JSONParamType to handle serialization of custom objects

[dalaoqi]

Tracking issue

Closes flyteorg/flyte#5985

Why are the changes needed?

Here are the errors I encountered while trying the cases mentioned above:

• TypeError: the JSON object must be str, bytes or bytearray, not Foo The issue arises when trying to serialize lists or dictionaries that contain custom objects, as non-serializable custom objects cause a TypeError by default. This prevents successful serialization of data structures with instances of user-defined classes, necessitating a solution to convert these objects into a JSON-compatible format.

• AttributeError: 'NoneType' object has no attribute 'literals' If lv.collection is None, attempting to access literals will raise an AttributeError. To mitigate this issue, the updated code introduces additional checks to ensure that the required attributes exist before attempting to access literals.

• Got multiple values The code raises an AssertionError when a duplicate value for an argument is encountered. Both positional and keyword arguments are provided for the same input, it disrupts the normal handling of arguments.

What changes were proposed in this pull request?

This pull request fixes:

• JSON serialization issues for lists and dictionaries containing custom objects by adding support to convert non-serializable objects to JSON-compatible dictionaries. Added default Parameter to json.dumps: Utilized default=lambda o: o.dict in json.dumps calls to convert non-serializable custom objects into a JSON-compatible dictionary format.
• Address a potential AttributeError when attempting to access the literals attribute from an object that might be None.
• Allow keyword arguments to take precedence over positional arguments without interrupting execution.

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

• [ ] I updated the documentation accordingly.
• [ ] All new and existing tests passed.
• [x] All commits are signed-off.

Related PRs

Docs link

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 46.75%. Comparing base (3f0ab84) to head (36f98ed). Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
flytekit/interaction/click_types.py	0.00%	4 Missing :warning:

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #2931       +/-   ##
===========================================
- Coverage   76.33%   46.75%   -29.59%     
===========================================
  Files         199      199               
  Lines       20840    20789       -51     
  Branches     2681     2684        +3     
===========================================
- Hits        15908     9719     -6189     
- Misses       4214    10594     +6380     
+ Partials      718      476      -242     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[dalaoqi]

After revising, I noticed that one of the unit tests, test_both_positional_and_keyword_args_task_raises_error, fails to pass:

def test_both_positional_and_keyword_args_task_raises_error():
    @task
    def t1(a: int) -> int:
        return a

    with pytest.raises(AssertionError, match="Got multiple values for argument"):
        t1(1, a=2)

The test fails because the function does not raise an error as expected.

Upon further consideration, I believe this behavior should not be treated as an error. In this scenario, since the keyword argument is explicitly provided, any conflicting value passed as a positional argument could simply be ignored. commit