Change model dictionaries to json format

[0xMimir]

Short description of current behavior

I have huggingface sentiment analyzer and sentiment explain comes as text in next format:

{'Neutral': 0.6581241488456726, 'Bullish': 0.34162577986717224, 'Bearish': 0.00025005039060488343}

which is not valid json and can't be converted to json or jsonb in postgresql,

This is also how most json is saved in project TRAINING_OPTIONS in table models is save as

{'target': 'sentiment', 'using': {'task': 'text-classification', 'model_name': 'ElKulako/cryptobert', 'input_column': 'content', 'max_length': 512}}

Video or screenshots

No response

Expected behavior

No response

How to reproduce the error

Create a model

CREATE MODEL mindsdb.sentiment_classifier
PREDICT sentiment
USING
  engine = 'huggingface',
  task = 'text-classification',
  model_name = 'cardiffnlp/twitter-roberta-base-sentiment',
  input_column = 'text_short',
  labels = ['negative', 'neutral', 'positive'];

Query the model

SELECT * from mindsdb.sentiment_classifier where text_short = 'this is good day'

sentiment|sentiment_explain                                                                                  |text_short      |
---------+---------------------------------------------------------------------------------------------------+----------------+
positive |{'positive': 0.9830580353736877, 'neutral': 0.014908154495060444, 'negative': 0.002033873461186886}|this is good day|

Anything else?

No response

[tomhuds]

thanks for this @0xMimir - we'll add to our backlog!

[paxcema]

I think there are two issues here, the JSON output from HuggingFace models and the TRAINING_OPTIONS format.

@StpMax @ea-rus should we open a new issue for the latter? We'll take a look at the HF formatting in the meantime.

[ZoranPandovski]

@paxcema should we keep this open?

[paxcema]

Seems important for user experience, so I would say yes.