scala-steward icon indicating copy to clipboard operation
scala-steward copied to clipboard

Rework `Update` JSON encoders and decoders

Open fthomas opened this issue 1 year ago • 5 comments

This is an attempt to clarify the various Encoder and Decoder instances that make up Encoder[Update] and Decoder[Update] that are used by the PullRequestRepository for writing/reading updates to/from disk.

I had a hard time understanding what was going on because

  • nearly all of the instances are implicit (for derivation) but only the instances for Update are used by other parts of the program
  • semi-derivation was mixed with manual modifications
  • decoders for compatibility with old caches depend on derivation and are defined ad-hoc

In this change I tried to bring more clarity to these instances by

  • giving compatibility decoders their own name
  • forgoing semi-derivation and define all instances manually. IMHO this makes it easier to evolve them if there are compatibility constraints because of old caches
  • making every instance except Encoder[Update] and Decoder[Update] private and non-implicit
  • only testing them via the public Encoder[Update] and Decoder[Update] instances

This might be a controversial change and I ackknowledge that one gains more understanding by doing this rework. I'm therefore biased that these changes actually clarify anything. I'm fine with letting this age for a while before merging to see if I still feel this is an improvement then.

fthomas avatar Jan 07 '24 08:01 fthomas

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (600d2a3) 91.18% compared to head (026eb23) 91.25%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3261      +/-   ##
==========================================
+ Coverage   91.18%   91.25%   +0.06%     
==========================================
  Files         167      167              
  Lines        3405     3432      +27     
  Branches      306      304       -2     
==========================================
+ Hits         3105     3132      +27     
  Misses        300      300              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jan 07 '24 08:01 codecov[bot]

@fthomas do we still need to keep compatibility for old caches? wouldn't have those already expire and one with the new encoder be created?

alejandrohdezma avatar Jan 07 '24 08:01 alejandrohdezma

wouldn't have those already expire and one with the new encoder be created?

This is true for caches that have been updated since https://github.com/scala-steward-org/scala-steward/pull/2898 (which was the last change to the Update encoding). But we can't be sure there are no older caches lingering around that will be used again in the future. If we break compatibility with those, the people who run the Scala Steward instance need to remove these old caches for it to work again. I know that this situation is very unlikely, but is unsatisfactory that it could exist.

On the other hand it is very tempting to just semi-derive all our instances again and remove the compatibility decoders. It would simplify that part of the code base as much as possible.

fthomas avatar Jan 07 '24 10:01 fthomas

wouldn't have those already expire and one with the new encoder be created?

This is true for caches that have been updated since #2898 (which was the last change to the Update encoding). But we can't be sure there are no older caches lingering around that will be used again in the future. If we break compatibility with those, the people who run the Scala Steward instance need to remove these old caches for it to work again. I know that this situation is very unlikely, but is unsatisfactory that it could exist.

On the other hand it is very tempting to just semi-derive all our instances again and remove the compatibility decoders. It would simplify that part of the code base as much as possible.

That makes sense. Thanks for the clarification 😊

alejandrohdezma avatar Jan 07 '24 10:01 alejandrohdezma

The other alternative would be to have matching case classes for all different versions, then use semiauto and convert afterwards into the current version.

That sounds interesting! I'll give it a try.

fthomas avatar Jan 08 '24 07:01 fthomas