cli icon indicating copy to clipboard operation
cli copied to clipboard

Implement `scoreTarget ` as CIL feature for categories

Open BioPhoton opened this issue 8 months ago • 2 comments

User story

As a user I want to have a scoreTarget configuration available to be able to control my KPIs better.

At the moment plugin-coverage implements something similar. A adopted version of this feature would be helpful as global CLI feature.

The behaviour should help to specify a value that acts as a gate in audit/score values to mark them as fulfilled/passed.

Existing similar ideas:

  • The largest-contentful-paint in the plugin-lighthouse plugin is scores with 1 (100) if the value is smaller than 2000 ms
  • The plugin-coverage plugin provides a perfectScoreThreshold option, to configure the score to be 1 (perfect) for all audits audits of the plugin.
  • The CategoryConfig object has a isBinary property that adds a checkmark if the score is equal to 1 (100)

This feature is applicable to the following types/data objects:

  • AuditOtuput
  • CategoryConfig

Relates parts Portal

  • Score display in the portal
    • The displayed score
    • The or mark on the score ATM configured over isBinary Relates parts CLI
  • The displayed score of AuditOutputs in all report formats
    • md as well as the json
    • and the stdout in the terminal
  • The displayed score of a category score in all report formats
    • md as well as the json
    • and the stdout in the terminal
  • Shared type to implement this property to all plugins

Related PRs:

Acceptance criteria

  • [x] https://github.com/code-pushup/cli/issues/592
graph TD
    #592 --> CLI_Preparation
    CLI_Preparation --> CLI_Client_Update
    Portal_Model_Adoption --> Portal_UI_Update
    Portal_Model_Adoption --> CLI_Client_Update

PR1 - CLI preparation

  • [ ] rename isBinary to targetScore and change the type to nonnegativeNumberSchema.max(1)
  • (TBD) adjust executePlugin to apply the new scoring logic after the normalization (for audits)
  • [ ] adopt categoryToGQL functions to convert the name back and the floating number back to boolean and mark it with a comment

PR2 - Portal model adoption

  • [ ] accept a number from 0-1 for new property targetScore and remove isBinary
  • [ ] release portal-client

PR3 - CLI portal-client update

  • [ ] adopt categoryToGQL functions for new type upload
  • [ ] add targetScore styling to all places
    • [ ] for categories, scoreTarget doesn't change the score (or score colour), but affects whether ✅ or ❌ is displayed
    • (TBD) for audits, scoreTarget means score is changed to perfect score, because it affects the category/group scoe calculation

PR4 - Portal UI updates

  • [ ] adopt score card for new targetScore logic

Implementation details


CategoryConfig adoption

const model = z.object({
      scoreTarget: nonnegativeNumberSchema
        .describe('Is this a binary category (i.e. only a perfect score considered a "pass")?')
        .max(1)
        .optional(),
    }),

scoreTarget suggestion

export const scoreTargetSchema = z.union([
  nonnegativeNumberSchema,
  z.record(z.string(), nonnegativeNumberSchema)
])
  .describe('one target treshold for all audits or a record with audit slug and treshold');
export type ScoreTarget = z.infer<typeof scoreTargetSchema>;

Part of other PR: #721

Related issues:

#592, #721

BioPhoton avatar Jun 15 '24 19:06 BioPhoton