sanity icon indicating copy to clipboard operation
sanity copied to clipboard

refactor: improve release note workflow

Open kmelve opened this issue 8 months ago • 10 comments
trafficstars

Summary

This PR refactors the release notes template generator script to improve the workflow, ensure Node.js compatibility, and enhance the documentation. The script now uses a more structured approach for generating release notes and changelog entries.

Key Changes

Code Structure and Dependencies

  • Converted from ES module imports to CommonJS requires for better compatibility
  • Added proper TypeScript type annotations
  • Structured the code with clear function separation and error handling

Functionality Enhancements

  • Added support for creating Sanity documents with changelog entries
  • Implemented automatic categorization of commits into features and bugfixes
  • Added dry run mode for template generation without document creation
  • Added debug mode for inspecting document structure

Template Generation

  • Changed from a static template to a dynamically generated one based on commit history
  • Added support for custom title, product tag, and version specification
  • Improved the GitHub release template to be more concise with links to detailed changelog

Authentication and Configuration

  • Hardcoded project ID and dataset for consistency
  • Added support for retrieving auth token from Sanity CLI config
  • Added option to provide a direct token via command line

Documentation

  • Created comprehensive README with usage examples and options
  • Added documentation on authentication, workflow, and URL formats
  • Included examples of command usage with different flags

Testing

  • Verified script execution with Node.js v23
  • Tested in dry run mode to ensure proper template generation
  • Validated document creation functionality
  • Confirmed URL generation for both public-facing and internal Studio links

Benefits

  • Improved Developer Experience: Clearer, more structured release notes generation
  • Better Compatibility: Works with latest Node.js versions without additional configuration
  • Enhanced Documentation: Comprehensive guide for using the script
  • Streamlined Workflow: Automated creation of changelog entries in Sanity Studio

Technical Details

The script now follows a more modular approach with separate functions for different aspects of the release notes generation process. It uses CommonJS requires for better compatibility with different Node.js versions and provides a more robust error handling mechanism.

kmelve avatar Mar 04 '25 00:03 kmelve

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 23, 2025 0:25am
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 23, 2025 0:25am
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 23, 2025 0:25am
3 Skipped Deployments
Name Status Preview Comments Updated (UTC)
e2e-studio ⬜️ Ignored (Inspect) Apr 23, 2025 0:25am
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Apr 23, 2025 0:25am
test-next-studio ⬜️ Ignored (Inspect) Apr 23, 2025 0:25am

vercel[bot] avatar Mar 04 '25 00:03 vercel[bot]

No changes to documentation

github-actions[bot] avatar Mar 04 '25 00:03 github-actions[bot]

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 43.34% 57296 / 132188
🔵 Statements 43.34% 57296 / 132188
🔵 Functions 47.87% 2871 / 5997
🔵 Branches 79.67% 10711 / 13444
File CoverageNo changed files found.
Generated in workflow #34041 for commit 318b9a6 by the Vitest Coverage Report Action

github-actions[bot] avatar Mar 04 '25 00:03 github-actions[bot]

⚡️ Editor Performance Report

Updated Wed, 23 Apr 2025 12:31:29 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 21.5 efps (47ms) 25.0 efps (40ms) -7ms (-14.0%)
article (body) 46.9 efps (21ms) 46.9 efps (21ms) -0ms (-0.0%)
article (string inside object) 24.7 efps (41ms) 26.3 efps (38ms) -3ms (-6.2%)
article (string inside array) 22.2 efps (45ms) 23.3 efps (43ms) -2ms (-4.4%)
recipe (name) 47.6 efps (21ms) 47.6 efps (21ms) +0ms (-/-%)
recipe (description) 54.1 efps (19ms) 54.1 efps (19ms) +0ms (-/-%)
recipe (instructions) 99.9+ efps (6ms) 99.9+ efps (6ms) +0ms (-/-%)
synthetic (title) 18.9 efps (53ms) 18.0 efps (56ms) +3ms (+4.7%)
synthetic (string inside object) 18.9 efps (53ms) 18.5 efps (54ms) +1ms (+1.9%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 47ms 53ms 61ms 317ms 549ms 12.1s
article (body) 21ms 25ms 43ms 268ms 592ms 6.4s
article (string inside object) 41ms 44ms 52ms 162ms 292ms 7.4s
article (string inside array) 45ms 48ms 51ms 279ms 404ms 7.7s
recipe (name) 21ms 22ms 25ms 41ms 0ms 7.2s
recipe (description) 19ms 19ms 20ms 21ms 0ms 4.6s
recipe (instructions) 6ms 8ms 10ms 28ms 0ms 3.1s
synthetic (title) 53ms 57ms 62ms 307ms 1115ms 15.2s
synthetic (string inside object) 53ms 57ms 60ms 216ms 989ms 8.3s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 40ms 42ms 45ms 147ms 205ms 10.6s
article (body) 21ms 26ms 48ms 115ms 173ms 6.3s
article (string inside object) 38ms 41ms 45ms 249ms 156ms 7.0s
article (string inside array) 43ms 48ms 57ms 300ms 391ms 7.6s
recipe (name) 21ms 23ms 28ms 51ms 6ms 7.4s
recipe (description) 19ms 20ms 22ms 33ms 0ms 4.7s
recipe (instructions) 6ms 8ms 9ms 22ms 0ms 3.0s
synthetic (title) 56ms 59ms 64ms 210ms 1026ms 14.8s
synthetic (string inside object) 54ms 57ms 67ms 346ms 806ms 8.7s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

github-actions[bot] avatar Mar 04 '25 00:03 github-actions[bot]

Component Testing Report Updated Mar 7, 2025 2:51 AM (UTC)

❌ Failed Tests (1) -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 8s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 13s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ❌ Failed (Inspect) 1m 15s 5 0 1
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 43s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 24s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 14s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 1m 6s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 1m 29s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 44s 21 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 13s 3 9 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 25s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 1m 35s 21 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

github-actions[bot] avatar Mar 04 '25 00:03 github-actions[bot]

@bjoerge Thanks for the feedback, and shame on me for not properly reviewing AI vibe code. But I have done a bit of refactoring.

  • The changelog entry is supposed to be edited from the raw commit messages that's inserted.
  • From now on, we can copy-paste the github release note from the studio.
  • I didn't figure out how to automate the semver version, so for now it has to be inserted manually.
  • I have noticed that we have "Notes for release" in the PRs, but these are not surfaced by the old or this new releaseNote script. Maybe there is a way to get those into the mix?

Changes

  1. Added GitHub release notes template as a field in the apiChange document:

    • Added githubReleaseNote field that includes:
      • Brief introduction
      • Link to full changelog
      • Installation instructions for multiple package managers (npm, pnpm, yarn, and bun)
      • Link to upgrade guide
      • Full commit log
  2. Improved TypeScript type safety:

    • Added PortableTextSpan and PortableTextBlock interfaces for proper typing of Portable Text structures
    • Added as const type assertions for _type fields to ensure literal type checking
    • Explicitly typed the return value of generateLongDescription
  3. Code cleanup:

    • Removed redundant console logging of GitHub release template
    • Simplified dry run output handling

Remaining Issues

  • There is a linter error indicating that SANITY_WEB_AUTH_TOKEN needs to be added to the root turbo.json file's globalEnv array, which it is. This should be addressed in a separate PR.

Testing

The script can be tested using:

# Dry run to verify document structure
node --no-warnings --experimental-strip-types scripts/printReleaseNotesTemplate.ts --dryRun
# Create actual documents
node --no-warnings --experimental-strip-types scripts/printReleaseNotesTemplate.ts

kmelve avatar Mar 07 '25 02:03 kmelve

@kmelve could you do a rebase against next and update the PR title so it matches semantic release? 🙏 ref: https://github.com/sanity-io/sanity/actions/runs/13712597273/job/38351775490?pr=8826

I think the other CI failures are flake and will go away after a rebase

bjoerge avatar Mar 25 '25 16:03 bjoerge

@kmelve could you do a rebase against next and update the PR title so it matches semantic release? 🙏 ref: https://github.com/sanity-io/sanity/actions/runs/13712597273/job/38351775490?pr=8826

I think the other CI failures are flake and will go away after a rebase

done!

kmelve avatar Mar 25 '25 16:03 kmelve

📊 Playwright Test Report

Download Full E2E Report

This report contains test results, including videos of failing tests.

github-actions[bot] avatar Mar 25 '25 16:03 github-actions[bot]

🧪 E2E Preview environment

🔑 Environment Variables for Local Testing

This is the preview URL for the E2E tests: https://e2e-studio-d53zjof2k.sanity.dev

To run the E2E tests locally, you can use the following environment variables, then run pnpm test:e2e --ui to open the Playwright test runner.

💬 Remember to build the project first with pnpm build:e2e.

  SANITY_E2E_PROJECT_ID=ittbm412
  SANITY_E2E_BASE_URL=https://e2e-studio-d53zjof2k.sanity.dev
  SANITY_E2E_DATASET="update depending the project you want to test (pr-chromium-8826 || pr-firefox-8826 )"
  SANITY_E2E_DATASET_CHROMIUM=pr-chromium-8826
  SANITY_E2E_DATASET_FIREFOX=pr-firefox-8826

github-actions[bot] avatar Apr 23 '25 12:04 github-actions[bot]