Generate Examples -- Continued

[trillium]

This is following on from the work of #940 Might resolve:

• #1744
• #215
• #866

This is currently a draft PR

• Last updated 2024-10-04

Latest updates

• Migrate more type usage from custom to @cursorless/common
• Refactor loadingFixture
• Rebased and fixed merge conflicts
• Resolved @cursorless/common import errors
• Switched to using some @cursorless/common types
  • More will need to be done on this

ToDos Immediate

• [ ] Render clipboard interactions in finalState
• [ ] Resolve type issues
  • [ ] Update generateHtml types
• [ ] Get during state rendering
• [ ] Get pendingDelete state to render
• [ ] Resolve some bugs in the generation around creating selections

ToDos IceBox

• [ ] Create animation out of initialState, during, and finalState
• [ ] Define some kind of internal API so it can be called properly or something like that

Screenshot

Older Screenshots

Current issues

• The during step that shows selection for deletion isn't being hit
  • Haven't updated the code to capture a pendingDeletion selection yet (loadFixtures.ts)
• Color scheme is very bad, will need to pick something better eventually

Older changes

Summary of older changes

• .yml files are successfully loaded via getStaticProps and passed to TestCaseComponentPage
• Updated to Shiki 1.4 from Shiki 0.x
• Brought in CSS from the codepen file linked in #940 (link)
• forked main
• Cherry picked the 3 commits from #940

4e0b81cc SimeonC         All working with tests                                                           
3d2e85ff pre-commit-ci.. [pre-commit.ci] auto fixes from pre-commit.com hooks                             
47ca6352 SimeonC         wip: Delete workspace.json for cherry-pick - git cherry pick 070c870 - Delete .. 

• Moved cursorless-rx/ ... /generate-exmples folder to packages/generate-examples to conform with monorepo
• Updated loadFixture directory to point correctly to cursorless-vscode-e2e recorded folder
• Resolved an error in packages/generate-examples/src/lib/generateHtml.ts @ SelectionParser.parse that made pnpm build fail
  • This might not be the right solution, but it the code it working again
• Populated results from loadFixture with original data -- as of yet unused
• Ran relevant pnpm install/setup commands

Checklist

• [ ] I have added tests
• [ ] I have updated the docs and cheatsheet
• [ ] I have not broken the cheatsheet

[SimeonC]

Thanks for picking this up! If you need any help figuring out what I was doing let me know. I can answer questions but completely have no time to code outside of work commitments 😞

[trillium]

All good, happy to see you're well :)

I'm definitely curious about the decisions you made to choose these libraries and what your vision was for how you wanted this feature to end up.

Recently I was looking for more opportunities for to learn what Cursorless can do and wanted a series of gifs that could demo its capabilities. We've already got all these test files so that seemed like a good place to start, and it turns out you already are plenty of the way there!

[SimeonC]

I think we just had the same desire, while I was learning Cursorless at the start I just wanted something like a "library" of examples of what each command could do. My overall thinking was just a "search command/scope/etc, see examples" type app, but never got past the "just make the HTML work" stage.

I don't think there was anything really "deep" in my decision-making, shiki lib had the ability to give me the objects/arrays that I could manipulate like a "middleware" to add all the highlighting/tokens before rendering the HTML (bonus that it had all the different styling options too). That was really the only decision I made, the rest is all just hard work figuring out how to make overlapping ranges highlight (no multi-ranges is the easy part), hence the tests cause I broke my own code so many times trying to fix it, I suddenly had a better appreciation for TDD 🤣

[trillium]

I think one of my main questions is where this will end up residing. Eg is it gonna be available locally via something like the cheat sheet or is it only online via the docs.

That might be a better question if Andreas or Pokey, but I think it affects how I would explore building this.

My first inclination was to see if anything could be reused from the Docusaurus space:

• Use a build system to turn the test YAML files into MDX
• Create a custom component to render the MDX with all the highlight, ranges, hats info in the grey matter area

If it's going to reside in the cheat sheet then there's no established components, theme etc to start from and something like shiki seems like the right choice.

I honestly have no idea though.

[SimeonC]

Sorry can't help you there, definitely a question for Pokey or Andreas as I didn't really figure out with them where was best.

[AndreasArvidsson]

With the caveat that I haven't really read through the previous pull request we definitely want code examples in the online documentation. We might link from the cheat sheet to the online docs, but I don't think we should actually cram all the documentation into the cheat sheet itself.

[pokey]

A package is the way to go, but I'd recommend starting from a clean checkout of main and then proceeding as follows:

1. Run the steps in the docs for creating a new package (your package is a library, so don't add private in step 4)
2. Lift just the typescript files from #940
3. Take only whatever else you need from #940 to actually make things work. There's a lot of nx cruft in that PR that we don't need anymore

I'm not sure generate-examples is the name I'd go with. I think the primary export of this package should be a react component whose input is a TestCaseFixture. So I think I'd maybe call the package test-case-component? @AndreasArvidsson any thoughts on name?

Fwiw you might want to look at the cheatsheet package, as that's an example of a package whose primary export is a react component

For local testing / experimentation, I'd just add a dummy page to our cursorless-org package so you can use our next.js local dev setup. That's what we do for our cheatsheet (https://github.com/cursorless-dev/cursorless/blob/main/packages/cursorless-org/src/pages/cheatsheet.tsx; actually deployed at https://www.cursorless.org/cheatsheet fwiw) You can just use a dummy fixture for now. See README at https://github.com/cursorless-dev/cursorless/tree/main/packages/cursorless-org for info on how to run local dev server

Once you have a working react component, we can look into getting it into docusaurus using mdx

Lmk if any of the above doesn't make sense, and feel free to drop into another meet-up if you get stuck!

[trillium]

I wanted to keep the commit record from @SimeonC if possible so I reconfigured a few things and it should be working properly. The code still builds in the same way it use to.

Still working on getting a test page to render similar to cheatsheet

[pokey]

ok no worries; looks like you managed to remove all the unnecessary nx stuff so that's fine 👍. I'd move all the lib/ stuff up to src/ dir tho; that lib subdir isn't a pattern we follow

[trillium]

Moved files from src/lib/* to src/*

Though, that same pattern is in packages/cheatsheet/, but it's literally the only one

# find . -type d
./cheatsheet
./cheatsheet/node_modules
./cheatsheet/node_modules/@types
./cheatsheet/node_modules/.bin
./cheatsheet/node_modules/@fortawesome
./cheatsheet/node_modules/@testing-library
./cheatsheet/src
./cheatsheet/src/lib
./cheatsheet/src/lib/components
./cheatsheet/src/lib/sampleSpokenFormInfos
./cheatsheet/src/lib/hooks

[pokey]

Oh interesting. I wonder why we do that there. Probably same reason tbh; was migrated from the old nx semi-mono-repo and never changed

[trillium]

I've been working on mirroring the format of the cheatsheet for a componentsheet, I currently am getting stuck at module/types import from @cursorless/test-case-component but I think it's because I don't understand how to generate types for the test-case-component module.

I was also having a few issues just trying to understand how everything worked together by running cursorless-org pnpm dev and editing text in packages/cheatsheet/src/cheatsheet.tsx to see how changes propagated

I couldn't get anything to change e.g.:

• edit packages/cheatsheet/src/cheatsheet.tsx
  • change h1 to Testy test Cursorless Cheatsheet
  • wait for next to confirm rebuild
  • see no changes on localhost:3000/cheatsheet

[pokey]

I've been working on mirroring the format of the cheatsheet for a componentsheet, I currently am getting stuck at module/types import from @cursorless/test-case-component but I think it's because I don't understand how to generate types for the test-case-component module.

could you possibly elaborate? not sure I follow

I was also having a few issues just trying to understand how everything worked together by running cursorless-org pnpm dev and editing text in packages/cheatsheet/src/cheatsheet.tsx to see how changes propagated

I couldn't get anything to change e.g.:

* edit `packages/cheatsheet/src/cheatsheet.tsx`
  
  * change `h1` to `Testy test Cursorless Cheatsheet`
  * wait for next to confirm rebuild
  * see no changes on `localhost:3000/cheatsheet`

Strange. I could have sworn that used to work 🤔. But yes I confirmed it's not working for me either. Here's a fix https://github.com/cursorless-dev/cursorless/pull/2286

[AndreasArvidsson]

I'm not sure generate-examples is the name I'd go with. I think the primary export of this package should be a react component whose input is a TestCaseFixture. So I think I'd maybe call the package test-case-component? @AndreasArvidsson any thoughts on name?

Test case component sounds good

[pokey]

fwiw https://github.com/cursorless-dev/cursorless/pull/2286 now merged so hot reload should work once you do a git merge

[pokey]

I see there's a conflict in your lockfile. Generally easiest to just revert the lockfile to main and then do a pnpm install

[trillium]

Haven't made that much progress on this since last check in. Got content to render from packages/test-case-component just now though so that's progress!

I do my best to copy the cheatsheet page format from existing code to where applicable to test-case-component Actions thusfar:

• Add dependency test-case-component to cursorless-org copying cheatsheet's format
• Create a text-case-component.tsx file for initial workings
  • Experiencing an error

import {
  TestCaseComponentPage
} from "@cursorless/test-case-component";
//              ~~~~~~~~~~~~~~~~~~~~~~~~~
import Head from "next/head";

// Cannot find module '@cursorless/test-case-component' or its corresponding type declarations.

I originally thought this was blocking me from getting the test-component-sheet page to render but that isn't blocking me any longer.

[trillium]

Thanks for taking some extra time before you all hopped off todays call :)

Notes

cursorless-org/component-sheet page

• get yaml coming into component
• hardcode ok for now
• pass as props
• can add a yaml dependency if needed
  • of type TestCaseFixture
    • let it be type:any for now
• okay to stick with Jest if needed
  • cheatsheet might use Jest?

Test file

• try to get tests to run before making any changes to structure
• Cursorless uses Mocha for most tests
• okay to stick with Jest if needed
  • cheatsheet might use Jest?

[pokey]

can add a yaml dependency if needed

* of type TestCaseFixture
  
  * let it be type:any for now

just to clarify this one: TestCaseComponent should declare the property type as TestCaseFixture, but in the TestCasePage in cursorless-org you should leave it as type any. Make sense?

[trillium]

Yes that makes sense

• Page should import the yaml but not care about the type
  • pass to Component as props
• Component should care about the type to prevent errors

[trillium]

Running into fs import errors when I try to move the yaml fetching into test-case-component:

Error

../test-case-component/src/index.ts:5:0
Module not found: Can't resolve 'fs'
  3 |
  4 | import * as yaml from "js-yaml";
> 5 | import fs from 'fs';
  6 | import path from 'path';
  7 |
  8 | const fixturesDir = path.join(

---

Works for getting the yaml (overview)

// packages/cursorless-org/src/pages/component-sheet.tsx

async function getData() {
  // Contents etc
  
  const data = [...dataActions, ...dataDecorations]
  return data
}

export async function getStaticProps() {
  const data = await getData();
  return { props: { data: data, bodyClasses: cheatsheetBodyClasses } };
}

export function App({data} : { data: any }) {
  return <TestCaseComponentPage data={data} />
}

Errors with `fs` import when geting yaml (overview)

// packages/cursorless-org/src/pages/component-sheet.tsx

import { getData } from '@cursorless/test-case-component';

// Everything else is the same

export async function getStaticProps() {
  const data = await getData();
  return { props: { data: data, bodyClasses: cheatsheetBodyClasses } };
}

export function App({data} : { data: any }) {
  return return <TestCaseComponentPage data={data} />
}

[pokey]

Huh. Why do you want to move it into test-case-component? I'm not sure we want a react component importing node stuff. I would just leave it in component-sheet.tsx for now. We can figure out where the yaml stuff should actually get loaded later. I'd argue the component just gets the TestCaseFixture already loaded, without caring where it comes from, so let's get that part working like we want it to and then figure out build pipeline

[trillium]

I meant the library of test case component, not the react component itself.

I wanted to make the import functions contained in package/test-case-component and then imported and used in getStaticProps.

Otherwise I'll have to start putting functionality into as a lib in /cursorless-org because it has access to fs

At least that's how I'm understanding it now

[trillium]

Basically as soon as I put an export in test-case-component/index.ts that has a 'fs' requirement it breaks even if that function is never called

[pokey]

I would argue for keeping everything node-related in cursorless-org. Could you elaborate on what problem you see with that approach? At the very least, it would solve your problem here, no? But I def may be missing something here; you're the closest to this code so would be good to understand your perspective

[pokey]

Update from meetup: we worked through various import issues and it looks like it should be fine to just keep node-related stuff in cursorless-org package

[trillium]

Thanks for looking into that. I'll make some more progress on this PR and show up to chat about it in a coming meetup :)

[pokey]

Oh just to be clear: my message wasn't referring to any new / recent discussion. I was just referring to the meet-up a week or two ago where you joined and we discussed imports. I always try to leave a quick note on PRs after we discuss them in case they linger long enough for us to forget that we discussed them 😅

[trillium]

Took a break from tech stuff for a minute, will work more on this and come to a Cursorless meeting in the coming days

[trillium]

Made some more progress

Screenshot

There's supposed to be a during step that isn't being hit. Will investigate more later.