breadboard icon indicating copy to clipboard operation
breadboard copied to clipboard

Published 20 hours ago • verified publisher icon breadboard-ai

Migrate old boards to new syntax

Open Mearman opened this issue 1 year ago • 10 comments

Current references to old syntax:

Typescript

  • https://github.com/breadboard-ai/breadboard/issues/2841
  • ~./packages/graph-playground/src/boards/endless-debate-with-voice.ts~
  • ~./packages/graph-playground/src/boards/endless-debate.ts~
  • https://github.com/breadboard-ai/breadboard/issues/2842
  • https://github.com/breadboard-ai/breadboard/issues/2703
  • https://github.com/breadboard-ai/breadboard/issues/2843
  • https://github.com/breadboard-ai/breadboard/issues/2543
  • https://github.com/breadboard-ai/breadboard/issues/2844
  • ~./packages/breadboard-web/src/boards/single-turn.ts~
  • https://github.com/breadboard-ai/breadboard/issues/2845
  • https://github.com/breadboard-ai/breadboard/issues/2718
  • https://github.com/breadboard-ai/breadboard/issues/2650
  • https://github.com/breadboard-ai/breadboard/issues/2846
  • https://github.com/breadboard-ai/breadboard/issues/2326
  • https://github.com/breadboard-ai/breadboard/issues/2327
  • https://github.com/breadboard-ai/breadboard/issues/2325
  • https://github.com/breadboard-ai/breadboard/issues/2847
  • https://github.com/breadboard-ai/breadboard/issues/2848
  • https://github.com/breadboard-ai/breadboard/issues/2817
  • https://github.com/breadboard-ai/breadboard/issues/2652
  • https://github.com/breadboard-ai/breadboard/issues/2849
  • https://github.com/breadboard-ai/breadboard/issues/2720
  • https://github.com/breadboard-ai/breadboard/issues/2850
  • [x] ./packages/breadboard-web/src/boards/math.ts
  • https://github.com/breadboard-ai/breadboard/issues/2851
  • https://github.com/breadboard-ai/breadboard/issues/2852
  • https://github.com/breadboard-ai/breadboard/issues/2853
  • https://github.com/breadboard-ai/breadboard/issues/2854
  • https://github.com/breadboard-ai/breadboard/issues/2855
  • https://github.com/breadboard-ai/breadboard/issues/2532
  • https://github.com/breadboard-ai/breadboard/issues/2818
  • https://github.com/breadboard-ai/breadboard/issues/2488
  • https://github.com/breadboard-ai/breadboard/issues/2489

Kits

  • [ ] ./packages/palm-kit/src/index.ts
  • https://github.com/breadboard-ai/breadboard/issues/2340

Nodes

  • [ ] ./packages/palm-kit/src/nodes/generate-text.ts
  • [ ] ./packages/palm-kit/src/nodes/embed-text.ts

Boards to delete

  • [x] ~./packages/breadboard-web/src/boards/json-generator.ts~
  • [x] ~./packages/breadboard-web/src/boards/json-validator.ts~
  • [x] ~./packages/breadboard-web/src/boards/retry.ts~
  • [x] ~./packages/breadboard-web/src/boards/healer.ts~
  • [x] ~./packages/breadboard-web/src/boards/math-chain.ts~
  • [x] ~./packages/breadboard-web/src/boards/math-imperative.ts~
  • [x] ~./packages/breadboard-web/src/boards/mock-text-generator.ts~
  • https://github.com/breadboard-ai/breadboard/issues/2856
  • [x] ~./packages/breadboard-web/src/boards/palm-text-generator.ts~
  • [x] ~./packages/breadboard-web/src/boards/retry.ts~
  • [x] ./packages/breadboard-web/src/boards/ask-user.ts
  • [x] ~./packages/breadboard-web/src/boards/board-as-function.ts~
  • [x] ~./packages/breadboard-web/src/boards/board-caller.ts~
  • https://github.com/breadboard-ai/breadboard/issues/2857
  • [x] ~./packages/breadboard-web/src/boards/gemini-pro-vision.ts~
  • [x] ./packages/breadboard-web/src/boards/agent.ts
  • [x] ~./packages/breadboard-web/src/boards/build-example.ts~
  • [x] ~./packages/breadboard-web/src/boards/choices.ts~
  • [x] ~./packages/breadboard-web/src/boards/json-agent.ts~
  • [x] ~./packages/breadboard-web/src/boards/data-board.ts~
  • [x] ~./packages/breadboard-web/src/boards/single-turn.ts~

Boards/projects to deprecate (no need to convert)

  • [ ] ./packages/coffee-bot-board/src/menu-agent.ts
  • [ ] ./packages/coffee-bot-board/src/menu-summary-agent.ts
  • [ ] ./packages/coffee-bot-board/src/order-agent.ts
  • [ ] ./packages/coffee-bot-board/src/schemish-generator.ts
  • [ ] ./packages/coffee-bot-board/src/template.ts
  • [ ] ./packages/coffee-bot-board/src/v1.ts
  • [ ] ./packages/graph-playground/src/boards/auto-simple-prompt.ts
  • [ ] ./packages/graph-playground/src/boards/call-react-with-lambdas.ts
  • [ ] ./packages/graph-playground/src/boards/call-react-with-slot.ts
  • [ ] ./packages/graph-playground/src/boards/find-file-by-similarity.ts
  • [ ] ./packages/graph-playground/src/boards/google-news.ts
  • [ ] ./packages/graph-playground/src/boards/infer-query.ts
  • [ ] ./packages/graph-playground/src/boards/just-search.ts
  • [ ] ./packages/graph-playground/src/boards/math-retry.ts
  • [ ] ./packages/graph-playground/src/boards/math.ts
  • [ ] ./packages/graph-playground/src/boards/output-secret.ts
  • [ ] ./packages/graph-playground/src/boards/pinecone-load.ts
  • [ ] ./packages/graph-playground/src/boards/pinecone-query.ts
  • [ ] ./packages/graph-playground/src/boards/react-with-include.ts
  • [ ] ./packages/graph-playground/src/boards/react-with-jsonata.ts
  • [ ] ./packages/graph-playground/src/boards/react-with-lambdas.ts
  • [ ] ./packages/graph-playground/src/boards/react-with-slot.ts
  • [ ] ./packages/graph-playground/src/boards/react.ts
  • [ ] ./packages/graph-playground/src/boards/relationship-json.ts
  • [ ] ./packages/graph-playground/src/boards/retry.ts
  • [ ] ./packages/graph-playground/src/boards/search-summarize.ts
  • [ ] ./packages/graph-playground/src/boards/simple-prompt.ts
  • [ ] ./packages/graph-playground/src/boards/simplest.ts
  • [ ] ./packages/graph-playground/wild/syntax/prop-6-accumulating-context.ts
  • [ ] ./packages/graph-playground/wild/syntax/prop-6-breadboard.ts
  • [ ] ./packages/graph-playground/wild/syntax/wires-proposal-2.ts
  • [ ] ./packages/graph-playground/wild/syntax/wires-proposal-5.ts
  • [ ] ./packages/pinecone-kit/src/boards/config.ts
  • [ ] ./packages/pinecone-kit/src/boards/query.ts
  • [ ] ./packages/pinecone-kit/src/boards/upsert.ts
  • [ ] ./packages/pinecone-kit/src/boards/vector.ts

Tests

  • [ ] ./packages/breadboard-cli/tests/index.ts
  • [ ] ./packages/breadboard-server/tests/server.ts
  • [ ] ./packages/breadboard/tests/board.ts
  • [ ] ./packages/breadboard/tests/kits.ts
  • [ ] ./packages/breadboard/tests/remote/http.ts
  • [ ] ./packages/breadboard/tests/remote/proxy.ts
  • [ ] ./packages/breadboard/tests/remote/transport.ts
  • [ ] ./packages/breadboard/tests/remote/worker.ts
  • [ ] ./packages/breadboard/tests/run.ts
  • [ ] ./packages/core-kit/tests/map.ts
  • [ ] ./packages/create-breadboard-kit/assets/tests/echo.ts
  • [ ] ./packages/create-breadboard-kit/assets/tests/index.ts
  • [ ] ./packages/node-nursery-web/tests/transform-stream.ts

Javascript

  • [ ] ./packages/breadboard-web/build/boards/healer.js
  • [ ] ./packages/breadboard-web/build/boards/json-generator.js
  • [ ] ./packages/breadboard-web/build/boards/json-validator.js
  • [ ] ./packages/breadboard-web/build/boards/retry.js
  • [ ] ./packages/breadboard/docs/tutorial/google-news-headlines.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-1.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-2.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-3.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-4.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-5.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-6a.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-7a.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-7b.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-8.js
  • [ ] ./packages/breadboard/docs/tutorial/tutorial-9.js
  • [ ] ./packages/graph-playground/examples/call-react-probe.js
  • [ ] ./packages/graph-playground/examples/load-react-wire-news.js
  • [ ] ./packages/graph-playground/examples/tools.js
  • [ ] ./packages/graph-playground/wild/memory/endless-debate-memory.js
  • [ ] ./packages/graph-playground/wild/memory/memory-pattern.js
  • [ ] ./packages/graph-playground/wild/memory/react-memory-pattern.js
  • [ ] ./packages/graph-playground/wild/syntax/wires-proposal-3-accumulating-context-b.js

Markdown

  • [ ] ./packages/breadboard/README.md
  • [ ] ./packages/breadboard/docs/api/README.md
  • [ ] ./packages/breadboard/docs/api/classes/Board.md
  • [ ] ./packages/breadboard/docs/kits/build-a-kit.md
  • [ ] ./packages/breadboard/docs/kits/libraries-as-kits.md
  • [ ] ./packages/breadboard/docs/nodes.md
  • [ ] ./packages/breadboard/docs/tutorial/README.md
  • [ ] ./packages/breadboard/docs/wires.md
  • [ ] ./packages/website/src/docs/kits/core.md

Mearman avatar May 22 '24 12:05 Mearman

Amazing! Let me go over the boards real quick and split out the ones we just want to delete.

dglazkov avatar May 22 '24 17:05 dglazkov

Awesome!

A prerequisite for converting a board is to migrate all of the nodes it uses first. (Note that migrated nodes work with both the old and new syntax, so this can happen incrementally). It would be cool to audit all of our nodes, too, and even cooler if there was a count of how often they are used in boards (after deleting the boards we don't care about, of course) to help prioritize.

aomarks avatar May 22 '24 17:05 aomarks

@aomarks Like I've got the gemini-generator board as an exemplar style with the new syntax. Do you have one for the nodes?

Mearman avatar May 23 '24 07:05 Mearman

@aomarks I'll see if I can get a usable dependency graph

Mearman avatar May 23 '24 07:05 Mearman

@aomarks I'll see if I can get a usable dependency graph

On second thought, I wouldn't spend too much time on analyzing the node dependencies, since the other approach is just to pick off one board that we know we want to keep, try to convert all its nodes, and repeat :)

@aomarks Like I've got the gemini-generator board as an exemplar style with the new syntax. Do you have one for the nodes?

Yep! Any file that contains "defineNodeType" is an example, but some interesting ones I recommend looking at are:

fetch: https://github.com/breadboard-ai/breadboard/blob/main/packages/core-kit/src/nodes/fetch.ts

template: https://github.com/breadboard-ai/breadboard/blob/main/packages/template-kit/src/nodes/prompt-template.ts (along with its tagged-template-literal helper https://github.com/breadboard-ai/breadboard/blob/main/packages/template-kit/src/nodes/prompt-template-tag.ts)

runJavascript: https://github.com/breadboard-ai/breadboard/blob/main/packages/core-kit/src/nodes/run-javascript.ts

aomarks avatar May 23 '24 16:05 aomarks

On second thought, I wouldn't spend too much time on analyzing the node dependencies, since the other approach is just to pick off one board that we know we want to keep, try to convert all its nodes, and repeat :)

Yeah I just had a look to see if I could pick them out with Madge. Not worth it

Mearman avatar May 24 '24 08:05 Mearman

@dglazkov I've been removing the boards which are listed for deletion, but I found a few which are currently used by other boards:

  • The 'Ask User' board (./packages/breadboard-web/src/boards/ask-user.ts) and the 'Agent' board (./packages/breadboard-web/src/boards/agent.ts) are both used by the 'Single turn' board (./packages/breadboard-web/src/boards/single-turn.ts).
  • The OpenAI Chunk Transformer board (./packages/breadboard-web/src/boards/openai-chunk-transformer.ts) is used by the 'OpenAI GTP 3.5 Turbo' board (./packages/breadboard-web/src/boards/openai-gpt-35-turbo.ts) and the 'OpenAI GTP 4 Vision' board (./packages/breadboard-web/src/boards/openai-gpt-4-vision.ts).
  • The 'Gemini Generator' board (./packages/breadboard-web/src/boards/gemini-generator.ts) is used by the 'Text Generator' board (./packages/breadboard-web/src/boards/text-generator.ts) and the 'Simplest' board (./packages/breadboard-web/src/boards/simplest.ts). Just to note, the 'Text Generator' board used two other boards which were removed (./packages/breadboard-web/src/boards/mock-text-generator.ts and ./packages/breadboard-web/src/boards/palm-text-generator.ts), so once the 'Gemini Generator' gets removed then the only generator option available will be the 'OpenAI GTP 3.5 Turbo' board.

Do you have any thoughts on deleting or updating any of those boards?

TinaNikou avatar May 30 '24 10:05 TinaNikou

Thank you!!

  • The 'Ask User' board (./packages/breadboard-web/src/boards/ask-user.ts) and the 'Agent' board (./packages/breadboard-web/src/boards/agent.ts) are both used by the 'Single turn' board (./packages/breadboard-web/src/boards/single-turn.ts).

Yes. Delete them all.

  • The OpenAI Chunk Transformer board (./packages/breadboard-web/src/boards/openai-chunk-transformer.ts) is used by the 'OpenAI GTP 3.5 Turbo' board (./packages/breadboard-web/src/boards/openai-gpt-35-turbo.ts) and the 'OpenAI GTP 4 Vision' board (./packages/breadboard-web/src/boards/openai-gpt-4-vision.ts).

Let's leave those.

  • The 'Gemini Generator' board (./packages/breadboard-web/src/boards/gemini-generator.ts) is used by the 'Text Generator' board (./packages/breadboard-web/src/boards/text-generator.ts) and the 'Simplest' board (./packages/breadboard-web/src/boards/simplest.ts). Just to note, the 'Text Generator' board used two other boards which were removed (./packages/breadboard-web/src/boards/mock-text-generator.ts and ./packages/breadboard-web/src/boards/palm-text-generator.ts), so once the 'Gemini Generator' gets removed then the only generator option available will be the 'OpenAI GTP 3.5 Turbo' board.

Sounds good.

I'd love to modernize the openai boards -- just make them work really well and simplify using the new syntax. If you're up for it 🙏🏻

dglazkov avatar May 31 '24 02:05 dglazkov

@dglazkov Hello! I recall you mentioned in discussion #2542 (https://github.com/orgs/breadboard-ai/discussions/2542#:~:text=We%20can%20just,1) that the Gemini Pro Vision board will be deleted, since the model has been deprecated. There are two boards that are awaiting migration that use this currently, Drawable (./packages/example-boards/src/boards/playground/drawable.ts) and Webcam (./packages/example-boards/src/boards/playground/webcam.ts), so I was wondering what you recommend we should do with these?

TinaNikou avatar Aug 01 '24 10:08 TinaNikou

The text one should work just fine for those! A drop-in replacement, I believe.

dglazkov avatar Aug 02 '24 04:08 dglazkov

Folding this into #3232

dglazkov avatar Sep 18 '24 23:09 dglazkov