venice icon indicating copy to clipboard operation
venice copied to clipboard

[doc] Venice Large Record Value Chunking Diagram

Open KaiSernLim opened this issue 1 year ago • 3 comments

Summary

Write-path diagram for large record value chunking in Venice.

Venice Large Record Value Chunking

How was this PR tested?

http://kaisernlim.github.io/venice/docs/dev_guide/venice_write_path

Does this PR introduce any user-facing changes?

  • [X] No. You can skip the rest of this section.

KaiSernLim avatar Aug 09 '24 17:08 KaiSernLim

Great work and diagram. You can build it on your personal fork and share the url with us for review.

Thanks @adamxchen. Added to http://kaisernlim.github.io/venice/docs/dev_guide/venice_write_path.

The order of manifest and chunks in diagrams may imply the order. IIRC, we get chunks first and then manifest. In the diagram, it looks like manifest first and then chunks. Can we double-check this?

I agree that the chunks are created first, then the manifest. For an arrow are pointed to the right, the chunks are on the right and the manifest is to the left. So, if all objects were to follow the direction of the arrow to the right, the chunks would arrive first.

Screenshot 2024-08-09 at 11 49 03 AM

I was thinking about creating an animation to illustrate this even further, because the diagram is really large.

KaiSernLim avatar Aug 09 '24 18:08 KaiSernLim

Thanks a lot for working on this! I am not familiar with this "Whimsical" tool, but I wonder, what is the workflow for modifying the diagram? Is there some sort of "source file" (in XML or whatever format) containing the definition of this diagram, that anybody could load into Whimsical, if they were to need to edit this diagram? For example, the other tool I've been using, draw.io, does let us save the source file, and I've been checking in those into the repo, so others could pick them up and edit them.

FelixGV avatar Aug 09 '24 19:08 FelixGV

Thanks a lot for working on this! I am not familiar with this "Whimsical" tool, but I wonder, what is the workflow for modifying the diagram?

@FelixGV In the free version, you'll have to create an account and copy the diagram / board on the web app (with a link) to get a copy that you can edit yourself. We may be able to secure a team board for the LinkedIn organization on Whimsical, in which case we could all collaborate and edit on the same board. Otherwise, it's behind a paid subscription service.

KaiSernLim avatar Aug 12 '24 18:08 KaiSernLim