Enhanced developer documentation

[jedevc]

This PR attempts to rework some of the existing developer documentation, while adding some additional pages.

The driving idea behind this is to have a central location in ./docs/dev where potential buildkit contributors and integrators can read and learn about the internal mechanisms without needing to dive into the code for all of the details. Ideally, we'd work to build this out over time even more, especially around areas that are especially complex, are areas that we expect developers to integrate with, such as attestations, frontends, etc.

@hinshun, @tonistiigi tells me you're the original author of the "Buildkit developer guide" google doc that I've taken the content for the request-lifecycle.md and dockerfile-llb.md pages from (though I've changed some of the examples to be more up-to-date). I'd like to add you as a co-author to those commits, could you confirm if you're happy with that?

[hinshun]

@jedevc Happy to be added. It's good to see more efforts being put towards BuildKit documentation. :+1:

@coryb also worked on some parts of the document.

[jedevc]

@coryb could you also confirm you'd be happy to be added as a co-author on the relevant import commits?

@hinshun is there a particular email address you'd like? In the buildkit history I see Edgar Lee <[email protected]>, is that still good?

[hinshun]

@jedevc Please use [email protected], I'm no longer at Netflix.

[jedevc]

Should be ready now:

• @coryb and @hinshun added as co-authors
• Comments resolved
• Diagrams converted to mermaid to easily view in-browser