Package documentation/article collaboration

[jimgar]

Hey @mdneuzerling, I have a proposal for you.

I’d like to collaborate on reworking/expanding the package documentation and articles/vignettes.

I think for people like me, who are “not-beginners” at R, but are totally new to Data Engineering, a guide/articles with the following would help make lambdr more accessible

• The basics of how Lambda works and why we need lambdr for R projects (contextualises the need for the package)
• A primer on Docker (could just be a few links to existing material)
• A rough overview of how to turn the code and Dockerfile into a Lambda on AWS
  • You’ve got one method in the docs already, but I would like to provide the outline for a CDK option too
• Example project when using an AWS parent image
• Example project when using an arbitrary image
• How to local test our lambdas when using lambdr (how do we actually configure the lambdr::lambda_config? Is this possible?)

The projects would be reproducible examples explained for newbies. They would actually be a single project, just recycled and presented slightly differently in each article due to the requirements of using a parent/arbitrary Docker image. They would have a slightly more realistic format than the existing “parity” example, i.e. at least one functions.R that gets sourced in runtime.R. R packages would also have their versions pinned during installation.

Why I’m proposing this

I originally came to the issues to ask if it is possible to use an arbitrary Docker image instead of an AWS parent one. Naturally, I saw the another, more explicit example thread.

I wanted to ask the question because I’ve just been going through the pain of trying to figure out why httpgd doesn’t correctly compile in the al2023:provided container, and it made me wonder if I should give up and use a different image, or if that is even possible.

Though I now see it is possible to use an arbitrary image, I still don’t fully understand why. In the past few months I’ve learned enough to have a rough idea. Still, there’s a lot to learn, and as someone who is still going through the process but has picked up some stuff, I thought I’d be a good candidate as a collaborator for expanding the package docs.

For example, after reading the “explicit example” thread, the instructions on the lambdr site, and the docs, I still don’t know if I should be able to use the runtime interface emulator for local testing or not. If I should, I can’t get it to work. I’ve just been running my handler function during development and then using paws or boto3 to invoke the lambda after idk deploying, which is a bit cumbersome. So if it is possible, I’d like to have some examples in the docs and potentially a fuller explanation about local testing in an article.

Perhaps separately, I also think there’s a community need for an explainer of at least one dev/local Dockerfile, and for those who use VS Code, a working example devcontainer that can turn said Dockerfile into a fully fledged R development environment. I could probably do those as a blog post on my own. I know enough that if you gave me a working Dockerfile (with httpgd correctly installed 😭) I could write up the appropriate devcontainer.json.

Let me know what you think of the proposal. If you’re interested then we can discuss further here or take it to email/a video call. No pressure though :)

Best, Jim

[mdneuzerling]

Hi Jim,

You're absolutely spot on with all of this. When I started the package I knew that it would be 10% code, 40% unit tests, and 50% documentation.

I'm super busy at work until mid-next week, but happy to connect after.

[jimgar]

Superb. Look forward to it, thanks David!

[jimgar]

This discussion is now happening in https://github.com/mdneuzerling/lambdr/issues/33.