For developers - onboarding documentation

[usingtechnology]

trafficstars

For new (and old?) developers, clearly document (and keep up to date) best practices for developing, building, and testing locally.

For instance, when opening a PR, there are a series of checks. These should be documented with instructions on how a developer would (if and where possible) run through these checks locally. We should use pre-commit hooks, VS Code dev containers etc, where possible, but those should not replace documentation and manual steps.

[hiteshgh]

Test

[Jsyro]

A few items on my radar

• [ ] Demos may need easy options to fully remove ngrok and run locally with two agents, rather than one agent that assumes a mobile wallet will play the other part. NGROK is taken for granted however certain corporate security policies block any traffic to the ngrok domain.

• /demo/run_demo faber assumes ngrok, so running with /demo/run_demo alice requires manual intervention to modify the service_endpoints to establish a connection\

• [ ] Unit test suite runs well as a whole, but using the unittest suite for the testing of individual files or tests are either very slow ,because they are configured to always run with coverage, or do not run at all. We should explore and refine a configuration that is fully supported by a given development tool (like VSCode)

• [ ] Find windows based python packages that allow for aca-py to be run natively on developers machine if possible.

[swcurran]

Helpful to list the demos that you tried that had ngrok requirements, so that we can target them.

Our experience has been trying to maintain a useful environment on every platform (and variation of platform) is very hard, so we suggest that developers find ways to work on containers vs. native. Maintaining things to run on all developer platforms is a lot of overhead. Doable, but not where we would like to spend our resources.

[swcurran]

I would open a ticket asking how the last can be done. How can a dev run a unit test in 2 seconds? Someone must know :-)

[usingtechnology]

I would like to see documentation simplified (lots and lots of different markdowns) and possibly gathered into streams: ACA-Py developer, ACA-Py user, ... business use cases?

The demos folder has a lot of stuff in it; possibly move things out (playground). Also, we should really clarify what the purpose is and who the demographic is... I feel like the demographic is more for business case, but was constantly getting pointed to the run_demo because it does X. Then it is a learning curve to go through the shell scripts to see how ACA-Py is configured and started, then work through the agent code to actually find out what is happening to connect two agents.

Do we need multi-demo? If so, it needs a better "script".

Also, BDD is inside demos; is that a demo? It is used in GitHub actions, but when I was looking for BDD tests, that is not where I expected to find them. As a developer, I would love to run the BDD scripts and be able to set breakpoints in the running ACA-Py code... the BDD scripts are at least reasonably easy to follow and understand what is going on. I find the BDD tests much more helpful than unit tests. There is so much mock code in unit tests that it is difficult to determine what is going on to gain knowledge - like what does a payload for issue credential v2 look like? I guess I'm saying I'd like to see those fleshed out and then promoted as a learning and testing tool.

ngrok - I agree with @swcurran about itemizing which demos absolutely need them (mobile) and which ones we can move away from the requirement. Moving everything to the same docker network is awkward but possibly doable. However, not all the components we want to use are in this repository... so we don't have control over other docker composes. Having a demo openshift with the components we could load (mediator, endorser, ledger) is useful, but a maintenance/dev ops headache - and not sure open source projects should heavily lean on bcgov or other for infrastructure. Most everything we need has a docker compose so we should stick with that and expect that devs could run everything (in non-HA) locally for developing / debugging. Perhaps it's as simple as having a second service in each docker compose that doesn't depend on ngrok and we document how to start with that and the limitations. Again, that may only apply to this repository, but could spread to others we contribute toward. (See issue #2275)

I know there is work to get past Python 3.6, so once that work is done I think we need to go through the various docker images in the repo and standardize them. The demos and bdd I don't think are exactly the same as a production image, that's a little concerning. We also need to update our requirements (I know that we can't update requests until we move to 3.9 - we don't use the code where the security flaw is but we are out of date).

In the devcontainer PR, @swcurran mentioned docker-in-docker... I think that is worth spending a little more time on since we do have (and rely) so many docker files and docker composes...

When we get to plugins as a more important feature, we must provide an easy way for dev teams to debug them. I know it is a real pain for the traction team, so I am sure it is for others. That can fold in with issue #2277.

[swcurran]

New structure of the https://aca-py.org site helps. An update path through the docs would be helpful...always.