singularityhub
OCI Compliance ? Thoughts?
Hello.
Have you looked at the OCI compliance / would you say that SRegisry is OCI compliant? (_ i think so since its based off docker v2 which is, but wanted to get your thoughts on it, directions you might be going_
Cheers, Daniel
Nope it's definitely not - it's "custom library API endpoint" compliant, as implemented by Sylabs. I'd be hugely in support of going for OCI compliance, as that would (in the long run) be easier to maintain than trying to follow what Sylabs is doing with their personal cloud. I'm definitely familiar with OCI, I was fairly regular a participant back when I was part of (open source) Singularity, and made several videos along with a Python library to wrap the image/digest specs. https://github.com/vsoch/oci-python. What is missing there is the most important part for the issue here - the distribution spec, which more so defines the endpoints that should be used for a registry.
Since the current library is intended for validating configuration files for images, how would you suggest implementing the distribution spec (which largely describes an API format) but for Python, so it could be used with Singularity Registry server? Likely we would need to have endpoints that conform to the OCI specification, and perhaps they could be validated separately by some integration of the distribution spec into opencontainers.
This is a fairly HUGE project, but I'm happy to think about taking it on for some point in the future, because OCI >>> pretty much everything we have implemented here.
The order that I'd approach these things is:
- fix issue here with your upload
- add distribution spec to opencontainers so minimally we can test endpoint returned values
- THEN django module for OCI distribution spec (this would be so hugely valuable to the community!)
- THEN add the django module here (and likely I'll use sregistry to help drive it's development, and then separate into a module).
Almost dinner, chat later this week!
hey @lmcdasm I am going to make some time to start looking into this! I think the first logical step (as I mentioned above) would be to integrate some representation (even if only functions to check some endpoint) with opencontainers python and then see if that is fittable into a Django module.
Hey there.. apologies.
i have been remiss - i wanted to get back to you both on the issue with SIF image file sizes (but i think i saw another issue that seemed to be similarly related).
What i have right now is a helm chart for our base OCI registry - its nothing fancy (its docker essentially as goHabour and others are not quite OCI compliant - the only two we have found is Docker (of course) and Azure Container Registry (funny enough) is OCI compliant. My target is told fold - i have a plethora of "object types" that we will wrap into a OCI format (dockerV2 essentially) regardless of the data inside (for our deliverables). - this is "today" in docker container, with copies and such, would rather normalize to singularity We will also (in our Kubernetes environments) use this as a direct docker registry for kube deploys (or orchestrated ones via a pipeline - helm calls, etc) We will also from our HPC side( where singularity is king :P) - store SIF's in there as well - which would be dockerV2 format, so essentially would the same in my view.
Cheers, D
On Tue, 24 Mar 2020 at 16:03, Vanessasaurus [email protected] wrote:
hey @lmcdasm https://github.com/lmcdasm I am going to make some time to start looking into this! I think the first logical step (as I mentioned above) would be to integrate some representation (even if only functions to check some endpoint) with opencontainers python and then see if that is fittable into a Django module.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/singularityhub/sregistry/issues/285#issuecomment-603476520, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACHMXRH7PSWRFBDIQ36SFUDRJEG23ANCNFSM4LD55S7Q .
hey @lmcdasm I'm implementing the django-oci module for this, and (outside of sregistry) wanted to get your feedback on what would be the most likely storage implementations in the wild. I just added Minio here #298 (uses S3 multipart upload API) but there are other choices with signed urls of course! If you have a chance, I opened an issue here to have some conversation. I'm generally looking for "What would someone creating a django app want for their storage" and then "how can I easily do that with a module without assuming a Minio container, for example). If you have any time in the coming weeks, please give feedback there! It's one of my projects I'll definitely be putting more time into soon, although there are some other stuffs that come before it.
hey @lmcdasm another update for you - the initial (conformance testing passing) version of django-oci (a plugin to implement oci in a registry) is ready! https://vsoch.github.io/django-oci/ There are many things to add (interface views for an example app, authentication, different storage backends), but if you want to take a look and express what is important to you / otherwise give feedback it would be appreciated!