Support Swift Package Manager

[0xTim]

Swift Package Manager (SwiftPM) is the canonical package manager and dependency solutions for building Swift applications on the server. Swift on Server applications are deployed and used by many companies, including Apple and Amazon so adding support for the SwiftPM ecosystem would be great.

Vapor, the most popular server-side framework already publishes security advisories on GitHub.

[kylebrowning]

APNSwift has published advisories as well.

[darakian]

Hey @0xTim and @kylebrowning 👋

We can certainly look into support and thank you for the interest! Do you have any docs on the package manager, backing package registry or registries and package publication you can share?

[0xTim]

@darakian sure! The main docs are here

It terms of registry and publication, SwiftPM is predominantly decentralised, like Go, however the main registry is https://swiftpackageindex.com

I know there's been some recent work on a registry speciciation - @tomerd do you have the latest docs for the package registry?

cc @finestructure - roping you in incase there's anything the SPI might be able to provide/you've investigated this

[darakian]

Many thanks for the docs. @0xTim. Looks like there's a list of packages. Is that the entire scope of the SwiftPackageIndex?

SwiftPM is predominantly decentralised, like Go

As in a user of some dependency can pull in an arbitrary url? ex.

  dependencies: [
    .package(url: "https://SomeCoolWebsite.com/SomeCoolOrg/SomeCoolRepo")
  ],

All the docs (that I've skimmed so far) seem to use git repos as examples but are other version control systems supported? Is version control even required? How does the SwiftPackageIndex fit in if url's are directly consumable? Sorry if these are rather basic questions, but I've never used swift before. I'll try to get some time to run myself through a basic project in the next few weeks to fix that 😄

[0xTim]

Is that the entire scope of the SwiftPackageIndex?

@finestructure can you confirm?

@darakian So yes, for the moment packages must be Git URLs, the SwiftPacakgeIndex is used for discovery rather than hosting. I'll try and find the information about using a registry

[kylebrowning]

Please forgive me for bringing this up but GitHub was supposed to do a hosted registry.

https://github.blog/2019-06-03-github-package-registry-will-support-swift-packages/

[finestructure]

Is that the entire scope of the SwiftPackageIndex?

Yes, that's our canonical list of package we track on a regular basis (analysis via various lookups in checked out repositories as well as Github API queries to get package info).

[0xTim]

I'll raise this issue at the next Swift Server Workgroup meeting to ensure we pull together everything else needed

[darakian]

@darakian So yes, for the moment packages must be Git URLs, the SwiftPacakgeIndex is used for discovery rather than hosting. I'll try and find the information about using a registry

Ok. Cool 👍 Looking through the rules https://swiftpackageindex.com/add-a-package I'm seeing

The packages must all have at least one release tagged as a semantic version.

Why only one release and which version of semver? Are only the semver releases indexed or would we need to deal with commit hash based versioning as well?

Yes, that's our canonical list of package we track on a regular basis (analysis via various lookups in checked out repositories as well as Github API queries to get package info).

Gotcha. If we were to restrict our scope to the swift package index would that be useful to the community at large?

Please forgive me for bringing this up but GitHub was supposed to do a hosted registry. https://github.blog/2019-06-03-github-package-registry-will-support-swift-packages/

😬 Not sure what to tell you there, but I'll ping around to see what I can find out.

[finestructure]

The packages must all have at least one release tagged as a semantic version.

Why only one release and which version of semver? Are only the semver releases indexed or would we need to deal with commit hash based versioning as well?

This is a criterion for inclusion that we're not actually enforcing at the moment. The idea with these criteria is to have some very basic thresholds in place when adding packages to the index.

Once a package has been added, we track all releases (tags) and the tip of the default branch. Ideally, we'd be able to associate advisories against these "versions".

[darakian]

This is a criterion for inclusion that we're not actually enforcing at the moment.

☹️

Once a package has been added, we track all releases (tags) and the tip of the default branch. Ideally, we'd be able to associate advisories against these "versions".

So, a version in this world is anything that git tag allows. That will make advisory curation rather difficult since we won't be able to rely on a specific ordering.

Doing a spot check I came across this one as an example https://github.com/Azure/azure-sdk-for-ios/tags Which if I read it right is hosting multiple independent packages from the same repo with the tag prefix denoting the difference.

[0xTim]

@darakian SwiftPM supports dependencies tagged with SemVer 2.0 tags only (you can point to a specific branch or commit but if you want versioned dependencies it has to be a SemVer tag). The repo linked looks like it auto generates other repos for you to actually use with SwiftPM, e.g. https://github.com/Azure/SwiftPM-AzureCore.

So if a repo has no semver tags you can assume it has no releases.

There are two ways to read dependencies. If a repo has a Package.resolved checked in, that points to the specific version that the project uses (like a lock file in other ecosystems). If that doesn't exist (many libraries don't check these in) you can run swift package dump-package to get information about the package including it's dependencies.

What is the work required to add support for SwiftPM and is there anything the Swift community can help with to enable this?

[0xTim]

Gotcha. If we were to restrict our scope to the swift package index would that be useful to the community at large?

Re 👆 - yes that would be immensely useful as a starting point!

[darakian]

@0xTim, ok so we would restrict versions to be semver 2.0.0. That will be feasible for sure. The azure sdk case is still strange to me. If we're using https://github.com/SwiftPackageIndex/PackageList/blob/main/packages.json as the canonical names for packages then https://github.com/Azure/SwiftPM-AzureCore doesn't exist as a package. It may be the case that https://github.com/Azure/azure-sdk-for-ios generates other repositories, but those don't appear to get populated into the swift package index and the base repo itself doesn't follow semver.

also sorry for the delayed reply on my end.

[finestructure]

If it helps, the Azure iOS SDK package is a bit of an outlier in terms of the package index. The only reason azure-sdk-for-ios is in the index right now is because it used to use SemVer tags - that's why we're still showing v0.5.0 as the latest stable. They detached themselves from our version tracking when they moved to a new process. If you tried to add it to the index now it would not pass validation.

For the same reason SwiftPM-AzureCore isn't a valid package and not tracked by the index at all.

Is the presence of azure-sdk-for-ios in the package list problematic for this feature? We should be able to produce a list that excludes package which don't pass validation without too much effort.

[0xTim]

@darakian great! Is there anything you need from me to proceed?

@finestructure is it worth just removing the Azure repo from SPI if it's not valid at all?

[finestructure]

Yeah, we'll remove it!

[darakian]

Seems like we have a clear path forward using urls from the package index as the canonical source of truth for "package names" and semver 2.0.0 as the implicit versioning scheme. I want to be clear that I can't promise you a timeline, but I'll try to get this on the roadmap internally and I can report back here when I have something for you. Please correct me if any of that is wrong 👍

Is there anything you need from me to proceed?

Can you start enforcing semver tags? 😃

[darakian]

Following up here with this https://github.blog/changelog/2023-06-19-dependency-graph-dependabot-alerts-and-advisory-database-now-support-swift-advisories/

🎉

[0xTim]

Going to close this since it's now supported! 🎉

[darakian]

@0xTim question for you; you don't have an example of a swift package hosted somewhere other than github do you? I'm trying to find an example just so we know how to format those and everything I'm finding is hosted here (hooray I guess 😕)

Edit: Found one from gitlab https://gitlab.com/rdall96/docker-swift-api but if you know of any other url forms those would help 🙇

[0xTim]

There's https://gitlab.com/swift-server-community/RediStack/ which is now a mirror to a GH project but still works. Have you tried using SSH URLs as well? Those are used sometimes

[darakian]

Have you tried using SSH URLs as well? Those are used sometimes

I have not. Please share if you have some examples 👍

[0xTim]

@darakian any package can be converted to an SSH URL, for instance you can pull in https://github.com/vapor/jwt-kit (note that sometimes people append .git on the end, so https://github.com/vapor/jwt-kit.git) or [email protected]:vapor/jwt-kit or [email protected]:vapor/jwt-kit.git. These are primarily used for private repos

[darakian]

These would still refer to the same code though right? I guess someone could setup their own sever and have https://some_domain/some_path point to a different set of code than git@some_domain/some_path, but would that only be the case for private code? For context our advisories are on public code only.

[0xTim]

Yes it would, it's partly a stylistic choice but you can use Git URLs for public repos without issue. I think keeping it as HTTPS repos for now would be an OK choice if it's problematic

[darakian]

Gotcha. Ya, we're focused not necessarily on http code sources, but on public ones and I have a hard time imagining a scenario where someone hosts of a public code base via ssh which wouldn't also be available via http. We can 100% revisit this if that pops up 👍