Incorrect URL generated by MavenPublishArtifactManager in azure snapshot builds

[jazminebarroga]

Summary

I wanted to distribute a snapshot build via Cocoapods but the url created by the MavenPublishArtifactManager.artifactPath is not the proper url to the snapshot build.

Details

We're using azure as our artifact repository for xcframeworks and one of its limitations is that you cannot overwrite published artifacts unless you append SNAPSHOT to its version. Snapshot builds have a different naming convention and do not follow the ...$kmmbridgeArtifactId-$version.zip url naming that the MavenPublishArtifactManager.artifactPath creates.

I also previously created a custom gradle task to make sure that the actual version in the generated podspec follows the proper naming convention so it passes pod lint such that if the version is 0.1.0-SNAPSHOT it becomes 0.1.0.beta in the podspec when generated.

Reproduction

1. Change artifact version to 0.1.0-SNAPSHOT
2. Use azure as artifact repository
3. Run kmmBridgePublish

Expected result

The generated podspec should refer to the actual URL of the maven artifact which is something like this ...sdk-kmmbridge/0.1.0-SNAPSHOT/sdk-kmmbridge-0.1.0-20230418.083616-1.zip

Current state

The generated podspec refers to ...sdk-kmmbridge/0.1.0-SNAPSHOT/sdk-kmmbridge-0.1.0-SNAPSHOT.zip which is not the actual URL of the maven artifact

[russhwolf]

We don't have any explicit snapshot support at the moment. I'm not sure I fully understand the azure issue, but you might be able to work around by defining a custom version writer like this:

class SuffixedVersionWriter(private val suffix: String, private val delegate: VersionWriter): VersionWriter by delegate {
    override fun scanVersions(project: Project, block: (Sequence<String>) -> Unit) {
        delegate.scanVersions(project) { sequence ->  
            block(sequence.map { it.removeSuffix(suffix) })
        }
    }

    override fun writeMarkerVersion(project: Project, version: String) {
        delegate.writeMarkerVersion(project, version + suffix)
    }

    override fun writeFinalVersion(project: Project, version: String) {
        delegate.writeFinalVersion(project, version + suffix)
    }
}

and then in your kmmbridge block do

versionWriter.set(SuffixedVersionWriter("-SNAPSHOT", GitRemoteVersionWriter()) // delegate to whatever version writer is currently being used

but I haven't tested that so it might not quite work as-is.

[jazminebarroga]

Let me provide more details. So for example, using kmmbridge, I am already able to upload the xcframework in an azure maven repository with the version 0.1.0-SNAPSHOT and so I am able to generate a podspec that looks like this

Pod::Spec.new do |spec|
    spec.name                     = 'SDK'
    spec.version                  = '0.1.0.beta'
    spec.homepage                 = 'https://www.google.com'
    spec.source                   = { 
                                      :http => 'https://pkgs.dev.azure.com/[redacted]/sdk-kmmbridge/0.1.0-SNAPSHOT/sdk-kmmbridge-0.1.0-SNAPSHOT.zip',
                                      :type => 'zip',
                                      :headers => ['Accept: application/octet-stream']
                                    }
    spec.authors                  = ''
    spec.license                  = ''
    spec.summary                  = 'API'
    spec.vendored_frameworks      = 'sdk.xcframework'
    spec.libraries                = 'c++'
    spec.ios.deployment_target = '14'
           
end

The url however is incorrect in the generated podspec, because the url for snapshot versions is actually like this:

https://pkgs.dev.azure.com/[redacted]/sdk-kmmbridge/0.1.0-SNAPSHOT/sdk-kmmbridge-0.1.0-20230420.015940-1.zip

The upload date is appended as suffix.

[russhwolf]

Can you show your kmmbridge gradle configuration? I want to understand how you're setting the url

[jazminebarroga]

Here

/**
 * Publishing configurations
 */
def localProperties = new Properties()
try {
    localProperties.load(project.rootProject.file("local.properties").newDataInputStream())
} catch (FileNotFoundException e) {
    println("Local properties file not found. $e")
}

def publishingGroupId = "group id"
def publishingVersion = localProperties.getProperty("publishing.version") ?: "0.0.0"
def azurePublishingUrl = localProperties.getProperty("azure.url")
def azurePublishingUser = localProperties.getProperty("azure.user")
def azurePublishingPw = localProperties.getProperty("azure.pw")
def cocoapodsSummary = localProperties.getProperty("cocoapods.summary") ?: ""
def cocoapodsHomepage = localProperties.getProperty("cocoapods.homepage") ?: ""

subprojects { project ->

    apply plugin: "maven-publish"
    apply plugin: libs.plugins.kmmbridge.get().pluginId
    apply plugin: "org.jetbrains.kotlin.native.cocoapods"
    apply plugin: "org.jetbrains.kotlin.multiplatform"

    project.group = publishingGroupId
    project.version = publishingVersion

    publishing {
        repositories {
            if (azurePublishingUrl != null && !azurePublishingUrl.isEmpty()) {
                maven {
                    name = "azure"
                    url = uri(azurePublishingUrl)
                    credentials {
                        username = azurePublishingUser
                        password = azurePublishingPw
                    }
                }
            }
        }
    }

    kmmbridge {
        mavenPublishArtifacts(project, null, null)
        manualVersions()

        tasks.register("overwriteKmmbridgeVersionFile", OverwriteKmmbridgeVersionFileTask) {
            buildDirectory = getBuildDir()
        }

        cocoapods(project, "specs url", true, false)

        afterEvaluate {
            tasks.overwriteKmmbridgeVersionFile.mustRunAfter tasks.uploadXCFramework
            tasks.generateReleasePodspec.dependsOn tasks.overwriteKmmbridgeVersionFile
        }
    }

    kotlin {
        cocoapods {
            summary = cocoapodsSummary
            homepage = cocoapodsHomepage
            name = "name"
            ios.deploymentTarget = "14"
        }
    }
}

abstract class OverwriteKmmbridgeVersionFileTask extends DefaultTask {

    @Input
    abstract Property<File> getBuildDirectory()

    @TaskAction
    def execute() {

        File versionFile = new File(buildDirectory.get().toString() + "/faktory/version")
        String newVersion = versionFile.readLines().first().replaceAll("-SNAPSHOT", ".beta")
        versionFile.write(newVersion)
    }
}

[arcadii-meister]

I have encountered the same issue with snapshot versions, but I am using Google Cloud Artifact Registry. However, it being a Maven repository, it is the same limitation.

I could overcome it by using this code, thank you @jazminebarroga for inspiration:

kmmbridge {
    mavenPublishArtifacts()
    manualVersions()
    noGitOperations()
    spm(useCustomPackageFile = true)

    val version = version.toString()

    if (!version.contains("-SNAPSHOT")) {
        return@kmmbridge
    }

    val fixKMMBridgeSnapshotVersion by tasks.registering {
        group = "kmmbridge"

        doLast {
            val snapShotVersion = project.dependencies
                .create(project.group.toString(), "${frameworkName.get()}-kmmbridge", version)
                .let { configurations.detachedConfiguration(it) }
                .apply { resolutionStrategy.cacheChangingModulesFor(0, TimeUnit.MINUTES) }
                .resolvedConfiguration.resolvedArtifacts.firstOrNull()
                ?.id?.componentIdentifier?.let { it as? MavenUniqueSnapshotComponentIdentifier }
                ?.timestamp
                ?: error("Cannot resolve component timestamp")

            with(file("$buildDir/faktory/url")) {
                readText()
                    .replace("SNAPSHOT.zip", "$snapShotVersion.zip")
                    .let(::writeText)
            }
        }
    }

    /**
     * Adds the snapshot fix task to the KMMBridge's task chain.
     */
    afterEvaluate {
        val uploadXCFramework by tasks.existing
        fixKMMBridgeSnapshotVersion.dependsOn(uploadXCFramework)

        val updatePackageSwift by tasks.getting
        updatePackageSwift.dependsOn(fixKMMBridgeSnapshotVersion)
        
        // Forces the task always rerun to write the new version to the package file.
        // Otherwise, it is cached and Package.swift is not updated.
        updatePackageSwift.outputs.upToDateWhen { false }
    }
}

The idea is to use the Maven snapshot resolution strategy to get the latest timestamp and replace -SNAPSHOT with it, so Package.swift will have a link to the specific file.

// BEGIN KMMBRIDGE VARIABLES BLOCK (do not edit)
let remoteKotlinUrl = "https://europe-maven.pkg.dev/.../common-kmmbridge/0.1.1-SNAPSHOT/common-kmmbridge-0.1.1-20230725.122208-80.zip"
let remoteKotlinChecksum = "..."
let packageName = "common"
// END KMMBRIDGE BLOCK

Replacing -SNAPSHOT only for "$buildDir/faktory/url" file is intentional as we need to fix versioning only for XCFramework, but we can leave it as SNAPSHOT for Android/Java since they are using Maven to resolve it when consuming.

At first, I wanted to try just to parse a Maven metadata file from a remote repository, but I decided to use Gradle API to reuse the already provided authentication to the repository.

I have also tried to use createArtifactResolutionQuery(), but I couldn't bypass the local cache, so after each uploadXCFramework, that created a new SNAPSHOT file, createArtifactResolutionQuery() was returning cached artifacts.

[kpgalligan]

We do indeed "guess" the maven url. The code above is interesting. I'd have to think through how this might impact SPM config, as there's a global "version" assumption in the code to some degree, and SPM can't deal well with non-semver versions, but it may not be a big deal.

[arcadii-meister]

SPM can't deal well with non-semver versions

@kpgalligan Exactly, because of the SPM semver requirements I have to tag my commits with versions excluding -SNAPSHOT postfix. So for the iOS devs, it looks like a regular version where only the commit revision is changing in the Package.resolved file.

Turns out Xcode doesn't like it very much and iOS devs need to constantly Reset Package Caches and sometimes clear the derived data folder, which in our case takes quite some time and is very annoying. Because of that, I am considering avoiding SNAPSHOT and just bumping up the version for each new commit. However, it looks like it is the SPM issue and it is not connected only to the KMP artifacts.