gitness icon indicating copy to clipboard operation
gitness copied to clipboard

Drone pipeline converter cache/memorizer?

Open nmapx opened this issue 1 year ago • 1 comments

I can't find anything related to pipeline cache in Drone docs but it seems there is such thing. I'm using a conversion extension (starlark - external one). When I update it's components Drone does not apply it in the pipelines. It's using the older version that was built by the extension before the change. It works if i trigger a different pipeline or just restart the drone server. Is it possible to force Drone to parse pipeline every time even if there is no change in the file (I mean the main .drone.yml/.drone.star)?

nmapx avatar Feb 15 '24 11:02 nmapx

Correct me if I'm wrong @bradrydzewski but I suspect this piece of code is the reason I'm looking for

package config

import (
	"context"
	"fmt"

	"github.com/drone/drone/core"

	lru "github.com/hashicorp/golang-lru"
	"github.com/sirupsen/logrus"
)

// cache key pattern used in the cache, comprised of the
// repository slug and commit sha.
const keyf = "%d|%d|%s|%s|%s|%s|%s"

// Memoize caches the conversion results for subsequent calls.
// This micro-optimization is intended for multi-pipeline
// projects that would otherwise covert the file for each
// pipeline execution.
func Memoize(base core.ConfigService) core.ConfigService {
	// simple cache prevents the same yaml file from being
	// requested multiple times in a short period.
	cache, _ := lru.New(10)
	return &memoize{base: base, cache: cache}
}

type memoize struct {
	base  core.ConfigService
	cache *lru.Cache
}
...

Any way to make it customized through env var? I would agree some degree of cache is necessary here but can't see any TTL set on this which creates more issues than it solves at least in my case.

nmapx avatar Feb 20 '24 00:02 nmapx

You can set DRONE_CONVERT_PLUGIN_CACHE_SIZE=0 to disable caching. See relevant issue: https://github.com/harness/gitness/pull/3292#issuecomment-1332623285

bradrydzewski avatar Feb 20 '24 16:02 bradrydzewski

Thanks! Couldn't find this env var in the docs but it does the job! 🎉

nmapx avatar Feb 20 '24 16:02 nmapx