artifactory-client icon indicating copy to clipboard operation
artifactory-client copied to clipboard

Using Artifactory.search() goes through the list of objects with GET -> slow performance

Open iiro opened this issue 9 years ago • 1 comments

Hi,

I'm trying to use the Artifactory.search() here like this:

Artifactory::Resource::Artifact.search(name: 'project*.gz', repos: 'project-artifacts-local')

And it does produce the list of objects yes - but it also goes through every object one by one with GET (at least by looking Artifactory's access.log) - which means it takes about a minute to run this (with current amount of artifacts on search scope).

Is this intentional?

iiro avatar Oct 24 '16 15:10 iiro

I encountered the same issue while using this library. The large number of request is due to from_url() method which is called for every single objects returned by a search request. This method calls Artifactory::Client::request at the end... For my needs I have extended the Resource::Artifact class with a new method (in order to deal with NPM packages). This method use an AQL query and then the from_url() method only once for the element which will be downloaded further... So performances are improved since only 2 requests to the server are needed to return the searched object.

Code exemple:

      def npm(options = {})
        client  = extract_client!(options)
        repos   = options[:repos]
        name    = options[:name]
        version = options[:version]

        aql = %Q(items.find({"repo":"#{repos}"},{"@npm.name":"#{name}"},{"@npm.version":{"$match":"#{version}"}}).include("repo","name","path").sort({"$desc":["name"]}))

        results = client.post('/api/search/aql', aql)['results']
        if not results.empty?
          sorted = results.sort {|x,y| Gem::Version.new(x['name'].match(/-(\d+[\.\d-]+)\.tgz/)[1].gsub('-', '.')) <=> Gem::Version.new(y['name'].match(/-(\d+[\.\d-]+)\.tgz/)[1].gsub('-', '.'))}
          npm = sorted[-1]

          # Call from_url only once in order to improve performances
          from_url(File.join('/api/storage/' + npm['repo'], npm['path'], npm['name']), client: client)
        else
          nil
        end
      rescue Error::HTTPError => e
        raise unless e.code == 404
        nil
      end

ThomVivet avatar Nov 30 '16 15:11 ThomVivet