Use net/http instead of puppetdb-ruby

[daenney]

trafficstars

The puppetdb-ruby gem provides a useful client but seems unable to encode more complex queries correctly causing APIError's to be thrown. Especially the classes query could benefit from a more efficient query on larger installations.

I'm having some trouble with Ruby right now but this should more or less be it:

require 'json'
require 'net/http'
require 'ghostbusters/version'


class PuppetGhostbuster
  class PuppetDB
    def self.client
      uri = URI.parse("#{ENV['PUPPETDB_URL'] || 'http://puppetdb:8080'}")

      @@client ||= Net::HTTP.new(uri.host, uri.port)
      @@client.key = ENV['PUPPETDB_KEY_FILE']
      @@client.cert = ENV['PUPPETDB_CERT_FILE']
      @@client.ca_file = ENV['PUPPETDB_CACERT_FILE']
    end

[..]

    def self.request(endpoint, query = nil)
      uri_prefix = '/pdb/query/v4'
      headers = {'user-agent' => "ghostbusters/#{PuppetGhostbuster::VERSION}", 'accept' => 'application/json' }
      if query.nil?
        request = Net::HTTP::Get.new("#{uri_prefix}/#{endpoint}", headers)
      else
        request = Net::HTTP::Get.new("#{uri_prefix}/#{endpoint}?query=#{URI.encode(query)}", headers)
      end
      resp = client.request(request)
      case resp
      when Net::HTTPSuccess
        JSON.parse resp.body
      else
        resp.value # calling value will raise an exception if we didn't get a 200
      end
    end
[..]

   def self.classes
      query = '["extract", [["function", "count"], "title"], ["=", "type", "Class"], ["group_by", "title"]]'
      @@classes ||= client.request('resources', query).select { |r| r.title }
  end

[mcanevet]

@daenney wouldn't it be possible to fix puppetdb-ruby? @nathanielksmith what do you think?

[daenney]

I took a look at that and it would be feasible but the project seems entirely abandoned. I'm not sure we'll ever see another release of it to be honest.

The easiest way for puppetdb-ruby would be to add an escape hatch, something along the lines of adding a raw_query parameter to the request method allowing you to escape the whole query builder thing so it doesn't have to be updated every time the PuppetDB query language changes. This would also allow it to support PQL with no further modifications. Albeit it's a bit less nice than when you can use query but it would stop blocking everyone.

[vilmibm]

Sadly, I haven't been able to get work time to maintain puppetdb-ruby and no one has volunteered to take over the project.

It's wisest to probably just use raw http for now :( sorry for this tragedy of the commons situation.

[mcanevet]

@daenney it will probably not be easy to do what this https://github.com/camptocamp/puppet-ghostbuster/blob/master/spec/spec_helper.rb#L6-L19 does for unit tests... I'm not sure it will be easily possible to convert complex puppetdb query to jgrep queries. But I like the idea of getting rid of an unmaintained piece of code.

[daenney]

It sounds like the would be better solved with VCR: https://github.com/vcr/vcr

[mcanevet]

@daenney did you already use VCR? I'm interested in using this instead of my hack with jgrep, but I can't find a simple doc about how to use it with rspec.

[daenney]

The usage example in the README has always worked for me. Is there something there you'd like me to clarify?