vessel icon indicating copy to clipboard operation
vessel copied to clipboard

Inherit settings

Open milk1000cc opened this issue 3 years ago • 5 comments

I want to inherit the settings of the parent class.

In the current master version, the following code will run in headless mode.

class ApplicationSpider < Vessel::Cargo
  driver :ferrum, headless: false
end

class ExampleCom < ApplicationSpider
  start_urls 'https://example.com/'

  def parse
    puts page.at_css('h1').text
  end
end

ExampleCom.run

milk1000cc avatar Jan 21 '22 05:01 milk1000cc

well, afaik it's good to avoid the using of inheritance in this way. take a look at these notes: https://github.com/serodriguez68/poodr-notes#coding-technique-6-create-shallow-hierarchies By the way, the case when we need shared settings is simpler to handle by this way:

settings = { driver_options: { headless: false } }
ExampleCom.run(settings)
ExampleCom2.run(settings)

Mifrill avatar Jan 21 '22 09:01 Mifrill

I understood it's not a good idea to use inheritance in this way. However, passing the shared settings to the #run method every time doesn't seem like a good idea either.

Is this better way?

class ApplicationSpider < Vessel::Cargo
  DEFAULT_SETTINGS = {
    driver_options: { headless: false }
  }

  def self.settings
    @settings ||= super.merge(DEFAULT_SETTINGS)
  end
end

class ExampleCom < ApplicationSpider
  start_urls 'https://example.com/'

  def parse
    puts page.at_css('h1').text
  end
end

milk1000cc avatar Jan 21 '22 14:01 milk1000cc

Is this better way?

yeah, probably...

Or this way:

      class ApplicationSpider < Vessel::Cargo
        driver :ferrum, headless: false

        def self.settings
          @settings ||= super.merge(ApplicationSpider.settings)
        end
      end

      class ExampleCom < ApplicationSpider
        start_urls 'https://example.com/'

        def parse
          puts page.at_css('h1').text
        end
      end
      ExampleCom.run

Mifrill avatar Jan 22 '22 06:01 Mifrill

Oh, thanks!

milk1000cc avatar Jan 22 '22 12:01 milk1000cc

I think I had an idea of inheriting all the settings from parent class, so if it doesn't work it's a bug. But I don't remember because I didn't finish full implementation of the driver mode. I'll check

route avatar Jan 24 '22 06:01 route