scrapy-mysql-pipeline icon indicating copy to clipboard operation
scrapy-mysql-pipeline copied to clipboard

Automatically configure MYSQL_TABLE from spider name

Open phelps-sg opened this issue 3 years ago • 0 comments

If the setting MYSQL_TABLE is not defined, then the table-name is now generated from spider name. Alternatively, MYSQL_TABLE can optionally contain a '%s' printf wildcard, which is replaced by the spider name. These modifications allow a configuration in which multiple spiders within the same scrapy project use the same DB schema, but each spider populates a different table.

phelps-sg avatar Dec 11 '21 14:12 phelps-sg