pgloader icon indicating copy to clipboard operation
pgloader copied to clipboard

Filename matcher cannot match exact file name inside an archive

Open volkanunsal opened this issue 4 years ago • 0 comments

  • [x] pgloader --version

    3.6.2
    
  • [x] did you search for other similar issues?

  • [x] how can I reproduce the bug?

LOAD ARCHIVE 
  FROM https://timezonedb.com/files/timezonedb.csv.zip
  INTO postgresql:///pgloader

  BEFORE LOAD EXECUTE 'tz.sql'


  LOAD CSV
    FROM FILENAME MATCHING ~/zone.csv/ 
    (zone_id,country_code,zone_name)
    
    INTO postgresql:///pgloader?zone (zone_id,country_code,zone_name)
    
    WITH truncate,
        drop indexes,
        skip header = 0,
        fields optionally enclosed by '"',
        fields escaped by double-quote,
        fields terminated by ','

    SET client_encoding to 'utf-8', work_mem to '12MB', standard_conforming_strings to 'on'

  AND LOAD CSV
    FROM FILENAME MATCHING ~/timezone.csv/ 
    (zone_id,abbreviation,time_start,gmt_offset,dst)
    
    INTO postgresql:///pgloader?timezone (zone_id,abbreviation,time_start,gmt_offset,dst)
    
    WITH truncate,
        drop indexes,
        skip header = 0,
        fields optionally enclosed by '"',
        fields escaped by double-quote,
        fields terminated by ','
    
    SET client_encoding to 'utf-8', work_mem to '12MB', standard_conforming_strings to 'on';
DROP TABLE IF EXISTS zone;
CREATE TABLE zone (
  zone_id numeric NOT NULL,
  country_code text NOT NULL,
  zone_name text NOT NULL
);

DROP TABLE IF EXISTS timezone;
CREATE TABLE timezone (
  zone_id varchar NOT NULL,
  abbreviation varchar NOT NULL,
  time_start DECIMAL(11,0) NOT NULL,
  gmt_offset INT NOT NULL,
  dst CHAR(1) NOT NULL
);

DROP TABLE IF EXISTS country;
CREATE TABLE country (
  country_code varchar NOT NULL,
  country_name varchar NOT NULL
);
  • [x] pgloader output you obtain

The issue is ~/zone.csv/ matches 2 files, and it's loading the wrong file into the table. I cannot use a string to match the file because the string does not work in the context of an archive.

volkanunsal avatar Nov 09 '21 02:11 volkanunsal