dimitri
I can not use multiple threads per table with mssql
I'm using last version compiled with the docker instance with debian version "3.6.3~devel"
This is my load file
load database
from mssql://psdbsausr:Z3n1us2!@eduardo-infostore-datalake.c0qo0u5xjnw9.us-east-1.rds.amazonaws.com:1433/infostore
into postgresql://cxp151stresstest:[email protected]:5432/GEduardoRequest
WITH
include drop, create tables, create indexes, reset sequences,
workers = 8, concurrency = 1,
alter schema 'dbo' rename to 'public'
including only table names like 'VOLDAGGRSV' in schema 'dbo'
BEFORE LOAD DO
$$ DROP TABLE IF EXISTS VOLDAGGRSV; $$,
$$ CREATE TABLE VOLDAGGRSV
(
MD_REP_SID NUMERIC(3) NOT NULL,
SRV_SID NUMERIC(5) NOT NULL,
VSC_SID NUMERIC(6) NOT NULL,
VSC_UID NUMERIC(7) NOT NULL,
YEAR_ID NUMERIC(4) NOT NULL,
MONTH_ID NUMERIC(6) NOT NULL,
DAY_ID NUMERIC(8) NOT NULL,
MINUTE_ID NUMERIC(4) NOT NULL,
SECOND_ID NUMERIC(2) NOT NULL,
UTC_OFFSET CHAR(6),
UTC_TIME TIMESTAMP NOT NULL,
TIMESLICE_ID NUMERIC(4),
TIMESLICE_MINUTE_ID NUMERIC(4),
NO_CON_CALLS NUMERIC(8),
NO_FIN_CALLS NUMERIC(8),
NO_ABO_CALLS NUMERIC(8),
NO_REJ_CALLS NUMERIC(8),
ACT_SES_GAR NUMERIC(8),
ACT_SES_FLO NUMERIC(8),
ACT_SES_LMT NUMERIC(8),
REQ_SES_GAR NUMERIC(8),
REQ_SES_LMT NUMERIC(8),
ROW_TS TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL,
CONSTRAINT VOLDAGGRSV_PK PRIMARY KEY (MD_REP_SID, SRV_SID, VSC_SID, DAY_ID, MINUTE_ID, SECOND_ID, ROW_TS)
);
$$
set work_mem to '16MB', maintenance_work_mem to '512 MB'
;
If I try to use any of this options
multiple readers per thread, rows per range = 50000
I always get the same error
2020-06-15T15:47:22.002906Z LOG pgloader version "3.6.3~devel"
KABOOM!
FATAL error: At
include drop, create tables, create indexes, reset sequences,
workers = 8, concurrency = 1, rows per range = 50000
^ (Line 6, Column 32, Position 345)
In context MSSQL-OPTIONS:While parsing MSSQL-OPTIONS. Expected: the character Tab or the character Newline or the character Return or the character Space or the string "--" or the string "/*" or the string "batch" or the string "concurrency" or the string "create" or the string "data" or the string "disable" or the string "downcase" or the string "drop" or the string "encoding" or the string "foreign" or the string "include" or the string "max" or the string "no" or the string "on" or the string "prefetch" or the string "preserve" or the string "quote" or the string "reset" or the string "schema" or the string "snake_case" or the string "truncate" or the string "uniquify" or the string "workers"
An unhandled error condition has been signalled: At
include drop, create tables, create indexes, reset sequences,
workers = 8, concurrency = 1, rows per range = 50000
^ (Line 6, Column 32, Position 345)
In context MSSQL-OPTIONS:While parsing MSSQL-OPTIONS. Expected: the character Tab or the character Newline or the character Return or the character Space or the string "--" or the string "/*" or the string "batch" or the string "concurrency" or the string "create" or the string "data" or the string "disable" or the string "downcase" or the string "drop" or the string "encoding" or the string "foreign" or the string "include" or the string "max" or the string "no" or the string "on" or the string "prefetch" or the string "preserve" or the string "quote" or the string "reset" or the string "schema" or the string "snake_case" or the string "truncate" or the string "uniquify" or the string "workers"
What I am doing here?
At
include drop, create tables, create indexes, reset sequences,
workers = 8, concurrency = 1, rows per range = 50000
^ (Line 6, Column 32, Position 345)
In context MSSQL-OPTIONS:While parsing MSSQL-OPTIONS. Expected: the character Tab or the character Newline or the character Return or the character Space or the string "--" or the string "/*" or the string "batch" or the string "concurrency" or the string "create" or the string "data" or the string "disable" or the string "downcase" or the string "drop" or the string "encoding" or the string "foreign" or the string "include" or the string "max" or the string "no" or the string "on" or the string "prefetch" or the string "preserve" or the string "quote" or the string "reset" or the string "schema" or the string "snake_case" or the string "truncate" or the string "uniquify" or the string "workers"
I'm using this freeTds file [global] # TDS protocol version tds version = 7.4 client charset = UTF-8
It looks like there's an extra comma (,) character at the end of your option list (the WITH clause) and that the parser chokes on that.
+1
Happens to me as well, MSSQL-OPTIONS does not recognise multiple readers per thread.
@dimitri He just formatted the conf he submitted in a bad way. What he meant is
WITH include drop, create tables, create indexes, reset sequences, workers = 8, concurrency = 1,
and to this appended
multiple readers per thread, rows per range = 50000
Which should produce legitimate conf. You can see his conf is legitimate from his error log:
An unhandled error condition has been signalled: At
include drop, create tables, create indexes, reset sequences,
workers = 8, concurrency = 1, rows per range = 50000
^ (Line 6, Column 32, Position 345)
In context MSSQL-OPTIONS:While parsing MSSQL-OPTIONS.
Might I add, that the parser also chokes on 'matching' as in 'excluding table names matching' for me? It only accepts 'like' in MSSQL, not 'matching' as proposed by documentation? So the official example from the documentation
excluding table names matching 'LocalAccount' in schema 'dbo'
chokes the parser which produces a similar fatal error.
