simple londiste3 replication standby server error

[ozanyurt]

I'm trying to configure simple replication and my standby server log:

2012-11-12 11:12:17,372 4435 ERROR Job st3_l3simple_db2 crashed: format requires a mapping Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/skytools/scripting.py", line 563, in run_func_safely return func() File "/usr/local/lib/python2.7/dist-packages/pgq/cascade/consumer.py", line 199, in work return Consumer.work(self) File "/usr/local/lib/python2.7/dist-packages/pgq/consumer.py", line 280, in work self._launch_process_batch(db, batch_id, ev_list) File "/usr/local/lib/python2.7/dist-packages/pgq/consumer.py", line 309, in _launch_process_batch self.process_batch(db, batch_id, list) File "/usr/local/lib/python2.7/dist-packages/pgq/cascade/consumer.py", line 172, in process_batch self.process_remote_batch(src_db, tick_id, event_list, dst_db) File "/usr/local/lib/python2.7/dist-packages/londiste/playback.py", line 363, in process_remote_batch self.restore_fkeys(dst_db) File "/usr/local/lib/python2.7/dist-packages/londiste/playback.py", line 926, in restore_fkeys self.log.info('Creating fkey: %(fkey_name)s (%(from_table)s --> %(to_table)s)' % row) TypeError: format requires a mapping

[ozanyurt]

Sorry

I'm using Ubuntu 12.04 and Python 2.7.3

[markokr]

please also say: skytools version, psycopg2 version, postgresql version.

and what does "select londiste.version()" show?

[ozanyurt]

Skytools 3.1.2 Postgresql 9.2.1 Python 2.7.3 python-psycopg2_2.4.5-1build1

I made some workaround maybe it helps I commented out line 926 self.log.info('Creating fkey: %(fkey_name)s (%(from_table)s --> %(to_table)s)' % row) and error gone. After that /python/londiste/compare.py:56 has throw same exception. And I change this line to not formatted version and everything works fine.

[ozanyurt]

londiste.version() show 3.1.1

[wbloos]

I have the same issue on: Ubuntu 12.10 (Desktop) 64 bit Skytools 2.1.13-2 Postgresql 9.1.6 python 2.7.3 and "python3" 3.2.3 python-psycopg2 2.4.5

The error is: 2012-11-27 17:29:41,377 24728 CRITICAL Job test_to_subcriber crashed: <type 'exceptions.TypeError'>: 'format requires a mapping' (<traceback object at 0x7f4598ffda28>: [' File "/usr/lib/python2.7/dist-packages/skytools/scripting.py", line 507, in run_once\n return self.work()\n', ' File "/usr/lib/python2.7/dist-packages/pgq/consumer.py", line 175, in work\n self._launch_process_batch(db, batch_id, list)\n', ' File "/usr/lib/python2.7/dist-packages/pgq/consumer.py", line 207, in _launch_process_batch\n self.process_batch(db, batch_id, list)\n', ' File "/usr/lib/python2.7/dist-packages/pgq/consumer.py", line 409, in process_batch\n self.process_remote_batch(db, batch_id, event_list, dst_db)\n', ' File "/usr/lib/python2.7/dist-packages/londiste/playback.py", line 313, in process_remote_batch\n self.restore_fkeys(dst_db)\n', ' File "/usr/lib/python2.7/dist-packages/londiste/playback.py", line 719, in restore_fkeys\n self.log.info('Creating fkey: %(fkey_name)s (%(from_table)s --> %(to_table)s)' % row)\n'])

[wbloos]

provider=# select londiste.version();

version

2.1.12 (1 row)

[markokr]

I cannot reproduce it with same version of components. Although all they are self-compiled, so maybe some ubuntu patches mess something up.

Please try to find out what kind of object is the 'row' value it complains on.

[wbloos]

Works fine (cannot reproduce) on our old: Debian 5 (64 bit) skytools 2.1.8 (backports) skytools-modules-8.4 2.1.8 (backports) PostgreSQL 8.4.7 (backports) python 2.5.2 python-psycopg2 2.0.7-4

[wbloos]

It's the foreign key. Please find reproduce info below.

If i don't create the foreign key in the db, the error does not occur. If i leave either of the 2 tables out of the replication (db schema remains the same, so yes fkey), the error does not occur either. Also, there's no problem when i add the fkey the the database after adding the tables to the replication.

Here's the test case:

• create new databases provider and subscriber --psql on the provider: create table a(aid serial primary key, aval integer); create table b(bid serial primary key, aid integer, bval integer, constraint aid_fkey foreign key (aid) references a(aid)); insert into a (aval) select generate_series(1, 1000, 10); insert into b (aid, bval) select generate_series(1, 100), generate_series(9, 100, 10);
• on the subscriber: pg_dump provider|psql subscriber
• install and start pgq and londiste
• add tables on provider
• add tables on subscriber

[markokr]

I understand it's supposed to be fkey. The question was about python object - why python % operator thinks its not a "mapping"? Please add print(repr(row)) or something there, to see what kind of object it is.

[wbloos]

looks like row is: ['public.b', 'public.a', 'b_aid_fkey', 'alter table only public.b add constraint b_aid_fkey FOREIGN KEY (aid) REFERENCES a(aid)']

[wbloos]

btw for anyone reading this that has the same issue: a workaround is adding the fkey to the provider+subscriber database after you added the table to the londiste subscriber service.