genna icon indicating copy to clipboard operation
genna copied to clipboard

database info collection query does not complete (timeout)

Open genez opened this issue 3 years ago • 0 comments

Hello, I am trying to map a third-party postres 9.5 database with more than 22000 (yes, twenty-two thousands) tables. My goal is to fetch some data in my Go application. Obviously I am interested in a couple of tables, the command line I am using is this:

genna model-named -c postgres://postgres:postgres@hostname/database?sslmode=disable -t hrdd_001employee00,hrdd_001employee02,hrdd_001employee11,codd_001subject03_m,codd_001subject04 -o models.go

Unfortunately the command never finishes processing. So I debugged the application and found that the query that is taking forever is at dizzyfool/genna/lib/store.go@180

In particular, the offending subquery is

select distinct
				 	kcu.table_schema as table_schema,
					kcu.table_name   as table_name,
					kcu.column_name  as column_name,
					array_agg((
						select constraint_type::text 
						from information_schema.table_constraints tc 
						where tc.constraint_name = kcu.constraint_name 
							and tc.constraint_schema = kcu.constraint_schema 
							and tc.constraint_catalog = kcu.constraint_catalog
						limit 1
					)) as constraint_types
				from information_schema.key_column_usage kcu
				group by kcu.table_schema, kcu.table_name, kcu.column_name

If I execute this statement on my database, this query never finishes If I add a where condition with table names filter, it is very slow but it completes.

What do you think about this change?

genez avatar May 07 '21 15:05 genez