rsontam-tc

FYI https://github.com/datahub-project/datahub/blob/33339e2c8933bb3b989b4052ed1b3d308624f2a0/metadata-ingestion/src/datahub/ingestion/source/ge_profiling_config.py#L80 Thanks!

Thanks @hsheth2 , I created two bugs in case _table_ and _column_ level denies had different methods for processing. https://github.com/datahub-project/datahub/issues/5589 is for _table_ level denies. https://github.com/datahub-project/datahub/issues/5590 is for _column_ level...

Hello @hsheth2, Some progress, the following seems to be working... ``` profile_pattern: allow: - .* deny: - 'dvdrental.public.test_to_exclude_table_from_datahub_[3-5]' #- 'dvdrental.public.test_to_exclude_table_columns_from_datahub.secret_value' - 'dvdrental.public.test_to_exclude_table_columns_from_datahub.secret_value2' ``` However, specifying either or both of `ignoreCase`...

OK, let me try that option, TY

Hi Harshal, specifying `ignoreCase` at the same level as `profile_pattern` as below... ``` profile_pattern: allow: - .* deny: [] ignoreCase: true ``` ... causes the following error: ``` [2022-08-18 14:45:14,084]...

This can be closed @mohdsiddique. It took me some time and attention to compare the changes and @hsheth2's recommendation was bang on. I will go ahead and mark it closed....

FYI https://github.com/datahub-project/datahub/blob/33339e2c8933bb3b989b4052ed1b3d308624f2a0/metadata-ingestion/src/datahub/ingestion/source/ge_profiling_config.py#L80 Thanks!

This is covered in https://github.com/datahub-project/datahub/issues/5590. Closing this bug.

As to server specs I tried initially with 4vCPU 8GB RAM and then with 8 vCPU and 8GB RAM. Results/errors were the same. Thanks.

The following observation may or may not be related, however, it is affecting the way my test results may be relied on. tables=50 ``` #time sysbench --db-driver=pgsql --pgsql-host=xx --pgsql-port=5432 --pgsql-user=sbtest...