fides icon indicating copy to clipboard operation
fides copied to clipboard

Not a bug but a help post

Open alisemihc opened this issue 3 years ago • 21 comments

I am using this code to generate YAML file for my synthetic data base to learn how to use fides:

fidesctl generate dataset db fides_resources/fidesfaker_dataset.yml --connection-string Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=fidesdatabase;Data Source=DESKTOP-ET8QFN2\SQLEXPRESS

image This is the image.

The error I get is at the bottom

Could someone please help me? Thank you.

  • Version:
  • OS:
  • Python Version:
  • Docker Version:

Additional context

Traceback (most recent call last): File "/fides/src/fidesctl/ctl/core/utils.py", line 59, in get_db_engine engine = sqlalchemy.create_engine(connection_string) File "", line 2, in create_engine File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned return fn(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 530, in create_engine u = _url.make_url(url) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/url.py", line 715, in make_url return _parse_rfc1738_args(name_or_url) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/url.py", line 776, in _parse_rfc1738_args raise exc.ArgumentError( sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string 'Provider=SQLOLEDB.1'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/bin/fidesctl", line 8, in sys.exit(cli()) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in call return self.main(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "/fides/src/fidesctl/cli/utils.py", line 192, in wrapper_func raise err File "/fides/src/fidesctl/cli/utils.py", line 188, in wrapper_func return ctx.invoke(func, ctx, *args, **kwargs) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/fides/src/fidesctl/cli/commands/generate.py", line 71, in generate_dataset_db _dataset.generate_dataset_db( File "/fides/src/fidesctl/ctl/core/dataset.py", line 302, in generate_dataset_db db_datasets = generate_db_datasets(connection_string=connection_string) File "/fides/src/fidesctl/ctl/core/dataset.py", line 273, in generate_db_datasets db_engine = get_db_engine(connection_string) File "/fides/src/fidesctl/ctl/core/utils.py", line 61, in get_db_engine raise Exception("Failed to create engine!") from err Exception: Failed to create engine! bash: Integrated: command not found bash: Persist: command not found bash: Initial: command not found bash: Data: command not found

alisemihc avatar Oct 03 '22 10:10 alisemihc

Also used this: fidesctl generate dataset db fides_resources/fidesfaker_dataset.yml --connection-string Server=C:\Program Files\Microsoft SQL Server\MSSQL15.SQLEXPRESS\MSSQL;Database=fidesdatabase;Uid=Administrator;Pwd=******;

the error I get with this is:

Usage: fidesctl generate dataset db [OPTIONS] OUTPUT_FILENAME Try 'fidesctl generate dataset db -h' for help.

Error: Got unexpected extra arguments (FilesMicrosoft SQL ServerMSSQL15.SQLEXPRESSMSSQL)

alisemihc avatar Oct 03 '22 10:10 alisemihc

try quoting the connection string, like Server=C:\Program Files\..., that might be the issue here

ThomasLaPiana avatar Oct 04 '22 20:10 ThomasLaPiana

@ThomasLaPiana

Quotations didnt work, it gave the same exception error as before.

Checked my error: Could not parse rfc1738 URL from string

My problem is that not the quotations but the connection string is wrong which I do not understand why.

image

This is my current database. I do not think something is wrong with it but it would be very nice to comment a document here or tell me how can I extract connection string from this. I am using Microsoft SQL Server Management Studio

alisemihc avatar Oct 04 '22 21:10 alisemihc

@LawyerDev68 generally the connection string should look something like found in this post

your current connection string is not valid as a URL which is where the error is coming from

ThomasLaPiana avatar Oct 04 '22 22:10 ThomasLaPiana

@ThomasLaPiana I installed postgresql to follow up fides tutorial properly.

image my postgresql database

--connection-string postgresql://postgres:[password]@localhost:5432/fidesdatabase

I checked the documentation here: https://hasura.io/learn/database/postgresql/installation/postgresql-connection-string/

and followed the exact steps

I am still getting the same error. Is there something I didnt see it properly? I am using Windows 10

alisemihc avatar Oct 07 '22 06:10 alisemihc

I'm running the following command and seeing it work:

root@64cc06461b53:/fides# fides scan dataset db --connection-string postgresql+psycopg2://postgres:fides@fides-db:5432/fides_test
Loading config from: /fides/.fides/fides.toml
Loading resource manifests from: .fides/
Taxonomy successfully created.
Loaded the following dataset manifests:
        fides_db
        fides_cache
Successfully scanned the following datasets:
        public

Annotation coverage: 100%

So my biggest guess here is that the password has something that isn't getting escaped properly 🤔

Do you get the same error when you quote it? This sounds like something we should be handling, if it is for sure the password causing this

ThomasLaPiana avatar Oct 07 '22 06:10 ThomasLaPiana

@ThomasLaPiana I had problems with password about this database, but then again if I'm able to enter the database with the password I typed through postgresql I think I should be able to do that in code too. But yeah I'll check couple of passwords, no password at all etc. Also yeah I tried to put it in the middle of the quotes but I'll try again with the new stuff too. Cant do that rn but will report here what happened

alisemihc avatar Oct 07 '22 06:10 alisemihc

image image

@ThomasLaPiana Something wrong with my SQL orFides ? Not sure. In the meantime I'll try to create YAML files manually

alisemihc avatar Oct 07 '22 10:10 alisemihc

@LawyerDev68 can you run the following command to verify postgres is up and ready at the expected host and port: pg_isready --host=localhost --port=5432? You should see a response of localhost:5432 - accepting connections

sanders41 avatar Oct 07 '22 12:10 sanders41

@sanders41 bash: pg_isready: command not found I got this, tried to add C:\Program Files\PostgreSQL\14 this to the path but it didnt work. I guess I made a mistake here or couldnt connect postgresql to the path. What do you think?

alisemihc avatar Oct 07 '22 12:10 alisemihc

This is an issue with the command. If it was an issue with connecting to the database the message would be localhost:5432 - no response. I think the path is C:\Program Files\PostgreSQL\14\bin. @ThomasLaPiana do you know if this is correct on Windows? You will also probably have to close and reopen your shell to pick up the new path.

sanders41 avatar Oct 07 '22 13:10 sanders41

@sanders41 tried to change the path to the way you proposed and restarted the shell, didnt work. image psql variable

alisemihc avatar Oct 07 '22 14:10 alisemihc

@sanders41 @ThomasLaPiana Also this is important to say, I installed postgresql plugin to my vs code and found my database through the plugin. So I think that means my connection string does not have any issue. It might be a bug. Problem still persists*

alisemihc avatar Oct 07 '22 15:10 alisemihc

Ok lets try one more thing. If you run netstat -a -o do you see 0.0.0.0:5432 in the Local Address list?

sanders41 avatar Oct 07 '22 17:10 sanders41

bash: netstat: command not found

@sanders41 Getting this error, but VS code is in the path, I can use shell to install other stuff like Faker etc.

alisemihc avatar Oct 07 '22 17:10 alisemihc

bash? You are on windows right? You should be able to run that command from powershell.

sanders41 avatar Oct 07 '22 17:10 sanders41

@sanders41 Yeah I'm on windows and I tried it on powershell but it didnt work. Hmm somethings missing in my system, what should I do? Any recommendations?

alisemihc avatar Oct 07 '22 18:10 alisemihc

@LawyerDev68 I tested locally with a postgres database at localhost in case that was the issue, but it completed successfully:

fides> fides scan dataset db --connection-string postgresql+psycopg2://postgres:fides@localhost:5432/fides_test
Loading config from: .fides/fides.toml
Loading resource manifests from: .fides/
Taxonomy successfully created.
Loaded the following dataset manifests:
        fides_db
        fides_cache
Successfully scanned the following datasets:
        public

Annotation coverage: 100%
fides> 

for us to better help you debug, can you show me the entire, unredacted command you're running for scanning now?

ThomasLaPiana avatar Oct 08 '22 03:10 ThomasLaPiana

@ThomasLaPiana Alright this is the entire code.

image

I have connected the database through postgresql extension in vs without a problem with the same parameters I entered below with the password etc.

My master password and postgres password are different from each other at pgAdmin4 if that means anything

PS C:\Users\Administrator\Desktop\Ders ve İş\VS> cd fides/ PS C:\Users\Administrator\Desktop\Ders ve İş\VS\fides> nox -s dev nox > Running session dev nox > Creating virtual environment (virtualenv) using python.exe in .nox\dev nox > docker build --target=dev --tag ethyca/fidesctl:local . [+] Building 4.3s (23/23) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 32B 0.0s => [internal] load .dockerignore 0.1s => => transferring context: 35B 0.0s => [internal] load metadata for docker.io/library/python:3.9-slim-bullseye 2.0s => [base 1/2] FROM docker.io/library/python:3.9-slim-bullseye@sha256:be8c64a08749503880e77755a1089272d4a201e395142aabd4bab54e7f547376 0.0s => [internal] load build context 2.0s => => transferring context: 51.49MB 2.0s => CACHED [base 2/2] RUN pip install -U pip 0.0s => CACHED [builder 1/15] RUN : && apt-get update && apt-get install -y --no-install-recommends curl git vim g++ gnupg gcc && apt-get clean && rm -rf /var/lib/apt/lists/* 0.0s => CACHED [builder 2/15] RUN : && apt-get update && apt-get install -y --no-install-recommends libpq-dev && apt-get clean && rm -rf /var/lib/apt/lists/* 0.0s => CACHED [builder 3/15] RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - 0.0s => CACHED [builder 4/15] RUN curl https://packages.microsoft.com/config/debian/11/prod.list | tee /etc/apt/sources.list.d/msprod.list 0.0s => CACHED [builder 5/15] RUN : && apt-get update && apt-get install -y --no-install-recommends apt-transport-https unixodbc-dev mssql-tools && apt-get clean && rm -rf /var/lib/apt/lists/* 0.0s => CACHED [builder 6/15] COPY dev-requirements.txt dev-requirements.txt 0.0s => CACHED [builder 7/15] RUN pip install -r dev-requirements.txt 0.0s => CACHED [builder 8/15] COPY requirements.txt requirements.txt 0.0s => CACHED [builder 9/15] RUN pip install -r requirements.txt 0.0s => CACHED [builder 10/15] COPY optional-requirements.txt optional-requirements.txt 0.0s => CACHED [builder 11/15] RUN pip install -r optional-requirements.txt 0.0s => CACHED [builder 12/15] COPY . /fides 0.0s => CACHED [builder 13/15] WORKDIR /fides 0.0s => CACHED [builder 14/15] RUN git rm --cached -r . 0.0s => CACHED [builder 15/15] RUN git reset --hard 0.0s => CACHED [dev 1/1] RUN pip install --no-deps -e ".[all,mssql]" 0.0s => exporting to image 0.1s => => exporting layers 0.0s => => writing image sha256:1b5d4328105ee2c37e83c454badf76a95e80d4bef86a2f55c6f69ea05a2edf8e 0.1s => => naming to docker.io/ethyca/fidesctl:local 0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them nox > docker build --target=frontend --tag ethyca/fidesctl:local-ui . [+] Building 2.1s (10/10) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 32B 0.0s => [internal] load .dockerignore 0.1s => => transferring context: 35B 0.0s => [internal] load metadata for docker.io/library/node:16 1.7s => [internal] load build context 0.1s => => transferring context: 28.09kB 0.0s => [frontend 1/5] FROM docker.io/library/node:16@sha256:b35e76ba744a975b9a5428b6c3cde1a1cf0be53b246e1e9a4874f87034222b5a 0.0s => CACHED [frontend 2/5] WORKDIR /fides/clients/ctl/admin-ui 0.0s => CACHED [frontend 3/5] COPY clients/ctl/admin-ui/ . 0.0s => CACHED [frontend 4/5] RUN npm install 0.0s => CACHED [frontend 5/5] RUN npm run export 0.0s => exporting to image 0.1s => => exporting layers 0.0s => => writing image sha256:a60ee6a6e094221ce4b10becd3ca4900365572af6173dfca8a9489a640bb3f2f 0.1s => => naming to docker.io/ethyca/fidesctl:local-ui 0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them nox > docker-compose up -d fidesctl-ui time="2022-10-08T09:49:47+03:00" level=warning msg="The "FIDESCTL__CLI__ANALYTICS_ID" variable is not set. Defaulting to a blank string." [+] Running 3/3

  • Container fides-fidesctl-db-1 Healthy 16.9s
  • Container fides-fidesctl-1 Started 18.2s
  • Container fides-fidesctl-ui-1 Started 19.6s nox > docker-compose run --rm -e FIDESCTL__CLI__ANALYTICS_ID --user=root fidesctl /bin/bash time="2022-10-08T09:50:07+03:00" level=warning msg="The "FIDESCTL__CLI__ANALYTICS_ID" variable is not set. Defaulting to a blank string." [+] Running 1/0
  • Container fides-fidesctl-db-1 Running 0.0s root@d06ab49436eb:/fides# fidesctl init

███████╗██╗██████╗ ███████╗███████╗ ██████╗████████╗██╗ ██╔════╝██║██╔══██╗██╔════╝██╔════╝██╔════╝╚══██╔══╝██║ █████╗ ██║██║ ██║█████╗ ███████╗██║ ██║ ██║ ██╔══╝ ██║██║ ██║██╔══╝ ╚════██║██║ ██║ ██║ ██║ ██║██████╔╝███████╗███████║╚██████╗ ██║ ███████╗ ╚═╝ ╚═╝╚═════╝ ╚══════╝╚══════╝ ╚═════╝ ╚═╝ ╚══════╝

Initializing Fidesctl...

Directory './.fides' already exists.

Configuration file already exists: ./.fides/fidesctl.toml

For example policies and help getting started, see: https://ethyca.github.io/fides/guides/policies/

Fidesctl initialization complete. root@d06ab49436eb:/fides# fides scan dataset db --connection-string postgresql://postgres:***********@localhost:5432/fidesdatabase bash: !@localhost: event not found root@d06ab49436eb:/fides# set +H root@d06ab49436eb:/fides# fides scan dataset db --connection-string postgresql://postgres:**********@localhost:5432/fidesdatabase Loading resource manifests from: .fides/ Taxonomy successfully created. Loaded the following dataset manifests: public Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3243, in _wrap_pool_connect return fn() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 310, in connect return _ConnectionFairy._checkout(self) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 476, in checkout rec = pool._do_get() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 146, in do_get self.dec_overflow() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get return self._create_connection() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection return _ConnectionRecord(self) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 371, in init self.__connect() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 666, in connect pool.logger.debug("Error on connect(): %s", e) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 661, in __connect self.dbapi_connection = connection = pool._invoke_creator(self) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 590, in connect return dialect.connect(*cargs, **cparams) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 584, in connect return self.dbapi.connect(*cargs, **cparams) File "/usr/local/lib/python3.9/site-packages/psycopg2/init.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) psycopg2.OperationalError: could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432? could not connect to server: Cannot assign requested address Is the server running on host "localhost" (::1) and accepting TCP/IP connections on port 5432?

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/fides/src/fidesctl/ctl/core/utils.py", line 64, in get_db_engine with engine.begin() as connection: File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3025, in begin conn = self.connect(close_with_result=close_with_result) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3197, in connect return self._connection_cls(self, close_with_result=close_with_result) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 96, in init else engine.raw_connection() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3276, in raw_connection return self._wrap_pool_connect(self.pool.connect, _connection) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3246, in _wrap_pool_connect Connection.handle_dbapi_exception_noconnection( File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2100, in handle_dbapi_exception_noconnection util.raise( File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3243, in _wrap_pool_connect return fn() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 310, in connect return _ConnectionFairy._checkout(self) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 476, in checkout rec = pool._do_get() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 146, in do_get self.dec_overflow() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get return self._create_connection() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection return _ConnectionRecord(self) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 371, in init self.__connect() File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 666, in connect pool.logger.debug("Error on connect(): %s", e) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 661, in __connect self.dbapi_connection = connection = pool._invoke_creator(self) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 590, in connect return dialect.connect(*cargs, **cparams) File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 584, in connect return self.dbapi.connect(*cargs, **cparams) File "/usr/local/lib/python3.9/site-packages/psycopg2/init.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432? could not connect to server: Cannot assign requested address Is the server running on host "localhost" (::1) and accepting TCP/IP connections on port 5432?

(Background on this error at: https://sqlalche.me/e/14/e3q8)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/bin/fides", line 8, in sys.exit(cli()) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in call return self.main(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "/fides/src/fidesctl/cli/utils.py", line 192, in wrapper_func raise err File "/fides/src/fidesctl/cli/utils.py", line 188, in wrapper_func return ctx.invoke(func, ctx, *args, **kwargs) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/fides/src/fidesctl/cli/commands/scan.py", line 75, in scan_dataset_db _dataset.scan_dataset_db( File "/fides/src/fidesctl/ctl/core/dataset.py", line 248, in scan_dataset_db db_datasets = generate_db_datasets(connection_string=connection_string) File "/fides/src/fidesctl/ctl/core/dataset.py", line 273, in generate_db_datasets db_engine = get_db_engine(connection_string) File "/fides/src/fidesctl/ctl/core/utils.py", line 67, in get_db_engine raise Exception(f"Database connection failed with engine:\n{engine}!") from err Exception: Database connection failed with engine: Engine(postgresql://postgres:@localhost:5432/fidesdatabase)! root@d06ab49436eb:/fides#

alisemihc avatar Oct 08 '22 07:10 alisemihc

can you try directly connecting to it via a python shell with sqlalchemy?

Follow this: https://docs.sqlalchemy.org/en/14/core/engines.html#postgresql

and then try to use that engine to talk to the database, such as:

engine = sqlalchemy.create_engine(connection_string)
with engine.begin() as connection:
    connection.execute("SELECT 1")

ThomasLaPiana avatar Oct 11 '22 02:10 ThomasLaPiana

I think it works @ThomasLaPiana
image Didnt receive any error message in any terminals

alisemihc avatar Oct 13 '22 11:10 alisemihc

Disclaimer: Writing this as a feedback and explaining my own experience. I think I wont be trying to use Fides for a while. Dealing with docker, linux systems etc. were too complicated for me. I am sure this is a good thing for senior devs and quality of the systems but for me, entrance bar was a bit high. I had a friend (Bachelors at Electrical Engineering, Masters at Data Science, has his own company, also a dev etc.) Said that this is too complicated for people like me and even mid-tier devs. Then again its an open-source system so I'm not even complaining. But yeah, just saying.

alisemihc avatar Oct 16 '22 12:10 alisemihc

Disclaimer: Writing this as a feedback and explaining my own experience. I think I wont be trying to use Fides for a while. Dealing with docker, linux systems etc. were too complicated for me. I am sure this is a good thing for senior devs and quality of the systems but for me, entrance bar was a bit high. I had a friend (Bachelors at Electrical Engineering, Masters at Data Science, has his own company, also a dev etc.) Said that this is too complicated for people like me and even mid-tier devs. Then again its an open-source system so I'm not even complaining. But yeah, just saying.

We really appreciate the candid feedback, and are actively working to make the user experience better so that it doesn't have that feeling of impenetrability. We believe privacy is a human right, and therefore the tooling to make that a reality should be as accessible as possible 🙂 Thank you again for taking the time to try things out and even open an issue to describe your experience, it's really valuable for us!

ThomasLaPiana avatar Oct 17 '22 08:10 ThomasLaPiana