fixes for running on local dev env and set postgres as default

[guymguym]

Signed-off-by: Guy Margalit [email protected]

Explain the changes

1. set default database to postgres
2. allow endpoint to conditionally start sts and metrics ports so that standalone nsfs can be used.

Issues: Fixed #xxx / Gap #xxx

1. NA

Testing Instructions:

# set env
cat <<EOF >>.env
LOCAL_MD_SERVER=true
CREATE_SYS_NAME=demo
[email protected]
CREATE_SYS_PASSWD=DeMo1
NOOBAA_ROOT_SECRET='AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
JWT_SECRET=123
EOF

mkdir -p db_storage/
mkdir -p noobaa_storage/

# start db
initdb -D db_storage/
npm run db
createuser postgres
createdb nbcore

# start core
npm run web
npm run bg
npm run hosted_agents

# start endpoint
npm run s3

# start nsfs (for s3 compat backing store)
node src/core nsfs --http_port 6003 --https_port 6004 .

# call apis to set up the bucket with nsfs backing store (token + conn + pool + tier + policy + bucket)
function nbapi() {
    curl localhost:5001/rpc -sd '{
        "api": "'$1'",
        "method": "'$2'",
        "params": '$3',
        "auth_token": "'$TOKEN'"
    }'
}
TOKEN=$(nbapi auth_api create_auth '{
    "role": "admin",
    "system": "demo",
    "email": "[email protected]",
    "password": "DeMo1"
}' | jq -r '.reply.token')
nbapi account_api add_external_connection '{
    "name": "conn1",
    "endpoint_type": "S3_COMPATIBLE",
    "endpoint": "http://localhost:6003",
    "identity": "unused",
    "secret": "unused"
}'
nbapi pool_api create_cloud_pool '{
    "name": "pool1",
    "connection": "conn1",
    "target_bucket": "noobaa_storage"
}'
nbapi tier_api create_tier '{
    "name": "tier1",
    "attached_pools": ["pool1"]
}'
nbapi tiering_policy_api create_policy '{
    "name": "policy1",
    "tiers": [{ "order": 1, "tier": "tier1" }]
}'
nbapi bucket_api create_bucket '{
    "name": "bucket1",
    "tiering": "policy1"
}'

# put objects
alias s3='AWS_ACCESS_KEY_ID=123 AWS_SECRET_ACCESS_KEY=abc aws --endpoint http://localhost:6001 s3'
s3 ls
for i in `seq 33`; do echo "hello $i" | s3 cp - s3://bucket1/file$i; done
for i in `seq 33`; do s3 cp s3://bucket1/file$i -; done
s3 ls bucket1

[tangledbytes]

I think we need to swap test and test-postgres recipes in Makefile or should change run_npm_test_on_test_container.sh to spin up postgres DB instead of Mongo. WDYT @guymguym?

[guymguym]

@utkarsh-pro You must be right, I had a brief look at the failing tests and didn't see exactly where that was located, so thank you for pointing it out.

[dannyzaken]

@guymguym any updates?

[liranmauda]

@guymguym Do we need anything else in this PR? Do you know what is pending?

[liranmauda]

@guymguym not sure, maybe the timeout is too small, the tests fail with:

+ main@./run_test_job.sh:98 kubectl wait --for=condition=complete job/sanity --timeout=500s -n noobaa-tests
error: timed out waiting for the condition on jobs/sanity
+ main@./run_test_job.sh:99 test_exit_code=1

[dannyzaken]

@guymguym is this PR related to https://github.com/noobaa/noobaa-core/pull/7182?

[guymguym]

Most of the changes were included in newer PRs. Closing.