Linux emulator failing on GHA ubuntu 20.04 and 22.04

[eddumelendez]

Using testcontainers cosmosdb module in this project and the cosmosdb linux emulator can not start when ubuntu 20.04 and 22.04 are used. Notice that using 18.04 everything is working fine. Logs attached here

[waszak]

Its probably this issue, so it would be best if you also update add coment there. Maybe if this issue will get enough attnetion then it will be fixed.

https://github.com/Azure/azure-cosmos-db-emulator-docker/issues/45

[DSpirit]

Any news on this? We want to introduce the CosmsoDB emulator in Azure DevOps and this would be a huge blocker due to EOL of Ubuntu 18.04.

[KrzychuR]

Lack of good CosmosDb emulation in general non-Windows environments is a big issue for us also. ~~Consumption~~ Serverless plan + kind of sandbox account on Azure is one off many workarounds but is far far away from proper docker base solution :( - and stable...

[mzurek-teldoc]

Same for me, please get it fixed

[DSpirit]

Until MS fixes this issue, my workaround would be to provide a self-hosted agent with vmSize D2_v5 or similar - according to #45 the emulator just fails on Intel Xeon E5-2673 chips, which are used in the D2_v3 series according to this: https://azure.microsoft.com/en-gb/pricing/details/virtual-machines/series/

For me the using Intel® Xeon® Platinum 8370C processor made the Emulator working again on Ubuntu 20.04 after setting up my own agent. However, I would like to switch to ubuntu-latest machine again, as having to manage build agents is unnecessary overhead.

[kostsamko]

Do we have any update on this? We are facing this issue for a long time and we cannot run our pipelines because now Ubuntu 18.04. is EOL.

[katperi]

Same problem here, as of this week ubuntu 18.04 got deprecated and azure pipelines stopped working

[Robinlievrouw]

Also have this issue, currently blocking us from running CosmosDB on azure devops pipelines

[LevYas]

The same problem, I had to disable tests in CI/CD. Any update?

[aomegax]

any news?

[DanteDeRuwe]

Any updates? This is causing issues in our azure pipelines too

[Bassonrichard]

We also have the same issue on Azure DevOps, tunning the emulator gets stuck in the starting state, think it might be something the start.sh script, is there maybe somewhere we can get the source for the Linux emulator dockerfile?

export PAL_NO_DEFAULT_PACKAGES=1
export PAL_LOADER_SNAPS=1
export PAL_NET_TCP_PORT_MAPPING=8081=8081:8900=8900:8901=8901:8902=8902:10250=10250:10251=10251:10252=10252:10253=10253:10254=10254:10255=10255:10350=10350

ipaddr="`hostname -I | grep -o '^[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+'`"

EMULATOR_PARTITION_SETTINGS="/masterpartitioncount=1 /partitioncount=${AZURE_COSMOS_EMULATOR_PARTITION_COUNT:-10} /defaultpartitioncount=${AZURE_COSMOS_EMULATOR_DEFAULT_PARTITION_COUNT:-0}"

EMULATOR_OTHER_ARGS="${AZURE_COSMOS_EMULATOR_ARGS:-/enablepreview}"

if [ ! -z "${AZURE_COSMOS_EMULATOR_ENABLE_MONGODB_ENDPOINT}" ]; then
        EMULATOR_OTHER_ARGS="$EMULATOR_OTHER_ARGS /EnableMongoDBEndpoint=${AZURE_COSMOS_EMULATOR_ENABLE_MONGODB_ENDPOINT}"
fi

EMULATOR_KEY="${AZURE_COSMOS_EMULATOR_KEY:-C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==}"

EMULATOR_OTHER_IP_ADDRESSES="`hostname -I | grep -o '[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+' | xargs | tr ' ' ','`"

EMULATOR_IP_ADDRESS=${AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE:-$ipaddr}

COSMOS_APP_HOME=/tmp/cosmos


EMULATOR_DEFAULT_CERTIFICATE="default.sslcert.pfx"
EMULATOR_CERTIFICATE_OPTION="/exportcert=c:\\${EMULATOR_DEFAULT_CERTIFICATE}"

if [ -z "${AZURE_COSMOS_EMULATOR_CERTIFICATE}${AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE}" ]; then
        # Work around to remove old emulator data since restarting with existing data does not work at this time.
        if test -d "${COSMOS_APP_HOME}/appdata"; then
            rm -fr "${COSMOS_APP_HOME}/appdata"
        fi

        mkdir -p ${COSMOS_APP_HOME}/appdata
else
        if test -d "${COSMOS_APP_HOME}/appdata"; then
            rm -fr "${COSMOS_APP_HOME}/appdata/log"
            rm -fr "${COSMOS_APP_HOME}/appdata/var"
            rm -fr "${COSMOS_APP_HOME}/appdata/wfroot"
            rm -fr "${COSMOS_APP_HOME}/appdata/Packages"
            rm -fr "${COSMOS_APP_HOME}/appdata/gateway.log"
        else
                mkdir -p ${COSMOS_APP_HOME}/appdata
        fi

        if [ ! -z "${AZURE_COSMOS_EMULATOR_CERTIFICATE}" ]; then
                if test -f ${AZURE_COSMOS_EMULATOR_CERTIFICATE}; then
                        cp -f ${AZURE_COSMOS_EMULATOR_CERTIFICATE} ${COSMOS_APP_HOME}/appdata/${EMULATOR_DEFAULT_CERTIFICATE}
                        EMULATOR_CERTIFICATE_OPTION="/importcert=c:\\${EMULATOR_DEFAULT_CERTIFICATE}"
                else
                        echo "ERROR: ${AZURE_COSMOS_EMULATOR_CERTIFICATE} not found"
                        exit 1
                fi
        else
                EMULATOR_CERTIFICATE_OPTION="/enabledatapersistence /importcert=c:\\${EMULATOR_DEFAULT_CERTIFICATE} /exportcert=c:\\${EMULATOR_DEFAULT_CERTIFICATE}"
                if test -f ${COSMOS_APP_HOME}/appdata/.system/profiles/Client/AppData/Local/CosmosDBEmulator/${EMULATOR_DEFAULT_CERTIFICATE}; then
                        cp -f ${COSMOS_APP_HOME}/appdata/.system/profiles/Client/AppData/Local/CosmosDBEmulator/${EMULATOR_DEFAULT_CERTIFICATE} ${COSMOS_APP_HOME}/appdata/${EMULATOR_DEFAULT_CERTIFICATE}
                else
                        if ! test -f ${COSMOS_APP_HOME}/appdata/${EMULATOR_DEFAULT_CERTIFICATE}; then
                                if test -d "${COSMOS_APP_HOME}/appdata/.system"; then
                                    rm -fr "${COSMOS_APP_HOME}/appdata/.system"
                                fi
                        fi
                fi
        fi
fi

./cosmosdb-emulator -w $COSMOS_APP_HOME/appdata -- /Microsoft.Azure.Cosmos.Emulator.exe /enablepreview /disableRIO /minimal $EMULATOR_PARTITION_SETTINGS /disablethrottling $EMULATOR_CERTIFICATE_OPTION /alternativenames=$EMULATOR_IP_ADDRESS,$EMULATOR_OTHER_IP_ADDRESSES /alternativeips=$EMULATOR_IP_ADDRESS,$EMULATOR_OTHER_IP_ADDRESSES /publicipaddressoverride=$EMULATOR_IP_ADDRESS /AllowNetworkAccess /Key=$EMULATOR_KEY $EMULATOR_OTHER_ARGS```

[Rick-van-Dam]

Can anyone from microsoft shed some light on this or is the emulator dead?

[KrzychuR]

I'm wondering if team responsible for this product understand that lack of proper non-Windows emulator, ideally as docker container, is one of important factor which might lead to the selection of an alternative to Azure Cosmos DB solution. Lack of clear information IF this issue will be solved is also alarming.
Sorry for being direct and blunt.

[drmarro]

Hi, have you found a workaround to the problem? I don't think you're pipelines have been disabled for months 😄

[LeeRothman1971]

This issue is not just effecting the Linux version. I can reproduce the issue locally on windows. The process is finishing before the emulator has started so no connections can be made. This is having a massive impact on our pipelines as our tests continuously fail because the emulator is not running. This has started to happen a few weeks ago when a new version of the emulator was deployed the azure hosted agents. Would it be possible to downgrade to the previous version until this issue has been resolved? In it's current state it is taking hours to get our pipelines to run successfully which doesn't really play with a continuously deployment strategy.

[diegosasw]

I'm new to Cosmos DB and it's being used in my new project, but if I were to be considering CosmosDb as a persistence mechanism and I saw that its docker emulator does not run on a Linux machine and the massive company behind that not precisely cheap product does not even answer in 2 years, I'd seriously consider alternative options.

How did you guys dealt with it? By simply not using the docker emulator? By disabling tests in CI/CD?

[alanta]

I had the test disabled in CI for a while but that was obviously not acceptable.

We're now using a free tier Cosmos DB instance with a 5 minute TTL on all collections.

We adjusted our tests to assume the collections are not 'clean'. We already adjusted the tests because the very lengthy startup of the emulator made it impossible to spin up the emulator on demand in a container. In local development we are still using the emulator, but we keep it running to eliminate the startup penalty.

[Rick-van-Dam]

We used an interface around cosmosdb and mocked that in memory. Of course that means queries might have different behavior but it seemed to be the best trade-off for us.

[razvangoga]

We have a dedicated azure instance we test against. Wach test run creates it's own collection with a random name and removes it when it finishes

On Mon, Jun 10, 2024 at 11:41 Rick van Dam @.***> wrote:

We used an interface around cosmosdb and mocked that in memory. Of course that means queries might have different behavior but it seemed to be the best trade-off for us.

— Reply to this email directly, view it on GitHub https://github.com/Azure/azure-cosmos-db-emulator-docker/issues/56#issuecomment-2157858361, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIETZRFCC3266DCJIQTCGLZGVYC7AVCNFSM6AAAAAAQBWEMCOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJXHA2TQMZWGE . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>