Make celery wait for geonode to launch

[EHJ-52n]

Description

This PR ensures that the celery container is waiting for the geonode container to launch before itself. By doing this, the migrations and other django init steps are only applied once.

This is achieved by two changes:

• Add netcat to the geonode container image used by django and celery service

• Add logic to wait via netcat for the geonode providing container. This is controlled by the following three environment variables:

  • CELERY_DONT_WAIT_FOR_GEONODE
  • CELERY_GEONODE_CONTAINER_HOSTNAME
  • CELERY_GEONODE_CONTAINER_PORT

  The first one can be used to disable the whole waiting procedure if not required. The other two are used by netcat to check for GeoNode

Checklist

Reviewing is a process done by project maintainers, mostly on a volunteer basis. We try to keep the overhead as small as possible and appreciate if you help us to do so by completing the following items. Feel free to ask in a comment if you have troubles with any of them.

For all pull requests:

• [x] Confirm you have read the contribution guidelines
• [ ] You have sent a Contribution Licence Agreement (CLA) as necessary (not required for small changes, e.g., fixing typos in the documentation)
• [x] Make sure the first PR targets the master branch, eventual backports will be managed later. This can be ignored if the PR is fixing an issue that only happens in a specific branch, but not in newer ones.

The following are required only for core and extension modules (they are welcomed, but not required, for contrib modules):

• [ ] There is a ticket in https://github.com/GeoNode/geonode/issues describing the issue/improvement/feature (a notable exemption is, changes not visible to end-users)
• [ ] The issue connected to the PR must have Labels and Milestone assigned
• [ ] PR for bug fixes and small new features are presented as a single commit
• [ ] Commit message must be in the form "[Fixes #<issue_number>] Title of the Issue"
• [ ] New unit tests have been added covering the changes, unless there is an explanation on why the tests are not necessary/implemented
• [ ] This PR passes all existing unit tests (test results will be reported by travis-ci after opening this PR)
• [ ] This PR passes the QA checks: flake8 geonode
• [ ] Commits changing the settings, UI, existing user workflows, or adding new functionality, need to include documentation updates
• [ ] Commits adding new texts do use gettext and have updated .po / .mo files (without location infos)

Submitting the PR does not require you to check all items, but by the time it gets merged, they should be either satisfied or inapplicable.

[cla-bot[bot]]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the users @EHJ-52n on file. In order for us to review and merge your code, please contact the project maintainers to get yourself added.

[cla-bot[bot]]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the users @EHJ-52n on file. In order for us to review and merge your code, please contact the project maintainers to get yourself added.

[codecov[bot]]

Codecov Report

Merging #8817 (9169485) into master (6363bac) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #8817   +/-   ##
=======================================
  Coverage   59.83%   59.83%           
=======================================
  Files         796      796           
  Lines       48039    48039           
  Branches     7410     7410           
=======================================
  Hits        28746    28746           
- Misses      17698    17700    +2     
+ Partials     1595     1593    -2     

[afabiani]

That's nice improvement @EHJ-52n thanks, I will test this and include in the geonode-project too.

[giohappy]

@EHJ-52n if the reason for this PR is to avoid double initialization we should first check why it happens to you. Celery already shouldn't re-initialize the DB, therefore either something is not correctly set up or something must be fixed for the solution already in place. I would avoid adding a new solution to the same problem if the only reason is that one.

If we find this feature useful beyond the initial reasons, ok, but in that case, we should avoid duplicated functionality.

[gannebamm]

Additionally, this could be backported to 3.3.x? @EHJ-52n Do you think the amount of work to backport it is doable?

[EHJ-52n]

Additionally, this could be backported to 3.3.x? @EHJ-52n Do you think the amount of work to backport it is doable?

AFAIK, this doesn't include any geonode specific changes, it is version agnostic.

@giohappy I agree with you. I remember that we cleaned up the tasks invocation, too. Hence, this might be somehow obsolete. In the master branch, this was fixed in 45a954bf6f01552743830c23167a00fd239421ff.

But it is still not fixed in the GeoNode project repository: https://github.com/GeoNode/geonode-project/blob/master/src/entrypoint.sh#L50-L95

[EHJ-52n]

@giohappy @afabiani @gannebamm

Any updates on this? Should I create a PR for the GeoNode project repository or just close this one?

[gitguardian[bot]]

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	Secret	Commit	Filename
-	Django Secret Key	c6100803e10a7dd082927c6c1479c647ae1e2930	.env_dev	View secret
-	Django Secret Key	c6100803e10a7dd082927c6c1479c647ae1e2930	.env_local	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secrets safely. Learn here the best practices.
3. Revoke and rotate these secrets.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

<sup>🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!</sup>

[giohappy]

Not needed anymore. This PR can be closed IMHO.

I agree