mozilla
Fix addons-dev->remote-settings integration
Around 2 month ago the integration with remote-settings that uploads built bloom filters (mlbf) stopped working.
see: https://mozilla.sentry.io/issues/4109568678/
Creating remote settings test account for addons_dev_allizom_org failed. [b'{"code":401,"errno":104,"error":"Unauthorized","message":"Please authenticate yourself to use this endpoint."}']
https://mozilla.sentry.io/issues/4109568698/
Remote settings account not created
https://mozilla.sentry.io/issues/4175705145 (this is the heartbeat)
Failed to contact Remote Settings server: 503 Server Error: Service Unavailable for url: https://settings-writer.prod.mozaws.net/v1/heartbeat
It may be connected to the changes made in https://github.com/mozilla/addons-server/pull/20593 - or to the changes in those services that prompted the pr to be be made.
┆Issue is synchronized with this Jira Task
Currently, we are using: Dev
REMOTE_SETTINGS_API_URL = 'https://remote-settings-dev.allizom.org/v1/'
REMOTE_SETTINGS_WRITER_URL = 'https://remote-settings-dev.allizom.org/v1/'
REMOTE_SETTINGS_WRITER_BUCKET = 'blocklists'
Stage
REMOTE_SETTINGS_API_URL = 'https://firefox.settings.services.allizom.org/v1/''
REMOTE_SETTINGS_WRITE_URL = 'https://remote-settings.allizom.org/v1/''
REMOTE_SETTINGS_WRITER_BUCKET = 'staging'
Prod
REMOTE_SETTINGS_API_URL = 'https://remote-settings.mozilla.org/v1/'
REMOTE_SETTINGS_WRITER_URL = 'https://remote-settings.mozilla.org/v1/'
REMOTE_SETTINGS_WRITER_BUCKET = 'staging'
Those seem correct. They are matching https://remote-settings.readthedocs.io/en/latest/getting-started.html
On dev though, we also have REMOTE_SETTINGS_IS_TEST_SERVER = True and some logic to create the remote settings account (presumably because we expect data to get wiped regularly), bucket and collection. I'm guessing that is what's broken...
I suspect we no longer need the REMOTE_SETTINGS_IS_TEST_SERVER logic. Remote settings documentation about the dev server mentions:
Until November 2021, we were using the Kinto demo server, which had no sign-off and was flushed everyday. This DEV instance is now running the same configuration as STAGE/PROD.
We've reset the credentials, and I've shared the new ones with Wei in https://mozilla-hub.atlassian.net/browse/SVCSE-2240 to get them installed on dev.
I see local-dev tag. I assume nothing has changed for QA, bloomfilter records are available for stage with https://firefox.settings.services.allizom.org/v1/buckets/blocklists/collections/addons-bloomfilters/records. (it's never been available on -dev)
It's now available on dev as well as part of this fix https://remote-settings-dev.allizom.org/v1/buckets/blocklists/collections/addons-bloomfilters/records
I've seen on slack the discussion about -stage.
I blocked this morning 10 unlisted versions on dev and stage.
It's working on -dev https://remote-settings-dev.allizom.org/v1/buckets/blocklists/collections/addons-bloomfilters/records I see all of versions blocked.
As for -stage https://remote-settings.allizom.org/v1/buckets/blocklists/collections/addons-bloomfilters/records is throwing a 403 forbidden.
The old url does not display any blocks https://firefox.settings.services.allizom.org/v1/buckets/blocklists/collections/addons-bloomfilters/records -> last time I've checked was on September 5th (I had no problems then)
I re-checked today dev and stage and further I've installed some add-ons and used remote settings for the bloomfilter records.
https://remote-settings-dev.allizom.org/v1/buckets/blocklists/collections/addons-bloomfilters/records was updated accordingly
same for https://firefox.settings.services.allizom.org/v1/buckets/blocklists/collections/addons-bloomfilters/records
The add-ons installed have been disabled