Andrea

Fri 17 May Update: - Provisioned the database on GCP - Completed the work on the HTTP server - Deployed the necessaries resources on GCP with monitoring, etc - Done...

Fri May 24 update: - Setup of the new telemetry subdomain: https://telemetry.nearone.org - Test with testnet canaries - Added the new datasources to grafana - Created a dashboard with the...

Redirecting telemetry traffic from Pagoda to N1 is non trivial, so I'm falling back to have both endpoints for a while and rotating out the Pagoda endpoint in the future.

The new telemetry endpoint is active and nodes are sending traffic to it. Also, automatic chain detection has been implemented.

# Problematic heights **Operational issue(s):** block 109913255 **First resharding:** block 114580308 **Second resharding:** block 115185108 # Known failing queries ### Height 109913260 JSON query: ``` curl -X POST https://archival-rpc.mainnet.near.org \...

Done: - Recovery of all data - Catching up node to mainnet tip - Reduced cold DB storage through Tayfun's changes Ongoing: - Recovering cold DB write lag - no...

@qnduncan We haven't published the fixed snapshot yet. We plan to release it in the next days.

From now on DB snapshots published by Pagoda will contain the recovered archival data. Also, The fleet of public RPC nodes has been updated to use the fixed snaphots.

I have deployed a mainnet "canary-like" node with this build: [link](https://console.cloud.google.com/compute/instancesDetail/zones/europe-west4-a/instances/rate-limit-canary?q=search&referrer=search&project=nearone-node&hl=en&tab=details&pageState=(%22timeRange%22:(%22duration%22:%22PT1H%22))) I checked manually that config overrides trigger rate limiting. Next I'll leave the node running for a while and...

[Draft document](https://docs.google.com/document/d/1Uo4211zkjgU7lHEnrEBBqD997Tn19eIh67P6Czl4JUg/edit?usp=sharing) with tentative rate limits.