fusionauth-issues icon indicating copy to clipboard operation
fusionauth-issues copied to clipboard

Issuing Signature failed tokens

Open avitsrimer opened this issue 2 years ago • 12 comments

Issuing Signature failed tokens

Description

It rarely issues Signature invalid tokens.

Guest 1-year token

eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2ODgzNjg3ODEsImlhdCI6MTY1NjgxMTg0MSwianRpIjoiNGE1ODNjZDMtZDllZS00ZjExLWE3M2YtYTJkOTk5YzNmNzBkIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AGwaLswrLuwvOHpwcf4ayrLp8OLBB2XLXSl05oKNpABGH7j4dRbOaRhmZD5vZm4eZxICWsaWCzSowNLzsLxL6g

Public key (the ES256 key was generated via Generate Elliptic in the FA interface)

-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEjnPvis4Oz4nOS3jqP0E1jzhR9prH
9ojU8rrL4Iua+8COu+qtxpQj2qtdEJ9uFSLg9SKfO+OKBjlPmLUh9AHHLg==
-----END PUBLIC KEY-----

Affects versions

v1.36.7 (current) v1.36.4

Steps to reproduce

It's happening only on production, and I have spent a few days reproducing it on the local environment - but without success. The validation step was also done manually with public key and token via python/php and jwt.io. So not related to the tool that validates it.

I'll explain how we get the wrong tokens. There are three flows when it happens.

Guest flow

  1. Request to POST /api/jwt/vend with usual data like strings and int values.
  2. Validate the access token from the response

Customer refresh flow

  1. Request to POST /api/jwt/refresh with a one-time refresh token.
  2. Validate the access token from the response

Customer refresh after patch flow

  1. Request to PATCH /api/user/{userId}
  2. Request to POST /api/jwt/refresh with a one-time refresh token.
  3. Validate the access token from the response

Expected behavior

Expected that FA would generate correct, but now it super rarely issues incorrect tokens - 0.0012% of all tokens daily (worst case).

Screenshots

image

Platform

Community guidelines

All issues filed in this repository must abide by the FusionAuth community guidelines.

Additional context

Forum link https://fusionauth.io/community/forum/topic/2115/issuing-signature-failed-tokens

avitsrimer avatar Jul 19 '22 14:07 avitsrimer

Thanks!

So, no reproduction steps, other than it sometimes happens in production. That's a tough one to debug.

Any commonalities across the various invalid tokens (contents, etc)? Does changing the key (rotating the signing key) resolve the issue?

mooreds avatar Jul 19 '22 23:07 mooreds

Hi, the tokens absolutely different and I don't see any commonalities. I gonna update my code a bit to rotate the signing key and will back to here once I have done it

avitsrimer avatar Jul 21 '22 11:07 avitsrimer

Any word, @avitsrimer ?

mooreds avatar Aug 16 '22 03:08 mooreds

hi @mooreds, sorry it's summer 🏖 It's still happening (~60 per month), but reissuing one more time fixes it for customers. I plan to rotate keys after 2 weeks.

avitsrimer avatar Aug 16 '22 07:08 avitsrimer

hi @mooreds, sorry it's summer

No problem, I get it!

It's still happening (~60 per month), but reissuing one more time fixes it for customers.

Are you saying that when you get an error with validation, you reissue the same exact JWT contents and the resulting token validates just fine? In that case, any chance you could log the contents of the JWTs that have the error so that you/we can look for a common thread between them (maybe they are all issued exactly on the minute or something else weird that is a timing issue).

mooreds avatar Aug 16 '22 14:08 mooreds

Are you saying that when you get an error with validation, you reissue the same exact JWT contents and the resulting token validates just fine?

Almost correct 😄 not on each validation - I added the validation step exactly after issuing to ensure. And did it for both classic issuing flow and vend-flow. I do validation via php library and do not use the validate FA endpoint

In that case, any chance you could log the contents of the JWTs that have the error

I'm logging the wrong tokens from the start of this problem. But the main thing is that I don't change anything in FA, I just call the same method to issue it one more time. I added logs if it would happen twice in a row, but it's not happening.

so that you/we can look for a common thread between them (maybe they are all issued exactly on the minute or something else weird that is a timing issue).

I can't share the customer tokens - because it has an email. But if you are keen to look at them I can share the guest's tokens. Anyway, I can add logs for each jwt issuing to see spikes 🤔 FYI my tz is UTC+4

avitsrimer avatar Aug 17 '22 16:08 avitsrimer

Ah, that's helpful data. So it isn't the JWT content that seems to cause the failure, it is the sequence of events.

Per the docs, https://fusionauth.io/docs/v1/tech/apis/jwt#validate-a-jwt

A valid JWT indicates the signature is valid and the payload has not be tampered with and the token is not expired.

Are you sure it is the signature check failing? Might it also be a timing issue (which will also fail JWT validation)?

mooreds avatar Aug 17 '22 18:08 mooreds

There are no spikes for broken tokens Screenshot 2022-08-17 at 20 11 44

The public key

-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEjnPvis4Oz4nOS3jqP0E1jzhR9prH
9ojU8rrL4Iua+8COu+qtxpQj2qtdEJ9uFSLg9SKfO+OKBjlPmLUh9AHHLg==
-----END PUBLIC KEY-----

A correct guest token

eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTAwMzAzMTAsImlhdCI6MTY1ODQ3MzM3MCwianRpIjoiMTNmZWNlY2ItYWI1MC00MDY5LWJiMDUtYTQ5ODcwZGMxZDI3IiwiYXBwbGljYXRpb25JZCI6ImQ2NDFmOTc4LWFmZTAtNGU4NS04NzdjLTY5Zjk1MTJiZTBjYSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjo0LCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MjQsIndlYnNpdGVJZCI6MTgsImxvY2FsZSI6ImVuX0dCIn0.82EW54B1It23N9Evw1jgGIPBcxCaSjVsqDnBqMJSNOCxkAftTS3eka-GM2uTZPbuRtniO7p5EYqLVM-ha_cLag

The list of guest's broken tokens:

[
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTIyODUwMTUsImlhdCI6MTY2MDcyODA3NSwianRpIjoiY2E2MWE1YjItZWU0MS00ZjllLWJiOWMtYjE3OGUxY2EyMTMxIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6Nywid2Vic2l0ZUlkIjo1LCJsb2NhbGUiOiJlbl9JRSJ9.um6Rwt4xJ9r9mqz9ZM_D04RBbbvYQu1YnJIMp7eYb3sANfTouTKTZzN9VBL1a5aHN5Zo3rxW4HuADEmJLos0AA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTIxNDY4OTIsImlhdCI6MTY2MDU4OTk1MiwianRpIjoiNjM2MTJjYmMtNTYxNy00NTgwLTk3NDgtNTc4ZWNjMTRmNGEyIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.0xCHsDMVS-UCZw7fLkzk1Ly06DMcNGTdJxIyHWs3QGIAJSvrMh8Lj33Cpm0fghJRlIpwsW7q6AmBEGFXLix4AA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTIxMjgxNTIsImlhdCI6MTY2MDU3MTIxMiwianRpIjoiYTEyYmY1ZjYtNDY5Zi00YzQ5LTlhNDAtOTU0OGE4NmExNDdjIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MTEsIndlYnNpdGVJZCI6OSwibG9jYWxlIjoiZGVfREUifQ.ZmlojenPKXTx5amJ3uJ6MtZA1e0k1pSE6X5745zPPUsALTcDqBd70uyW9WGWSkWuf1CIApBFLh4E5EhsZALcAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTIwMzE4MTIsImlhdCI6MTY2MDQ3NDg3MiwianRpIjoiZTBhODkwYzQtZTdlYS00M2MyLTk2YTEtMmJhMGE1OGM0MDFkIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.kD87jVyYBUL6wtza1Ch_Y_7znjcIpb8_iiYKREGMG_cAKHDjqQHxrsYfGMti2NyJBgb-AqZQa94zqGVf9zdrAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTIwMjQyNTEsImlhdCI6MTY2MDQ2NzMxMSwianRpIjoiNzhmMzU0NzItYzFmOC00ZTFiLTlmNDEtYTI5MzhkZjZkMTg0IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MTEsIndlYnNpdGVJZCI6OSwibG9jYWxlIjoiZGVfREUifQ.94gjoL_z5ujD5rHhW-v5G9wQmwcQ7JO8rMxZo49eQM4AbQfq9DcKqnt_nOWmvF_5OKlJiv84sP14-773m1U_AA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE5ODgyNzEsImlhdCI6MTY2MDQzMTMzMSwianRpIjoiZmFmNTdlZGYtNzU4NC00ZjNlLTk1ZDktMDk0Y2IwZmRhMmNlIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AEvbVA4Bx3iKq06yAIvyYRI4ur7ptSP8y3UVX89W8QBn4eiRiSl1rHQy61W7JjYsN_5glhzx9i-DbuKhEtUlmw",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE4MTA4MjIsImlhdCI6MTY2MDI1Mzg4MiwianRpIjoiZjU3ZDMzZjUtM2M1My00NDNjLWE4MzQtZjQyZmNmODAxZjI4IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.ABHGFzvLc_cbf5F4FsVlOLYX5VnHgn_TRbDuDBUK8QDhvogxfj2bNTxM2W0akpPGxU--fqBWNB_8dl8EUrzq_Q",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE3Mjg2MjUsImlhdCI6MTY2MDE3MTY4NSwianRpIjoiNDcyYjk1Y2ItNTM4ZC00OTE3LTgzNDYtZmFlMDQ0MzZmYzYzIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6Nywid2Vic2l0ZUlkIjo1LCJsb2NhbGUiOiJlbl9JRSJ9.ACKWH6dA4BsGM9f3zuO-X2FTXW7uck6bOqFjjUJN_wBw1YqKwGy3GV2M0d-pDjzupl2o2WC5bqP8KSwWjPXXLw",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE3MjU5NDYsImlhdCI6MTY2MDE2OTAwNiwianRpIjoiNGQ0NGFiZDYtNDZlZi00NDE3LWFhMzQtOTBmYjRlOGQ3NzZjIiwiYXBwbGljYXRpb25JZCI6ImQ2NDFmOTc4LWFmZTAtNGU4NS04NzdjLTY5Zjk1MTJiZTBjYSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjo0LCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MjQsIndlYnNpdGVJZCI6MTgsImxvY2FsZSI6ImVuX0dCIn0.-VXMzsEeYy3QKVv68e_EqgmZGf-0xOCsAhgSLEKwip0AA8xrjDAxZk2EHCumjKMhMqm-IQuJRT4eKtSEIDnuAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE2ODcwODcsImlhdCI6MTY2MDEzMDE0NywianRpIjoiMDViOGE2MjctMDAxYS00MDdjLTkyMWUtZmQxMDRhMGI4NzQ2IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.ewPbg-5QV7I5g-2qK99Lp3MEscvHKR4tejVpdOMr9o8AEGaRM4erqjoiab1gFXesL7XfL0cJkOtRj6O8GkTnAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE2NzQ0MjYsImlhdCI6MTY2MDExNzQ4NiwianRpIjoiMmRmNmViZTUtYzBiMS00ZmIwLTlkYjgtZGZlOGZiOGUyNDI4IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AB6Jiko-Kl389ohEPS8oigu9Ym7SN_xYarIxSkRXHwAznp_PWYlQfkEpEPVKVerByHgzvLXapPGD4ubGeHxfJg",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE0NTg2MDEsImlhdCI6MTY1OTkwMTY2MSwianRpIjoiMzgwZGE2ZTAtNGM4NC00YTJhLTg4N2YtMjBiYzAzZjE4MjNjIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6Nywid2Vic2l0ZUlkIjo1LCJsb2NhbGUiOiJlbl9JRSJ9.ADNupQFuSEpvC6P_jbkT24aoACblSS_m4K1ZMrvVVQBIt8ifygFlv8k6rrzxE5x1Zqx6LcAdSIjHwUS-sjkE9A",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE0NTE3MjAsImlhdCI6MTY1OTg5NDc4MCwianRpIjoiZDMwZmU4MjMtMjUwOC00M2RjLWEwM2ItYTFlZWFkMTI3YzEzIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AEhrbruPOo3eNs9cW-9nu9gV_F6vb_9i-CJzVrdMBgC1XUG6iTa26tERWhfabvmlTvSstYb1U05eZBvV8Hz2ug",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE0MzE4NDAsImlhdCI6MTY1OTg3NDkwMCwianRpIjoiZjE3ZDU1MjYtYTA4Yy00YjBlLThiMDYtZDM5MjkyNzc0MGJhIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AGGLOfB1LTAr0QIbmKn7h5idGxBYyGTJNYM8rVPwYwBT3EXUom-VtFpcyRPeOKRjI6EtSwIr-ydi4UNbHcb6Dw",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTE0MTE5MDMsImlhdCI6MTY1OTg1NDk2MywianRpIjoiZGFmOWI0NzYtYmVlZi00YzFlLTkzMzktZTliYzEyNTIxZjNkIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.FVI1WoFsmPvQHH_eTae2B45NYgNFLtvQ3_8OrH5pFssACu1ljRGe8oIL4z0M43K6_WABThzj52Vi1MwI9CREAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTEzNTUxMDQsImlhdCI6MTY1OTc5ODE2NCwianRpIjoiZWUzNGM2NDEtM2Q5Mi00ZDJiLTg2N2MtZmI2ODRhYzU1MmJhIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.WZdvY82szn4AYHIZL5o_xPD262JCg5kqc5kSFhRP2FYAAxS8H8jiQ7d353854uJ3LM6XvZXimOXdAsfz2bNBAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTEzMzIxMTksImlhdCI6MTY1OTc3NTE3OSwianRpIjoiY2VkYWZiMzUtOGEyZS00YjBlLWFlM2MtOWY3YWRmOTVlODgzIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.j51LWNjKmiF5PUIZZdLcifflenoWPp3NKw9vTkLtBfEAS_ZQpksVx2mvNTPpkArPOYNb21g34u4tag-2K8ULAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTEzMTE3OTQsImlhdCI6MTY1OTc1NDg1NCwianRpIjoiNTEwZTIzY2EtNTViNS00MTdkLWJhMzMtYjJhOWMyZTJkMzEzIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AADvVJXb50IEWNkUvKMKsiIsXfdYnBChLxsDGaXtQwCcjIUD3u11XPPWH1Gqe-nJ8L6_GIoiU0SaqWkMajuFGg",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTEwOTg2MzgsImlhdCI6MTY1OTU0MTY5OCwianRpIjoiODBhYzMxMGYtN2Q3My00ZmE2LWE0YjgtZDUyODRjNTc0MWYxIiwiYXBwbGljYXRpb25JZCI6ImQ2NDFmOTc4LWFmZTAtNGU4NS04NzdjLTY5Zjk1MTJiZTBjYSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjo0LCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MjQsIndlYnNpdGVJZCI6MTgsImxvY2FsZSI6ImVuX0dCIn0.Gs30-NgsNtTyzrffdRwrouif0cIcPDcRuJ3Ix3KeyTUAPSAj-AO3uKdme45A_EA3HIjQC1dnzshFOX8UMinzAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTEwNjMyNTUsImlhdCI6MTY1OTUwNjMxNSwianRpIjoiYzBkNmIyZDAtMmZlMS00OGVlLTk1M2MtNWJmZjYwZjI5NTI1IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AFc1P1ifzsTgB3kJ9GyVG_PN2QaqJaR0lSmCbbqGKgCoHGPnJhdiBpM95BQx5Z9ACaH8E6ysFF8xW8pxr1r1Gg",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTA5ODEwMzQsImlhdCI6MTY1OTQyNDA5NCwianRpIjoiOTI1ZGZhZDUtZWMyZi00MzhiLTk3MTYtZWEyODQ4NThhNzU2IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6Nywid2Vic2l0ZUlkIjo1LCJsb2NhbGUiOiJlbl9JRSJ9.AEiclhA5IaPv5pJj5Hx8rw9woYhEhvi7IShe-VBm4gAt1BCLlvf0UVF7NmcZiEPW0zMOqRUViHXfGfSWhBTXQw",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTA5NzIzMDQsImlhdCI6MTY1OTQxNTM2NCwianRpIjoiNmMxMjdlZTItNDgyZS00ZWQyLTgwOGQtMWQ4YzhmNWFhNzY5IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.ACsS9QGF97Ggx5_IWaGpyz6FOb5fBLYUS68yuD5koQDhRHtYOP1vu9cJSg-M-GGqtV-dQgjgU7hAt8WEr4-slw",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTA4MjUwMjIsImlhdCI6MTY1OTI2ODA4MiwianRpIjoiMGU0ZTU2M2EtNjNhZC00OWVmLWIyMGItODU4ZDQ5NWRiZTY3IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AB-zrUMszpZiT_mNdxYX2LbhHCnKAY4pXcwuULNwyADJcdJIic1HwVRUTyJxo1IKFgXeAVUB8ATGkqie_mvp9g",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTA4MTAyOTgsImlhdCI6MTY1OTI1MzM1OCwianRpIjoiN2UxNDkxNDgtYjgwMC00MGQ3LWFiYmYtZjUxM2ZhNTBhZDI5IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.i0nEYjyv6LzT2dc3ecfhsjmX-eu5IzkJmsxj_YTiQeYAOMZ22IoMYEVQGtxpc5zhX-Gl3XV79FQTNSdAdV7aAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTA2NjYzMTcsImlhdCI6MTY1OTEwOTM3NywianRpIjoiNTRlZTE4YzUtNGFjNi00YzUwLTlhNjUtYjRkYjk3YmY3MGE4IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.vHWecNjPQ9UXHoG6ulUMnweuieTA228HHmpT8tX6ZdsAfl9lvDL4zn4Smd6zHqdjbg2mh6hq3MzZoJbGdHlrAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTA1MTAyMTQsImlhdCI6MTY1ODk1MzI3NCwianRpIjoiZDRiMGY0MTgtZTgwYS00YzU5LTgxMDMtZmFlZDAwNGY4YWEzIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AG2ugFzBcem29hyE6S-OWUF4FFDDRE5qZLhYQCviPgAqbJ2RER2oloifAvHb49UMAGAIqAnMnqU_xOJKC8AHNg",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTA0NjU1NjEsImlhdCI6MTY1ODkwODYyMSwianRpIjoiZmVjNGZmOTQtYjJjOS00NzM4LWIwMjktMGI2NmYwNzQ2NWUyIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AE1ju9l-jPpu6O-bFwt4SXK7sMdvCuIDZyJbmWLrhgApdqeVnqc9uxBpP4kuhpcvqizzrTD9wQ4ZD0lr1M4qhQ",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTA0MzE1ODcsImlhdCI6MTY1ODg3NDY0NywianRpIjoiMWM2Y2U1ZGEtMTU3YS00OGMxLWE3ZjEtMzk5NTUxNDBkZWEwIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.xDmbUUudNdQRWRN9njUViCYyzUIExuLUbqysxrqCRPUAUgL_ClxWq3NXQh6enVIuxeXpyJnbeom51LBphLwPAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTAyNTIwMTgsImlhdCI6MTY1ODY5NTA3OCwianRpIjoiNTU0MjY5ZmMtMmQxYy00Yzc4LWExZjAtYjQ1ZGQ3ZmNiZDQ0IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MTEsIndlYnNpdGVJZCI6OSwibG9jYWxlIjoiZGVfREUifQ.KsUiG4vo4e3uhhQZu2fbLF3bZlfCMuPLznfKrOeBst0ANjSiSybImPJuaYr3d3mLcRUvxNfoSTrDKdgNMKd4AA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTAxNjMxMjYsImlhdCI6MTY1ODYwNjE4NiwianRpIjoiYmU5NjlkNGYtOTE2Zi00N2RmLTkyYmQtNTAyZDQ2Y2UxZmIwIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AENuYVh69Upf2Pm5B7v4BPHMh0HHk_Sk00ziptTPwgBOnHniuDtK_oev-NJbwBlqYHn0ax_PoJ3j4GgcXapTaA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTAxNTk5OTAsImlhdCI6MTY1ODYwMzA1MCwianRpIjoiZDEyNGQ4YzMtMGRjMy00YjFmLWExOTAtZmNlNjI1MGMwMDgwIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MSwid2Vic2l0ZUlkIjoxLCJsb2NhbGUiOiJlbl9HQiJ9.AAX5vpmxFFwuaDmzkVJ8OG-Ov48yprLzlDOulbwsRwAapvOUlfjkeplFhiR1h7E5Pbd1tYcejlukHh928Ja3ug",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTAxMTc2NTAsImlhdCI6MTY1ODU2MDcxMCwianRpIjoiNGVjOGM3OWUtYzYwNi00ODdjLTk2YTAtM2FiY2JhMWU0MDI3IiwiYXBwbGljYXRpb25JZCI6ImQ2NDFmOTc4LWFmZTAtNGU4NS04NzdjLTY5Zjk1MTJiZTBjYSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjo0LCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MjQsIndlYnNpdGVJZCI6MTgsImxvY2FsZSI6ImVuX0dCIn0.AGsALOutyvjOOJgUBntZLo3TORZxPr_8A8LU0T4pbwC-wf19-tvDpC5y_x4VrjZVXXV7m8-jTPaCAyFRClPFLQ",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTAxMTQ4NDIsImlhdCI6MTY1ODU1NzkwMiwianRpIjoiNWFkYWNlZDYtNzEzZC00MGU5LTljN2EtNmMyNDE0YjExMDIwIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MTEsIndlYnNpdGVJZCI6OSwibG9jYWxlIjoiZGVfREUifQ.ABZv2x5lzpck6d4iBljFZ4pIW91L61zqgwETmo96VAA1CIp28r0zx9I4AqzWuq65ihwKmsnCjIKNTVbWIwolGw",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTAwOTgwNTAsImlhdCI6MTY1ODU0MTExMCwianRpIjoiOTY1OTFhMjgtOGNhMS00ODVjLWEyMmUtOTJjMmE5YWQ2YTY0IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6MTEsIndlYnNpdGVJZCI6OSwibG9jYWxlIjoiZGVfREUifQ.0lFViRd2RByeyxpx3Rzxb5fhvQ1L5azEi1QmPSsE4qMAGLq7orAAQuFCd_zFzKvLhNTy_g5_HLXSDaKcX9fpAA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTAwMzY3MTIsImlhdCI6MTY1ODQ3OTc3MiwianRpIjoiZTFkZDU5ODMtYjliZi00YTAwLWI2MzUtM2QxMTQ3OGZmMGE0IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6Nywid2Vic2l0ZUlkIjo1LCJsb2NhbGUiOiJlbl9JRSJ9.ACYXCsOR01_uBjqHJurdw9SxTmy0eFtWrCn3SuZosQBKGq9ImUVJ1effAV6mr3OC9A-MOP8M54QA0sos3g_9CQ",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2OTAwMzE4MTMsImlhdCI6MTY1ODQ3NDg3MywianRpIjoiNzgwZmNlMzktYTUwNi00ZjQxLWEwMjktNjMxYzcwNjE1ZjA5IiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6Nywid2Vic2l0ZUlkIjo1LCJsb2NhbGUiOiJlbl9JRSJ9.K8lfPkp5prGlDTj4STrsysWLTdbVK6ZDjRduQwOcHVcANo2sAREJUH0Xw-l-JX93Nz9gBxGe8_Yxp3g-puQ_AA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2ODk5NzUwNTgsImlhdCI6MTY1ODQxODExOCwianRpIjoiMWRkMmZmZGQtYmFiOC00ZWJiLTg1M2MtYTRjZjAwNjMwMWFlIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6Nywid2Vic2l0ZUlkIjo1LCJsb2NhbGUiOiJlbl9JRSJ9.ABYFFoicAVMVAi5oWuIPNsgPEwSGRhBzQPcac_jGLABxrtI4ly5MmhbWO-loNTIlazXQ-C646vKr-azQgEDOfA",
    "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Il9YZkZkRUlwN24yTEdkcVJjUGdtQVcxM1NMWSJ9.eyJleHAiOjE2ODk5Mjg4ODEsImlhdCI6MTY1ODM3MTk0MSwianRpIjoiYjMwYTAxZTMtYWRhNS00MDA4LTgwODgtNDA4NDljNmE1MDkxIiwiYXBwbGljYXRpb25JZCI6ImMyNDMyMWZhLTc5OTctNDRjMy04ZGVkLTI1MGEyMjNkNDQ5YSIsImN1c3RvbWVySWQiOjAsImN1c3RvbWVyR3JvdXBJZCI6MCwic3RvcmVUeXBlIjoxLCJmYVVzZXJJZCI6MCwic3RvcmVJZCI6Nywid2Vic2l0ZUlkIjo1LCJsb2NhbGUiOiJlbl9JRSJ9.ABOobjwzeeB0eSEY9UEZIjDqyOUpv_rxLjVghnrncQD6mzOyRGUj5BctEnLUwyWvv3iHMAvUcjB0F1sDb6bMQg"
]

avitsrimer avatar Aug 18 '22 07:08 avitsrimer

I've deployed logs for each call to FA from my service, will back to you on Monday with a useful graph - I hope 🤞

avitsrimer avatar Aug 19 '22 11:08 avitsrimer

unfortunately, it's unrelated to any spikes, graph to prove image and cpu/ram usage also pretty low

avitsrimer avatar Aug 23 '22 07:08 avitsrimer

Hmmm. It sounds like we'll have to try to replicate locally with some load testing; I'm not sure when we'll be able to get to this, unfortunately. I'm glad you've found a workaround and really appreciate the details you've been able to share. I'm going to leave this issue open until we can triage it.

mooreds avatar Aug 24 '22 16:08 mooreds

Thank you, i will share the singing key with you once we rotate it.

avitsrimer avatar Aug 24 '22 18:08 avitsrimer