InvalidRequestError: Request req_IfgzVoS7q6iToF: No such customer: cus_EgFIfB3ith04Dq; a similar object exists in live...

[Gregoor]

https://sentry.prod.mozaws.net/operations/mdn-stage/issues/7559508/

Resolver404: {'tried': [[<URLPattern '^media/(?:redesign/)?css/(?P<doc>.*)-min.css$'>], [<URLPattern '^media/(?:redesign/)?js/(?P<doc>.*)-min.js$'>], [<URLPattern '^media/(?:redesign/)?img(?P<suffix>.*)$'>], [<URLPattern '^media/(?:redesign/)?css(?P<suffix>.*)$'>], [<URLPattern '^media/(?:redesign/)?js(?P<suffix>.*)$'>], [<URLPattern '^media/(?:redesign/)?fonts(?P<suffix>.*)$'>], [<URLPattern '^media/uploads/demos/(?:.*)$'>], [<URLPattern '(?i)^(?P<one>.*)//(?P<two>.*)//(?P<three>.*)$'>], [<URLPattern '(?i)^(?P<one>.*)//(?P<two>.*)$'>], [<URLPattern '(?i)^samples/canvas-tutorial/2_1_canvas_rect.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/2_2_canvas_moveto.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/2_3_canvas_lineto.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/2_4_canvas_arc.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/2_5_canvas_quadraticcurveto.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/2_6_canvas_beziercurveto.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/3_1_canvas_drawimage.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/3_2_canvas_drawimage.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/3_3_canvas_drawimage.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/3_4_canvas_gallery.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_1_canvas_fillstyle.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_2_canvas_strokestyle.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_3_canvas_globalalpha.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_4_canvas_rgba.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_5_canvas_linewidth.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_6_canvas_linecap.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_7_canvas_linejoin.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_8_canvas_miterlimit.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_9_canvas_lineargradient.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_10_canvas_radialgradient.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/4_11_canvas_createpattern.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/5_1_canvas_savestate.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/5_2_canvas_translate.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/5_3_canvas_rotate.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/5_4_canvas_scale.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/6_1_canvas_composite.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/6_2_canvas_clipping.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/globalCompositeOperation.html$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/backdrop.png$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/bg_gallery.png$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/gallery_1.jpg$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/gallery_2.jpg$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/gallery_3.jpg$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/gallery_4.jpg$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/gallery_5.jpg$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/gallery_6.jpg$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/gallery_7.jpg$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/gallery_8.jpg$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/picture_frame.png$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/rhino.jpg$'>], [<URLPattern '(?i)^samples/canvas-tutorial/images/wallpaper.png$'>], [<URLPattern '(?i)^samples/domref/mozGetAsFile.html$'>], [<URLPattern '(?i)^samples/raycaster/input.js$'>], [<URLPattern '(?i)^samples/raycaster/Level.js$'>], [<URLPattern '(?i)^samples/raycaster/Player.js$'>], [<URLPattern '(?i)^samples/raycaster/RayCaster.html$'>], [<URLPattern '(?i)^samples/raycaster/RayCaster.js$'>], [<URLPattern '(?i)^samples/raycaster/trace.css$'>], [<URLPattern '(?i)^samples/raycaster/trace.js$'>], [<URLPattern '(?i)^samples/webgl/sample1$'>], [<URLPattern '(?i)^samples/webgl/sample1/index.html$'>], [<URLPattern '(?i)^samples/webgl/sample1/webgl-demo.js$'>], [<UR...
  File "redirect_urls/middleware.py", line 14, in __call__
    resolver_match = self.resolver.resolve(request.path_info)
  File "newrelic/hooks/framework_django.py", line 600, in wrapper
    return _wrapped(*args, **kwargs)
  File "newrelic/hooks/framework_django.py", line 588, in _wrapped
    result = wrapped(path)
  File "django/urls/resolvers.py", line 567, in resolve
    raise Resolver404({'tried': tried, 'path': new_path})

InvalidRequestError: Request req_IfgzVoS7q6iToF: No such customer: cus_EgFIfB3ith04Dq; a similar object exists in live mode, but a test mode key was used to make this request.
(9 additional frame(s) were not displayed)
...
  File "stripe/api_resources/abstract/api_resource.py", line 16, in refresh
    self.refresh_from(self.request("get", self.instance_url()))
  File "stripe/stripe_object.py", line 243, in request
    response, api_key = requestor.request(method, url, params, headers)
  File "stripe/api_requestor.py", line 122, in request
    resp = self.interpret_response(rbody, rcode, rheaders)
  File "stripe/api_requestor.py", line 373, in interpret_response
    self.handle_error_response(rbody, rcode, resp.data, rheaders)
  File "stripe/api_requestor.py", line 152, in handle_error_response
    raise err

InvalidRequestError: Request req_IfgzVoS7q6iToF: No such customer: cus_EgFIfB3ith04Dq; a similar object exists in live mode, but a test mode key was used to make this request.

[Gregoor]

Quite puzzled by this one atm, it suggests (to me) that we've used prod stripe keys on stage for a while?! Yet the mdn infra log shows that the keys have been there for 2 years now: https://github.com/mdn/infra/blame/7137c70345ea158426076600012a7b4c9133c8e4/apps/mdn/mdn-aws/k8s/regions/oregon/stage.mm.sh#L145

[Gregoor]

I wonder if this also happens to the newly signed up. So far we only have cases for Kadir and Schalk, who have both been loyal MDN stripe customers for a while. It doesn't seem to happen to me (anymore?). Is this maybe as simple as wiping record from people who got onto stripe at a point in time when Kuma was misconfigured on staging?

[schalkneethling]

Kadir and Schalk, who have both been loyal MDN stripe customers for a while.

Wait I have? I am not aware of that. Perhaps I used a test card? But that cannot be right?

[Gregoor]

Kadir and Schalk, who have both been loyal MDN stripe customers for a while.

Wait I have? I am not aware of that. Perhaps I used a test card? But that cannot be right?

If you have access to stripe: https://dashboard.stripe.com/customers/cus_EgFIfB3ith04Dq Your customer was created in March of last year.

[schalkneethling]

Kadir and Schalk, who have both been loyal MDN stripe customers for a while.

Wait I have? I am not aware of that. Perhaps I used a test card? But that cannot be right?

If you have access to stripe: https://dashboard.stripe.com/customers/cus_EgFIfB3ith04Dq Your customer was created in March of last year.

I wonder whether @peterbe experiences the same problem?

[tobinmori]

Quite puzzled by this one atm, it suggests (to me) that we've used prod stripe keys on stage for a while?! Yet the mdn infra log shows that the keys have been there for 2 years now: https://github.com/mdn/infra/blame/7137c70345ea158426076600012a7b4c9133c8e4/apps/mdn/mdn-aws/k8s/regions/oregon/stage.mm.sh#L145

Looks like the stage key, it has the pk_test* prefix. right?

[Gregoor]

Quite puzzled by this one atm, it suggests (to me) that we've used prod stripe keys on stage for a while?! Yet the mdn infra log shows that the keys have been there for 2 years now: https://github.com/mdn/infra/blame/7137c70345ea158426076600012a7b4c9133c8e4/apps/mdn/mdn-aws/k8s/regions/oregon/stage.mm.sh#L145

Looks like the stage key, it has the pk_test* prefix. right?

yup, exactly, which is good. That means stage did have the right key from the get-go. Which is what puzzles me. Would be really curious to hear whether someone apart from Kadir and Schalk experience that error.

@peterbe @escattone @tobinmori @mindy can you see the user edit page on staging without a server error? How about after creating a subscription?

[mindy]

I am able to see the user edit page on staging. Did not have a subscription, so made a subscription, cancelled it, and made a new one, and didn't run into any errors.

[Gregoor]

Thank you Mindy!

Just had a good chat with Peter about it who pointed me towards the obvious (in hindsight that is) source of the problem: Staging runs on production dumps! In an optimal world we wouldn't migrate any data to stage that is environment specific (i.e. stripe IDs, oauth tokens). For now I'd recommend that you @schalkneethling and @atopal use different accounts / delete those on stage.

Longer term, I'd gladly work with @limed to work on a partial migration strategy.