platform-samples icon indicating copy to clipboard operation
platform-samples copied to clipboard

Add checkpoint feature to find_inactive_members.rb

Open asendia opened this issue 4 years ago • 5 comments

Script find_inactive_members.rb sometimes breaks while parsing large organization either caused by timeout or connection error. Checkpoint flag --checkpoint helps by storing Octokit API responses as json files in data/, so that during re-run it will first try to use stored json files instead of calling Github API from the beginning.

asendia avatar Nov 09 '19 12:11 asendia

Hi @asendia. I like this idea, and I have run into the problem that you are describing myself.

What help do you need to get this PR over the finish line? Do you need a review, or something else?

spier avatar Mar 20 '20 13:03 spier

I would also support this change and merge it once it has been tested by somebody else like @spier . I resolved the conflicts to upstream to the best of my knowledge already.

jonico avatar Mar 20 '20 14:03 jonico

I will try this out locally, and leave comments on this PR.

spier avatar Mar 20 '20 23:03 spier

I was wondering if instead of creating a custom caching solution, one could also use something like http://octokit.github.io/octokit.rb/#Caching instead? Don't have time right now to try it unfortunately, but maybe over the weekend.

spier avatar Mar 20 '20 23:03 spier

I was wondering if instead of creating a custom caching solution, one could also use something like http://octokit.github.io/octokit.rb/#Caching instead? Don't have time right now to try it unfortunately, but maybe over the weekend.

I am not too familiar with ruby as you can see from the code in my PR 🙇

My reasons:

  1. If we use eTag based cache it will still call github API during retry and it will still eat the request bandwidth.
  2. Does that cache persist after the script closes? When I used this script I experienced few numbers of unknown errors that made the script force closed.

CMIIW

asendia avatar Apr 20 '20 16:04 asendia