canvas-data-cli
canvas-data-cli copied to clipboard
instructure
Two data canvas data exports in the same night
The sync command doesn't know how to handle two separate data dumps on the same night performed by Instructure, if the second dump is just for the requests table and doesn't include the other tables. The sync will clear out all the rest of the datafiles locally and just sync the new requests table data only. You then have to use the fetch command to grab the other updated tables manually.
Why there is an export for just requests tables? Are those duplicate data or new data? Thanks for any clarification.
Thanks for the report and sorry for the issue! The Sync API in the portal is intended to account for this but appears we might be missing an edge case. Will look into it but it should be fixed later today naturally when the next export is complete.
As for why we need a separate export for just requests, Requests data is (at the moment) the only table that is delivered incrementally, the rest are full tables daily. Due to the way we try and expire old exports out of the API, we need to be able to roll up all the requests into a "full" snapshot of requests. We split up the export into its own export for visibility and ease, but as mentioned the sync api is intended to compensate for it so that it doesn't cause additional complexity for the end user.
Will look into this though and hopefully prevent it from happening again!
Addison
On Fri, Feb 10, 2017, 10:03 AM Mu [email protected] wrote:
Why there is an export for just requests tables? Are those duplicate data or new data? Thanks for any clarification.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/instructure/canvas-data-cli/issues/9#issuecomment-279000842, or mute the thread https://github.com/notifications/unsubscribe-auth/AAm1gwnrmzmAA_KW-6i3LVXm64auNf_Jks5rbJhmgaJpZM4L9gIL .
Thanks Addison, I was hoping my explanation was enough for someone to understand. I appreciate the quick response. If it 's fixed today in the sync api I can always test before the next export happens tonight.