aws-connected-device-framework icon indicating copy to clipboard operation
aws-connected-device-framework copied to clipboard

assetlibrary-export: doesn't handle large volumes of data

Open aaronatbissell opened this issue 4 years ago • 1 comments

Aws Connected Device Framework Affected Module(s):

assetlibrary-export

I'm submitting a ...

  • [x] bug report
  • [ ] feature request

Description:

The asset library export tool runs into a timeout issue when trying to list all the devices in our system. We have 1M+ devices and that may be causing issues.

Current behavior:

Load 1M+ devices in the system. Try to run the asset library export tool. See a timeout error.

Screen Shot 2021-11-08 at 7 59 18 AM

Expected behavior:

No timeouts - it should export the data

Steps to reproduce:

Load 1M+ devices in the system. Try to run the asset library export tool. See a timeout error.

Additional Information: I think this is already a know issue, I just wanted something in place to track it...

aaronatbissell avatar Nov 08 '21 13:11 aaronatbissell

The current assetlibrary-export module is not scalable in its current form for larger datasets. We are in the process of removing this module from CDF and replacing with a more native Neptune method of exporting. Will close this ticket once it has been released.

boardthatpowder avatar Dec 08 '21 20:12 boardthatpowder