onestop icon indicating copy to clipboard operation
onestop copied to clipboard

Onestop-clients update scripts

Open erinreeves opened this issue 3 years ago • 3 comments

Summary

  • [x] Update script constructors where used to take config location and now takes a dict
  • [x] Changed constructor parameters of metadata_type to classname_metadata_type so could have two metadata_types in same config if needed by user.
As a user
I want to be able to execute the scripts, in the scripts directory
So that I can perform the different supported actions onestop-clients has to offer.

Additional Tasks: Due to being unable to run the full OneStop stack locally I had to execute the scripts on bastion(cedar-devs). This required building an image of onestop-python-client, onestop-s3-handler, and onestop-sme to hub.docker so I could install that on bastion and have it pull the image down and deploy it to an EKS node (which is not able to see local files on bastion).

  • [x] Build image of onestop-python-client, onestop-s3-handler, and onestop-sme to hub.docker
  • [x] Able to use helm to create sme pod
  • [x] Scripts execute on bastion

erinreeves avatar May 20 '21 21:05 erinreeves

Need to revisit image names

erinreeves avatar Jun 01 '21 18:06 erinreeves

2021-06-05 03:30:20.364 INFO 1 --- [-StreamThread-1] org.apache.kafka.streams.KafkaStreams : stream-client [psi-registry-ad13de8e-a5f2-4c4a-95e4-daa0a3746902] State transition from RUNNING to ERROR 2021-06-05 03:30:20.366 ERROR 1 --- [-StreamThread-1] o.c.o.kafka.common.util.KafkaHelpers : Streams app entered a bad state: ERROR 2021-06-05 03:30:20.367 ERROR 1 --- [-StreamThread-1] org.apache.kafka.streams.KafkaStreams : stream-client [psi-registry-ad13de8e-a5f2-4c4a-95e4-daa0a3746902] All stream threads have died. The instance will be in error state and should be closed. 2021-06-05 03:30:20.367 INFO 1 --- [-StreamThread-1] o.a.k.s.p.internals.StreamThread : stream-thread [psi-registry-ad13de8e-a5f2-4c4a-95e4-daa0a3746902-StreamThread-1] Shutdown complete

erinreeves avatar Jun 05 '21 03:06 erinreeves

Pull request https://github.com/cedardevs/onestop-clients/pull/58

erinreeves avatar Jun 15 '21 15:06 erinreeves