kinesis-python icon indicating copy to clipboard operation
kinesis-python copied to clipboard

Low level, multiprocessing based AWS Kinesis producer & consumer library

Results 17 kinesis-python issues
Sort by recently updated
recently updated
newest added

Hi, when i try to initialise the KinesisProducer object, it throws this error. ``` self.kproducer = KinesisProducer(stream_name=produce_stream_name) ``` Traceback: ``` ('2018-07-28 23:58:19,660 - botocore.loaders - MainThread - DEBUG - Loading...

This code is very helpful for understanding how to work with Kinesis Data Streams - thank you! While sifting through it, I noticed that `retries`, which controls the `ShardReader`'s back-off...

Makes it possible to use this library with https://github.com/localstack/localstack I just copied #28 and removed the change that mutates `item`, all credit to @nicholasball

- Add the ability to save a custom consumer state along with the log offset, allowing for stateful consumers to stop and resume gracefully - Maintains the original API when...

Hi, We are looking to use this library in our project and would be happy to pick up some of the project maintenance if needed. However we do need a...

- Add custom endpoint_url to consumer/producer and state - Return 'ShardId' in item

https://github.com/NerdWalletOSS/kinesis-python/blob/2bf355173de05a38f685df0f5a6000cff9a2076a/src/kinesis/producer.py#L126 By default, failure of individual records within a request does not stop the processing of subsequent records in a boto3 kinesis client put_records() request. This means that a returned...

While doing some testing of kinesis-python I was evaluating how well it handles live resharding. As old shards remained kinesis-python would spin up a ShardReader and shut it down for...

As documented here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kinesis.html#Kinesis.Client.get_shard_iterator I think those lines may be enough to support AT_TIMESTAMP behavior. It is not tested yet. It should be used this way: ```python3 from kinesis.consumer import...

I am new to this tech stack and have been going through the code. I am trying to find the method `should_start_shard_reader` [self.should_start_shard_reader](https://github.com/NerdWalletOSS/kinesis-python/blob/5305f7bda5ec0db72a3ff24ad141d29de115e191/src/kinesis/state.py#L126) in DynamoDB. Have I missed anything?