airbyte
airbyte copied to clipboard
fix(amazon-seller-partner): ReportsAmazonSPStream, dataStartTime is after dataEndTime
…
What
We noticed that for records with timezones other than utc in some cases the ReportsAmazonSPStream.stream_slices method was returning dataStartTime > dataEndTime which was resulting in 400 Client Error from amazon-seller-partner's API with message dataStartTime is after dataEndTime.
The reason is this code:
while start_date < end_date:
end_date_slice = start_date.add(days=self.period_in_days)
yield {
"dataStartTime": start_date.strftime(DATE_TIME_FORMAT),
"dataEndTime": min(end_date_slice.subtract(seconds=1), end_date).strftime(DATE_TIME_FORMAT),
}
start_date = end_date_slice
because the timezone of start_date and end_date can be different, end_date can be in utc but the date from the records could have other time zone, in our case this timezone was +09:00
when calling strftime it converts the datetime object to "%Y-%m-%dT%H:%M:%SZ" format no matter the timezone it is in.
So even if the calculated dates where start_date was less than min(end_date_slice.subtract(seconds=1), end_date)
example 2024-10-14T10:51:34+09:00 and 2024-09-14T10:00:00+00:00 but the datestring created from it which is
2024-10-14T10:51:34 & 2024-09-14T10:00:00 results in 400 errors with dataStartTime is after dataEndTime
How
By adding
end_date = end_date.in_timezone(start_date.timezone)
We sync up the end_date timezone with the start_date timezone, so the stringified version of the datetime object is also relatively correct
Review guide
User Impact
Can this PR be safely reverted and rolled back?
- [x] YES 💚
- [ ] NO ❌
@adityamohta is attempting to deploy a commit to the Airbyte Growth Team on Vercel.
A member of the Team first needs to authorize it.