google-cloud-go icon indicating copy to clipboard operation
google-cloud-go copied to clipboard

bigtable: Allow microseconds for TimestampRangeFilter (again)

Open mmetto opened this issue 2 years ago • 6 comments

Is your feature request related to a problem? Please describe. Could you please remove code rejecting microseconds when defining TimestampRangeFilters?

Describe the solution you'd like

  1. Remove if-block here
  2. Update tests here (remove these 2 lines)

Describe alternatives you've considered Detect which API version is used (v1 vs. v2) and deny or allow microseconds for TimestampRangeFilter accordingly.

Additional context It seems v2 APIs/clients use microseconds now when defining TimestampRangeFilters. This breaks most tests we have after upgrading gcp packages to get our dev environments working on Mac M1.


The v2 proto defines TimestampFilter using micros:

# google.bigtable.v2.data:
// Specified a contiguous range of microsecond timestamps.
message TimestampRange {
  // Inclusive lower bound. If left empty, interpreted as 0.
  int64 start_timestamp_micros = 1;

  // Exclusive upper bound. If left empty, interpreted as infinity.
  int64 end_timestamp_micros = 2;
}

and here is Filters.java coming from com.google.cloud.bigtable.data.v2.models package:

        public static final class TimestampRangeFilter
      extends AbstractTimestampRange<TimestampRangeFilter> implements Filter {
    ...
    public RowFilter toProto() {
      com.google.bigtable.v2.TimestampRange.Builder builder =
          com.google.bigtable.v2.TimestampRange.newBuilder();

      switch (getStartBound()) {
        case CLOSED:
          builder.setStartTimestampMicros(getStart());
          break;
        case OPEN:
          builder.setStartTimestampMicros(getStart() + 1);
          break;
        case UNBOUNDED:
          break;
        default:
          throw new IllegalStateException("Unknown start bound: " + getStartBound());
      }

mmetto avatar Apr 22 '22 02:04 mmetto

Hi @telpirion can we get traction on this issue? Thanks in advance!

mmetto avatar May 18 '22 17:05 mmetto

Hello @telpirion kindly reminder about this issue.

mmetto avatar Jun 01 '22 20:06 mmetto

This is affecting my project as well. Thanks for the fix @mmetto!

fredsadaghiani avatar Jun 01 '22 20:06 fredsadaghiani

@telpirion @igorbernstein2 can I get your eyes on the linked PR? This is blocking our testing and requires manual changes to get the tests working properly with newest bigtable client/models locally, while also locking us to use old version of dependencies for our service. If you're not happy with the changes introduced, please feel free to reach out to me and let's get things worked out.

mmetto avatar Aug 02 '22 18:08 mmetto

This obscure error has caused me some issues too.

icio avatar Aug 10 '22 16:08 icio

@telpirion @igorbernstein2 any chance we can get traction on this issue?

mmetto avatar Aug 22 '22 17:08 mmetto

Closing as the Bigtable service behaves similarly to the emulator.

telpirion avatar Oct 26 '22 21:10 telpirion