OpenSearch icon indicating copy to clipboard operation
OpenSearch copied to clipboard

[Remove] Remaining Joda and Joda Dependency

Open nknize opened this issue 1 year ago • 23 comments

Joda was deprecated in Elasticsearch 7.x. OpenSearch should have proactively removed in 2.0. This PR removes the remaining Joda dependency and formatting logic in favor of java 8 time. It adds a new LegacyFormatName class, however, to ensure camelCase DateFormat is supported for indexes created in ElasticSearch 7.x (and carried through OpenSearch 1.x).

relates #5910 relates #8110

Closes https://github.com/opensearch-project/OpenSearch/issues/3995

nknize avatar Oct 26 '23 18:10 nknize

@github-actions Pull Request Checks / Verify Description Checklist (pull_request) Failing after 2s 🔫 (this PR template check is burdensome. It shouldn't fail PRs.)

nknize avatar Oct 26 '23 18:10 nknize

Gradle Check (Jenkins) Run Completed with:

  • RESULT: FAILURE :x:
  • URL: https://build.ci.opensearch.org/job/gradle-check/29090/
  • CommitID: 853e4668c83e1f44e6e57ce036bc05e657571b6f Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Oct 26 '23 18:10 github-actions[bot]

Compatibility status:

Checks if related components are compatible with change 07b930e

Incompatible components

Incompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git]

github-actions[bot] avatar Oct 26 '23 18:10 github-actions[bot]

@github-actions Pull Request Checks / Verify Description Checklist (pull_request) Failing after 2s 🔫 (this PR template check is burdensome. It shouldn't fail PRs.)

We try to address few things with this check:

  • follow the template (no matter maintainer or contributor)
  • ask people to report flaky test failure (no matter maintainer or contributor) till we get situation improved

reta avatar Oct 26 '23 19:10 reta

@github-actions Pull Request Checks / Verify Description Checklist (pull_request) Failing after 2s 🔫 (this PR template check is burdensome. It shouldn't fail PRs.)

@nknize This is a good criticism of the template for pull requests, want to make an issue to channel this discussion?

Note; this check on is triggered if you update the description, no commits/pushes required to go from failing to passing.

peternied avatar Oct 27 '23 16:10 peternied

@github-actions Pull Request Checks / Verify Description Checklist (pull_request) Failing after 2s 🔫 (this PR template check is burdensome. It shouldn't fail PRs.)

We try to address few things with this check:

  • follow the template (no matter maintainer or contributor)
  • ask people to report flaky test failure (no matter maintainer or contributor) till we get situation improved

I opened #10970

We shouldn't conflate flaky test failures with this issue. They're a separate issue tied to other bad behaviors (e.g., only relying on PRs for testing instead of testing locally) .

nknize avatar Oct 27 '23 17:10 nknize

Gradle Check (Jenkins) Run Completed with:

  • RESULT: FAILURE :x:
  • URL: https://build.ci.opensearch.org/job/gradle-check/29160/
  • CommitID: d8bab00690eaea767af276f9e4faaff89a9777af Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Oct 27 '23 17:10 github-actions[bot]

Gradle Check (Jenkins) Run Completed with:

  • RESULT: FAILURE :x:
  • URL: https://build.ci.opensearch.org/job/gradle-check/29159/
  • CommitID: be835c553fc374821992b4667ad911931e6d3f39 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Oct 27 '23 17:10 github-actions[bot]

@nknize minor one, I think this change request deserves changelog entry

reta avatar Oct 27 '23 17:10 reta

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE :grey_exclamation:
  • TEST FAILURES:
      1 org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica
  • URL: https://build.ci.opensearch.org/job/gradle-check/29380/
  • CommitID: 8dba7067615e18e327a00ef334f19174e7afb9d1 Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions[bot] avatar Oct 31 '23 15:10 github-actions[bot]

Codecov Report

Attention: Patch coverage is 80.16878% with 47 lines in your changes are missing coverage. Please review.

Project coverage is 71.23%. Comparing base (b01e483) to head (07b930e). Report is 598 commits behind head on main.

Files Patch % Lines
...a/org/opensearch/script/expression/DateObject.java 0.00% 40 Missing :warning:
...java/org/opensearch/common/time/DateFormatter.java 70.00% 3 Missing :warning:
...ain/java/org/opensearch/search/DocValueFormat.java 50.00% 1 Missing and 1 partial :warning:
...earch/script/expression/DateObjectValueSource.java 0.00% 1 Missing :warning:
...a/org/opensearch/index/mapper/DateFieldMapper.java 85.71% 0 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #10949      +/-   ##
============================================
+ Coverage     71.17%   71.23%   +0.05%     
+ Complexity    58810    58631     -179     
============================================
  Files          4883     4879       -4     
  Lines        277152   276187     -965     
  Branches      40281    40037     -244     
============================================
- Hits         197263   196735     -528     
+ Misses        63430    63004     -426     
+ Partials      16459    16448      -11     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Oct 31 '23 15:10 codecov[bot]

@nknize nice cleanup, curious why you've opted to use ZonedDateTime instead of OffsetDateTime (which it is recommended by the standard library):

  • It is intended that {@code ZonedDateTime} or {@code Instant} is used to model data
  • in simpler applications. {@code OffsetDateTime} may be used when modeling date-time concepts in
  • more detail, or when communicating to a database or in a network protocol.

AFAIK the ZonedDateTime is susceptible to daylight saving rules whereas OffsetDateTime is not (small example below):

ZonedDateTime.now().plusDays(7) -> 2023-11-07T13:44:56.092955-05:00[America/Toronto]
OffsetDateTime.now().plusDays(7) -> 2023-11-07T13:44:56.093087-04:00

reta avatar Oct 31 '23 17:10 reta

curious why you've opted to use ZonedDateTime instead of OffsetDateTime

Because ZonedDateTime is the same concept used in Joda which works off the TimeZone and with a change this large I wanted to preserve the logic (and minimize the blast radius) as much as possible for bwc. There's all sorts of interesting blog posts out about this very topic.

nknize avatar Oct 31 '23 18:10 nknize

Because ZonedDateTime is the same concept used in Joda which works off the TimeZone and with a change this large I wanted to preserve the logic (and minimize the blast radius) as much as possible for bwc.

That makes sense, for sure, thanks!

reta avatar Oct 31 '23 19:10 reta

:x: Gradle check result for 5599d3efe5dc1ee0e8bb738e040f375e3adc5cf6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] avatar Nov 13 '23 15:11 github-actions[bot]

:white_check_mark: Gradle check result for 07b930e073c1616f70fbc1419420b49f1d2d0686: SUCCESS

github-actions[bot] avatar Nov 13 '23 17:11 github-actions[bot]

This PR is stalled because it has been open for 30 days with no activity.

Hi @nknize, the PR is stalled. Is this being worked upon?

ticheng-aws avatar Jan 08 '24 23:01 ticheng-aws

These PRs have been ready for review / approval for quite a while. Either no one has been comfortable approving them or everyone is busy with their own tasks. I'll rebase when there's a need (approval or change) otherwise I'll be rebasing in perpetuity.

nknize avatar Jan 09 '24 03:01 nknize

These PRs have been ready for review / approval for quite a while. Either no one has been comfortable approving them or everyone is busy with their own tasks. I'll rebase when there's a need (approval or change) otherwise I'll be rebasing in perpetuity.

To be fair, @reta reviewed it on November 13th, asked a question and got no reply.

msfroh avatar Jan 09 '24 07:01 msfroh

This PR is stalled because it has been open for 30 days with no activity.

This PR is stalled because it has been open for 30 days with no activity.

This PR is stalled because it has been open for 30 days with no activity.

won't be actively working unless needed

nknize avatar Jul 01 '24 21:07 nknize