terraform-provider-databricks icon indicating copy to clipboard operation
terraform-provider-databricks copied to clipboard

Fix model serving resource

Open arpitjasa-db opened this issue 1 year ago • 6 comments

Changes

This PR fixes the provisioned throughput (PTP), external model (EM), and inference tables journeys via Terraform.

We also remove invalid validation checks and planned invalid checks. We do not add new checks since the source-of-truth should be left to the API backend, which will throw an informative error as necessary when invalid parameters are provided.

Should resolve https://github.com/databricks/terraform-provider-databricks/issues/3676.

Tests

We add new acceptance tests to cover the different endpoint types that can be created.

  • [X] make test run locally
  • [X] relevant change in docs/ folder
  • [X] covered with integration tests in internal/acceptance
  • [x] relevant acceptance tests are passing
  • [X] using Go SDK

arpitjasa-db avatar Jun 22 '24 01:06 arpitjasa-db

@mgyucht @alexott @tanmay-db @hectorcast-db I don't have access to the integration tests, so can someone let me know if they're passing or not?

Locally when I make the provider, the resources seem to be working fine, except for the external_model endpoint, which seems to get created in Databricks just fine, but Terraform never seems to realize it and waits for its "creation" indefinitely until it times out. Would appreciate some help looking into this as well thanks!

arpitjasa-db avatar Jun 22 '24 01:06 arpitjasa-db

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Please upload report for BASE (main@66a881f). Learn more about missing BASE report.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #3690   +/-   ##
=======================================
  Coverage        ?   81.59%           
=======================================
  Files           ?      196           
  Lines           ?    19699           
  Branches        ?        0           
=======================================
  Hits            ?    16074           
  Misses          ?     2668           
  Partials        ?      957           
Files Coverage Δ
serving/resource_model_serving.go 79.72% <100.00%> (ø)

codecov-commenter avatar Jun 22 '24 01:06 codecov-commenter

@mgyucht @alexott @tanmay-db @hectorcast-db I don't have access to the integration tests, so can someone let me know if they're passing or not?

Locally when I make the provider, the resources seem to be working fine, except for the external_model endpoint, which seems to get created in Databricks just fine, but Terraform never seems to realize it and waits for its "creation" indefinitely until it times out. Would appreciate some help looking into this as well thanks!

@alexott I have a theory the EM functionality broke with this commit, since the functionality has been broken since version 1.39.0, and the only other change in that version was the commit that @840 wrote to support EM in the first place (and I imagine that @840 validated it was working).

arpitjasa-db avatar Jun 22 '24 02:06 arpitjasa-db

Triggered integration tests for this

edwardfeng-db avatar Jun 25 '24 12:06 edwardfeng-db

Synced offline about integration test failures

edwardfeng-db avatar Jun 25 '24 13:06 edwardfeng-db

@edwardfeng-db looks like the PTP integration test is now passing. The EM test is still failing but that's because of the underlying Go SDK methods for create/wait that are causing this issue where the EM endpoint is being created, but the status is not being fetched correctly by the wait method. How can we fix that? @alexott @mgyucht @hectorcast-db @tanmay-db ?

arpitjasa-db avatar Jun 28 '24 19:06 arpitjasa-db