Vertex AI Model Garden is unusable due to ignored endpoint configuration

[rolurq]

Checked other resources

• [x] I added a very descriptive title to this issue.
• [x] I searched the LangChain documentation with the integrated search.
• [x] I used the GitHub search to find a similar question and didn't find it.
• [x] I am sure that this is a bug in LangChain rather than my code.
• [x] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain_google_vertexai import VertexAIModelGarden

llm = VertexAIModelGarden(project="PROJECT_ID", endpoint_id="ENDPOINT_ID")
llm.invoke("What's the capital of France?")

And also

from langchain_google_vertexai import VertexAIModelGarden

llm = VertexAIModelGarden(project="PROJECT_ID", endpoint_id="ENDPOINT_ID", api_endpoint="DEDICATED_ENDPOINT")
llm.invoke("What's the capital of France?")

Error Message and Stack Trace (if applicable)

Traceback (most recent call last): File "/code/.venv/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable return callable_(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/code/.venv/lib/python3.12/site-packages/grpc/_interceptor.py", line 277, in call response, ignored_call = self._with_call( ^^^^^^^^^^^^^^^^ File "/code/.venv/lib/python3.12/site-packages/grpc/_interceptor.py", line 332, in _with_call return call.result(), call ^^^^^^^^^^^^^ File "/code/.venv/lib/python3.12/site-packages/grpc/_channel.py", line 440, in result raise self File "/code/.venv/lib/python3.12/site-packages/grpc/_interceptor.py", line 315, in continuation response, call = self._thunk(new_method).with_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/code/.venv/lib/python3.12/site-packages/grpc/_channel.py", line 1198, in with_call return _end_unary_response_blocking(state, call, True, None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/code/.venv/lib/python3.12/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking raise _InactiveRpcError(state) # pytype: disable=not-instantiable ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.FAILED_PRECONDITION details = "Dedicated Endpoint cannot be accessed through the shared Vertex AI domain aiplatform.googleapis.com. Please access the endpoint using its dedicated domain name '...'" debug_error_string = "UNKNOWN:Error received from peer ipv4:172.217.20.74:443 {grpc_message:"Dedicated Endpoint cannot be accessed through the shared Vertex AI domain aiplatform.googleapis.com. Please access the endpoint using its dedicated domain name '...'", grpc_status:9, created_time:"2025-02-19T15:51:50.346416714+00:00"}"

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "", line 1, in File "/code/.venv/lib/python3.12/site-packages/langchain_core/language_models/llms.py", line 387, in invoke self.generate_prompt( File "/code/.venv/lib/python3.12/site-packages/langchain_core/language_models/llms.py", line 760, in generate_prompt return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/code/.venv/lib/python3.12/site-packages/langchain_core/language_models/llms.py", line 963, in generate output = self._generate_helper( ^^^^^^^^^^^^^^^^^^^^^^ File "/code/.venv/lib/python3.12/site-packages/langchain_core/language_models/llms.py", line 784, in _generate_helper self._generate( File "/code/.venv/lib/python3.12/site-packages/langchain_google_vertexai/model_garden.py", line 105, in _generate response = self.client.predict(endpoint=self.endpoint_path, instances=instances) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/code/.venv/lib/python3.12/site-packages/google/cloud/aiplatform_v1/services/prediction_service/client.py", line 966, in predict response = rpc( ^^^^ File "/code/.venv/lib/python3.12/site-packages/google/api_core/gapic_v1/method.py", line 131, in call return wrapped_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/code/.venv/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable raise exceptions.from_grpc_error(exc) from exc google.api_core.exceptions.FailedPrecondition: 400 Dedicated Endpoint cannot be accessed through the shared Vertex AI domain aiplatform.googleapis.com. Please access the endpoint using its dedicated domain name '...'

Description

I'm trying to use langchain connected with Vertex AI Model Garden. I have a Llama model deployed in model garden that I'm able to access using the google provided endpoint. However when I try using langchain it throws the above error regarding the use of an incorrect domain endpoint.

I also tried providing the correct domain using api_endpoint=... but it throws the same error.

System Info

System Information

OS: Linux OS Version: #60 SMP Wed Feb 12 20:25:12 UTC 2025 Python Version: 3.12.9 (main, Feb 7 2025, 01:03:02) [GCC 12.2.0]

Package Information

langchain_core: 0.3.36 langsmith: 0.3.8 langchain_google_vertexai: 2.0.13

Optional packages not installed

langserve

Other Dependencies

anthropic[vertexai]: Installed. No version info available. google-cloud-aiplatform: 1.81.0 google-cloud-storage: 2.19.0 httpx: 0.28.1 httpx-sse: 0.4.0 jsonpatch<2.0,>=1.33: Installed. No version info available. langchain-mistralai: Installed. No version info available. langsmith-pyo3: Installed. No version info available. langsmith<0.4,>=0.1.125: Installed. No version info available. orjson: 3.10.15 packaging<25,>=23.2: Installed. No version info available. pydantic: 2.10.6 pydantic<3.0.0,>=2.5.2;: Installed. No version info available. pydantic<3.0.0,>=2.7.4;: Installed. No version info available. pytest: Installed. No version info available. PyYAML>=5.3: Installed. No version info available. requests: 2.32.3 requests-toolbelt: 1.0.0 rich: 13.9.4 tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available. typing-extensions>=4.7: Installed. No version info available. zstandard: 0.23.0