`Estimator` environment variables are not passed to entry point script when using `HyperparameterTuner`

[yuntan]

Describe the bug Environment variables specified at Estimator class are not passed to the entry point script when running jobs by HyperparameterTune class.

To reproduce Dummy entry point script and notebook running sagemaker SDK

Expected behavior Training jobs run without failure and HyperParameterTuning job finishes without error.

Screenshots or logs Logs is shown in above link.

System information

• SageMaker Python SDK version: 2.59.1 (local)
• Framework name (eg. PyTorch) or algorithm (eg. KMeans): TensorFlow
• Framework version: 2.4.1
• Python version: 3.9.4 (local)
• CPU or GPU: CPU
• Custom Docker image (Y/N): N

related: #2311

[lorenzwalthert]

This bug already costed me hours. -.- Any ETA of a fix @ahsan-z-khan or at least no slient failing?

[alar0330]

@ahsan-z-khan, I also have this issue. Do we have any updates on this?

[lorenzwalthert]

I think the problem is that the underlaying botocore functionality does not exist, not even sure it exists on the API level.

[DougTrajano]

Hi folks, a workaround solution for that is to use the hyperparameters instead of environment in the Estimator API.

To don't change my estimator implementation, I added the code below in the hyperparameters code.

for k, v in estimator.environment.items():
    estimator._hyperparameters[k] = v

In the entry_point script, I added all unexpected arguments as environment variables

if len(remaining_args) > 0:
    for arg in range(len(remaining_args)):
        if remaining_args[arg].startswith("--"):
            os.environ[remaining_args[arg].strip("--")] = remaining_args[arg+1]

[lorenzwalthert]

I think we can close this as per #3614.

[trungleduc]

Closing as completed