[Bug Report] Pretrained Llama 2 Model URI leads to a .tar.gz with dummy.txt file

[abdullahsych]

trafficstars

Link to the notebook Add the link to the notebook.

N/A

Describe the bug A clear and concise description of what the bug is.

I get the error: not a gzip file when I run the following code:

model_id, model_version = (
    "meta-textgeneration-llama-2-7b-f",
    "1.2.0",
)
scope = "inference" 

model_uri = model_uris.retrieve(
    model_id=model_id,
    model_version=model_version,
    model_scope=scope,
)

print(f"model_url: {model_uri}\n")

script_uri = script_uris.retrieve(
    model_id=model_id,
    model_version=model_version,
    script_scope=scope,
)

print(f"script_uri: {script_uri}\n")

instance_type = instance_types.retrieve_default(
    model_id=model_id, model_version=model_version, scope=scope
)

print(f"instance_type: {instance_type}\n")

image_uri = image_uris.retrieve(
    region=None,
    framework=None,
    image_scope=scope,
    model_id=model_id,
    model_version=model_version,
    instance_type=instance_type,
)

print(f"image_uri: {image_uri}\n")

model = Model(
    image_uri=image_uri,
    model_data=model_uri,
    source_dir=script_uri,
    entry_point="inference.py",
    role=role,
    name="some-model-name",
    predictor_cls=Predictor,
)

health_check_timeout = 300

predictor = model.deploy(
    initial_instance_count=1,
    instance_type=instance_type,
    endpoint_name="some-endpoint-name",
)

After inspecting the model uri on s3 https://s3.console.aws.amazon.com/s3/object/jumpstart-cache-prod-us-east-1?region=us-east-1&prefix=meta-infer%2Finfer-meta-textgeneration-llama-2-7b-f.tar.gz I noticed that it only contains a dummy.txt file.

To reproduce A clear, step-by-step set of instructions to reproduce the bug.

Logs If applicable, add logs to help explain your problem. You may also attach an .ipynb file to this issue if it includes relevant logs or output.

[WilliamDidier]

Hi,

I have the exact same issue, trying to setup some InferenceRecommencer job with this model. If you fetch the s3 file at the model_uri adress, you'll see that it is indeed not a gzip file. If you un-tar it using tar -xvf you'll see it only contains a dummy txt file.

I've seen this issue that seems to be related : https://github.com/aws/amazon-sagemaker-examples/issues/4337 but I can't work out how the fix that has been produced is related to the problem

I currently do not know how to make this work, if anyone is able to provide guidance regarding this issue it would be great.

[abdullahsych]

It only seems to work if I use the JumpStartModel class by only passing model_id and mode_version. But JumpStartModel is very limited and I am hoping someone can shed light on why this model_uri is not publicly available.