Is this normal to use 12G GPU memory to load the 200m model with default parameter?

[keefeleen]

Our code (following the sample)

model = timesfm.TimesFm(
    context_len=128,
    horizon_len=5,
    input_patch_len=32,
    output_patch_len=128,
    num_layers=20,
    model_dims=1280,
    backend=backend,
)
model.load_from_checkpoint(repo_id="google/timesfm-1.0-200m")

then our process gives the GPU memory usage around 12237MiB

[blackcat1402]

that is a lot

[R3xpook]

I have 64gb of ram and 64 of swap and it oom XD sooooo

[rajatsen91]

Can you try setting the environment variable: XLA_PYTHON_CLIENT_PREALLOCATE=false ?