Why is length of test_dataloader 2, but the equal size is 200

[hagersalehahmed]

Why is length of test_dataloader is 2, but the equal size of testing set is 200

Coding

max_encoder_length = 5
max_prediction_length = 3


training = TimeSeriesDataSet(
    train_df,
    time_idx="time_idx",
    target=["RR", "SpO2"],
    group_ids=["subject_id"],
    #min_encoder_length=max_encoder_length // 2,
    max_encoder_length=max_encoder_length,
    #min_prediction_length=1,
    max_prediction_length=max_prediction_length,
    static_categoricals=[],
    #time_varying_known_reals=["time_idx"],
    time_varying_unknown_reals=["RR", "SpO2"],
    target_normalizer=MultiNormalizer(
        [
            GroupNormalizer(groups=["subject_id"], scale_by_group=True)
            for _ in ["RR", "SpO2"]
        ]
    ),
    add_relative_time_idx=True,
    add_target_scales=True,
    add_encoder_length=True,
)


testing = TimeSeriesDataSet.from_dataset(
    training,
    test_df,
    predict=True,
   
)


batch_size = 16

train_dataloader = training.to_dataloader(
    train=True, batch_size=batch_size, num_workers=0
)

test_dataloader = testing.to_dataloader(
    train=False,  batch_size=len(test_df), num_workers=0
)

[fnhirwa]

Hey @hagersalehahmed, would you provide more context to this issue with a full reproducible code example?

[PoorRican]

@hagersalehahmed, predict_mode=True only produces 1 sample for validation per-group. I see you only have 1 group so I cannot explain why you get len(testing) == 2, but that should point you in the right direction.