Fix evaluate_boxes TypeError when ground truth has non-default index

[Copilot]

evaluate_boxes() raises TypeError when ground truth annotations have a non-default index and no predictions exist for that image. The issue occurs at lines 239-250 where Series with mismatched indices are passed to pd.DataFrame().

# This fails before the fix
ground_truth = gpd.GeoDataFrame(
    {"image_path": ["img.jpg", "img.jpg"], "label": ["Tree", "Tree"], ...},
    index=[100, 200]  # Non-default index
)
predictions = gpd.GeoDataFrame({"image_path": ["other.jpg"], ...})
evaluate_boxes(predictions, ground_truth)  # TypeError

Changes:

• Reset group index before DataFrame construction in the empty predictions branch (line 240), matching the existing pattern in the else branch (line 258)
• Add test case test_evaluate_boxes_no_predictions_for_image() with non-default indices

The fix ensures Series alignment by converting group to a default integer index before mixing with newly created Series.

[!WARNING]

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• astral.sh
  • Triggering command: /usr/bin/curl curl -LsSf REDACTED (dns block)
• huggingface.co
  • Triggering command: /usr/bin/python python -m pytest tests/test_evaluate.py -v (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Evaluate boxes fails when there are some ground that have predictions, but others do not.</issue_title> <issue_description>

Issue Summary

When evaluate_boxes encounters ground truth annotations for an image but no predictions for that image, it raises a TypeError during DataFrame construction at lines 239-250 in deepforest/evaluate.py.

Minimal Reproducible Example

import pandas as pd
import geopandas as gpd
from shapely.geometry import box
from deepforest.evaluate import evaluate_boxes

# Create ground truth with annotations for an image
ground_truth = gpd.GeoDataFrame(
    {
        "image_path": ["image1.jpg", "image1.jpg"],
        "label": ["Tree", "Tree"],
        "xmin": [10, 50],
        "ymin": [10, 50],
        "xmax": [30, 70],
        "ymax": [30, 70],
    }
)
ground_truth["geometry"] = ground_truth.apply(
    lambda row: box(row["xmin"], row["ymin"], row["xmax"], row["ymax"]), axis=1
)

# Create empty predictions (no predictions for image1.jpg)
predictions = gpd.GeoDataFrame(
    {
        "image_path": ["image2.jpg"],  # Different image
        "label": ["Tree"],
        "xmin": [10],
        "ymin": [10],
        "xmax": [30],
        "ymax": [30],
        "score": [0.9],
    }
)
predictions["geometry"] = predictions.apply(
    lambda row: box(row["xmin"], row["ymin"], row["xmax"], row["ymax"]), axis=1
)

# This raises TypeError
results = evaluate_boxes(
    predictions=predictions,
    ground_df=ground_truth,
    iou_threshold=0.4
)

Error Traceback

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File ".../pandas/core/frame.py", line 782, in __init__
    mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
  File ".../pandas/core/internals/construction.py", line 503, in dict_to_mgr
    return arrays_to_mgr(arrays, columns, index, dtype=dtype, typ=typ, consolidate=copy)
  File ".../pandas/core/internals/construction.py", line 114, in arrays_to_mgr
    ...
TypeError: ...

Root Cause

The issue occurs in evaluate_boxes function at lines 239-250. When image_predictions.empty is True, the code attempts to create a DataFrame using group.index.values from a groupby operation. The index from groupby may not align properly with the DataFrame constructor, especially when the group has a non-standard index structure.

Proposed Fix

Reset the index of group before using it in DataFrame construction:

# If empty, add to list without computing IoU
if image_predictions.empty:
    # Reset index to ensure proper alignment with DataFrame construction
    group = group.reset_index(drop=True)
    result = pd.DataFrame(
        {
            "truth_id": group.index.values,
            "prediction_id": pd.Series([None] * len(group), dtype="object"),
            "IoU": pd.Series([0.0] * len(group), dtype="float64"),
            "predicted_label": pd.Series([None] * len(group), dtype="object"),
            "score": pd.Series([None] * len(group), dtype="float64"),
            "match": pd.Series([False] * len(group), dtype="bool"),
            "true_label": group.label.astype("object"),
            "geometry": group.geometry,
        }
    )

This fix ensures that group.index.values produces a clean integer index that aligns properly with the other Series in the DataFrame constructor.

Agent statement

I identified the error and found the trace, but could not identify which error within the set on L240 that caused the error. I asked the agent to create the situation I knew was causing the error and document a reproducible example, which I tested. </issue_description>

<agent_instructions>Check the logic, if reasonable, follow the proposed fix, and add the reproducible example as a new test. Keep it short and minimal. </agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes weecology/DeepForest#1220

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

---

[!NOTE] Fixes evaluate_boxes crash with non-default ground-truth indices and no predictions; adds image_path to empty-results and a targeted test.

• Evaluation (src/deepforest/evaluate.py):
  • Fix: Reset group index in the empty-predictions branch of evaluate_boxes to avoid TypeError with non-default ground-truth indices.
  • Result schema: Add image_path to the per-image results when predictions are empty.
• Tests (tests/test_evaluate.py):
  • Add test_evaluate_boxes_no_predictions_for_image validating behavior with no predictions and non-default indices.
  • Add import of shapely.geometry.box for test geometry creation.

^{Written by Cursor Bugbot for commit 403634e776f6121e27a51d3cdf95b63c3dd87ad6. This will update automatically on new commits. Configure here.}