Cast inferred label to data type of label column

[rgleaton]

Overview

Prevents errors around trying to set ints in incompatible columns.

Description of changes

Cast the int values to be the same value as the label column in infer_label.

Author Checklist

• [x] Linting passes; run early with pre-commit hook.
• [x] Tests added for new code and issue being fixed.
• [ ] Added type annotations and full numpy-style docstrings for new methods.
• [ ] Draft your news fragment in new changelog/ISSUE.TYPE.rst files; see changelog/README.md.

[diehlbw]

I think #92 + #100 obviates the need for this update.
Let me know if you think it is still necessary, otherwise we should close this