I'm getting an error when trying to save my SLEAP file as .slp after running inference that says "An error occured when attempting to save: Unable to create attribute (object header message is too large). Try saving your project with a different filename or in a different format". I saved my project with a shorter file name but still got the same error. I exported my data as csv and I had data but I need the sleap file to save the inference in the .slp format.

On the Anaconda prompt, it says "RuntimeError: unable to create attribute (object header message is too large)." I tried to re run it but still got the same issue. I am currently doing this with 150 videos at once which I have run through SLEAP in the past with no issue in saving. I tried to make another SLEAP file with just one video and see if that saved (with my normal length file name) after running inference with the same model and it worked without any issues.

Not sure if an update is needed, need to run fewer videos at once, or something else. Please advise!

Feb 17 '25 17:02 leo8163

Hi @leo8163,

From your description, it sounds like an error with how much data is being written to a single SLP file. But, I have a feeling there is more that we can do to handle storing this large a project... (see action items).

Action Items (to help us)

To help us pin down the issue, are you able to include more information from you terminal (this would let us see where in the SLEAP code the error occurs - i.e. in saving videos/provenance/labels)? A few lines ~5 before you see Traceback should be sufficient.

Work-around (as you've already discovered)

For training a model, it is good idea to sparsely annotate frames from across all videos, but then, for running inference on all frames, splitting slp files by video (or smaller video sets) would be the work around here.

Thanks, Liezl

Related (External) Posts:

Feb 18 '25 21:02 roomrys

Hi @roomrys thank you for your prompt response! This is the only image I have from the terminal. Unfortunately I have closed out of anaconda and am not able to go further back from the "Traceback" function. I might have to rerun that whole file to see if I get the same error. I have since then tried to run half the amount of videos (I started with 153 and now am doing 88) and the sleap file seems to be saving the inference like normal, so perhaps it is a file size issue. I have attached the image I was referring to earlier below. Again, thank you for your help!

    'start_timestamp': '2025-02-16 14:39:34.826657',
    'finish_timestamp': '2025-02-16 14:40:16.825106'
}

Saved output: C:/SLEAP/Retrieval Project AIM/Retrieval G1 Test\predictions\Ret_G1_MF_Test_v16_021625_AIM.slp.250216_1439
31.predictions.slp

Process return code: 0
Traceback (most recent call last):
    File "C:\Users\eisch\anaconda3\envs\sleap\lib\site-packages\sleap\gui\commands.py", line 1078, in _try_save
        Labels.save_file(labels=labels, filename=filename, as_format=extension)
    File "C:\Users\eisch\anaconda3\envs\sleap\lib\site-packages\sleap\io\dataset.py", line 1997, in save_file
        write(filename, labels, *args, **kwargs)
    File "C:\Users\eisch\anaconda3\envs\sleap\lib\site-packages\sleap\io\format\main.py", line 162, in write
        return disp.write(filename, source_object, *args, **kwargs)
    File "C:\Users\eisch\anaconda3\envs\sleap\lib\site-packages\sleap\io\format\dispatch.py", line 79, in write
        return adaptor.write(filename, source_object, *args, **kwargs)
    File "C:\Users\eisch\anaconda3\envs\sleap\lib\site-packages\sleap\io\format\hdf5.py", line 341, in write
        meta_group.attrs["json"] = np.string_(json_dumps(d))
    File "h5py\_objects.pyx", Line 54, in h5py._objects.with_phil.wrapper
    File "h5py\_objects.pyx", Line 55, in h5py._objects.with_phil.wrapper
    File "C:\Users\eisch\anaconda3\envs\sleap\lib\site-packages\h5py\_hl\attrs.py", line 103, in __setitem__
        self.create(name, data=value)
    File "C:\Users\eisch\anaconda3\envs\sleap\lib\site-packages\h5py\_hl\attrs.py", line 197, in create
        attr = h5a.create(self._id, self._e(tempname), htype, space)
    File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
    File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
    File "h5py\h5a.pyx", line 50, in h5py.h5a.create
RuntimeError: Unable to create attribute (object header message is too large)

Feb 20 '25 16:02 leo8163

That is perfect for us, thank you. I'll begin my analysis (and keep updating this comment) on what we can do to save larger projects.

Analysis

Starting point

All .slp and .pkg.slp files (excluding .analysis.slp files) are serialized from complex SLEAP objects to primitive types via the LabelsV1Adaptor.write method. This is the line of interest inside this method where things fail: https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/format/hdf5.py#L383-L384 .

Now, what is `d` and why is it so big - too big?

Initial `d`

The initial d contains everything but the labeled frame data.

d is initially defined at the start of the LabelsV1Adaptor.write method as: https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/format/hdf5.py#L285-L286 where the Labels.to_dict method returns the following dict: https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/dataset.py#L1931-L1946 , skipping the actual LabeledFrame data - which is good news since that would easily make our metadata WAY too large. Some other keys that could scale with project size are "suggestions" (all the SuggestionFrames from the Labeling Suggestions), "tracks" (all the Track objects in the project - used or unused), and "videos" (all the Videos in the project).

(Lack of) Modifications based on `save_frame_data`

There are no modifications for saving an slp to another slp.

In the next few lines, we modify d based on save_frame_data. However, we know that save_frame_data is False here because we call Labels.save_file (which then calls this LabelsV1Adaptor.write method) from this SaveProjectAs command: https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/gui/commands.py#L1071-L1079 where we opt not to pass in save_frame_data (this is only passed in as True when saving a .pkg.slp file). Hence, we use a default value of https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/format/hdf5.py#L270 and instead of entering the https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/format/hdf5.py#L288 condition, we enter: https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/format/hdf5.py#L307-L315 which has no affect for our analysis (see #462 for reasoning behind this condition).

Modifications based on `append`

We seem to remove all the scale-with-dataset keys here.

Next, we do or don't do some appending - potentially rewriting the entire dictionary d. Following the calls from Labels.save_file in the aformentioned SaveProjectAs command, we see that there is no append keyword argument passed in, so we use the default value of: https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/format/hdf5.py#L269C9-L269C15 . Thus, we skip the condition: https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/format/hdf5.py#L325 , and instead enter: https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/format/hdf5.py#L368-L381 . Hmm... and here we seem to remove all the keys that we had said before would scale with the project:

Some other keys that could scale with project size are "suggestions" (all the SuggestionFrames from the Labeling Suggestions), "tracks" (all the Track objects in the project - used or unused), and "videos" (all the Videos in the project).

. This leaves us scratching our heads since the very next line is where we run into the error.

Is it possible we enter append accidentally? Nein.

https://github.com/talmolab/sleap/blob/653144760882aa3f5c7f1199366d45aba0bccaed/sleap/io/format/hdf5.py#L325 ... say if we reason if (append and "json") in metadata_group.attrs? Nein.

In [1]: append = False

In [2]: append and "json"
Out[2]: False

In [3]: import h5py

In [4]: filename = "test.slp"

In [5]: with h5py.File(filename, "a") as f:
   ...:     meta_group = f.require_group("metadata")
   ...:     meta_group.attrs["format_id"] = 1.2
   ...:     if append and "json" in meta_group.attrs:
   ...:         print("True")
   ...: 

In [6]: with h5py.File(filename, "a") as f:
   ...:     meta_group = f.require_group("metadata")
   ...:     meta_group.attrs["format_id"] = 1.2
   ...:     if not (append and "json" in meta_group.attrs):
   ...:         print("True")
   ...: 
True

In [7]: with h5py.File(filename, "a") as f:
   ...:     meta_group = f.require_group("metadata")
   ...:     meta_group.attrs["format_id"] = 1.2
   ...:     if (append and "json") in meta_group.attrs:
   ...:         print("True")
   ...: 
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[7], line 4
      2 meta_group = f.require_group("metadata")
      3 meta_group.attrs["format_id"] = 1.2
----> 4 if (append and "json") in meta_group.attrs:
      5     print("True")

File h5py/_objects.pyx:54, in h5py._objects.with_phil.wrapper()

File h5py/_objects.pyx:55, in h5py._objects.with_phil.wrapper()

File ~/micromamba/envs/sa0/lib/python3.9/site-packages/h5py/_hl/attrs.py:282, in AttributeManager.__contains__(self, name)
    279 @with_phil
    280 def __contains__(self, name):
    281     """ Determine if an attribute exists, by name. """
--> 282     return h5a.exists(self._id, self._e(name))

File ~/micromamba/envs/sa0/lib/python3.9/site-packages/h5py/_hl/base.py:200, in CommonStateObject._e(self, name, lcpl)
    198 else:
    199     try:
--> 200         name = name.encode('ascii')
    201         coding = h5t.CSET_ASCII
    202     except UnicodeEncodeError:

AttributeError: 'bool' object has no attribute 'encode'

What keys (other than `"suggestions"`, `"tracks"`, or `"videos"`) could be too large?

? Still processing...

Feb 20 '25 18:02 roomrys

Error saving: Unable to create attribute (object header message is too large)

Action Items (to help us)

Work-around (as you've already discovered)

Analysis

Starting point

Now, what is d and why is it so big - too big?

Initial d

(Lack of) Modifications based on save_frame_data

Modifications based on append

What keys (other than "suggestions", "tracks", or "videos") could be too large?

Now, what is `d` and why is it so big - too big?

Initial `d`

(Lack of) Modifications based on `save_frame_data`

Modifications based on `append`

What keys (other than `"suggestions"`, `"tracks"`, or `"videos"`) could be too large?