Memory issues with recursive script calls

[jjatria]

I've been trying to write a script to make lists of all files and directories under a given path. I've been doing that using recursive calls to script files (with runScript), and it's been working well (although maybe the problem is with the recursive script calls and not the Strings objects).

I came across two odd errors throwing segmentation faults or core dumps. And both seem to be triggered by lines in my script that would otherwise be pretty innocuous tasks:

1. Inserting a string with Insert string...
2. Removing a Strings object, both through the GUI or with commands like Remove or removeObject

The problems will come up for specific directory structures, but it's hard to predict which ones will be problematic. I don't think the problem is with the script, though, since the same script that will work in a directory with 2 subdirectories, crashes in a directory with 3 subdirectories. And the problem I find when removing the Strings object happens after my script has otherwise successfully finished.

Among the errors I get:

*** Error in `praat': free(): invalid next size (fast): 0x0e705d28 ***
Aborted (core dumped)

*** Error in `praat': corrupted double-linked list: 0x10148d20 ***
Aborted (core dumped)

*** Error in `praat': double free or corruption (out): 0x0edcd830 ***
Aborted (core dumped)

Other times, I just get a segmentation fault.

The problems are erratic, in that they appear only when querying some directories (with specific sub-directory structures), and sometimes only when using specific values for the max_depth argument, which modifies how deep the recursion should be. In these cases, running the script with a max_depth of 0 throws no errors, but specifying a value of eg. 2 might give me the double free or corruption error above.

This I tested on a machine running Kubuntu 14.04 (32-bit) using Praat 5.4.22. The problems exist both from the command line and from the GUI.

[PaulBoersma]

I was trying to install your plugins. Your website says that plugin_utils requires plugin_selection, but the reverse also seems to be the case.

[PaulBoersma]

OK, I have it working nicely, and it doesn't crash on my first tries. That was to be expected, sort of. The problem sounds like a nasty bug with freeing objects, and the cause could be anywhere. Does it still occur with 6.0.04 at yours?

[jjatria]

Your website says that plugin_utils requires plugin_selection, but the reverse also seems to be the case.

Really? That's a mistake. "utils" should only require "testsimple", and that only for testing. Where does it say so?

I have it working nicely, and it doesn't crash on my first tries.

Thanks for taking a look!

I've made changes to the script above that seem to work around the memory problems, so if you're trying the most recent version I would expect you not to find the problem. If you cloned the repository, you can try the "buggy" version of the script at v0.0.9.

I just tried that out with 6.0.04 and it still crashes for me. But using the more recent version of the script does not.

There were two parts in the script that could cause the error: inserting a string on some Strings objects, and removing certain Strings objects. The first part was solved by replacing this with this. The changes basically avoid using the Insert string... command.

After doing that, the problem with the Remove command disappeared, so I imagine the script was no longer creating those problematic Strings objects. Although what is it that I did that fixed it, I'm not entirely sure.

Basically, the broken version was making a list of all the directories below the directory given as root, including that directory. This list was then used to get the files under each of those directories.

The "fixed" version makes a list of all the subdirectories under the root directory, gets the files under those directories, and then appends a list of all the files immediately under the root directory. The change is small, but somehow it works.

[PaulBoersma]

"utils" should only require "testsimple", and that only for testing. Where does it say so?

I first installed only "utils", and one of the two commands ("Save selected objects") complained about not having "selection". I can see that plugin_utils/scripts/save_all.praat includes ../../plugin_selection/procedures/selection.proc

[jjatria]

Oops! Thanks for pointing that out.