Custom configurations are not passed to derivatives indexer

[pierre-nedelec]

Hello!

I'm trying to use a new Entity element, and I can't figure it out by looking at the documentation. Here's what I have, but it doesn't work:

from bids.layout import BIDSLayout, add_config_paths, Config, Entity
afq_entity = Entity('algorithm', pattern=r"[_/\\]+algo-([a-zA-Z0-9]+)", dtype=str)
my_config = Config('afq_bids_config', entities=afq_entity)
layout = BIDSLayout(root_dir, config=[ my_config])

Returns TypeError: Object of type Config is not JSON serializable. Also, I'm not trying to replace all entities but add one to the existing default ones.

I've looked at:

• https://bids-standard.github.io/pybids/generated/bids.layout.Config.html
• https://bids-standard.github.io/pybids/generated/bids.layout.Entity.html

[Remi-Gau]

What I usualy do in cases like this is to create a separate config.json file (starting from this one) and to load this config before running BIDSLayout.

[pierre-nedelec]

Thanks for the tip! However I can't quite get it to work.

Here's the file I created:

{
  "name": "bidsafq",
  "entities": [
    {
      "name": "algorithm",
      "pattern": "[_/\\\\]+algo-([a-zA-Z0-9]+)"
    }
  ]
}

And here's the code I've been using. I tried a few different things, to no avail so far. It currently returns an empty list.

add_config_paths(**{'bidsafq':'full/path/to/bids.json'}) # with or without that line -- although if run twice in the same session it errors out: ConfigError: Configuration 'bidsafq' already exists
layout = BIDSLayout(
    root_dir,
    derivatives=True,
    config=[
        "bids",
        "full/path/to/bids.json"  # or just "bidsafq" if `add_config_paths` was run
    ],
)
layout.get(algorithm="AFQ", return_type="filename")
>> []

# the regex works well though
import re
rex = re.compile(r"[_/\\]+algo-([a-zA-Z0-9]+)")
rex.findall(
    "main/derivatives/afq/sub-01/ses-01/sub-01_ses-01_run-01_dwi_space-RASMM_model-probCSD_algo-AFQ_desc-profiles_dwi.csv"
)
>> ['AFQ']

[effigies]

You will need to use validate=False in BIDSLayout(). If validate=True, then files that do not pass the BIDS validator regular expressions are dropped.

[pierre-nedelec]

Adding validate=False in BIDSLayout() didn't change the result: still []... Also I'm not sure that would be the issue, as I see these files if I just call layout.get(). Any thought?

[effigies]

If you call layout.get() and take the index of one of those files, you should be able to use file.get_entities() to see what BIDS attached to it.

[pierre-nedelec]

Just checked: it didn't attach the "algo" entity. Do I need to add a default_path_patterns in the afq_bids_config.json? Right now I only define entities.

[effigies]

Oh, I think this may be a bug where the configs are getting reset when you descend into derivatives. Try the following:

add_config_paths(**{'bidsafq':'full/path/to/bids.json'})
layout = BIDSLayout(root_dir)
layout.add_derivatives(Path(root_dir) / 'derivatives' / 'afq', config=['bids', 'derivatives', 'afq'])

[pierre-nedelec]

That works, thank you! Even with a slightly simplified version, which allows to include all derivative directories at once.

layout.add_derivatives(Path(root_dir) / 'derivatives', config=['bids', 'derivatives', 'bidsafq'])

What is the 'derivatives' for here? config=['bids', 'derivatives', 'bidsafq'] I tried without with no apparent changes.

[effigies]

If you're not using any other derivatives entities, there should be no difference.