Add depth option to 'Create groups based on directory names'

[cratcliff]

Is it possible to add a depth option to the 'Create groups based on directory names' in the configuration tab.

This would enable the user to control how many nested groups are created. For example, I have a directory structure like MyProject/Site1/project_data/Terrain. A depth option of 2 would create a single group called 'Site1' .

I can then quickly load data for all my sites without excess subgroups.

[gacarrillor]

That's a nice suggestion. Sometimes I've also ended up with too many subgroups.

There might be many possibilities here, @cratcliff. Could you explain a little bit your use case? Perhaps we can figure out a generic solution.

[cratcliff]

@gacarrillor before I found your plugin I had a script which would loop through and load known extensions. To help explain look at the following picture which shows the folder/data structure on the left and the resulting QGIS layer groupings on the right.

I used the following code with depths of 1 & 3 for the shapefiles and 1 & 4 for the tif files to produce the groupings on the right. I started using Load Them all for the the bounding box and alphanumeric filters.

import glob
import os
base_folder = r"C:\_data\MyProject"
load_ext = ['*.shp','*.tif']
depth_list = [2]

root = QgsProject.instance().layerTreeRoot()

for eaExt in load_ext:
    # find the list of files to load
    file_list = glob.glob(os.path.join(base_folder,'**',eaExt),recursive=True)
    file_list = sorted(file_list)
    print ('Found {} of file types {} to load'.format(len(file_list), eaExt))
    
    #loop through each file
    for eaFile in file_list:
        
        eaFile = os.path.normpath(eaFile)
        
        # break the folder structure into a list omiting the base folder
        sub = os.path.dirname(eaFile).replace(base_folder,'').split(os.path.sep)
        
        nested_groups=[]   # stores nested groups
        thisGroup=None
        for eaDepth in depth_list:
            if eaDepth <= len(sub)-1:       # make sure the depth folder is valid
                if len(nested_groups) == 0:  # ie top level group
                    # see if group already exists, if not create it
                    thisGroup = root.findGroup( sub[eaDepth])
                    if thisGroup is None:
                          thisGroup = root.addGroup(sub[eaDepth])
                else:
                    #same for further sub groups starting with the last in the list
                    thisGroup = nested_groups[-1].findGroup(sub[eaDepth])
                    if thisGroup is None:
                        thisGroup = nested_groups[-1].addGroup( sub[eaDepth])
                
                nested_groups+=[thisGroup]    # Track nested group layers

        if eaExt == '*.shp':
            layer = QgsVectorLayer( eaFile, os.path.basename(eaFile), "ogr" )
        else:
            layer = QgsRasterLayer(eaFile, os.path.basename(eaFile))
        
        if thisGroup is not None:
            QgsProject.instance().addMapLayer(layer, False)
            thisGroup.addLayer(layer)
        else:
            QgsProject.instance().addMapLayer(layer, True)
        
        if len(nested_groups) > 0:
            print ('Added layer {} to {}'.format(layer.name(),os.path.join (*[gp.name() for gp in nested_groups])))
        else:
            print ('Added layer {} to {}'.format(layer.name(),'Top'))

[gacarrillor]

In your example, what happened to modified_data/Site_Extent.shp? Since it has also a depth of 3, how was it filtered out?

On the other hand, how should we handle files that lie in between of the specified depths? For instance, let's imagine that SpatialData had a Shapefile (out of Transport and Hidrology folders). Would you load to QGIS Layer Tree with Site1 as its parent folder?

And finally, but very related, what if Transport had a subfolder (say, Subfolder) with another Shapefile. Would you load it to QGIS Layer Tree with Transport as its parent folder?

[cratcliff]

@gacarrillor sorry I forgot the modified_data/Site_Extent.shp when creating the picture.

On all your other points I would be putting the layers into the parent group. If there isn't a parent group then just load the layer. Using the updated code above if depth_list=[] just loads the layers without any grouping and depth_list=[0] a parent group called '' (blank) is created. You could also use the base folder name (ie MyProject) instead of blank.

I have updated the file structure and created couple of other variations as shown in the picture. The order of the layers will vary depending on how the file_list is sorted. These were created using the updated code above.

[gacarrillor]

Now, that makes sense. And I think it is the expected behavior.

Consider joining efforts and contributing in terms of code (as a Pull Request, since you have a script already) or even funding the feature.