Folders should be optional

[wezell]

Getting your head around it

In a nutshell, think about how git or s3 or Google Drive treats folders - they basically don't. This is because they are object stores and care about objects, not folders and because

1. versioning folders and paths is hard
2. really, all any object needs to know is its path and using these paths, a system can build out a hierarchical folder tree for browsing

In dotCMS, a content object store, we've relied on the idea that a Folder is a content dependency, a thing that needs to exist before content can be pathed under it. Our issue in dotCMS is that Folders have multiple fields that have business meaning, fields like

• folder.inode
• identifier
• identifier.parent_path
• folder.name
• identifier.asset_name

all of which in some ways act like unique ids and make them hard to sync and have presented challenges when managing content. In dotCMS, Folders are a fragile entity.

Folders should be Optional

Every piece of content in dotCMS already has a non-optional path. Because of this, a piece of content should not care if its parent Folder exists. If the parent folders do not exist, that is fine, we can still render a hierarchical folder tree appropriately (maybe they are only entries in the identifier table with an optional entry in the folder table).

Folders can still be created in dotCMS

If a Folder does exist in data, they should work as they do now, and when doing permission lookups we should still recurse to find the closest existing parent folder. When we are looking for a file type we should do the same thing. In essence, folders should be thought of more like a type of categorization or tag which if it exists can be used to hang things off of, like permissions and default file types all of which should work like they do now.

Folder id generation - no collisions

~~We need to remove the Folder inheritance and Inode foreign key. If a folder is created or needed, for the folder's ID need to use some sort of hash to generate a consistent folder id based on the folders url, e.g. sha3(lowercase(host.hostname + parent_path + asset_name)) . This new id should be used as the id in both the folder table and the identifier table. We should also migrate the inode.idate and owner columns to the folder table creation_date and owner columns to the folder table.~~

~~If someone wants to "move" a folder that does exist, we should not update the folder record. Instead, we should create a new folder record that has the id generated as specified above for the new path and use this, deleting the old folder / identifier when done.~~

The same is true if a customer renames the hostname - we will create new folders with the proper ids and delete the old ones.

~~We will need a startup task that removes the inode/folder fk~~

We will need a startup task that updates all folder ids to match the pattern above.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity within the past 90 days. It will be closed in 30 days no further activity occurs. Thank you.

[john-thomas-dotcms]

First, just a note about a few things we may need to put some thought into for this conversion/transition, since folders are used in many areas:

• URL resolution
  • This might need some thought if we allow creation of Pages that are not in a folder
• Content browsing
  • e.g. Site Browser & file selection/upload modals
• Permissions & permission inheritance
• PP dependencies
• Menu generation
  • e.g. navtool and Menu Links, folder sort order, etc.
• Special folder paths
  • e.g. /applications/apivtl, /applications/themes and how it might affect resolution of things like DOTSASS, containers-as-files, etc.
• Velocity viewtools and directives
  • e.g. #dotParse(), #dotInclude(), etc.
• Site Search job include/exclude paths
• Handling of the path property in file/page objects that aren't in a folder

[john-thomas-dotcms]

Second, a question: As part of this transition, would it make sense to eventually convert folders into content (including a new Folder base type)?

That would allow users to do things like:

• Add custom fields to folders (a commonly requested feature).
• Create relationships between folders, between folders and content, etc.
• Easily get and navigate folder structures, using an ES or Lucene query (another commonly requested feature).

I'm not saying we should do this, just that it's probably a good time to ask the question.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.