Add FilePattern param to Import-DatabricksFolder method

[NowinskiK]

Would be good to have param like -FilePattern for this command to be able filtering files in the selected path. Example: Import-DatabricksFolder -BearerToken $token -Region $region -LocalPath 'c:\repo\Databricks\' -DatabricksPath '/Shared/Project1' -FilePattern 'ag*.py'

[simondmorias]

What would be the use case for this? Currently it imports all files in the folder that match a valid type:

$FileType = @{".py"="PYTHON";".scala"="SCALA";".r"="R";".sql"="SQL" }

If there is something else (like a .txt file) you will get a warning that it could not be uploaded.

My view is that you want the local folder to match the target folder - therefore all files should be uploaded.

I'm open to ideas though.

[NowinskiK]

Yes, I know. I checked the code, but that would help if one (or more) of your files are valid but you don't want to push it/them to the notebook. It might be many various scenarios for people; for me right now is a simple thing: I have 1 file (test.ipynb) which is for tests purposes and I don't want to push it to Production environment (non-prod is fine). Obviously, the parameter is optional. BTW 'ipynb' is not supported by the module right now, but I will create a separate issue for this.

[matallanas]

Is there a way to only upload a file? Because it seems that you maybe want to upload an specific notebook to databricks.

Thank you.

[simondmorias]

If the -FilePattern parameter just matches a single file then yes it will upload just the one file.

[matallanas]

But there is no -FilePattern, right?

[ekdz]

@simondmorias As there is a possibility to add files with extensions other than those listed above, is there any chance to update the code to deploy files, not only notebooks?