semantic-link-labs
semantic-link-labs copied to clipboard
Add notebook to list large semantic models
When migrating capacities, extra action is needed when moving workspaces between capacities in different regions if large semantic models exist in the workspace. It would be useful to have a notebook to list those semantic models.
I don't really think this warrants its own notebook. It can be done in 2 lines of code. Maybe in a script repository would be a better idea.
I do this when I run my daily inventory notebook (Get All Datasets):
#Get a list of all the datasets in the entire tenant
dfDatasets = spark.createDataFrame(labs.admin.list_datasets())
display(dfDatasets)
#Filter that list for only the datasets that are of the "large storage format" variety
dfLSFDatasets = dfDatasets[dfDatasets["Target Storage Mode"] == "PremiumFiles"]
lsfdatasetscnt = dfLSFDatasets.count()
print(f"There are {lsfdatasetscnt} large-storage format datasets in the tenant.")
display(dfLSFDatasets)
I just write this as a separate delta table, but you could easily create another column that designates a dataset as "Large" or "Small" instead.