dlt-hub
Corrupted dlt pipelines metadata
dlt version
1.7.0
Describe the problem
I've somehow managed to corrupt my dlt metadata at: "/Users/{username}/.dlt/pipelines" ( I don't know how )
Due to this dlt stopped creating it's tables in databases and I was getting the following error:
"(sqlite3.OperationalError) no such table: main._dlt_loads"
I had a hard time debugging what was the problem with my dlt afterwards as reinstalling dlt with pip, making fresh environments and etc. didn't help. When I manually deleted the dlt pipelines data dlt managed to work again.
Maybe consider also removing this pipeline data when dlt in uninstalled? Or maybe have a suggestion to clean this dlt pipelines metadata in the traceback message.
It's not an easy issue to find, debug and resolve for users currently.
Expected behavior
No response
Steps to reproduce
Corrupt the dlt pipelines metadata at: "/Users/username/.dlt/pipelines"
Operating system
macOS
Runtime environment
Local
Python version
3.12
dlt data source
No response
dlt destination
No response
Other deployment details
No response
Additional information
No response
Note: we should probably include pipeline dir location in improved PipelineStepFailed message
