turbinia icon indicating copy to clipboard operation
turbinia copied to clipboard
verified publisher icon google

image_export.py failed for artifact JupyterConfigFile.

Open wajihyassine opened this issue 4 years ago • 3 comments

When running processing for test disk, I keep getting Task failure for * FileArtifactExtractionTask: image_export.py failed for artifact JupyterConfigFile.

When enabling debug_logs:

2021-09-13 21:40:59,647 [DEBUG] (MainProcess) PID:120696 <image_export_tool> Starting preprocessing.
2021-09-13 21:41:00,243 [DEBUG] (MainProcess) PID:120696 <image_export_tool> Preprocessing done.
2021-09-13 21:41:00,244 [DEBUG] (MainProcess) PID:120696 <engine> building find specification based on artifacts: JupyterConfigFile
2021-09-13 21:41:00,783 [DEBUG] (MainProcess) PID:120696 <artifact_filters> building find spec from artifact definition: JupyterConfigFile
2021-09-13 21:41:00,783 [DEBUG] (MainProcess) PID:120696 <artifact_filters> building find spec from path glob: %%users.homedir%%/.jupyter/jupyter_notebook_config.py
2021-09-13 21:41:00,783 [WARNING] (MainProcess) PID:120696 <image_export> Unable to build collection filters with error: No valid file system find specifications were built from artifacts.

wajihyassine avatar Sep 13 '21 21:09 wajihyassine

Interestingly i do get the message in the FinalizeRequestTask like this * FileArtifactExtractionTask: image_export.py failed for artifact JupyterConfigFile.

But if i turn on debug and verbose and I check the logs from the FileArtifactExtractionTask i see that it succeeded:

**Running image_export as [sudo image_export.py --no-hashes --logfile /evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/5c1031787c90471bb4f6ed1440a6ac95.log -w /evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/export --partitions all --volumes all --unattended --artifact_filters LinuxScheduleFiles -d /evidence/evidence_1]
Writing stderr to /evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/stderr-yhee7go0.txt
Writing stdout to /evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/stdout-5x2wr9vb.txt
Output log file found at /evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/5c1031787c90471bb4f6ed1440a6ac95.log
Output log file found at /evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/stderr-yhee7go0.txt
Output log file found at /evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/stdout-5x2wr9vb.txt
Execution of [['sudo', 'image_export.py', '--no-hashes', '--logfile', '/evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/5c1031787c90471bb4f6ed1440a6ac95.log', '-w', '/evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/export', '--partitions', 'all', '--volumes', 'all', '--unattended', '--artifact_filters', 'LinuxScheduleFiles', '-d', '/evidence/evidence_1']] succeeded
Extracted 0 new LinuxScheduleFiles artifacts**

Is that the expected bahavior?

HolzmanoLagrene avatar Aug 30 '22 14:08 HolzmanoLagrene

Hmm interesting, did you by chance see any error in evidence/dc71d6e4bd18428180e14e945b54b6e5/1661867888-5c1031787c90471bb4f6ed1440a6ac95-FileArtifactExtractionTask/stderr-yhee7go0.txt?

wajihyassine avatar Sep 02 '22 17:09 wajihyassine

Sorry for taking so long!

Turbinia spawns multiple FileArtifactExtractionTasks. Actually it starts 8 such processes...only one really fails and that is the one trying to extract JupyterConfigFile. Here are the generated files.

073e73a8de8843bca968797890c17da8.log

2022-10-10 11:56:46,539 [DEBUG] (MainProcess) PID:59 <image_export_tool> Starting preprocessing.
2022-10-10 11:56:47,331 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: DetermineOperatingSystemPlugin with artifact definition: N/A
2022-10-10 11:56:47,331 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: LinuxHostnamePlugin with artifact definition: LinuxHostnameFile
2022-10-10 11:56:47,332 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: LinuxDistributionPlugin with artifact definition: LinuxDistributionRelease
2022-10-10 11:56:47,333 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: LinuxIssueFilePlugin with artifact definition: LinuxIssueFile
2022-10-10 11:56:47,333 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: LinuxStandardBaseReleasePlugin with artifact definition: LinuxLSBRelease
2022-10-10 11:56:47,333 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: LinuxSystemdOperatingSystemPlugin with artifact definition: LinuxSystemdOSRelease
2022-10-10 11:56:47,333 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: LinuxTimeZonePlugin with artifact definition: LinuxLocalTime
2022-10-10 11:56:47,334 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: LinuxUserAccountsPlugin with artifact definition: LinuxPasswdFile
2022-10-10 11:56:47,334 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: MacOSHostnamePlugin with artifact definition: MacOSSystemConfigurationPreferencesPlistFile
2022-10-10 11:56:47,334 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: MacOSKeyboardLayoutPlugin with artifact definition: MacOSKeyboardLayoutPlistFile
2022-10-10 11:56:47,334 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: MacOSSystemVersionPlugin with artifact definition: MacOSSystemVersionPlistFile
2022-10-10 11:56:47,334 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: MacOSTimeZonePlugin with artifact definition: MacOSLocalTime
2022-10-10 11:56:47,335 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: MacOSUserAccountsPlugin with artifact definition: MacOSUserPasswordHashesPlistFiles
2022-10-10 11:56:47,335 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: WindowsSystemRootEnvironmentVariablePlugin with artifact definition: WindowsEnvironmentVariableSystemRoot
2022-10-10 11:56:47,336 [DEBUG] (MainProcess) PID:59 <manager> Running file system preprocessor plugin: WindowsWinDirEnvironmentVariablePlugin with artifact definition: WindowsEnvironmentVariableWinDir
2022-10-10 11:56:47,336 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsAvailableTimeZones
2022-10-10 11:56:47,338 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsCodePage
2022-10-10 11:56:47,339 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsComputerName
2022-10-10 11:56:47,339 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsCurrentVersion
2022-10-10 11:56:47,340 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsEnvironmentVariableAllUsersProfile
2022-10-10 11:56:47,340 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsEnvironmentVariableProgramData
2022-10-10 11:56:47,340 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsEnvironmentVariableProgramFiles
2022-10-10 11:56:47,341 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsEnvironmentVariableProgramFilesX86
2022-10-10 11:56:47,341 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsEventLogPublishers
2022-10-10 11:56:47,342 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsEventLogSources
2022-10-10 11:56:47,342 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsLanguage
2022-10-10 11:56:47,343 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsMountedDevices
2022-10-10 11:56:47,343 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsProductName
2022-10-10 11:56:47,343 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsRegistryProfiles
2022-10-10 11:56:47,344 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsServices
2022-10-10 11:56:47,344 [DEBUG] (MainProcess) PID:59 <manager> Running Windows Registry preprocessor plugin: WindowsTimezone
2022-10-10 11:56:47,345 [DEBUG] (MainProcess) PID:59 <manager> Running knowledge base preprocessor plugin: WindowsAllUsersAppDataKnowledgeBasePlugin
2022-10-10 11:56:47,345 [DEBUG] (MainProcess) PID:59 <manager> Running knowledge base preprocessor plugin: WindowsAllUsersAppProfileKnowledgeBasePlugin
2022-10-10 11:56:47,345 [DEBUG] (MainProcess) PID:59 <manager> Running knowledge base preprocessor plugin: WindowsProgramDataKnowledgeBasePlugin
2022-10-10 11:56:47,346 [DEBUG] (MainProcess) PID:59 <image_export_tool> Preprocessing done.
2022-10-10 11:56:47,346 [DEBUG] (MainProcess) PID:59 <engine> building find specification based on artifacts: JupyterConfigFile
2022-10-10 11:56:48,146 [DEBUG] (MainProcess) PID:59 <artifact_filters> building find spec from artifact definition: JupyterConfigFile
2022-10-10 11:56:48,146 [DEBUG] (MainProcess) PID:59 <artifact_filters> building find spec from path glob: %%users.homedir%%/.jupyter/jupyter_notebook_config.py
2022-10-10 11:56:48,146 [WARNING] (MainProcess) PID:59 <image_export> Unable to build collection filters with error: No valid file system find specifications were built from artifacts.

stderr-9svgjze6.txt Empty

stdout-6cb09xid.txt Empty

worker-log.txt

Running image_export as [sudo image_export.py --no-hashes --logfile /evidence/9bd6068d10be4fc099a96672e44dc57c/1665403005-073e73a8de8843bca968797890c17da8-FileArtifactExtractionTask/073e73a8de8843bca968797890c17da8.log -w /evidence/9bd6068d10be4fc099a96672e44dc57c/1665403005-073e73a8de8843bca968797890c17da8-FileArtifactExtractionTask/export --partitions all --volumes all --unattended --artifact_filters JupyterConfigFile -d /evidence/test_data/artifacts]
Writing stderr to /evidence/9bd6068d10be4fc099a96672e44dc57c/1665403005-073e73a8de8843bca968797890c17da8-FileArtifactExtractionTask/stderr-9svgjze6.txt
Writing stdout to /evidence/9bd6068d10be4fc099a96672e44dc57c/1665403005-073e73a8de8843bca968797890c17da8-FileArtifactExtractionTask/stdout-6cb09xid.txt
Output log file found at /evidence/9bd6068d10be4fc099a96672e44dc57c/1665403005-073e73a8de8843bca968797890c17da8-FileArtifactExtractionTask/stderr-9svgjze6.txt
Output log file found at /evidence/9bd6068d10be4fc099a96672e44dc57c/1665403005-073e73a8de8843bca968797890c17da8-FileArtifactExtractionTask/073e73a8de8843bca968797890c17da8.log
Output log file found at /evidence/9bd6068d10be4fc099a96672e44dc57c/1665403005-073e73a8de8843bca968797890c17da8-FileArtifactExtractionTask/stdout-6cb09xid.txt
Execution of [['sudo', 'image_export.py', '--no-hashes', '--logfile', '/evidence/9bd6068d10be4fc099a96672e44dc57c/1665403005-073e73a8de8843bca968797890c17da8-FileArtifactExtractionTask/073e73a8de8843bca968797890c17da8.log', '-w', '/evidence/9bd6068d10be4fc099a96672e44dc57c/1665403005-073e73a8de8843bca968797890c17da8-FileArtifactExtractionTask/export', '--partitions', 'all', '--volumes', 'all', '--unattended', '--artifact_filters', 'JupyterConfigFile', '-d', '/evidence/test_data/artifacts']] failed with status 1
image_export.py failed for artifact JupyterConfigFile.

It seems that there are interesting part is at the bottom of the log file:

2022-10-10 11:56:48,146 [WARNING] (MainProcess) PID:59 <image_export> Unable to build collection filters with error: No valid file system find specifications were built from artifacts.

Does anyone know what that means?

HolzmanoLagrene avatar Oct 10 '22 12:10 HolzmanoLagrene