IPED
IPED copied to clipboard
sepinf-inc
ALeapp task
Hi @lfcnassif , I decided to create this pull request, but marked it as draft, as I am leaving on vacation
Thanks @patrickdalla! I think it is good to create the PR in draft mode, since this is a big implementation and others can see the progress and comment on.
I've made some more tests and pontual enhancements and corrections. I think I should some third developer evaluation/opinion before continuing, @lfcnassif. I fact, if the design is good and no errors found, I think this can be merged as is.
Thank you very much @patrickdalla! This is a very important feature, I'll try to test it after other ready PRs scheduled for 4.2 in the queue.
One question: if an UFDR is processed with this PR, will we get duplicated results coming from PA decoding and from ALeapp decoding? If yes, I think it should be avoided or be configurable. Maybe with a similar approach used for WhatsApp today, or maybe with the approach that would be implemented for #2012.
Aleapp processes data from FFS. It expects data files in its original path, not in the way they are in UFDR.
Em seg., 20 de mai. de 2024, 21:01, Luis Filipe Nassif < @.***> escreveu:
Thank you very much @patrickdalla https://github.com/patrickdalla! This is a very important feature, I'll try to test it after other ready PRs scheduled for 4.2 in the queue.
One question: if an UFDR is processed with this PR, will we get duplicated results coming from PA decoding and from ALeapp decoding? If yes, I think it should be avoided or be configurable. Maybe with a similar approach used for WhatsApp today, or maybe with the approach that would be implemented on #2012 https://github.com/sepinf-inc/IPED/issues/2012.
— Reply to this email directly, view it on GitHub https://github.com/sepinf-inc/IPED/pull/2095#issuecomment-2121504987, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG247S7KCJ75V56GGLYFXADZDKMGZAVCNFSM6AAAAABDVRWLH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRRGUYDIOJYG4 . You are receiving this because you were mentioned.Message ID: @.***>
Anyway, there is already a config file to inform which Aleapp parser not to run.
Em seg., 20 de mai. de 2024, 22:22, Patrick Bernardina < @.***> escreveu:
Aleapp processes data from FFS. It expects data files in its original path, not in the way they are in UFDR.
Em seg., 20 de mai. de 2024, 21:01, Luis Filipe Nassif < @.***> escreveu:
Thank you very much @patrickdalla https://github.com/patrickdalla! This is a very important feature, I'll try to test it after other ready PRs scheduled for 4.2 in the queue.
One question: if an UFDR is processed with this PR, will we get duplicated results coming from PA decoding and from ALeapp decoding? If yes, I think it should be avoided or be configurable. Maybe with a similar approach used for WhatsApp today, or maybe with the approach that would be implemented on #2012 https://github.com/sepinf-inc/IPED/issues/2012.
— Reply to this email directly, view it on GitHub https://github.com/sepinf-inc/IPED/pull/2095#issuecomment-2121504987, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG247S7KCJ75V56GGLYFXADZDKMGZAVCNFSM6AAAAABDVRWLH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRRGUYDIOJYG4 . You are receiving this because you were mentioned.Message ID: @.***>
Also, thinking twice, the task searches for items from already processed items and its paths. As IPED uses UFDR xml info to restore original path in cel FS, maybe, with some simple modification, Aleapp parser can find these items too. I will check it.
Em seg., 20 de mai. de 2024, 22:22, Patrick Bernardina < @.***> escreveu:
Aleapp processes data from FFS. It expects data files in its original path, not in the way they are in UFDR.
Em seg., 20 de mai. de 2024, 21:01, Luis Filipe Nassif < @.***> escreveu:
Thank you very much @patrickdalla https://github.com/patrickdalla! This is a very important feature, I'll try to test it after other ready PRs scheduled for 4.2 in the queue.
One question: if an UFDR is processed with this PR, will we get duplicated results coming from PA decoding and from ALeapp decoding? If yes, I think it should be avoided or be configurable. Maybe with a similar approach used for WhatsApp today, or maybe with the approach that would be implemented on #2012 https://github.com/sepinf-inc/IPED/issues/2012.
— Reply to this email directly, view it on GitHub https://github.com/sepinf-inc/IPED/pull/2095#issuecomment-2121504987, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG247S7KCJ75V56GGLYFXADZDKMGZAVCNFSM6AAAAABDVRWLH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRRGUYDIOJYG4 . You are receiving this because you were mentioned.Message ID: @.***>
Anyway, there is already a config file to inform which Aleapp parser not to run.
Great!
Also, thinking twice, the task searches for items from already processed items and its paths. As IPED uses UFDR xml info to restore original path in cel FS, maybe, with some simple modification, Aleapp parser can find these items too. I will check it.
I think it can be useful, for example, for an application supported by ALeapp but not supported by PA, or if PA decoding brings incomplete or eventually wrong results. For this last example, disabling PA results importing per application may be needed, but that is related to #2012.
PS: Running ALeapp into AB backups (when #2079 is merged) should be very useful too.
Yes, it worked with the modifications of last commit, ALeapp plugins found and processed items from UFDR.
I'm eager to see this merged into main. Is this bringing ileapp too? Or is this planned for another PR?
I'm eager to see this merged into main. Is this bringing ileapp too? Or is this planned for another PR?
AFAIK this is just about ALeapp integration, iLeapp should be done later. Would you like to help testing? There is a snapshot with this support below, you should be logged in github to see the download link: https://github.com/sepinf-inc/IPED/actions/runs/9180006157
Hi @patrickdalla, an user/developer is trying to test this, but got an error "no module named geopy". What is the updated python dependency list needed to run this PR?
Just found this list in Teams, posting here for those willing to help testing, let me know if it is outdated:
bcrypt==3.2.0
beautifulsoup4==4.8.2
bencoding
blackboxprotobuf
fitdecode==0.10.0
folium==0.14.0
geopy==2.3.0
packaging==20.1
pillow
polyline==2.0.0
protobuf==3.10.0
PyCryptodome
PySimpleGUI
pytz
simplekml
wheel
xlsxwriter==3.1.1
xmltodict
python-magic
libmagic
python-magic-bin
filetype
Just put those into a requirements.txt file and run from iped embedded python:
pip install -r requirements.txt