bulk_extractor icon indicating copy to clipboard operation
bulk_extractor copied to clipboard

Stop Lists not working

Open fcgreg opened this issue 2 years ago • 7 comments

First, thanks for all of the great work that you all have put into the new 2.x version. I have been putting release 2.0.0 through some functional testing with real-world data, using a Linux-based environment. After running multiple tests against the same evidence extraction, it appears that the Stop List functionality is not working. Results, repeated multiple times:

  1. None of the special _*stopped.txt files are created.
  2. Entries in the Stop List show up in the output Feature Files (such as "[email protected]" )
  3. The total number of entries in the Feature Files seems to be the same between my default run and subsequent runs using the Stop List.

Environment notes:

  • For a Stop List, I am using the BE15_stoplists.zip file downloaded from the Digital Corpora site.
  • I am using BE version 2.0.0 compiled from source on a modern variant of Ubuntu 22.04 LTS.

Let me know if I can help test this further. Thanks in advance.

Quick aside: I have noticed the Alert List functionality doesn't seem to be working, either--I'll file that as a separate issue.

fcgreg avatar Jun 01 '22 04:06 fcgreg

Thank you. I did a survey earlier and did not find anyone using the stop list or alert list functionality. Are you using it in a production setting, or did you simply notice that it is missing?


Sent from my phone.

On Jun 1, 2022, at 12:32 AM, Greg Tassone @.***> wrote:

 First, thanks for all of the great work that you all have put into the new 2.x version. I have been putting release 2.0.0 through some functional testing with real-world data, using a Linux-based environment. After running multiple tests against the same evidence extraction, it appears that the Stop List functionality is not working. Results, repeated multiple times:

None of the special _stopped.txt files are created. Entries in the Stop List show up in the output Feature Files (such as @.**" ) The total number of entries in the Feature Files seems to be the same between my default run and subsequent runs using the Stop List. Environment notes:

For a Stop List, I am using the BE15_stoplists.zip file downloaded from the Digital Corpora site. I am using BE version 2.0.0 compiled from source on a modern variant of Ubuntu 22.04 LTS. Let me know if I can help test this further. Thanks in advance.

Quick aside: I have noticed the Alert List functionality doesn't seem to be working, either--I'll file that as a separate issue.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

simsong avatar Jun 01 '22 09:06 simsong

I appreciate the quick reply. I'm sorry that I missed your earlier feature survey.

I am using BE in a production setting but I am using v1.5.5 on Windows for that purpose (and using BE_Viewer with Java 8/1.8). I am generally using it with Stop Lists and occasionally with Alert Lists. I primarily use it for parsing memory extractions for passwords and other memory-only artifacts, as well as for rapid triage on extractions and loose devices.

I haven't started using 2.0.0 in production yet until I do more functional testing and there is more validation on the new code-base, but I'm excited about using it in the future. If I had more time I'd offer to assist in the development (or perhaps porting BE_Viewer), but I'm too limited on time for that presently. I am able to do some testing, though.

Thanks again.

fcgreg avatar Jun 01 '22 15:06 fcgreg

If you think these are useful features, we can add them back in. You are the only person who has asked for them!


Sent from my phone.

On Jun 1, 2022, at 11:07 AM, Greg Tassone @.***> wrote:

 I appreciate the quick reply. I'm sorry that I missed your earlier feature survey.

I am using BE in a production setting but I am using v1.5.5 on Windows for that purpose (and using BE_Viewer with Java 8/1.8). I am generally using it with Stop Lists and occasionally with Alert Lists. I primarily use it for parsing memory extractions for passwords and other memory-only artifacts, as well as for rapid triage on extractions and loose devices.

I haven't started using 2.0.0 in production yet until I do more functional testing and there is more validation on the new code-base, but I'm excited about using it in the future. If I had more time I'd offer to assist in the development (or perhaps porting BE_Viewer), but I'm too limited on time for that presently. I am able to do some testing, though.

Thanks again.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

simsong avatar Jun 01 '22 15:06 simsong

I do think these features are useful. Actually, I've recently been presenting/teaching about BE to other colleagues because there has been renewed interest in using fast/powerful forensic tools that are open source. In my sessions, my colleagues were unaware that these features existed... maybe that explains the lack of requests in your survey. Once I described how these features worked (e.g. I likened Stop Lists to an NSRL filter, which made sense to them), they were very interested and wanted copies of the NIST generated list.

Thanks again for your work. Let me know if I can help further (testing, etc.).

fcgreg avatar Jun 01 '22 16:06 fcgreg

Are you familiar with the context-specific stop lists? That should still be there.

And yes, there is help that I need, specifically on the packaging of the Windows installer and the Java GUI.

simsong avatar Jun 01 '22 16:06 simsong

Are you familiar with the context-specific stop lists? That should still be there.

Yes, I am familiar with that feature but I haven't used it much. Perhaps I should use that instead? I'm generally using Stop Lists to ignore items globally that have no nexus to my searches, so I've been using them without context. I could probably get away with using them in-context, though (less ideal for my use-cases).

fcgreg avatar Jun 01 '22 16:06 fcgreg

Are you familiar with the context-specific stop lists? That should still be there.

Quick follow-up: I ran another test using the context-specific stop list but it doesn't seem to be functional. It matched nothing on an extraction from a Windows laptop (e.g. no *_stopped.txt files). Let me know if you'd like more analysis about this.

fcgreg avatar Jun 02 '22 19:06 fcgreg