misp-modules icon indicating copy to clipboard operation
misp-modules copied to clipboard

Published 20 hours ago • verified publisher icon MISP

misp-modules not working

Open Phil-ThePower-Pearce opened this issue 3 years ago • 12 comments

Hi,

Ive noticed that the misp modules arent working. I followed the instructions here, but all looks ok https://github.com/MISP/misp-modules/issues/354

root@hostname:/usr/local/src/faup/src/lib/bindings/python# service misp-modules status
● misp-modules.service - System-wide instance of the MISP Modules
     Loaded: loaded (/etc/systemd/system/misp-modules.service; enabled; vendor preset: enabled)
     Active: inactive (dead) since Tue 2022-01-11 11:44:58 UTC; 7s ago
    Process: 15994 ExecStart=/var/www/MISP/venv/bin/misp-modules -l 127.0.0.1 -s (code=exited, status=0/SUCCESS)
   Main PID: 15994 (code=exited, status=0/SUCCESS)

Jan 11 11:44:58 hostname misp-modules[15994]:     auth_param_name = token.copy().setName("auth-param-name").addParseAction(pp.downcaseTokens)
Jan 11 11:44:58 hostname misp-modules[15994]: AttributeError: module 'pyparsing' has no attribute 'downcaseTokens'
Jan 11 11:44:58 hostname misp-modules[15994]: uwhois module not installed.
Jan 11 11:44:58 hostname misp-modules[15994]: 2022-01-11 11:44:58,729 - misp-modules - INFO - Launch MISP modules server from package.
Jan 11 11:44:58 hostname misp-modules[15994]: INFO:misp-modules:Launch MISP modules server from package.
Jan 11 11:44:58 hostname misp-modules[15994]: 2022-01-11 11:44:58,731 - misp-modules - INFO - Helper loaded cache
Jan 11 11:44:58 hostname misp-modules[15994]: INFO:misp-modules:Helper loaded cache
Jan 11 11:44:58 hostname misp-modules[15994]: 2022-01-11 11:44:58,731 - misp-modules - INFO - Unable to load MISP modules from package.
Jan 11 11:44:58 hostname misp-modules[15994]: INFO:misp-modules:Unable to load MISP modules from package.
Jan 11 11:44:58 hostname systemd[1]: misp-modules.service: Succeeded.

I tried installing the module that it says is missing

root@hostname:/usr/local/src/faup/src/lib/bindings/python# pip3 install uwhois
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
ERROR: Could not find a version that satisfies the requirement uwhois (from versions: none)
ERROR: No matching distribution found for uwhois

Any ideas? Im on ubuntu 20.04

Phil-ThePower-Pearce avatar Jan 11 '22 11:01 Phil-ThePower-Pearce

Hi, I had the same error

zero060699 avatar Jan 19 '22 06:01 zero060699

You can do the following as a workaround pip install pyparsing==2.4.7. Let us know if it solves your issue.

adulau avatar Jan 19 '22 07:01 adulau 

root@host:/home/ubuntu# pip install pyparsing==2.4.7
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Requirement already satisfied: pyparsing==2.4.7 in /usr/local/lib/python3.8/dist-packages (2.4.7)

Phil-ThePower-Pearce avatar Jan 19 '22 14:01 Phil-ThePower-Pearce

I did an install of misp-modules on a standard Ubuntu:

adulau@kolmogorov ~/git/misp-modules (main)$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
Codename:	focal

I installed the requirements pip3 install -r REQUIREMENTS (along with the packages required).

It works out of the box. I didn't installed uwhois as it's not required.

Could you run misp-modules in a terminal? It's Python 3.8.10 (standard for LTS).

adulau avatar Jan 19 '22 21:01 adulau

I ran it out of the box too on an updated aws ubuntu 20.04 t2.xlarge instance. Exactly the same command. Every time the same kind of issues.

This is the result. Its paused "INFO:misp-modules:MISP modules server started on localhost port 6666"

root@host:/etc# misp-modules
uwhois module not installed.
ERROR:root:No module named 'pytesseract'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/misp_modules/__init__.py", line 41, in <module>
    from .modules import *  # noqa
  File "/usr/local/lib/python3.8/dist-packages/misp_modules/modules/__init__.py", line 1, in <module>
    from .expansion import *  # noqa
  File "/usr/local/lib/python3.8/dist-packages/misp_modules/modules/expansion/ocr_enrich.py", line 5, in <module>
    import pytesseract
ModuleNotFoundError: No module named 'pytesseract'
2022-01-20 11:20:52,707 - misp-modules - INFO - Launch MISP modules server from current directory.
INFO:misp-modules:Launch MISP modules server from current directory.
2022-01-20 11:20:52,708 - misp-modules - INFO - Helpers loaded cache.py 
INFO:misp-modules:Helpers loaded cache.py 
2022-01-20 11:20:52,711 - misp-modules - INFO - MISP modules apivoid imported
INFO:misp-modules:MISP modules apivoid imported
2022-01-20 11:20:52,711 - misp-modules - INFO - MISP modules cve_advanced imported
INFO:misp-modules:MISP modules cve_advanced imported
2022-01-20 11:20:52,712 - misp-modules - WARNING - MISP modules ocr_enrich failed due to No module named 'pytesseract'
WARNING:misp-modules:MISP modules ocr_enrich failed due to No module named 'pytesseract'
2022-01-20 11:20:52,712 - misp-modules - INFO - MISP modules cve imported
INFO:misp-modules:MISP modules cve imported
2022-01-20 11:20:52,713 - misp-modules - INFO - MISP modules bgpranking imported
INFO:misp-modules:MISP modules bgpranking imported
2022-01-20 11:20:52,717 - misp-modules - INFO - MISP modules joesandbox_query imported
INFO:misp-modules:MISP modules joesandbox_query imported
2022-01-20 11:20:52,717 - misp-modules - INFO - MISP modules domaintools imported
INFO:misp-modules:MISP modules domaintools imported
2022-01-20 11:20:52,975 - misp-modules - INFO - MISP modules assemblyline_submit imported
INFO:misp-modules:MISP modules assemblyline_submit imported
2022-01-20 11:20:52,975 - misp-modules - INFO - MISP modules onyphe_full imported
INFO:misp-modules:MISP modules onyphe_full imported
2022-01-20 11:20:52,976 - misp-modules - INFO - MISP modules threatminer imported
INFO:misp-modules:MISP modules threatminer imported
2022-01-20 11:20:53,007 - misp-modules - INFO - MISP modules pdf_enrich imported
INFO:misp-modules:MISP modules pdf_enrich imported
pyeti module not installed.
2022-01-20 11:20:53,009 - misp-modules - INFO - MISP modules yeti imported
INFO:misp-modules:MISP modules yeti imported
2022-01-20 11:20:53,010 - misp-modules - INFO - MISP modules shodan imported
INFO:misp-modules:MISP modules shodan imported
2022-01-20 11:20:53,010 - misp-modules - INFO - MISP modules onyphe imported
INFO:misp-modules:MISP modules onyphe imported
2022-01-20 11:20:53,012 - misp-modules - INFO - MISP modules lastline_query imported
INFO:misp-modules:MISP modules lastline_query imported
2022-01-20 11:20:53,013 - misp-modules - INFO - MISP modules cuckoo_submit imported
INFO:misp-modules:MISP modules cuckoo_submit imported
2022-01-20 11:20:53,013 - misp-modules - INFO - MISP modules virustotal imported
INFO:misp-modules:MISP modules virustotal imported
2022-01-20 11:20:53,018 - misp-modules - INFO - MISP modules securitytrails imported
INFO:misp-modules:MISP modules securitytrails imported
2022-01-20 11:20:53,019 - misp-modules - INFO - MISP modules countrycode imported
INFO:misp-modules:MISP modules countrycode imported
2022-01-20 11:20:53,020 - misp-modules - INFO - MISP modules lastline_submit imported
INFO:misp-modules:MISP modules lastline_submit imported
2022-01-20 11:20:53,020 - misp-modules - INFO - MISP modules rbl imported
INFO:misp-modules:MISP modules rbl imported
2022-01-20 11:20:53,021 - misp-modules - INFO - MISP modules macaddress_io imported
INFO:misp-modules:MISP modules macaddress_io imported
2022-01-20 11:20:53,021 - misp-modules - INFO - MISP modules wiki imported
INFO:misp-modules:MISP modules wiki imported
2022-01-20 11:20:53,021 - misp-modules - INFO - MISP modules eupi imported
INFO:misp-modules:MISP modules eupi imported
2022-01-20 11:20:53,022 - misp-modules - INFO - MISP modules btc_scam_check imported
INFO:misp-modules:MISP modules btc_scam_check imported
2022-01-20 11:20:53,030 - misp-modules - INFO - MISP modules mcafee_insights_enrich imported
INFO:misp-modules:MISP modules mcafee_insights_enrich imported
2022-01-20 11:20:53,033 - misp-modules - INFO - MISP modules qintel_qsentry imported
INFO:misp-modules:MISP modules qintel_qsentry imported
2022-01-20 11:20:53,033 - misp-modules - INFO - MISP modules qrcode imported
INFO:misp-modules:MISP modules qrcode imported
2022-01-20 11:20:53,034 - misp-modules - INFO - MISP modules urlhaus imported
INFO:misp-modules:MISP modules urlhaus imported
2022-01-20 11:20:53,035 - misp-modules - INFO - MISP modules vulndb imported
INFO:misp-modules:MISP modules vulndb imported
2022-01-20 11:20:53,036 - misp-modules - INFO - MISP modules urlscan imported
INFO:misp-modules:MISP modules urlscan imported
2022-01-20 11:20:53,036 - misp-modules - INFO - MISP modules circl_passivedns imported
INFO:misp-modules:MISP modules circl_passivedns imported
2022-01-20 11:20:53,066 - misp-modules - INFO - MISP modules trustar_enrich imported
INFO:misp-modules:MISP modules trustar_enrich imported
2022-01-20 11:20:53,066 - misp-modules - INFO - MISP modules vulners imported
INFO:misp-modules:MISP modules vulners imported
2022-01-20 11:20:53,067 - misp-modules - INFO - MISP modules vmray_submit imported
INFO:misp-modules:MISP modules vmray_submit imported
2022-01-20 11:20:53,067 - misp-modules - INFO - MISP modules yara_syntax_validator imported
INFO:misp-modules:MISP modules yara_syntax_validator imported
2022-01-20 11:20:53,067 - misp-modules - INFO - MISP modules intel471 imported
INFO:misp-modules:MISP modules intel471 imported
2022-01-20 11:20:53,087 - misp-modules - INFO - MISP modules vmware_nsx imported
INFO:misp-modules:MISP modules vmware_nsx imported
2022-01-20 11:20:53,088 - misp-modules - INFO - MISP modules dns imported
INFO:misp-modules:MISP modules dns imported
2022-01-20 11:20:53,759 - misp-modules - INFO - MISP modules ods_enrich imported
INFO:misp-modules:MISP modules ods_enrich imported
2022-01-20 11:20:53,762 - misp-modules - INFO - MISP modules clamav imported
INFO:misp-modules:MISP modules clamav imported
2022-01-20 11:20:53,763 - misp-modules - INFO - MISP modules assemblyline_query imported
INFO:misp-modules:MISP modules assemblyline_query imported
2022-01-20 11:20:53,764 - misp-modules - INFO - MISP modules xlsx_enrich imported
INFO:misp-modules:MISP modules xlsx_enrich imported
2022-01-20 11:20:53,764 - misp-modules - INFO - MISP modules passivetotal imported
INFO:misp-modules:MISP modules passivetotal imported
2022-01-20 11:20:53,764 - misp-modules - INFO - MISP modules yara_query imported
INFO:misp-modules:MISP modules yara_query imported
2022-01-20 11:20:53,765 - misp-modules - INFO - MISP modules cpe imported
INFO:misp-modules:MISP modules cpe imported
uwhois module not installed.
2022-01-20 11:20:53,765 - misp-modules - INFO - MISP modules whois imported
INFO:misp-modules:MISP modules whois imported
2022-01-20 11:20:53,766 - misp-modules - INFO - MISP modules sigma_syntax_validator imported
INFO:misp-modules:MISP modules sigma_syntax_validator imported
2022-01-20 11:20:53,766 - misp-modules - INFO - MISP modules threatfox imported
INFO:misp-modules:MISP modules threatfox imported
2022-01-20 11:20:53,778 - misp-modules - INFO - MISP modules recordedfuture imported
INFO:misp-modules:MISP modules recordedfuture imported
2022-01-20 11:20:53,778 - misp-modules - INFO - MISP modules backscatter_io imported
INFO:misp-modules:MISP modules backscatter_io imported
2022-01-20 11:20:53,779 - misp-modules - INFO - MISP modules virustotal_public imported
INFO:misp-modules:MISP modules virustotal_public imported
2022-01-20 11:20:53,779 - misp-modules - INFO - MISP modules eql imported
INFO:misp-modules:MISP modules eql imported
2022-01-20 11:20:53,780 - misp-modules - INFO - MISP modules dbl_spamhaus imported
INFO:misp-modules:MISP modules dbl_spamhaus imported
Censys module not installed. Try 'pip install censys'
2022-01-20 11:20:53,781 - misp-modules - INFO - MISP modules censys_enrich imported
INFO:misp-modules:MISP modules censys_enrich imported
2022-01-20 11:20:53,781 - misp-modules - INFO - MISP modules reversedns imported
INFO:misp-modules:MISP modules reversedns imported
2022-01-20 11:20:53,782 - misp-modules - INFO - MISP modules iprep imported
INFO:misp-modules:MISP modules iprep imported
2022-01-20 11:20:53,782 - misp-modules - INFO - MISP modules sigma_queries imported
INFO:misp-modules:MISP modules sigma_queries imported
2022-01-20 11:20:53,789 - misp-modules - INFO - MISP modules odt_enrich imported
INFO:misp-modules:MISP modules odt_enrich imported
2022-01-20 11:20:53,789 - misp-modules - INFO - MISP modules farsight_passivedns imported
INFO:misp-modules:MISP modules farsight_passivedns imported
2022-01-20 11:20:53,790 - misp-modules - INFO - MISP modules threatcrowd imported
INFO:misp-modules:MISP modules threatcrowd imported
2022-01-20 11:20:53,797 - misp-modules - INFO - MISP modules socialscan imported
INFO:misp-modules:MISP modules socialscan imported
GoogleAPI not installed. Command : pip install git+https://github.com/abenassi/Google-Search-API
2022-01-20 11:20:53,798 - misp-modules - INFO - MISP modules google_search imported
INFO:misp-modules:MISP modules google_search imported
2022-01-20 11:20:53,798 - misp-modules - INFO - MISP modules geoip_asn imported
INFO:misp-modules:MISP modules geoip_asn imported
2022-01-20 11:20:53,800 - misp-modules - INFO - MISP modules ransomcoindb imported
INFO:misp-modules:MISP modules ransomcoindb imported
2022-01-20 11:20:53,804 - misp-modules - INFO - MISP modules html_to_markdown imported
INFO:misp-modules:MISP modules html_to_markdown imported
2022-01-20 11:20:54,063 - misp-modules - INFO - MISP modules pptx_enrich imported
INFO:misp-modules:MISP modules pptx_enrich imported
2022-01-20 11:20:54,064 - misp-modules - INFO - MISP modules circl_passivessl imported
INFO:misp-modules:MISP modules circl_passivessl imported
2022-01-20 11:20:54,064 - misp-modules - INFO - MISP modules geoip_city imported
INFO:misp-modules:MISP modules geoip_city imported
2022-01-20 11:20:54,065 - misp-modules - INFO - MISP modules passive-ssh imported
INFO:misp-modules:MISP modules passive-ssh imported
2022-01-20 11:20:54,065 - misp-modules - INFO - MISP modules crowdstrike_falcon imported
INFO:misp-modules:MISP modules crowdstrike_falcon imported
2022-01-20 11:20:54,066 - misp-modules - INFO - MISP modules joesandbox_submit imported
INFO:misp-modules:MISP modules joesandbox_submit imported
2022-01-20 11:20:54,067 - misp-modules - INFO - MISP modules hibp imported
INFO:misp-modules:MISP modules hibp imported
2022-01-20 11:20:54,067 - misp-modules - INFO - MISP modules otx imported
INFO:misp-modules:MISP modules otx imported
2022-01-20 11:20:54,154 - misp-modules - INFO - MISP modules docx_enrich imported
INFO:misp-modules:MISP modules docx_enrich imported
2022-01-20 11:20:54,155 - misp-modules - INFO - MISP modules xforceexchange imported
INFO:misp-modules:MISP modules xforceexchange imported
2022-01-20 11:20:54,155 - misp-modules - INFO - MISP modules greynoise imported
INFO:misp-modules:MISP modules greynoise imported
2022-01-20 11:20:54,156 - misp-modules - INFO - MISP modules cytomic_orion imported
INFO:misp-modules:MISP modules cytomic_orion imported
2022-01-20 11:20:54,157 - misp-modules - INFO - MISP modules sophoslabs_intelix imported
INFO:misp-modules:MISP modules sophoslabs_intelix imported
2022-01-20 11:20:54,177 - misp-modules - INFO - MISP modules apiosintds imported
INFO:misp-modules:MISP modules apiosintds imported
2022-01-20 11:20:54,178 - misp-modules - INFO - MISP modules geoip_country imported
INFO:misp-modules:MISP modules geoip_country imported
2022-01-20 11:20:54,178 - misp-modules - INFO - MISP modules hashdd imported
INFO:misp-modules:MISP modules hashdd imported
2022-01-20 11:20:54,178 - misp-modules - INFO - MISP modules ipasn imported
INFO:misp-modules:MISP modules ipasn imported
2022-01-20 11:20:54,178 - misp-modules - INFO - MISP modules macvendors imported
INFO:misp-modules:MISP modules macvendors imported
2022-01-20 11:20:54,179 - misp-modules - INFO - MISP modules hashlookup imported
INFO:misp-modules:MISP modules hashlookup imported
2022-01-20 11:20:54,180 - misp-modules - INFO - MISP modules sourcecache imported
INFO:misp-modules:MISP modules sourcecache imported
2022-01-20 11:20:54,194 - misp-modules - INFO - MISP modules mwdb imported
INFO:misp-modules:MISP modules mwdb imported
2022-01-20 11:20:54,194 - misp-modules - INFO - MISP modules btc_steroids imported
INFO:misp-modules:MISP modules btc_steroids imported
2022-01-20 11:20:54,195 - misp-modules - INFO - MISP modules malwarebazaar imported
INFO:misp-modules:MISP modules malwarebazaar imported
2022-01-20 11:20:54,195 - misp-modules - INFO - MISP modules stix2_pattern_syntax_validator imported
INFO:misp-modules:MISP modules stix2_pattern_syntax_validator imported
2022-01-20 11:20:54,197 - misp-modules - INFO - MISP modules openiocimport imported
INFO:misp-modules:MISP modules openiocimport imported
2022-01-20 11:20:54,198 - misp-modules - INFO - MISP modules joe_import imported
INFO:misp-modules:MISP modules joe_import imported
2022-01-20 11:20:54,198 - misp-modules - INFO - MISP modules stiximport imported
INFO:misp-modules:MISP modules stiximport imported
2022-01-20 11:20:54,209 - misp-modules - INFO - MISP modules vmray_summary_json_import imported
INFO:misp-modules:MISP modules vmray_summary_json_import imported
2022-01-20 11:20:54,211 - misp-modules - INFO - MISP modules cuckooimport imported
INFO:misp-modules:MISP modules cuckooimport imported
2022-01-20 11:20:54,212 - misp-modules - INFO - MISP modules testimport imported
INFO:misp-modules:MISP modules testimport imported
2022-01-20 11:20:54,214 - misp-modules - INFO - MISP modules lastline_import imported
INFO:misp-modules:MISP modules lastline_import imported
2022-01-20 11:20:54,215 - misp-modules - INFO - MISP modules vmray_import imported
INFO:misp-modules:MISP modules vmray_import imported
2022-01-20 11:20:54,217 - misp-modules - INFO - MISP modules csvimport imported
INFO:misp-modules:MISP modules csvimport imported
2022-01-20 11:20:54,222 - misp-modules - INFO - MISP modules cof2misp imported
INFO:misp-modules:MISP modules cof2misp imported
2022-01-20 11:20:54,223 - misp-modules - INFO - MISP modules mispjson imported
INFO:misp-modules:MISP modules mispjson imported
2022-01-20 11:20:54,224 - misp-modules - INFO - MISP modules goamlimport imported
INFO:misp-modules:MISP modules goamlimport imported
2022-01-20 11:20:54,225 - misp-modules - INFO - MISP modules threatanalyzer_import imported
INFO:misp-modules:MISP modules threatanalyzer_import imported
2022-01-20 11:20:54,226 - misp-modules - INFO - MISP modules email_import imported
INFO:misp-modules:MISP modules email_import imported
2022-01-20 11:20:54,227 - misp-modules - INFO - MISP modules ocr imported
INFO:misp-modules:MISP modules ocr imported
2022-01-20 11:20:54,228 - misp-modules - INFO - MISP modules liteexport imported
INFO:misp-modules:MISP modules liteexport imported
2022-01-20 11:20:54,229 - misp-modules - INFO - MISP modules mass_eql_export imported
INFO:misp-modules:MISP modules mass_eql_export imported
2022-01-20 11:20:54,230 - misp-modules - INFO - MISP modules defender_endpoint_export imported
INFO:misp-modules:MISP modules defender_endpoint_export imported
2022-01-20 11:20:54,231 - misp-modules - INFO - MISP modules cisco_firesight_manager_ACL_rule_export imported
INFO:misp-modules:MISP modules cisco_firesight_manager_ACL_rule_export imported
2022-01-20 11:20:54,232 - misp-modules - INFO - MISP modules goamlexport imported
INFO:misp-modules:MISP modules goamlexport imported
2022-01-20 11:20:54,232 - misp-modules - INFO - MISP modules threatStream_misp_export imported
INFO:misp-modules:MISP modules threatStream_misp_export imported
2022-01-20 11:20:54,233 - misp-modules - INFO - MISP modules osqueryexport imported
INFO:misp-modules:MISP modules osqueryexport imported
2022-01-20 11:20:54,235 - misp-modules - INFO - MISP modules cef_export imported
INFO:misp-modules:MISP modules cef_export imported
2022-01-20 11:20:54,235 - misp-modules - INFO - MISP modules threat_connect_export imported
INFO:misp-modules:MISP modules threat_connect_export imported
2022-01-20 11:20:54,236 - misp-modules - INFO - MISP modules nexthinkexport imported
INFO:misp-modules:MISP modules nexthinkexport imported
2022-01-20 11:20:54,237 - misp-modules - INFO - MISP modules pdfexport imported
INFO:misp-modules:MISP modules pdfexport imported
2022-01-20 11:20:54,238 - misp-modules - INFO - MISP modules testexport imported
INFO:misp-modules:MISP modules testexport imported
2022-01-20 11:20:54,256 - misp-modules - INFO - MISP modules vt_graph imported
INFO:misp-modules:MISP modules vt_graph imported
2022-01-20 11:20:54,261 - misp-modules - INFO - MISP modules server started on localhost port 6666
INFO:misp-modules:MISP modules server started on localhost port 6666


root@host:/etc# pip install pytesseract
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Collecting pytesseract
  Using cached pytesseract-0.3.8.tar.gz (14 kB)
Requirement already satisfied: Pillow in /usr/local/lib/python3.8/dist-packages (from pytesseract) (8.3.2)
Building wheels for collected packages: pytesseract
  Building wheel for pytesseract (setup.py) ... done
  Created wheel for pytesseract: filename=pytesseract-0.3.8-py2.py3-none-any.whl size=14063 sha256=2c683a44efa518655e41a5f5e875cb196738494bf64237a3423557fa1550e882
  Stored in directory: /root/.cache/pip/wheels/ed/8a/c6/40c7ec06c2dd3df636832537238128a1471bf9c1b6a3a9bf40
Successfully built pytesseract
Installing collected packages: pytesseract
Successfully installed pytesseract-0.3.8

If I try and start the service afterwards

root@host:/etc# service misp-modules start
root@host:/etc# service misp-modules status
● misp-modules.service - System-wide instance of the MISP Modules
     Loaded: loaded (/etc/systemd/system/misp-modules.service; enabled; vendor preset: enabled)
     Active: inactive (dead) since Thu 2022-01-20 11:25:52 UTC; 6s ago
    Process: 4340 ExecStart=/var/www/MISP/venv/bin/misp-modules -l 127.0.0.1 -s (code=exited, status=0/SUCCESS)
   Main PID: 4340 (code=exited, status=0/SUCCESS)

Jan 20 11:25:52 host misp-modules[4340]:     auth_param_name = token.copy().setName("auth-param-name").addParseAction(pp.downcaseTokens)
Jan 20 11:25:52 host misp-modules[4340]: AttributeError: module 'pyparsing' has no attribute 'downcaseTokens'
Jan 20 11:25:52 host misp-modules[4340]: uwhois module not installed.
Jan 20 11:25:52 host misp-modules[4340]: 2022-01-20 11:25:52,129 - misp-modules - INFO - Launch MISP modules server from package.
Jan 20 11:25:52 host misp-modules[4340]: INFO:misp-modules:Launch MISP modules server from package.
Jan 20 11:25:52 host misp-modules[4340]: 2022-01-20 11:25:52,130 - misp-modules - INFO - Helper loaded cache
Jan 20 11:25:52 host misp-modules[4340]: INFO:misp-modules:Helper loaded cache
Jan 20 11:25:52 host misp-modules[4340]: 2022-01-20 11:25:52,130 - misp-modules - INFO - Unable to load MISP modules from package.
Jan 20 11:25:52 host misp-modules[4340]: INFO:misp-modules:Unable to load MISP modules from package.
Jan 20 11:25:52 host systemd[1]: misp-modules.service: Succeeded.

Phil-ThePower-Pearce avatar Jan 20 '22 11:01 Phil-ThePower-Pearce

So if you run it manually, misp-modules are running successfully.

But it seems you didn't install the required modules in the /var/www/MISP/venv/ , you need to activate the venv and then do the install. This should solve your issue.

adulau avatar Jan 20 '22 13:01 adulau

Hi,

I followed the instructions "How to install and start MISP modules in a Python virtualenv? (recommended)" On this: sudo -u www-data /var/www/MISP/venv/bin/pip install -I -r REQUIREMENTS

 ERROR: Command errored out with exit status 1:
     command: /var/www/MISP/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-4w3i5dba/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-4w3i5dba/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-req-build-4w3i5dba/pip-egg-info
         cwd: /tmp/pip-req-build-4w3i5dba/
    Complete output (28 lines):
    Traceback (most recent call last):
      File "/var/www/MISP/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2451, in resolve
        return functools.reduce(getattr, self.attrs, module)
    AttributeError: module 'setuptools.dist' has no attribute 'invalid_unless_false'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-4w3i5dba/setup.py", line 10, in <module>
        setup(
      File "/var/www/MISP/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.8/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/var/www/MISP/venv/lib/python3.8/site-packages/setuptools/dist.py", line 447, in __init__
        _Distribution.__init__(self, {
      File "/usr/lib/python3.8/distutils/dist.py", line 292, in __init__
        self.finalize_options()
      File "/var/www/MISP/venv/lib/python3.8/site-packages/setuptools/dist.py", line 740, in finalize_options
        ep.load()(self)
      File "/var/www/MISP/venv/lib/python3.8/site-packages/setuptools/dist.py", line 747, in _finalize_setup_keywords
        ep.load()(self, ep.name, value)
      File "/var/www/MISP/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load
        return self.resolve()
      File "/var/www/MISP/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2453, in resolve
        raise ImportError(str(exc))
    ImportError: module 'setuptools.dist' has no attribute 'invalid_unless_false'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Same error if I go in as root (venv) root@hostname:/usr/local/src/misp-modules# pip install -I -r REQUIREMENTS

Phil-ThePower-Pearce avatar Jan 26 '22 15:01 Phil-ThePower-Pearce

Hello, I'm currently running into the same issue on a Fresh Ubuntu Installation. What I noticed in syslog is, that Python has an Error "AttributeError: module 'pyparsing' has no attribute 'downcaseTokens'".

This led me to this three Stack Overflow Discussions: [1] https://stackoverflow.com/questions/69694833/frappe-installation-error-attributeerror-module-pyparsing-has-no-attribute [2] https://stackoverflow.com/questions/69780351/attributeerror-module-pyparsing-has-no-attribute-downcasetokens-rasa [3] https://stackoverflow.com/questions/33412974/how-to-uninstall-a-package-installed-with-pip-install-user/56948334#56948334

While pip install pyparsing==2.4.2 / pip install pyparsing==2.4.7 did not solve the Issue, I noticed that a user said [2] that for him this issue got thrown because httplib2 was incompatible. On my Ubuntu, httplib2 0.14.0 seems the latest. Maybe that's a part of the Issue?

What I noticed as well is, when I follow the "How to install and start MISP modules in a Python virtualenv" Guide but change the commands, so it's not in context of www-data but my SUDO User instead (so chown /usr/local/src/ to my SUDO User and using sudo pip install -I -r REQUIREMENTS inside /usr/local/src/misp-modules/ with sudo pip install .) it finishes successfully. So for me, with my basic knowledge, it seems that it's an Issue with the Virtual Environment. Someone I talked to said, that it can happen, that the Virtual Environment gets "confused" when installing something like pyparsing into it with another Version. He pointed me to this Post and Issue: https://stackoverflow.com/a/56948334 https://github.com/pypa/pip/issues/3352#issuecomment-183746948

My Knowledge of Python Virtual Environments is very Basic, but I thought it's a good Idea to share the Information maybe it gives you a clue what it could be. Is maybe the Guide "How to install and start MISP modules in a Python virtualenv" not suitable anymore for the current Ubuntu Version? or is a Step Missing?

JustCodeIt4Head avatar Jan 30 '22 00:01 JustCodeIt4Head

Hello everyone,

I am trying to start the misp-modules and unfortunately, I am receiving the following error. I can't seem to find python library call misp_modules. Please advice

Started System-wide instance of the MISP Modules.
Traceback (most recent call last):
File "/var/www/MISP/venv/bin/misp-modules", line 5, in <module>
from misp_modules import main
ModuleNotFoundError: No module named 'misp_modules'
misp-modules.service: Main process exited, code=exited, status=1/FAILURE
misp-modules.service: Failed with result 'exit-code'.

asgharali1 avatar Feb 17 '22 21:02 asgharali1

Sorry to add another one in here. I'm on ubuntu18 and after fixing versions in the REQUIREMENTS file to older versions that worked with python3.6 i got modules installed but it wont start up.

I can start manually though but it looks like there is some error with a ''requote'

 /var/www/MISP/venv/bin/misp-modules
ERROR:root:__init__() got an unexpected keyword argument 'requote'
Traceback (most recent call last):
  File "/var/www/MISP/venv/lib/python3.6/site-packages/misp_modules/__init__.py", line 41, in <module>
    from .modules import *  # noqa

I've got that error in a few places

2022-02-24 19:35:06,619 - misp-modules - WARNING - MISP modules assemblyline_query failed due to __init__() got an unexpected keyword argument 'requote'
WARNING:misp-modules:MISP modules assemblyline_query failed due to __init__() got an unexpected keyword argument 'requote'

2022-02-24 19:35:08,188 - misp-modules - WARNING - MISP modules ocr_enrich failed due to cannot import name '_registerMatType'
WARNING:misp-modules:MISP modules ocr_enrich failed due to cannot import name '_registerMatType'

And a few others. Is this a python3.6 issue? I am planning on upgrading python at some point

cybgit avatar Feb 24 '22 19:02 cybgit

Same here:misp@misp:/usr/local/src/misp-modules$ /var/www/MISP/venv/bin/misp-modules -l 127.0.0.1 -s & [1] 5228 misp@misp:/usr/local/src/misp-modules$ ERROR:root:No module named 'pybgpranking' Traceback (most recent call last): File "/var/www/MISP/venv/lib/python3.6/site-packages/misp_modules/init.py", line 41, in from .modules import * # noqa File "/var/www/MISP/venv/lib/python3.6/site-packages/misp_modules/modules/init.py", line 1, in from .expansion import * # noqa File "/var/www/MISP/venv/lib/python3.6/site-packages/misp_modules/modules/expansion/bgpranking.py", line 6, in from pybgpranking import BGPRanking ModuleNotFoundError: No module named 'pybgpranking' 2022-03-20 15:27:56,608 - misp-modules - INFO - Launch MISP modules server from package. INFO:misp-modules:Launch MISP modules server from package. 2022-03-20 15:27:56,610 - misp-modules - INFO - Helper loaded cache INFO:misp-modules:Helper loaded cache 2022-03-20 15:27:56,610 - misp-modules - INFO - Unable to load MISP modules from package. INFO:misp-modules:Unable to load MISP modules from package.

(everything else went smooth/ no errors)

ChezDaniela avatar Mar 20 '22 14:03 ChezDaniela

To resolve the pybgpranking issue, I had to install python3.8 and then install the pybgpranking2 library via pip.

pip3 install pybgpranking2

The other installation command listed in the requirements.txt git+https://github.com/D4-project/BGP-Ranking.git/@68de39f6c5196f796055c1ac34504054d688aa59#egg=pybgpranking&subdirectory=client did not work and returned the following error in python 3.6.

root@4259d61c07d0:/var/www/MISP# pip3 install git+https://github.com/D4-project/BGP-Ranking.git/@68de39f6c5196f796055c1ac34504054d688aa59#egg=pybgprankin
g&subdirectory=client
[1] 1637
root@4259d61c07d0:/var/www/MISP# Collecting pybgpranking
  Cloning https://github.com/D4-project/BGP-Ranking.git/ (to revision 68de39f6c5196f796055c1ac34504054d688aa59) to /tmp/pip-install-x6xdbws6/pybgpranking_679465c17bfc42e182dfc86186c4206d
  Running command git clone --filter=blob:none --quiet https://github.com/D4-project/BGP-Ranking.git/ /tmp/pip-install-x6xdbws6/pybgpranking_679465c17bfc42e182dfc86186c4206d
  Running command git rev-parse -q --verify 'sha^68de39f6c5196f796055c1ac34504054d688aa59'
  Running command git fetch -q https://github.com/D4-project/BGP-Ranking.git/ 68de39f6c5196f796055c1ac34504054d688aa59
  Running command git checkout -q 68de39f6c5196f796055c1ac34504054d688aa59
  Resolved https://github.com/D4-project/BGP-Ranking.git/ to commit 68de39f6c5196f796055c1ac34504054d688aa59
  Preparing metadata (setup.py) ... done
  WARNING: Generating metadata for package pybgpranking produced metadata for project name bgpranking. Fix your #egg=pybgpranking fragments.
Discarding git+https://github.com/D4-project/BGP-Ranking.git/@68de39f6c5196f796055c1ac34504054d688aa59#egg=pybgpranking: Requested bgpranking from git+https://github.com/D4-project/BGP-Ranking.git/@68de39f6c5196f796055c1ac34504054d688aa59#egg=pybgpranking has inconsistent name: filename has 'pybgpranking', but metadata has 'bgpranking'
ERROR: Could not find a version that satisfies the requirement pybgpranking (unavailable) (from versions: none)
ERROR: No matching distribution found for pybgpranking (unavailable)

When looking at the BGP-Ranking project, I no longer see the client folder (referenced in the pip install path) present. A commit recent commit from Jan 2022 shows they upgraded pybgpranking to version 2. See this commit - https://github.com/D4-project/BGP-Ranking/commit/5c10ce75b70f9e104ba765d01cc942a445fe36e2

The pypi package shows that it only works with Python 3.8 and above. Upgrading my python to version 3.8 and installing the pybgpranking2 library resolved my issue.

billmurrin avatar Jul 11 '22 23:07 billmurrin