matrix-eno-bot
matrix-eno-bot copied to clipboard
Published
20 hours ago •
8go
8go
Docker container is stuck at "Performing initial database setup..."
Running on debian bullseye amd64. When I start the container via "docker compose up" after building it I get this message over and over
matrix-eno-bot | 2022-10-07 17:31:20,115 | storage [INFO] Performing initial database setup...
matrix-eno-bot exited with code 1
Below are my configs:
commands.yaml
#
# ***** Initialization *****
# This is an example command dictionary configuration. Mind you, this is an
# example configuration meant to convey the structure of the configuration
# file. It will probably not work as is in your environment.
# So, copy this file to "commands.yaml". Then modify and adjust
# "commands.yaml" to suit your needs. E.g. On Linux you would do
# $ cp commands.yaml.example commands.yaml
# $ nano commands.yaml # adjust to your needs, especially the paths!
# ***** Paths *****
# The bot needs to find your commands. For that the bot uses the PATH
# variable. By adjusting the "paths" values below you can add additional
# paths to the system PATH variable. If the bot is using commands that
# are not already on the system PATH, you must add the additional command
# paths below.
# Furthermore, make sure the bot has read and execute access
# to the commands, otherwise they won't work.
# Here we show two example command paths. Adjust them as needed.
paths:
- /home/minnix/.local/lib/matrix-eno-bot/eno/scripts
- /home/minnix/.local/etc/matrix-eno-bot/bin
# ***** Commands *****
# Commands are a list of specifications for your bot commands.
# The children of the "commands" key are the command names.
# Command names are nick names like "alert" or "backup".
# Each command, listed by its name, has the following
# keys (or properties):
#
# cmd: string: name of the command, executable, or script.
# The bot will attempt to execute this command when
# triggered. The bot will call this "cmd" and pass
# the given arguments to it.
# Example: "date" (a command provided by the OS)
# help: string: help string explaining the command.
# This is useful when you forget how use the command.
# The help string provided will be listed and shown with
# the 'help' command of the bot.
# Example: "returns the current date of the server"
# regex: string: regex pattern whose matches are valid ways to call the
# command.
# If a bot message matches a regular expression, then the
# corresponding command will be executed.
# If a bot message matched multiple regular expressions,
# only the first matching command will be executed!
# It is recommended that the regular expressions are used
# such that they are mutually exclusive to avoid
# situation where one messages matches multiple regular
# expressions.
# When the "regex" matches the "cmd" will be triggered.
# Example: "^date$" (only the exact string of "date"
# will match)
# markdown_convert: true or false: Specifies whether the message reply has
# been formatted in markdown.
# The bot will convert this markdown-formatted input
# and convert it into an HTML-like format understood by
# Matrix, so that the bot reply shows up visually as
# nice as the markdown input.
# Defaults to true.
# Example: Your command output is a markdown formatted
# string such as "I *really* like it!" Set
# markdown_convert to true and the receiver gets the
# text "I really like it!" with the word "really"
# visually in italic. In short, use "true", whenever
# your command output is a markdown-formatted string;
# false otherwise.
# formatted: true or false: Specifies whether message reply will
# be sent as a formatted message.
# Defaults to true.
# code: true or false: Specifies whether message reply will
# be formatted as a code block with fixed-size font.
# If set to "true", "markdown_convert" will be ignored.
# Defaults to false.
# split: string: if this string is set, splits the message reply
# into multiple messages wherever the string specified in
# split occurs.
# Defaults to None, no message splitting by default.
# Example: "\n\n\n" (Wherever the command output contains
# two empty lines, the output will be split. Each piece
# will be sent as a separate reply message.
commands:
# There are 3 kinds of commands.
# a) built-in commands
# b) pre-packed commands provided by the matrix-eno-bot repo
# c) your custom commands that you can add
# To add your custom commands go to the end of file, you will
# find a "Custom commands" header there. Add them there.
# Built-in commands
# ---------------
# help : "help" is a reserved word, so don't use it as a custom command!
# reload : "reload" is a reserved word, so don't use it as a custom command!
# Pre-packed commands provided by the matrix-eno-bot repo
# -------------------------------------------------------
# alert if too many resources are used, best to use with cron
alert:
regex: "alert$|^alert .*$|^alarm$|^alarm .*|^alert.sh$"
cmd: alert.sh
help: shows if any CPU, RAM, or disk thresholds have been exceeded
markdown_convert: false
formatted: true
code: true
# perform a backup to disk
backup:
regex: "^backup$|^backup .*$|^backup.sh$"
cmd: backup.sh
help: performs backup on server
markdown_convert: false
formatted: true
code: true
# get BTC ticker
btc:
regex: "^btc$|^btc .*$|^bitcoin$|^btc.sh$"
cmd: btc.sh
help: gives Bitcoin BTC price info
markdown_convert: false
formatted: true
code: true
# get cheatsheets, see https://github.com/cheat/cheat
cheatsheet:
regex: "^cheat$|^cheatsheet$|^chuleta$|^cheat.sh$|^cheat .*$|^cheatsheet .*$|^chuleta .*$|^cheat.sh .*$"
cmd: cheat
help: get cheatsheets, see https://github.com/cheat/cheat
markdown_convert: false
formatted: true
code: true
# check status and look for updates
# see also: upgrade
check:
regex: "^check$|^chk$|^status$|^state$|^check .*$|^chk .*|^status .*$|^state .*$|^check.sh$|^check.sh .*"
cmd: check.sh
help: check status, health status, updates, etc.
markdown_convert: false
formatted: true
code: false
# CPU temperature, to monitor the CPU temperatures
cputemp:
regex: "^cpu$|^temp$|^temperature$|^celsius$|^cputemp.*$|^hot$|^chaud$"
cmd: cputemp.sh
help: give the current CPU temperatures
markdown_convert: false
formatted: true
code: false
# get date and time
datetime:
regex: "^date$|^time$|^tiempo$|^hora$|^temps$|^heure$|^heures$|^datum$|^zeit$|^datetime.sh$"
cmd: datetime.sh
help: give current date and time of server
markdown_convert: false
formatted: true
code: true
# duckduckgo
ddg:
regex: "^ddg$|^ddg .*$|^duck$|^duck .*$|^duckduckgo$|^duckduckgo .*$|^search$|^search .*|^ddg.sh$|^ddg.sh .*"
cmd: ddg.sh
help: search the web with DuckDuckGo search
markdown_convert: false
formatted: true
code: false
# disk space, monitor disk space
disks:
regex: "^disks$|^disk$|^full$|^space$|^disks.sh$"
cmd: disks.sh
help: see how full your disks or mountpoints are
markdown_convert: false
formatted: true
code: true
# echo, trivial example to have the bot respond
echo:
regex: "^echo$|^echo .*"
cmd: echo.py
help: bot echoes back your input
markdown_convert: false
formatted: true
code: false
# get ETH ticker
eth:
regex: "^eth$|^eth .*$|^ethereum$|^eth.sh$"
cmd: eth.sh
help: gives Ethereum price info
markdown_convert: false
formatted: true
code: true
# get firewall settings
firewall:
regex: "^firewall$|^fw$|^firewall .*$|^firewall.sh$"
cmd: firewall.sh
help: list the firewall settings and configuration
markdown_convert: false
formatted: true
code: true
# get a compliment, hello
hello:
regex: "^salut$|^ciao$|^hallo$|^hi$|^servus$|^hola$|^hello$|^hello .*$|^bonjour$|^bonne nuit$|^hello.sh$"
cmd: hello.sh
help: gives you a friendly compliment
markdown_convert: false
formatted: true
code: false
# Hacker News
hn:
regex: "^hn$|^hn .*$|^hn.sh$|^hn.sh .*"
cmd: hn.sh
help: read Hacker News, fetches front page headlines from Hacker News
markdown_convert: false
formatted: true
code: false
# Messari News
mn:
regex: "^mn$|^mn .*$|^mn.sh$|^mn.sh .*"
cmd: mn.sh
help: read Messari News, fetches the latest news articles from Messari
markdown_convert: false
formatted: true
code: false
split: "\n\n\n"
# message of the day
motd:
regex: "^motd|^motd .*|^motd.sh$"
cmd: motd.sh
help: gives you the Linux Message Of The Day
# platform info
platforminfo:
regex: "^platform$|^platform .*|^platforminfo.py$"
cmd: platforminfo.py
help: give hardware and operating system platform information
# ps, host status
ps:
regex: "^ps$|^ps .*|^ps.sh$"
cmd: ps.sh
help: print current CPU, RAM and Disk utilization of server
markdown_convert: false
formatted: true
code: true
# restart, reset
restart:
regex: "^restart$|^reset$|^restart .*$|^reset .*$|^restart.sh$|^restart.sh .*"
cmd: restart.sh
help: restart the bot itself, or Matrix services
markdown_convert: false
formatted: true
code: false
# RSS
rss:
regex: "^rss$|^feed$|^rss .*$|^feed .*$|^rss.sh$|^rss.sh .*"
cmd: rss.sh
help: read RSS feeds
markdown_convert: false
formatted: true
code: false
split: "\n\n\n"
# Stock-to-flow
s2f:
regex: "^s2f$|^mys2f.py.*|^flow$|^s2f|^flow .*$|^s2f .$|^s-to-f$|^stock-to-flow .*$|^eyf$|^eyf .*$|^e-y-f$"
cmd: s2f.sh
help: give Stock-to-flow info
markdown_convert: false
formatted: true
code: true
# tides
tides:
regex: "^tide$|^tides$|^marea|^mareas|^tide .*$|^tides .*$|^marea .*$|^mareas .*$|^gehzeiten .*$|^tides.sh$|^tides.sh .*"
cmd: tides.sh
help: give tidal forecast
markdown_convert: false
formatted: true
code: false
# top CPU, MEM consumers
top:
regex: "^top$|^top .*|^top.sh$|^top.sh .*"
cmd: top.sh
help: list 5 top CPU and RAM consuming processes
markdown_convert: false
formatted: true
code: true
# get TOTP 2FA pin
totp:
regex: "^otp$|^totp$|^otp .*$|^totp .*$"
cmd: totp.sh
help: get 2FA Two-factor-authentication TOTP PIN via bot message
markdown_convert: false
formatted: true
code: false
# twitter
twitter:
regex: "^tweet$|^twitter$|^tweet .*$|^twitter .*$|^twitter.sh$|^twitter.sh .*"
cmd: twitter.sh
help: read latest user tweets from Twitter
markdown_convert: false
formatted: true
code: false
# update components
update:
regex: "^update$|^upgrade$|^update .*$|^upgrade .*$|^update.sh$|^update.sh .*"
cmd: update.sh
help: update operating sytem
markdown_convert: false
formatted: true
code: false
# list matrix users by issuing a REST API query
users:
regex: "^usr$|^user$|^users$|^users .*$|^users.sh$"
cmd: users.sh
help: list registered Matrix users
markdown_convert: false
formatted: true
code: true
# wake up PC via wake-on-LAN
wake:
regex: "^wake$|^wakeup$|^wake .*$|^wakeup .*$|^wakelan .*$|^wake.sh$|^wake .*"
cmd: wake.sh
help: wake up another PC via LAN
markdown_convert: false
formatted: true
code: false
# waves and surf conditions
# see also: tides
waves:
regex: "^wave$|^waves$|^wave .*$|^waves .*$|^surf$|^surf .*$|^waves.sh$"
cmd: waves.sh
help: give waves and surf forecast
markdown_convert: false
formatted: true
code: true
# get weather forecast
weather:
regex: "^weather$|^tiempo$|^wetter$|^temps$|^weather .*$|^tiempo .*$|^eltiempo .*$|^wetter .*$|^temps .*$|^weather.sh$|^weather.sh .*"
cmd: weather.sh
help: give weather forecast
markdown_convert: false
formatted: true
code: true
# fetch web pages
web:
regex: "^www$|^web$|^web .*$|^www .*$|^browse$|^browse .*|^web.sh$|^web.sh .*"
cmd: web.sh
help: surf the web, get a web page (JavaScript pages not supported)
markdown_convert: false
formatted: true
code: false
# whoami
whoami:
regex: "^w$|^who$|^whoami$"
cmd: whoami.py
help: return information about the user, whose unix account is running the bot
markdown_convert: false
formatted: true
code: false
# Custom commands
# ---------------
# add your custom commands here
# End of commands configuration file
config.yaml
# Welcome to the sample config file
# Below you will find various config sections and options
# Default values are shown
# The string to prefix messages with to talk to the bot in group chats
# Changed from original "!c" to "1z! because on cell phone "!c" seems too
# complicated.
command_prefix: "1z"
# Options for connecting to the bot's Matrix account
matrix:
# The Matrix User ID of the bot account
user_id: "@lugbot:minnix.dev"
# Matrix account password
user_password: "<password>"
# Matrix account access token
# To create a new Matrix device for the bot, use and set the `user_password`
# field. Once the device exists, you can optionally replace the
# `user_password` with the `access_token` field. Using the `access_token`
# field is slightly safer as it does not expose the password of the bot
# account. You can use only one or the other: either use the `user_password`
# or the `access_token` field. You can find the access token in the Matrix
# client where you have registered the bot account or you can find it in
# the bot log file. If the logging options are set accordingly the access
# token will be logged to the bot log file.
# Default: commented out
# access_token: "PutYourLongAccessTokenHere"
# The URL of the homeserver to connect to
homeserver_url: https://matrix.minnix.dev
# The device ID that is **non pre-existing** device
# If this device ID already exists, messages will be dropped
# silently in encrypted rooms
device_id: lugbot_serverid
# What to name the device? Often referred to as device name or display name.
device_name: lugbot
# Should the bot trust all the devices of its own Matrix account?
# Default: false
# If false, nothing is done. After login, no device will be automatically
# trusted.
# If true, once at startup, after logging in, the bot device will
# automatically establish trust to all other devices of the bot account.
trust_own_devices: false
# Do you want to change the device_name of the already existing bot?
# Default: false
# If false, nothing is done. After creation, device_name will be ignored.
# If true, device_name of bot will be changed to value given in device_name.
change_device_name: false
# encrytion is enabled by default
storage:
# The path to the database
database_filepath: "bot.db"
# The path to a directory for internal bot storage
# containing encryption keys, sync tokens, etc.
store_filepath: "./store"
# The path to the command dictionary configuration file
command_dict_filepath: "./commands.yaml"
# Logging setup
logging:
# Logging level
# Allowed levels are 'INFO', 'WARNING', 'ERROR', 'DEBUG'
# where DEBUG is most verbose
level: INFO
# Configure logging to a file
file_logging:
# Whether logging to a file is enabled
enabled: false
# The path to the file to log to. May be relative or absolute
filepath: bot.log
# Configure logging to the console output
console_logging:
# Whether logging to the console is enabled
enabled: true
docker-compose.yaml
version: '3'
services:
matrix-eno-bot:
container_name: matrix-eno-bot
image: 'matrix-eno-bot:latest'
build: '.'
restart: always
volumes:
- /home/minnix/lugbot/matrix-eno-bot/config.yaml:/bot/config.yaml
- /home/minnix/lugbot/matrix-eno-bot/commands.yaml:/bot/commands.yaml
- /home/minnix/lugbot/matrix-eno-bot/bot.db:/bot/bot.db
- /home/minnix/lugbot/matrix-eno-bot/store/:/bot/store/
stop_signal: SIGINT
Any help would be appreciated thanks.
I do not use docker frequently, so I cannot be of help.
Anyone else out there who can assist or provide some feedback? Please comment.
Also:
vim config.yaml # modify the logging parameters, increase logging
See in the log output helps you further.
Also:
vim config.yaml # modify the logging parameters, increase loggingSee in the log output helps you further.
Ok I changed to DEBUG instead of INFO and was able to get a little more info:
matrix-eno-bot | 2022-10-10 08:24:10,313 | storage [INFO] Performing initial database setup...
matrix-eno-bot | 2022-10-10 08:24:10,314 | __main__ [DEBUG] Traceback (most recent call last):
matrix-eno-bot | File "./main.py", line 169, in <module>
matrix-eno-bot | asyncio.get_event_loop().run_until_complete(main())
matrix-eno-bot | File "/usr/local/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
matrix-eno-bot | return future.result()
matrix-eno-bot | File "./main.py", line 59, in main
matrix-eno-bot | store = Storage(config.database_filepath)
matrix-eno-bot | File "/bot/storage.py", line 26, in __init__
matrix-eno-bot | self._initial_setup()
matrix-eno-bot | File "/bot/storage.py", line 33, in _initial_setup
matrix-eno-bot | self.conn = sqlite3.connect(self.db_path)
matrix-eno-bot | sqlite3.OperationalError: unable to open database file
matrix-eno-bot |
matrix-eno-bot exited with code 1
The debug log says: sqlite3 tries to create a database and it fails. wherever your bot.db there is a problem. Either the file exists and is not a real sqlite3 db, or the dir is not writeable, etc.
As a detail test: test that bot.db does not exist, but that the location is writeable. So, check if a file is there. Do a cat to create a tiny file, check file, (this verifies you can write), then remove the file (verify that it does not exist), then start bot.
Alternatively, run it without docker to learn more about it and then debug your docker yaml.
Ok the bot.db directory is there but is empty. If I try to write to the directory as a regular user there is a permission error, but using sudo is ok. I run docker as sudo so I would think it should be ok, unless something was supposed to be written there during the docker first run. I can prune all the docker stuff and re-run and see what happens.
bot.db is not a directory, should not be a directory. bot.db is a file to be created.
database_filepath: "/foo/bar/bot.db" then /foo/bar must be an existing dir with write permissions.
database_filepath: "bot.db" then . must be an existing dir with write permissions. . obviously depends, where you are, from where you execute the command, ...
So I removed the bot.db directory that was created from docker compose and just created a blank bot.db file to see what would happen than ran docker compose up again. This is the error:
[+] Running 1/0
⠿ Container matrix-eno-bot Created 0.0s
Attaching to matrix-eno-bot
Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/home/minnix/matrix-eno-bot/bot.db" to rootfs at "/bot/bot.db": mount /home/minnix/matrix-eno-bot/bot.db:/bot/bot.db (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
So it looks like it is expecting bot.db to be a directory
bot.db as directory: definitely wrong
bot.db as a blank file: definitely wrong
Your Docker file is not correct. If done correctly, the eno bot creates the bot.db file.
Dockerfile
FROM python:3.7-slim
RUN apt update && apt upgrade -y && \
apt install -y \
wget \
libmagic1 \
build-essential
WORKDIR /bot
COPY *.py /bot/
COPY *.yaml /bot/
COPY *.txt /bot/
COPY eno /bot/eno/
# download libolm3 from Ubuntu focal distribution
# https://packages.ubuntu.com/focal/libolm3
# https://packages.ubuntu.com/focal/libolm-dev
RUN wget http://mirrors.kernel.org/ubuntu/pool/universe/o/olm/libolm-dev_3.1.3+dfsg-2build2_amd64.deb
RUN wget http://mirrors.kernel.org/ubuntu/pool/universe/o/olm/libolm3_3.1.3+dfsg-2build2_amd64.deb
RUN dpkg -i ./*.deb
RUN pip install -r requirements.txt
# clean up apt cache and remove gcc
RUN apt purge -y build-essential && \
apt autoremove -y && apt clean && \
rm -rf /var/lib/apt/lists/*
CMD [ "python", "./main.py" ]
docker-compose.yaml
version: '3'
services:
matrix-eno-bot:
container_name: matrix-eno-bot
image: 'matrix-eno-bot:latest'
build: '.'
restart: always
volumes:
- /home/minnix/matrix-eno-bot/config.yaml:/bot/config.yaml
- /home/minnix/matrix-eno-bot/commands.yaml:/bot/commands.yaml
- /home/minnix/matrix-eno-bot/bot.db:/bot/bot.db
- /home/minnix/matrix-eno-bot/store/:/bot/store/
stop_signal: SIGINT
If there's anything amiss you can find please let me know.
I am not expert enough on Docker to make comments on Docker files.
Run it without Docker. You will see, that if bot.db is a directory, it will fail. If bot.db is an empty file it will also fail. And if bot.db does not exist it will create it. And if bot.db is a valid Sqlite file it will read it and use it.
Ok, trying to run it without docker. I'm having problems understanding what the paths should be in the systemctl file.
[Unit]
Description=matrix-eno-bot
[Service]
# change user name to fit your needs
User=matrix-neo-bot
Group=users
Environment=PYTHONUNBUFFERED=1
# change this to match your server's timezone
Environment=TZ=UTC
# change this PATH to fit your needs
Environment=PATH=/home/matrix-eno-bot/matrix-eno-bot/eno/scripts:/home/matrix-eno-bot/bin:/home/matrix-eno-bot/.local/bin:/home/matrix-eno-bot/Scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# change this PATH to fit your PATH
ExecStart=/home/matrix-eno-bot/matrix-eno-bot/main.py /home/matrix-eno-bot/matrix-eno-bot/config.yaml
ExecStop=/bin/kill -9 $MAINPID
Restart=on-failure
RestartSec=30
[Install]
WantedBy=multi-user.target
the matrix-eno-bot folder is in my user folder, /home/minnix/. how should Environment=PATH= and ExecStart= read? Also do I need to create a User called matrix-neo-bot?
To start with you do not need any of this, this is if and only if you want to run it as a service. But you can run it without a service as well, just starting it by hand.
Environment=PATH= should include paths to all the scripts and binaries that your bot is going to call.
ExecStart= just has a full path to program and to 1 argument.
In your case likely: ExecStart=/home/minnix/matrix-eno-bot/main.py /home/minnix/matrix-eno-bot/config.yaml
You should be able to run this is your terminal /home/minnix/matrix-eno-bot/main.py /home/minnix/matrix-eno-bot/config.yaml
User called matrix-neo-bot? No, not needed.