temboard
temboard copied to clipboard
Published
20 hours ago •
dalibo
dalibo
Statements didnt show up data or records on Temboard UI
Hello friends i have an issue as u can see, page Statements on Temboard UI didnt show any data or records, and i did set preload libraries and enable pg_stat_statement already. Is there solution for my issue? Thanks
Hi @fathoniinsansilmi , do you have some information in logs ? In both agent and UI ?
Hi @fathoniinsansilmi , do you have some information in logs ? In both agent and UI ?
yes i do have logs both agen and ui here it is
agent log
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: {'error': 'Invalid session.'}
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: Traceback (most recent call last):
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: File "/usr/lib/python3.6/site-packages/temboardagent/api.py", line 36, in check_sessionid
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: session = sessions.get_by_sessionid(xsession)
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: File "/usr/lib/python3.6/site-packages/temboardagent/sharedmemory.py", line 48, in get_by_sessionid
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: raise SharedItem_not_found()
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: temboardagent.errors.SharedItem_not_found
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR:
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: During handling of the above exception, another exception occurred:
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR:
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: Traceback (most recent call last):
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: File "/usr/lib/python3.6/site-packages/temboardagent/httpd.py", line 131, in response
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: (code, message) = self.route_request()
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: File "/usr/lib/python3.6/site-packages/temboardagent/httpd.py", line 253, in route_request
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: username = check_sessionid(self.headers, self.sessions)
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: File "/usr/lib/python3.6/site-packages/temboardagent/api.py", line 42, in check_sessionid
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: raise HTTPError(401, "Invalid session.")
2021-12-24 10:33:26,011 temboardagent[11781]: [httpd] ERROR: temboardagent.errors.HTTPError: Invalid session.
2021-12-24 10:33:26,012 temboardagent[11781]: [httpd] ERROR: {'error': 'Invalid session.'}
ui log
2021-12-24 10:38:00,013 temboardui[9021]: [statements] ERROR: ^
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: relation "statements_src_tmp" does not exist
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: LINE 2: INSERT INTO statements_src_tmp
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: ^
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: Traceback (most recent call last):
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: File "/usr/local/lib/python2.7/dist-packages/temboardui/plugins/statements/__init__.py", line 827, in pull_data_for_instance
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: add_statement(session, instance, json_decode(body))
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: File "/usr/local/lib/python2.7/dist-packages/temboardui/plugins/statements/__init__.py", line 788, in add_statement
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: raise TemboardUIError(400, e.message)
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: TemboardUIError: relation "statements_src_tmp" does not exist
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: LINE 2: INSERT INTO statements_src_tmp
2021-12-24 10:38:00,014 temboardui[9021]: [statements] ERROR: ^
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: <class 'sqlalchemy.exc.InternalError'>: (psycopg2.errors.InFailedSqlTransaction) current transaction is aborted, commands ignored until end of transaction block
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR:
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: [SQL: SELECT application.plugins.agent_address AS application_plugins_agent_address, application.plugins.agent_port AS application_plugins_agent_port, application.plugins.plugin_name AS application_plugins_plugin_name
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: FROM application.plugins
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: WHERE %(param_1)s = application.plugins.agent_address AND %(param_2)s = application.plugins.agent_port ORDER BY application.plugins.plugin_name]
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: (Background on this error at: http://sqlalche.me/e/14/2j85)
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: Traceback (most recent call last):
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/temboardui/toolkit/taskmanager.py", line 882, in exec_worker
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: res = getattr(sys.modules[module], function)(*args, **kws)
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/temboardui/toolkit/taskmanager.py", line 1046, in wrapper
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: return function(app=self.app, *a, **kw)
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/temboardui/plugins/statements/__init__.py", line 800, in pull_data_worker
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: plugin_names = [plugin.plugin_name for plugin in instance.plugins]
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 480, in __get__
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: return self.impl.get(state, dict_)
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 931, in get
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: value = self.callable_(state, passive)
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/strategies.py", line 881, in _load_for_state
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: extra_criteria,
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/strategies.py", line 1033, in _emit_lazyload
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: stmt, params, execution_options=execution_options
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1689, in execute
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: result = conn._execute_20(statement, params or {}, execution_options)
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1582, in _execute_20
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: return meth(self, args_10style, kwargs_10style, execution_options)
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/lambdas.py", line 482, in _execute_on_connection
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: self, multiparams, params, execution_options
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1461, in _execute_clauseelement
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: cache_hit=cache_hit,
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1814, in _execute_context
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: e, statement, parameters, cursor, context
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1995, in _handle_dbapi_exception
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: sqlalchemy_exception, with_traceback=exc_info[2], from_=e
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1771, in _execute_context
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: cursor, statement, parameters, context
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 717, in do_execute
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: cursor.execute(statement, parameters)
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR: InternalError: (psycopg2.errors.InFailedSqlTransaction) current transaction is aborted, commands ignored until end of transaction block
2021-12-24 10:38:00,050 temboardui[9021]: [taskmanager] ERROR:
@fathoniinsansilmi what is the output of temboard --version ?
sudo temboard --version
INFO: Starting temboard 7.7.
temBoard 7.7
System Ubuntu 18.04.5 LTS (Bionic Beaver)
Python 2.7.17 (/usr/bin/python)
psycopg2 2.8.6 (dt dec pq3 ext lo64)
Tornado 4.5.3
SQLAlchemy 1.4.18
alembic 1.6.5
sudo temboard-agent --version
[sudo] password for ngufathoni:
INFO: Starting temboard-agent 7.8.
7.8
Thanks. What is the output of
DEBUG=Y temboard-migratedb check?
where should i execute this command?
ngufathoni@gateway:~$ export DEBUG=Y
ngufathoni@gateway:~$ temboard-migratedb check
2021-12-27 16:47:35,546 [11511] [app ] INFO: Starting temboard-migratedb 7.7.
2021-12-27 16:47:35,550 [11511] [app ] INFO: No config file found.
2021-12-27 16:47:35,566 [11511] [migratedb ] DEBUG: Using Python 2.7.17 (/usr/bin/python).
2021-12-27 16:47:35,567 [11511] [migratedb ] DEBUG: Using Psycopg2 2.8.6 (dt dec pq3 ext lo64), Alembic 1.6.5 and SQLAlchemy 1.4.18
2021-12-27 16:47:35,577 [11511] [app ] CRITI: Failed to check schema: (psycopg2.OperationalError) FATAL: Peer authentication failed for user "temboard"
2021-12-27 16:47:35,577 [11511] [app ] CRITI:
2021-12-27 16:47:35,577 [11511] [app ] CRITI: (Background on this error at: http://sqlalche.me/e/14/e3q8).
ngufathoni@gateway:~$
ngufathoni@gateway:~$ sudo -u temboard temboard-migratedb check INFO: Starting temboard-migratedb 7.7. INFO: Found config file /etc/temboard/temboard.conf. INFO: temBoard database is up-to-date.
Your temboard database seems corrupted. In temboard database, what is the output of \d in psql ?
postgres=# \d List of relations Schema | Name | Type | Owner --------+----------------------+----------+---------- public | accounts | table | postgres public | accounts_user_id_seq | sequence | postgres public | pg_stat_statements | view | postgres public | roles | table | postgres public | roles2 | table | postgres public | roles2_role_id_seq | sequence | postgres public | roles_role_id_seq | sequence | postgres (7 rows)
@fathoniinsansilmi no. Use the database found by temboard in its configuration. Did you use auto_configure.sh ?
oh u mean psql on same environment temboard service, sory i thought that was supposed to execute on temboard-agent,
temboard=# \d
Did not find any relations.
temboard=# \dn+
List of schemas
Name | Owner | Access privileges | Description
-------------+----------+----------------------+------------------------
application | postgres | postgres=UC/postgres+|
| | temboard=UC/postgres |
monitoring | postgres | postgres=UC/postgres+|
| | temboard=UC/postgres |
public | postgres | postgres=UC/postgres+| standard public schema
| | =UC/postgres |
statements | postgres | |
(4 rows)
temboard=# \d statements.*
Table "statements.metas"
Column | Type | Collation | Nullable | Default
---------------+--------------------------+-----------+----------+---------------------------------------
agent_address | text | | not null |
agent_port | integer | | not null |
coalesce_seq | bigint | | not null | 1
snapts | timestamp with time zone | | not null | '-infinity'::timestamp with time zone
aggts | timestamp with time zone | | not null | '-infinity'::timestamp with time zone
purgets | timestamp with time zone | | not null | '-infinity'::timestamp with time zone
error | text | | |
Indexes:
"metas_pkey" PRIMARY KEY, btree (agent_address, agent_port)
Foreign-key constraints:
"metas_agent_address_agent_port_fkey" FOREIGN KEY (agent_address, agent_port) REFERENCES application.instances(agent_address, agent_port) ON UPDATE CASCADE ON DELETE CASCADE
Index "statements.metas_pkey"
Column | Type | Key? | Definition
---------------+---------+------+---------------
agent_address | text | yes | agent_address
agent_port | integer | yes | agent_port
primary key, btree, for table "statements.metas"
Table "statements.statements"
Column | Type | Collation | Nullable | Default
---------------+---------+-----------+----------+---------
agent_address | text | | not null |
agent_port | integer | | not null |
queryid | bigint | | not null |
query | text | | not null |
dbid | oid | | not null |
datname | text | | not null |
userid | oid | | not null |
rolname | text | | not null |
Indexes:
"statements_pkey" PRIMARY KEY, btree (agent_address, agent_port, queryid, dbid, userid)
Foreign-key constraints:
"statements_agent_address_agent_port_fkey" FOREIGN KEY (agent_address, agent_port) REFERENCES application.instances(agent_address, agent_port) ON UPDATE CASCADE ON DELETE CASCADE
Referenced by:
TABLE "statements.statements_history" CONSTRAINT "statements_history_agent_address_agent_port_queryid_dbid_u_fkey" FOREIGN KEY (agent_address, agent_port, queryid, dbid, userid) REFERENCES statements.statements(agent_address, agent_port, queryid, dbid, userid) ON UPDATE CASCADE ON DELETE CASCADE
TABLE "statements.statements_history_current" CONSTRAINT "statements_history_current_agent_address_agent_port_queryi_fkey" FOREIGN KEY (agent_address, agent_port, queryid, dbid, userid) REFERENCES statements.statements(agent_address, agent_port, queryid, dbid, userid) ON UPDATE CASCADE ON DELETE CASCADE
Table "statements.statements_history"
Column | Type | Collation | Nullable | Default
----------------+----------------------------------------+-----------+----------+---------
agent_address | text | | not null |
agent_port | integer | | not null |
queryid | bigint | | not null |
dbid | oid | | not null |
userid | oid | | not null |
coalesce_range | tstzrange | | not null |
records | statements.statements_history_record[] | | not null |
mins_in_range | statements.statements_history_record | | not null |
maxs_in_range | statements.statements_history_record | | not null |
Indexes:
"statements_history_agent_address_agent_port_dbid_idx" btree (agent_address, agent_port, dbid)
"statements_history_coalesce_range_idx" gist (coalesce_range)
Foreign-key constraints:
"statements_history_agent_address_agent_port_queryid_dbid_u_fkey" FOREIGN KEY (agent_address, agent_port, queryid, dbid, userid) REFERENCES statements.statements(agent_address, agent_port, queryid, dbid, userid) ON UPDATE CASCADE ON DELETE CASCADE
Index "statements.statements_history_agent_address_agent_port_dbid_idx"
Column | Type | Key? | Definition
---------------+---------+------+---------------
agent_address | text | yes | agent_address
agent_port | integer | yes | agent_port
dbid | oid | yes | dbid
btree, for table "statements.statements_history"
Index "statements.statements_history_coalesce_range_idx"
Column | Type | Key? | Definition
----------------+-----------+------+----------------
coalesce_range | tstzrange | yes | coalesce_range
gist, for table "statements.statements_history"
Table "statements.statements_history_current"
Column | Type | Collation | Nullable | Default
---------------+--------------------------------------+-----------+----------+---------
agent_address | text | | not null |
agent_port | integer | | not null |
queryid | bigint | | not null |
dbid | oid | | not null |
userid | oid | | not null |
record | statements.statements_history_record | | not null |
Indexes:
"statements_history_current_agent_address_agent_port_dbid_idx" btree (agent_address, agent_port, dbid)
Foreign-key constraints:
"statements_history_current_agent_address_agent_port_queryi_fkey" FOREIGN KEY (agent_address, agent_port, queryid, dbid, userid) REFERENCES statements.statements(agent_address, agent_port, queryid, dbid, userid) ON UPDATE CASCADE ON DELETE CASCADE
Index "statements.statements_history_current_agent_address_agent_port_dbid_idx"
Column | Type | Key? | Definition
---------------+---------+------+---------------
agent_address | text | yes | agent_address
agent_port | integer | yes | agent_port
dbid | oid | yes | dbid
btree, for table "statements.statements_history_current"
Table "statements.statements_history_current_db"
Column | Type | Collation | Nullable | Default
---------------+--------------------------------------+-----------+----------+---------
agent_address | text | | not null |
agent_port | integer | | not null |
dbid | oid | | not null |
datname | text | | not null |
record | statements.statements_history_record | | not null |
Foreign-key constraints:
"statements_history_current_db_agent_address_agent_port_fkey" FOREIGN KEY (agent_address, agent_port) REFERENCES application.instances(agent_address, agent_port) ON UPDATE CASCADE ON DELETE CASCADE
Table "statements.statements_history_db"
Column | Type | Collation | Nullable | Default
----------------+----------------------------------------+-----------+----------+---------
agent_address | text | | not null |
agent_port | integer | | not null |
dbid | oid | | not null |
datname | text | | not null |
coalesce_range | tstzrange | | not null |
records | statements.statements_history_record[] | | not null |
mins_in_range | statements.statements_history_record | | not null |
maxs_in_range | statements.statements_history_record | | not null |
Foreign-key constraints:
"statements_history_db_agent_address_agent_port_fkey" FOREIGN KEY (agent_address, agent_port) REFERENCES application.instances(agent_address, agent_port) ON UPDATE CASCADE ON DELETE CASCADE
Composite type "statements.statements_history_record"
Column | Type | Collation | Nullable | Default
---------------------+--------------------------+-----------+----------+---------
ts | timestamp with time zone | | |
calls | bigint | | |
total_exec_time | double precision | | |
rows | bigint | | |
shared_blks_hit | bigint | | |
shared_blks_read | bigint | | |
shared_blks_dirtied | bigint | | |
shared_blks_written | bigint | | |
local_blks_hit | bigint | | |
local_blks_read | bigint | | |
local_blks_dirtied | bigint | | |
local_blks_written | bigint | | |
temp_blks_read | bigint | | |
temp_blks_written | bigint | | |
blk_read_time | double precision | | |
blk_write_time | double precision | | |
total_plan_time | double precision | | |
wal_records | bigint | | |
wal_fpi | bigint | | |
wal_bytes | numeric | | |
Index "statements.statements_pkey"
Column | Type | Key? | Definition
---------------+---------+------+---------------
agent_address | text | yes | agent_address
agent_port | integer | yes | agent_port
queryid | bigint | yes | queryid
dbid | oid | yes | dbid
userid | oid | yes | userid
primary key, btree, for table "statements.statements"
temboard=#
That's odd. You're database miss some database.
If you don't mind resetting statements data, you can reexecute /usr/lib/temboard/lib/python2.7/site-packages/temboardui/model/alembic/versions/statements.sql with psql, in temboard database.
$ sudo -u temboard psql -f /usr/lib/temboard/lib/python2.7/site-packages/temboardui/model/alembic/versions/statements.sql
Hi @bersace , do we need upgrade version first before execute this command: $ sudo -u temboard psql -f /usr/lib/temboard/lib/python2.7/site-packages/temboardui/model/alembic/versions/statements.sql
Hi @bersace , do we need upgrade version first before execute this command: $ sudo -u temboard psql -f /usr/lib/temboard/lib/python2.7/site-packages/temboardui/model/alembic/versions/statements.sql
@fathoniinsansilmi your temBoard database is inconsistent. Can you restart from scratch ?
Hi @bersace i create new temboard UI with 7.9 version, first day statement shows the graph, but next day until now statement not showing any graph again. Any solution for this issue?
hi @bersace i have another question, is it possible to conect temboard agent 7.10 version with temboard ui 7.9 version? if that possible, will there be any issue for that?
Hi @fathoniinsansilmi Yes, temBoard UI 7.9 is compatible with agent 7.10. However, I strongly suggest you to use temBoard UI 7.10.
This issue has been inactive for several weeks. Unless you can provide new elements to the discussion, we will close it in a few days.
FTR Temboard 7.11 was published a few weeks ago.
