dbcli
dbcli copied to clipboard
Windows/Linux/OSX/MSYS command line utility for Oracle/MySQL/DB2/PostgreSQL
DBCLI : A CLI utility for DBA (Oracle/MySQL/DB2)
For more detailed usage, please refer to the PowerPoint file under the docs
directory
Description
DBCLI is a portable database command line utility, for DBA or application support to easily manage and run their common used scripts, similar to database built-in CLI tools but is more flexible.
It is mainly developed by Lua language, and bridges to JDBC to access database, many of the files are SQL scripts that easy to maintain.
It is designed to support multiple databases, and includes 2 types of modules, one is the public module whose functions are visible across all database platforms,
and another one is the platform-specific module.
It is not intended to support all features of the existing CLI utilities that embedded by the specific database(Oracle SQL*Plus
, DB2 CLP
, etc), but to provide the features other than those tools:
- Pretty-print of the select statement and the flexible control of the grid layouts(see set command)
- Automatically generate help documents for customized SQL/shell/etc scripts.
- The
alias
command to reduce the frequent inputs - The powerful
snap
andora
commands as the replacement of the standard CLI scripts, so that DBA is able to quickly define and execute the SQL templates with the smallest input - Support SSH operations
- Provides the graph charts of the database performance, refer to the "chart" command
- Easy to extend, modify or deploy, because most of the source code is not required to compile
- Easily switch to the db built-in cli utility
Download
DBCLI does not require compile or installation, but depends on JRE 1.8+. Click Download ZIP
,extract and use. Or you can also download from the published releases
Branch With-JRE
contains the shrinked JRE bundle, if you don't have JRE installed,
you may switch to that branch to download. Due to the region and charset differences, this JRE bundle is not guaranteed to work well.
Windows: Configure terminal window
For Windows OS, to avoid the word wrap in the terminal window which affects the print layout, following settings are recommended in Windows command window:
- In the
Layout
tab, set buffer width and buffer height as500+
- In the
Font
tab, change the font size as smaller value.
Linux: Pre-Requisitions
Linux must be the 64-bit
version, and make sure glibc >= v2.4
(use ldd --version
to verify) are installed. You can use command <command> | more
to vertically/horizontally scroll the command output, or more last
or scroll the last command output.
OSX: Pre-Requisitions
Only tested on OSX 10.12, lower version should be fine. Java 1.8+ is required. It's recommended to add entry <hostname>.local
into /etc/hosts
to speed up the JVM performance.
For Example:
127.0.0.1 localhost mac-pc.local
::1 localhost mac-pc.local
And sometimes you may run into below exception when try to execute dbcli.sh
:
"luajit" cannot be opened because the developer cannot be verified
The solution is to disable the restriction: sudo spctl --master-disable
Configure environment
Before running dbcli, make sure that you have installed JRE 1.8+ in your local PC. If you are using the version of "With-JRE" branch, this step can be skipped.
Create file init.cfg
(or init.conf
for non-Windows) under the "data" directory with following content(for non-Windows OS use export
instead):
SET JRE_HOME=<JRE HOME>
SET TNS_ADM=<location of tnsnames.ora>
Of which TNS_ADM
is optional unless you need to connect Oracle via tnsnames.ora, or may also refer to init_sample.cfg
for more settings. An example is:
SET JRE_HOME=d:\soft\java
SET TNS_ADM=d:\Soft\InstanceClient\network\admin
The default charset is UTF-8
, to use another charset, just change the DBCLI_ENCODING
environment variable globally or in the mentioned config file. For example:
-
SET DBCLI_ENCODING=GBK
(for Windows) -
DBCLI_ENCODING=GBK ./dbcli.sh
Launch DBCLI Console
After the above configurations, you are able to start DBCLI by executing dbcli.bat
or bin\ConsoleZ\Console.exe
.
dbcli.bat
also supports arguments in "<command>=<args>"
format, all available commands can be found by typing help
or help -a
.
For example: dbcli.bat "connect=tiger/scott@orcl"
Switch Database Platform
DBCLI supports multiple database platforms(oracle,mysql,db2,etc), default is Oracle
.
To permanently change the preferred platform other than Oracle
, run set -p platform <platform>
after launching the console.
For example, set -p platform db2
Without the -p
option, the change only takes affect on current console window.
Type set platform
to see all available options.
Command Overview
-
Public Commands: visible in all database platforms
-
Instruction
-
help
: Show help information,Use"help <command>"
to show the detail information of other commands, or"help help"
to see more usage of thehelp
command.
The default database platform isOracle
, to switch to other platform, use the"set [-p] platform"
command -
ansi
:Show the usage of console color solution -
hotkeys
:Show the available hotkeys in console|
-
-
Core
-
set
:A command set, show or change all kinds of preferences of console environment or other commands. The changed settings can be either permanent or temporary. -
alias
:A command set, similar to the Unixalias
command, mainly used to define the shortcut of other command line, the definition is automatially stored in folderaliases
for permanent reused| -
history/his
:Check or run historical commands, and the historial commands are only kept in memory and available within the current window -
r
or/
:Rerun previous command -
sleep
:Sleep for specific seconds, mainly used in script file -
repeat/itv
:Periodically run other commands -
host
or!
:Run OS command -
reload
:Close all connections and reload the console environment, everything will be reset. -
exit
:Close all connections and close the console
-
-
Screen
-
clear/cls
:Clear screen -
spool/spo
:Sync screen output to file, the command name can be different regarding to different db platforms -
column/col
:Control the layout format of grid(i.e. query) field -
grep
:Similar to the Unixgrep
command but only supports piping from other command -
more
:Similar to the Unixmore
command but only supports piping from other command -
tee
:Similar to the Unixtee
command but only supports piping from other command, also can export the screen output intocsv
orhtml
format regarding to the file extension, the command name can be different regarding to different db platforms -
prompt/pro/echo
:Print text, if the text contains variables, then also convert as the the vairable values, the command name can be different regarding to different db platforms
-
-
Database
-
connect/conn
:Connect to database, Usage:"conn user/password@connection-string"
The connection string can be either the name that pre-defined indata\jdbc_url.cfg
, or follow the JDBC thin URL format, and for different database platform, the URL format can be different, normally inhost:port/database-or-else
format. But forOracle
database, it also supportsTNS-NAME
andEZConnect
. Typehelp connect
for more information -
login/logon
:A command set, view or logon to a connection history, when a connection is successfully established, the logon info would be automatially stored indata\password.dat
, so that user doesn't have to manually input the a/c for each time -
disconnect/disc
:Disconnect the current connection| -
Commit
: Commit DML changes, only available when autocommit mode ison
-
Rollback
: Rollback DML changes, only available when autocommit mode ison
-
-
Scripting
-
*
:A command set, run customized script file which includes a batch of commands that supported in DBCLI, supports the features of variables, options and pre-compiled conditions. The engine maps the input parameters asV1 - V20
variables except the parameter in-<option>
format. This is the interface for some other commands to implement or extend, such assnap
/shell
/ora
/sys
/chart
/sql
/etc, different command owns different folder to store the script files, and those file names are regconized as the sub-command names. -
snap
:A command set,used in following scenario:[capture db pre-stats] => [run other commands / sleep for seconds] => [capture db post-stats] => [generate db delta-stats]
-
-
Graphing
-
chart/ch
:A command set, run customized script to generate graph chart from SQL statement -
graph/gr
:Generate graph chart from input parameter which is a SQL text or a CSV file
-
-
Variable
-
define/def
:Define variable with initial value, can be used in either SQL bindings or text substitution in command or script file -
variable/var
:Define variable and its data type without value assignment, can be used as the output binding parameter of the SQL statement, mainly used in customized script file -
accept/acc
:Assign user-input or file to a new variable -
save
:Save a specific variable value into target file -
print/pri
:Print target variable
-
-
Unloading
-
sql2file
:Generate the result of a query or cursor into SQL file, supports auto-compression regarding to the target file extension -
sql2csv
:Generate the result of a query or cursor into CSV file andsqlldr
control file, supports auto-compression regarding to the target file extension -
csv2sql
:Convert CSV file into SQL file andsqlldr
control file, supports auto-compression regarding to the target file extension
-
-
Shell
-
ssh
:A command set to communicate between client and SSH server -
shell
:A command set, run in remote SSH sever with local shell/python/perl script(to be enhanced), this command is inherited from the scripting modules
-
-
Internal
-
event
:Show event bindings of the console code -
luajit
:Switch into theaLuaJIT
console -
ilua
:Enter into Lua console, can access environment objects -
search
:Search the source code -
-p
:Test a command and verify its input parameters -
help -a
orset -a
: see where the commands/settings are defined
-
-
Instruction
-
Oracle Database
-
SQL
-
*
:Supports almost all Oracle database SQL statements
-
-
Report
-
awrdump
:Generate AWR report regarding to the input time range or snap_id -
awrdiff
:Generate AWR diff report regarding to the input time range or snap_id -
addmdump
:Generate ADDM report regarding to the input time range or snap_id -
addmdiff
:Generate ADDM diff report regarding to the input time range or snap_id -
ashdump
:Generate ASH report regarding to the input time range or snap_id
-
-
Extraction
-
loadtrace/dumptrace
:Download the self/alert/other trace file from db server,requires thecreate/drop any directory
access rights if unable to access the target OS path with existing Oracle directories -
sqlprof
:Generate SQL profile script from sqlarea/awr/spm/profile/plan_table/etc -
unwrap
:Extract PL/SQL code or view from database into local file, if the code is encrypted, then auto-decrypt it
-
-
Scripting
-
ora
:A command set that implement the scripting engine, to run predefined or customized script that used for DBA or db developer, this command has pre-defined many useful scripts -
sys
:A command set that inherited fromora
and similar toora
, except can only used by SYSDBA -
show
:A command set that inherited fromora
and similar toora
, except that it doesn't accept input parameters -
exa
: A command set that inherited fromora
, used to queryExadata
cell configurations and statisticses
-
-
SQL*Plus
-
sqlplus
:Launchsqlplus
console with the same login in native mode or embbed mode, plus other input parameters -
@
orsp
:Run sqlplus script in embbed mode with the same login
-
-
Settings
-
set instance
:Auto-limit the instance regarding to any SQL that references thegv$/x$/dba_hist
views -
set container
:Auto-limit the container ID regarding to any SQL that references the views that have thecon_id
field -
set starttime
:Auto-limit the default begin time of the predefined SQLs that specify the time range predicate -
set endtime
:Auto-limit the default end time of the predefined SQLs that specify the time range predicate
-
-
Others
-
describe/desc
:Describe Oracle object, provides more features than that ofsqlplus
-
xplan
:Show execution plan for inputSQL text/ID
, as well as downloading the 10053 trace file if specify the-10053
option
-
-
SQL
-
MySQL Database(to be enhanced)
-
SQL
-
*
:Supports almost all MySQL database SQL statements
-
-
Console
-
mysql
:Switch tomysql.exe
with same login plus other input parameters -
mysqluc
:Switch tomysqluc.exe
with same login plus other input parameters -
source
or\.
orms
:Switch tomysql.exe
with same login to execute the target script
-
-
Scripting
-
sql
:Inherited from the scripting engine, to run the script which contains a batches of all available commands -
ps
:Similar tosql
except that it's based onperformance_schema
-
ti
:Similar tosql
except that it's forTiDB
only -
list
:Similar tosql
except that it accepts no parameters
-
-
Others
-
?
or\?
: A bit different from the native\?
, provides more features -
N/A
: Some commands are similar to the nativemysql
console -
info
: Describes object more detail
-
-
SQL
-
PostgreSQL Database(to be enhanced)
-
SQL
-
*
:Supports almost all PostgreSQL database SQL statements
-
-
Console
-
psql
:Switch topsql.exe
with same login plus other input parameters -
file
or-f
:Switch topsql.exe
with same login and execute the target script
-
-
Scripting
-
sql
:Inherited from the scripting engine, to run scripts which contains a batches of all available command
-
-
Others
-
N/A
: Some commands are similar to the nativepsql
console
-
-
SQL
Command Types
The command rules are similar to SQL*Plus:
- DDL commands that related to creating source code(
create trigger/package/function/procedure/type
) are cross-lines commands and should be end with/
in a stand-alone line.create table/index/etc
are not of this kind because they are related to structures. - Other DML,DDL,DCL commands are also cross-lines commands that end with either
;
, or/
in a new line. - Command
SET COMMAND_ENDMARKS
is used to customize those 2 symbols. - Apart from the above commands, most of other commands are single line commands and will be executed after the
Enter
key is pressed. Typehelp -a
and refer to theCross-lines
field for the type of each command.
Line Wrap
For the output lines that wider than the buffer width(in Posix System it's the screen width), the overflow part will be trimmed. To view the complete information:
- use
<command> | less
to enter Linux-styleless
mode, and pressarrow
keys to scroll the screen - use
less last
to view the last output inless
mode - run command
out
to launch the default editor to see the historical output. If the default editor isvi
, execute:set nowrap
and use hotkeye/b/zL/zH
to horizontally scroll the screen. To change the default editor, useset [-p] editor <executable name>
. For example,set editor code
Color Solutions
dbcli
has defined some colors that used in different scenario, those colors can be found by set color
.
You are able to change the colors by executing set [-p] <color name> <new code>
, of which the code can be referenced by command ansi
, and -p
means permanently.
For example, set promptcolor BLU;BWHT;UDL
is to change the prompt color as Blue Foreground + White Background + Underlined
, and set promptcolor default
to change as default.
It's possible to turn of using colors, which is done by executing set ansicolor off
Releases
There are 5 zip files in the releases
page that used in different scenario, all those releases don't include the src
directory:
-
dbcli_all.zip
: includes everything -
dbcli_win.zip
: excludes thejre_linux
directory and includeJRE 1.8
forWindows
-
dbcli_linux.zip
: excludes thejre
directory and includeJRE 1.8
forLinux
-
dbcli_nojre.zip
: excludes thejre/jre_linux
directories -
dbcli_oracle_lite.zip
: excludes thejre/jre_linux/bin/docs
directories, you can remove more sub-directories underlib
depending on your OS.
References
The utility depends on some binary files of open-source projects, the authors own the rights. The binaries have been included into this project. Below is the list:
JLine3 : (BSD) https://github.com/jline/jline3
JSch : (BSD) http://www.jcraft.com/jsch/
ConsoleZ : (GNU) https://github.com/cbucher/console
luajit v2.1 : (MIT) https://github.com/LuaJIT/LuaJIT
jnlua : (MIT) https://github.com/hyee/JNLuaJIT(revised from jnlua.googlecode.com)
OpenCSV : (MIT) https://github.com/hyee/OpenCSV(revised from opencsv.sourceforge.net)
MessagePack : (MIT) https://github.com/fperrad/lua-MessagePack (pure lua)
dygraphs : (MIT) https://github.com/danvk/dygraphs
JsonForLua : (MIT) https://github.com/craigmj/json4lua
PSCP : (MIT) http://www.putty.org/
LPeg : (MIT) http://www.inf.puc-rio.br/~roberto/lpeg
xml2lua : (MIT) https://github.com/manoelcampos/xml2lua
JNA : (LGPL) https://github.com/java-native-access/jna
NuProcess : (Apache) https://github.com/brettwooldridge/NuProcess
Luv(libuv) : (Apache) https://github.com/luvit/luv
ConEmu : (BSD) https://github.com/Maximus5/ConEmu
juniversalchardet : https://github.com/albfernandez/juniversalchardet
Besides, some SQL scripts come from internet.
Customize Commands
Customize new simple commands
You are able to use command alias
to define the shortcut of the simple command, for instance:
alias sf select * from (select * from $*) where rownum<=50;
In this case, you can execute sf dba_objects where object_id<1000
to simplify the input.
The alias
command supports the $1-$9
and $*
wildcard characters, of which $n
corresponds to the n
th parameter, and $*
means the concatenation of $n+1
-$9
via space.
Type alias
to see more usage.
Customize new sub-commands
Similar to SQL*Plus script, a sub-command script is the combination of one or more existing commands that supports user-input parameters.
Take command ora
for example, to define a sub-command xxx
, create file oracle\ora\xxx.sql
and fill with following content:
select * from (select * from &V1) where rownum<=50;
After that, run ora -r
to take effect, then you can run ora xxx dba_objects
to query the view.
The utility has created some pre-defined commands, if you want to modify the those commands without concerning of overriding back by the new version, just create a sub-folder under the ora
directory, and put the updated file into it, because for the scripts with same name, the one in the sub directory will be treated as higher priority. Or you may also use ora -l <path>
to link to another work dir.
Commands ora/show/sys/snap/chart/sql/shell/etc
follow the similar rules:
- Parameters: Accept
:V1-:V20
or&V1-&V20
as the input parameters, of which:Vn
means binding parameters, and&Vn
means variable substitution. - Help comment:
/*[[...]]*/
is optional, as the help or usage information - Options:
--[[...]]--
inside the help comment is also optional, normally used to specify the command options(i.e.,ora actives -m
) and access validation, refer to other sub-commands for more examples.
Customize new root commands
Different from sub-commands, the root command must be a lua script. To plug a user-define lua script into the utility, just add its path in data\plugin.cfg
, you may refer to data\plugin_sample.cfg
for the example.
Below are the common interfaces that can be used in the script:
- Define a new command:
env.set_command(...)
- Define a new setting:
env.set.init_cfg(...)
- Add trigger to an event:
env.snoop('<event_name>', <parameters>)
- Get current database:
env.getdb()
You may also:
- Execute
help -a
orset -a
orevent
to see how those interfaces are used in other scripts - Execute
help -e env[.module] 2
to see the available interfaces, i.e.:help -e env.oracle 2