DevOps tools

A curated list of development and operational tools for Unix-like operating systems.

All tools are Open Source / Free Software.

Contents

• Backup
• Databases
• Data filtering and transformations
• Desktop
• Dev stubbing
• Development aids
• Documentation
• Filesystems
• Linting
• Networking
• Monitoring
• Security
• Source control
• Other lists

Backup

• borgbackup: BorgBackup is a deduplicating backup program. Optionally, it supports compression and authenticated encryption.
• borgmatic: Simple, configuration-driven backup software for servers and workstations. A user-friendly wrapper around borgbackup.
• restic: Fast, encrypted remote incremental backups that works without a remote server in a chrooted remote SFTP. Single binary written in Go. Warning: Pruning old backups is very slow.

Databases

• squirrelsql: Multi-database graphical SQL client. Supports anything JDBC supports (MySQL, Postgres, MSSQL, etc).
• dbeaver: Free universal database manager and SQL client.
• apache directory studio: A complete GUI LDAP management tool.

Data filtering and transformations

• xmlstarlet: Commandline tools to work with XML. Validate (DTD, XSD), XPath, XSLT, modify and visualize XML documents.
• xlstproc: Process XSLT on the commandline.
• xmllint: parses one or more XML files, specified on the command line as xmlfile. It prints various types of output, depending upon the options selected. It is useful for detecting errors both in XML code and in the XML parser itself.
• jq: a lightweight and flexible command-line JSON processor.
• meld: a visual diff and merge tool targeted at developers. Meld helps you compare files, directories, and version controlled projects. It provides two- and three-way comparison of both files and directories, and has support for many popular version control systems.
• expect: Systematically interact with programs that can't be scripted but require human interaction.
• fop: XSL-driven documentation generator. Basically takes XML and XSL as input and generates PDF, PS, etc.
• dhex: Console / curses-based hex editor with diffing mode.

Desktop

• tilix: A tiling terminal emulator for Linux.
• zim: Powerful desktop wiki / note taking application.

Dev stubbing

• fakesmtp: a Fake SMTP Server with GUI for testing emails in applications easily.
• mailslurper: a Fake SMTP server with a web interface for testing emails without actually sending them. Simple single-package installation.
• mailhog: a Fake SMTP server with a web interface for testing emails without actually sending them. Simple single-package installation.
• tcconfig: Easy to set up traffic control of network bandwidth/latency/packet loss/packet-corruption to a network interface.

Development aids

• soapui: A SOAP and REST testing tool.
• skajla: Command-line JMX client.
• insomnia: a cross-platform REST client, built on top of Electron.
• virtualbox: Powerful x86 and AMD64/Intel64 virtualization lets you run full-fledged Windows, Linux, Solaris and BSD Virtual machines on your computer.
• vagrant: Quickly bring up Virtual Machines for development and testing using a single declarative configuration file.
• lxdock: Quickly bring up lightweight containers for development and testing.
• when-changed: Automatically run commands when files change on disk.
• entr: Automatically run commands when files change on disk.

Documentation

• pandoc: convert files from one markup format into another. Supports a large number of formats.
• asciidoc: a text document format for writing notes, documentation, articles, books, ebooks, slideshows, web pages, man pages and blogs. AsciiDoc files can be translated to many formats including HTML, PDF, EPUB, man page.
• mkdocs: a fast, simple and downright gorgeous static site generator that's geared towards building project documentation. Documentation source files are written in Markdown, and configured with a single YAML configuration file.
• yed: cross-platform diagram editor.
• plantuml: Generate UML diagrams from a simple text-based description.
• graphviz: Use a simple text language to generate diagrams of abstract graphs and networks.
• wkhtmltopdf: Convert HTML to PDF.

Filesystems

• archivemount: Mount tar archives, including write-support. (changes are written when archive is unmounted).
• mhddfs: join several filesystems together to form a single larger one.
• sshfs: allows you to mount a remote filesystem using SFTP. Most SSH servers support and enable this SFTP access by default, so SSHFS is very simple to use - there's nothing to do on the server-side.
• squashfs: Create and mount compressed filesystem images.
• xdiskusage: Visually represent disk usage in and below a directory.
• rsync: Provides fast, reliable, configurable incremental file transfer on local disk or over the network. Archive, mirror, etc.
• ncdu: A disk usage analyzer with an ncurses interface. It is designed to find space hogs on a remote server where you don't have an entire graphical setup available.
• fdupes: FDUPES is a program for identifying or deleting duplicate files residing within specified directories.

Linting

• httpolice: a validator for HTTP requests and responses. It can spot bad header syntax, inappropriate status codes, and other interoperability problems in your HTTP server or client.
• shellcheck: a tool that gives warnings and suggestions for bash/sh shell scripts.

Networking

• mtr: My TraceRoute. combines the functionality of the 'traceroute' and 'ping' programs in a single network diagnostic tool.
• lft: display the route packets take to a network host/socket using one of several layer-4 protocols and methods. Basically traceroute for TCP, UDP and ICMP.
• netcat: a computer networking utility for reading from and writing to network connections using TCP or UDP. Replacement for telnet. Can also act as a server.
• sshuttle: Sshutle is VPN over SSH without requiring a remote VPN server or admin rights. Instead, it builds up an ssh session and than locally forwards traffic over it by creating local PREROUTING firewall rules.
• stunnel: a proxy designed to add TLS encryption functionality to existing clients and servers without any changes in the programs' code.
• tcptraceroute: A traceroute implementation using TCP packets
• wavemon: an ncurses-based monitoring application for wireless network devices.
• rinetd: Redirects TCP connections from one IP address and port to another.
• dig: The multitool for DNS enquiries.
• ipcalc: IP network calculator. Available as online tool as well as a downloadable package. (Ubuntu users: apt install ipcalc).
• netalyzr: Commandline tool that runs various network related tests and generates a report for potential problems. (careful: sends the results to Berkeley for research purposes).
• mitmproxy: An interactive console program that allows HTTP traffic flows to be intercepted, inspected, modified and replayed.
• iftop: iftop does for network usage what top(1) does for CPU usage. It listens to network traffic on a named interface and displays a table of current bandwidth usage by pairs of hosts. Handy for answering the question "why is our ADSL link so slow?".
• tinc: A auto-routing meshing VPN.
• proxychains: a tool that forces any TCP connection made by any given application to follow through proxy like TOR or any other SOCKS4, SOCKS5 or HTTP(S) proxy.

Monitoring

• monit: a small utility for managing and monitoring Unix systems. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.
• monitorix: Lightweight monitoring tool that collects and displays many system usage graphs through a web interface.
• iotop: Show disk and swap I/O per process.
• logtail: Watch multiple log files on multiple machines.
• lnav: An advanced log file viewer for the small-scale. Watch and analyze your log files from a terminal. No server. No setup. Still featureful.
• sentry: Open-source error tracking that helps developers monitor and fix crashes in real time.
• ts: Add timestamps to piped output (stdout). E.g. rsync x y | ts '[%Y-%m-%d %H:%M:%S]'.
• collectd: Collectd is a daemon which collects system and application performance metrics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. Has many plugins and many frontends are available for viewing the data.
• osquery: Use SQL to query information from your device / system.

Security

• fail2ban: Fail2ban scans log files and bans IPs that show malicious signs. Easily protects your SSH ports against attacks.
• ferm: a tool to maintain complex firewalls, without having the trouble to rewrite the complex rules over and over again.
• lynus: Lynis is a battle-tested security tool for systems running Linux, macOS, or Unix-based operating system. It performs an extensive health scan of your systems to support system hardening and compliance testing.
• pwgen: Generate pronouncable and easy to type passwords.
• keystore explorer: Java GUI for managing Java Key stores (SSL certificate databases). Also lets you convert all kinds of certificate formats.
• testssl.sh: a free command line tool which checks a server's service on any port for the support of TLS/SSL ciphers, protocols as well as recent cryptographic flaws and more.
• qualys ssl server test: Online tool for deep analysis of the configuration of any SSL web server on the public Internet. Commandline also available.
• mozilla ssl configuration generator: Generate secure SSL configurations for the most common browsers.
• mozilla observatory: Scan hosts for HTTPS, TLS and SSH to see if they are configured safely and securely. Warns about missing and improperly configured Content Security Policies for websites.
• zed attack proxy: A java desktop gui to perform PEN tests against web applications.
• simple event correlator: an event correlation tool for advanced event processing which can be harnessed for event log monitoring, for network and security management, for fraud detection, and for any other task which involves event correlation.
• age: age is a simple, modern and secure file encryption tool, format, and library. It features small explicit keys, no config options, and UNIX-style composability. Does symmetric and asymmetric encryption (public / private keys). Way simpler than GPG.

Source control

• tig: an ncurses-based text-mode interface for git.
• git cola: a sleek and powerful graphical user interface for Git.
• multi-git-status: Show uncommitted, untracked and unpushed changes for multiple Git repos.

Other lists

• awesome-sysadmin: A curated list of amazingly awesome open source sysadmin resources inspired by Awesome PHP.