Hackerbeat icon indicating copy to clipboard operation
Hackerbeat copied to clipboard

An Elastic Beats for indexing HackerNews posts

Hackerbeat

Hackerbeat indexes the top stories of HackerNews into an ElasticSearch instance.

Ensure that this folder is at the following location: ${GOPATH}/src/github.com/Ullaakut/hackerbeat

Getting Started with Hackerbeat

Requirements

Build

To build the binary for Hackerbeat run the command below. This will generate a binary in the same directory with the name hackerbeat.

make

Run

To run Hackerbeat with debugging output enabled, run:

./hackerbeat -c hackerbeat.yml -e -d "*"

Test

To test Hackerbeat, run the following command:

make testsuite

alternatively:

make unit-tests
make system-tests
make integration-tests
make coverage-report

The test coverage is reported in the folder ./build/coverage/

Update

Each beat has a template for the mapping in elasticsearch and a documentation for the fields which is automatically generated based on fields.yml by running the following command.

make update

Cleanup

To clean up the build directory and generated artifacts, run:

make clean

Clone

To clone Hackerbeat from the git repository, run the following commands:

mkdir -p ${GOPATH}/src/github.com/Ullaakut/hackerbeat
git clone https://github.com/Ullaakut/hackerbeat ${GOPATH}/src/github.com/Ullaakut/hackerbeat

For further development, check out the beat developer guide.

Packaging

The beat frameworks provides tools to crosscompile and package your beat for different platforms. This requires docker and vendoring as described above. To build packages of your beat, run the following command:

make package

This will fetch and create all images required for the build process. The hole process to finish can take several minutes.