Hackerbeat
Hackerbeat copied to clipboard
An Elastic Beats for indexing HackerNews posts
Hackerbeat
Hackerbeat indexes the top stories of HackerNews into an ElasticSearch instance.
Ensure that this folder is at the following location:
${GOPATH}/src/github.com/Ullaakut/hackerbeat
Getting Started with Hackerbeat
Requirements
- Golang 1.7
Build
To build the binary for Hackerbeat run the command below. This will generate a binary in the same directory with the name hackerbeat.
make
Run
To run Hackerbeat with debugging output enabled, run:
./hackerbeat -c hackerbeat.yml -e -d "*"
Test
To test Hackerbeat, run the following command:
make testsuite
alternatively:
make unit-tests
make system-tests
make integration-tests
make coverage-report
The test coverage is reported in the folder ./build/coverage/
Update
Each beat has a template for the mapping in elasticsearch and a documentation for the fields
which is automatically generated based on fields.yml
by running the following command.
make update
Cleanup
To clean up the build directory and generated artifacts, run:
make clean
Clone
To clone Hackerbeat from the git repository, run the following commands:
mkdir -p ${GOPATH}/src/github.com/Ullaakut/hackerbeat
git clone https://github.com/Ullaakut/hackerbeat ${GOPATH}/src/github.com/Ullaakut/hackerbeat
For further development, check out the beat developer guide.
Packaging
The beat frameworks provides tools to crosscompile and package your beat for different platforms. This requires docker and vendoring as described above. To build packages of your beat, run the following command:
make package
This will fetch and create all images required for the build process. The hole process to finish can take several minutes.