aws-kinesis-agent fails to start in Ubuntu 18.04.3 LTS

[vikramaditya234]

I installed the aws-kinesis-agent using the script below:

git clone https://github.com/awslabs/amazon-kinesis-agent.git cd amazon-kinesis-agent/ update-java-alternatives --jre-headless --jre --set java-1.8.0-openjdk-amd64 ./setup --install update-java-alternatives --jre-headless --jre --set java-1.11.0-openjdk-amd64

The configuration JSON is:

{ "kinesis.endpoint": "https://kinesis.eu-west-1.amazonaws.com", "flows": [ { "filePattern": "/var/log/binu/delivery.log", "kinesisStream": "a-valid-kinesis-stream" } ] }

Started the agent by: service aws-kinesis-agent start

But when I check the log: /var/log/aws-kinesis-agent/aws-kinesis-agent.log, I get the error:

2019-11-04 05:58:43.475+0000 (main) com.amazon.kinesis.streaming.agent.Agent [INFO] Reading configuration from file: /etc/aws-kinesis/agent.json 2019-11-04 05:58:43.478+0000 (main) com.amazon.kinesis.streaming.agent.Agent [INFO] Found 1 configured flow(s) 2019-11-04 05:58:43.512+0000 (main) com.amazon.kinesis.streaming.agent.Agent [INFO] null: Agent will use up to 24 threads for sending data. 2019-11-04 05:58:43.526+0000 (main) com.amazon.kinesis.streaming.agent.Agent [ERROR] Unhandled error. java.lang.RuntimeException: Failed to create or connect to the checkpoint database. at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.connect(SQLiteFileCheckpointStore.java:108) at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.(SQLiteFileCheckpointStore.java:69) at com.amazon.kinesis.streaming.agent.Agent.(Agent.java:190) at com.amazon.kinesis.streaming.agent.Agent.main(Agent.java:96) Caused by: java.nio.file.AccessDeniedException: /var/run/aws-kinesis-agent at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) at java.nio.file.Files.createDirectory(Files.java:674) at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) at java.nio.file.Files.createDirectories(Files.java:767) at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.connect(SQLiteFileCheckpointStore.java:102) ... 3 more

There is no such file /var/run/aws-kinesis-agent, hence following instruction from https://github.com/awslabs/amazon-kinesis-agent/issues/39 I created the file /var/aws-kinesis-agent/checkpoints and changed agent.json to:

{ "kinesis.endpoint": "https://kinesis.eu-west-1.amazonaws.com", "checkpointFile": "/var/run/aws-kinesis-agent", "flows": [ { "filePattern": "/var/log/binu/delivery.log", "kinesisStream": "a-valid-kinesis-stream" } ] }

On restarting the service I am getting, this error:

2019-11-04 06:04:13.503+0000 (main) com.amazon.kinesis.streaming.agent.Agent [INFO] Reading configuration from file: /etc/aws-kinesis/agent.json 2019-11-04 06:04:13.506+0000 (main) com.amazon.kinesis.streaming.agent.Agent [INFO] Found 1 configured flow(s) 2019-11-04 06:04:13.545+0000 (main) com.amazon.kinesis.streaming.agent.Agent [INFO] null: Agent will use up to 24 threads for sending data. 2019-11-04 06:04:13.696+0000 (main) com.amazon.kinesis.streaming.agent.Agent [ERROR] Unhandled error. java.lang.RuntimeException: Failed to configure the checkpoint database. at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.connect(SQLiteFileCheckpointStore.java:123) at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.(SQLiteFileCheckpointStore.java:69) at com.amazon.kinesis.streaming.agent.Agent.(Agent.java:190) at com.amazon.kinesis.streaming.agent.Agent.main(Agent.java:96) Caused by: org.sqlite.SQLiteException: [SQLITE_READONLY] Attempt to write a readonly database (attempt to write a readonly database) at org.sqlite.core.DB.newSQLException(DB.java:909) at org.sqlite.core.DB.newSQLException(DB.java:921) at org.sqlite.core.DB.throwex(DB.java:886) at org.sqlite.core.NativeDB._exec_utf8(Native Method) at org.sqlite.core.NativeDB._exec(NativeDB.java:87) at org.sqlite.jdbc3.JDBC3Statement.executeUpdate(JDBC3Statement.java:116) at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.connect(SQLiteFileCheckpointStore.java:112) ... 3 more `

Can anyone please help in fixing this?

[apogiatzis]

This is a permissions issue. Try adding a different checkpoint path in your agent config like this: "checkpointFile": "/opt/aws-kinesis-agent/run/checkpoints", (create the directoris up to run if they don't exist) and change the permissions:

chmod ugo+rwx /opt/aws-kinesis-agent/run