maxbuffer size exceeded error

[mayusuk]

hi, i am using git2consul in docker and ,it working fine but for bulk data push it throws error maxbuffer size exceeded, any suggestions

[calvn]

Can you include the actual logs for more information?

[mayusuk]

these are the logs i get

{"name":"git2consul","hostname":"git-2-consul","pid":6,"level":30,"msg":"Adding /etc/git2consul.d/config.json to KV git2consul/config as: \n{\n\"version\": \"1.0\",\n \"repos\": [{\n \"name\": \"localhiiiost\",\n,\n \"include_branch_name\": false,\n \"ignore_file_extension\": false,\n \"branches\": [\"master\"],\n \"hooks\": [{\n \"type\": \"polling\",\n \"interval\": \"1\"\n }]\n }]\n}\n","time":"2016-08-11T08:24:17.991Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":6,"level":30,"msg":"git2consul is running","time":"2016-08-11T08:24:18.538Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":6,"level":30,"msg":"Initting repo localhiiiost","time":"2016-08-11T08:24:18.546Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":6,"level":30,"msg":"Initting branch /tmp/localhiiiost /tmp/localhiiiost/master","time":"2016-08-11T08:24:18.547Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":6,"level":40,"msg":"Purging branch cache /tmp/localhiiiost/master for branch master in repo localhiiiost","time":"2016-08-11T08:24:18.548Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":6,"level":30,"msg":"Initialized branch master from localhiiiost","time":"2016-08-11T08:24:32.035Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":6,"level":30,"msg":"Loaded repo localhiiiost","time":"2016-08-11T08:24:32.037Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":6,"level":40,"msg":"Failed to handle pull due to Error: Error: stdout maxBuffer exceeded. Total 258 (delta 66), reused 249 (delta 66) branch master -> FETCH_HEAD\n 3e50106..6aeb332 master -> origin/master, clearing cache and retrying","time":"2016-08-11T08:27:54.005Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":6,"level":40,"msg":"Purging branch cache /tmp/localhiiiost/master for branch master in repo localhiiiost","time":"2016-08-11T08:27:54.005Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":6,"level":50,"err":{"message":"Error: stdout maxBuffer exceeded. ","name":"Error","stack":"Error: Error: stdout maxBuffer exceeded. \n at /usr/lib/node_modules/git2consul/lib/git/commands.js:20:17\n at ChildProcess.exithandler (child_process.js:758:5)\n at ChildProcess.emit (events.js:110:17)\n at maybeClose (child_process.js:1016:16)\n at Process.ChildProcess._handle.onexit (child_process.js:1088:5)"},"msg":"Error: stdout maxBuffer exceeded. ","time":"2016-08-11T08:27:59.312Z","v":0}

[calvn]

This is probably due to the default maxBuffer being exceeded. It should solved by explicitly bumping up the value, specifically in command.js.

exec should be changed to the following:

exec(cmd, {cwd: cwd, maxBuffer: 500 * 1024}, function(err, stdout, stderr) {
    ...
}

Can you try that locally and see if it fixes the problem?

[mayusuk]

yeah I will try that and most probably that will solve the problem

[mayusuk]

thanks, it actually worked, can i increase more size because still sometimes it throws error.

[mayusuk]

Hi , I found another error , i deleted all my repo and i waited to git2consul to poll and i got this error

{"name":"git2consul","hostname":"git-2-consul","pid":7,"level":30,"msg":"Initialized branch master from git2consul","time":"2016-08-13T18:18:15.557Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":30,"msg":"Loaded repo git2consul","time":"2016-08-13T18:18:15.560Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":40,"msg":"Failed to handle pull due to Error: Error: stdout maxBuffer exceeded. Total 2 (delta 0), reused 2 (delta 0)\nFrom * branch master -> FETCH_HEAD\n e40c80b..1157666 master -> origin/master, clearing cache and retrying","time":"2016-08-13T18:19:34.881Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":40,"msg":"Purging branch cache /tmp/git2consul/master for branch master in repo git2consul","time":"2016-08-13T18:19:34.883Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":50,"msg":"Some consul updates failed:\nFailed to read key /tmp/git2consul/master/ due to Error: EISDIR, read","time":"2016-08-13T18:19:42.704Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":40,"msg":"Failed to handle pull due to Some consul updates failed:\nFailed to read key /tmp/git2consul/master/ due to Error: EISDIR, read, clearing cache and retrying","time":"2016-08-13T18:20:34.421Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":40,"msg":"Purging branch cache /tmp/git2consul/master for branch master in repo git2consul","time":"2016-08-13T18:20:34.422Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":50,"msg":"Some consul updates failed:\nFailed to read key /tmp/git2consul/master/ due to Error: EISDIR, read","time":"2016-08-13T18:20:41.344Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":40,"msg":"Failed to handle pull due to Some consul updates failed:\nFailed to read key /tmp/git2consul/master/ due to Error: EISDIR, read, clearing cache and retrying","time":"2016-08-13T18:21:34.571Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":40,"msg":"Purging branch cache /tmp/git2consul/master for branch master in repo git2consul","time":"2016-08-13T18:21:34.571Z","v":0} {"name":"git2consul","hostname":"git-2-consul","pid":7,"level":50,"msg":"Some consul updates failed:\nFailed to read key /tmp/git2consul/master/ due to Error: EISDIR, read","time":"2016-08-13T18:21:42.183Z","v":0}

in idea??

[calvn]

According to this Stackoverflow post, ESDIR is caused by a mismatch in expected and actual file type. In your case, it's triggering on /tmp/git2consul/master/

[mayusuk]

I guess that is a problem when we delete whole repo

[margueritepd]

Would you accept a pull request to increase maxbuffer?