PowerDNS
Remote backend fails during 'list' call with (not that) large zone
- Program: Authoritative
- Issue type: Bug report
Short description
Trying to transfer a zone using the remote backend with a Unix connector with a zone that "isn't that big" (~1722095 bytes < 1.7 MiB) fails. Transferring significantly more data over the HTTP connector works.
Environment
- Operating system: Ubuntu 14.04.5 LTS (Trusty)
- Software version: PowerDNS auth 4.0.3
- Software source: PowerDNS repository
Steps to reproduce
A bit more about how I have this set-up, I have PowerDNS auth talking to a Unix socket, I have a listener on the other end that responds to the API requests and issues JSON responses. This all works fine when I have a smaller zone (~1291275 bytes < 1.3 MiB).
I can't provide the exact data I'm using, no specific Lua scripts. I can try to reproduce with fake data if that will be useful.
Expected behaviour
I would have expected the zone to transfer!
Actual behaviour
The transfer fails with Backend signals error condition in the logs.
Other information
I took a look at some of the code and had an IRC conversation, copied below:
3:38:12 PM - clokep: Hi! I'm messing around with the remote backend using a unix connector. I'm running into an issue doing an AXFR where it seems to just drop my connection when I have an OK amount of data. The same amount of data seems to work OK when using the HTTP connector. Is there some maximum size of data I should expect to be able to transfer in this situation? 3:38:27 PM - Habbie: clokep, smaller zones do work? 3:38:33 PM - clokep: Habbie: Yep. 3:38:46 PM - Habbie: 'interesting' 3:39:02 PM - clokep: Of course it's possible that the bigger zone has some edge-case that my code doesn't handle...but I'm not seeing any exceptions or anything. 3:39:41 PM - Habbie: cmouse, you here? :) 3:40:05 PM - clokep: The zone is ~4.9 MiB of data when converted to JSON... 3:40:16 PM - Habbie: not small 3:40:19 PM - clokep: Haven't tried to figure out at what point it breaks. 3:40:25 PM - Habbie: uhuh 3:40:26 PM - clokep: But that was my next plan. 3:40:33 PM - Habbie: can't hurt 3:40:47 PM - Habbie: and/or cut your zone in half and see if either half fails 3:40:50 PM - Habbie: in case it's not about size 3:41:29 PM - clokep: Ah, bisect the data? Good idea! 3:41:37 PM - Habbie: yes 3:41:44 PM - Habbie: if you're feeling really fancy, use ddmin 3:41:45 PM - sarnold: :) 3:44:40 PM - clokep: Using 1/4 of the zone seemed to work, but it was super slow. :-
3:45:38 PM - clokep: Looks like the data gets read into a std::string, then appended onto another std::string -- I wonder if that causes a lot of allocs. 3:46:30 PM - Habbie: oh that probably hurts yes 3:46:57 PM - clokep: Looking at https://github.com/PowerDNS/pdns/blob/b7eeb5059d756f93a8858f5234800e3c7459b382/modules/remotebackend/unixconnector.cc#L60-L121 3:48:10 PM - Habbie: eck 3:49:15 PM - clokep: Does that mean I should file an issue? :P 3:49:25 PM - Habbie: yes 3:55:07 PM - HM: in the very least move 'temp' outside the loop 3:55:13 PM - HM: that way it'll get reused 3:55:46 PM - HM: worst case is then 2n memory
@Habbie I can try to test it, I'll need to dust off some code. Is there any easy way to install a version of PowerDNS from the repo that has #5308 in it?
@clokep as it was merged to master, just pick the master repo from repo.powerdns.com
@clokep @Habbie is this assigned to some one. Can I try this? This is first bug for me, but sounds interesting.
This particular issue was likely resolved. If you see a similar issue, open a new ticket with all the details: version, platform, logs, etc.
