puppet-wildfly
puppet-wildfly copied to clipboard
voxpupuli
503 Service Unavailable
The code designed to retry api connections if Wildfly hasn't started up yet isn't working for me.
Error: /Stage[main]/Profile::Keycloak/Wildfly::Resource[/socket-binding-group=standard-sockets/socket-binding=proxy-https]/Wildfly_resource[/socket-binding-group=standard-sockets/socket-binding=proxy-https]: Could not evaluate: 757: unexpected token at '<html><head><title>Error</title></head><body>503 - Service Unavailable</body></html>'
/opt/puppetlabs/puppet/lib/ruby/2.1.0/json/common.rb:155:in `parse'
/opt/puppetlabs/puppet/lib/ruby/2.1.0/json/common.rb:155:in `parse'
/opt/puppetlabs/puppet/cache/lib/puppet_x/wildfly/api_client.rb:58:in `submit'
/opt/puppetlabs/puppet/cache/lib/puppet_x/wildfly/operation_request.rb:40:in `exists?'
/opt/puppetlabs/puppet/cache/lib/puppet/provider/wildfly_resource/http_api.rb:18:in `exists?'
It appears the @http_client.request authz_request get in authz_header works fine, but the post in submit gets a 503. The service is 'partially' started??
Furthermore, the 503 isn't returned as json so I get the unexpected token exception instead of the "Failed with: #{response['failure-description']} for #{body.to_json}" one.
I'm using the keycloak distribution on CentOS 7 with puppet 4.10.3. I think you should be able to reproduce the issue with the following and puppet apply.
class { '::wildfly':
version => '10.1.0',
distribution => 'wildfly',
config => 'standalone-ha.xml',
install_cache_dir => '/tmp',
install_source => 'https://downloads.jboss.org/keycloak/3.1.0.Final/keycloak-3.1.0.Final.tar.gz',
mgmt_user => {
username => 'admin',
password => 'admin',
},
}
wildfly::resource { '/socket-binding-group=standard-sockets/socket-binding=proxy-https':
content => {
port => 443,
},
notify => Wildfly::Reload['Reload if necessary'],
}
wildfly::reload { 'Reload if necessary':
retries => 2,
wait => 15,
}
Many thanks, Alex
I have seen this as well using the standard wildfly distribution, so it's not related to keycloak. Could perhaps be solved by checking if response is an instance of Net::HTTPSuccess, and if not trigger a retry?
Just confirmed this. Don't even need wildfly::reload to reproduce it.
It's not happening during retry - and the retried request needs to return 401 (Net::HTTPUnauthorized) with data to perform digest authentication.
This is happening afterwards during the actual Management Request - as far as I understood, Management API is ready to process authentication, but Wildfly is not ready to receive any management operations, hence the 503.
Sysvinit scripts used to work by checking Wildfly's logs for JBAS015874, but they replaced it with a markerfile approach. Bundled systemd scripts does not use any of this.
I need another way to assert that Wildfly is ready cause I can't retry actual management requests since some of them are not idempotent. It was OK to retry authentication.
One alternative would be to not use bundled systemd script but systemd/sysvinit wrapper (like Wildfly < 10: https://github.com/biemond/biemond-wildfly/blob/master/templates/wildfly.sysvinit.service.epp) or create a new script.
