500 error on getNextInvoiceTopLevelBillingItems for large accounts

[cbreden]

I have a repeatable failure for SoftLayer accounts with large numbers of instances.

require 'rubygems'
require 'softlayer_api'
require 'pp'

client = SoftLayer::Client.new(
    :username => ENV["SL_API_USER"],  ## API Username
    :api_key => ENV["SL_API_KEY"],    ## API Key
    :timeout => 120
)

BILLING_ITEM = "mask[
                  id,
                  description,
                  categoryCode,
                  category,
                  createDate,
                  cancellationDate,
                  lastBillDate,
                  nextBillDate,
                  orderItemId,
                  parentId,
                  recurringMonths,
                  serviceProviderId,
                  invoiceItem[hostName,domainName,resourceTableId,notes],
                  item,
                  location[longName],
                  nextInvoiceTotalOneTimeAmount,
                  nextInvoiceTotalOneTimeTaxAmount,
                  nextInvoiceTotalRecurringAmount,
                  nextInvoiceTotalRecurringTaxAmount,
                  bundledItems,
                  currentHourlyCharge,
                  hourlyRecurringFee,
                  hoursUsed,
                  orderItem[order[userRecord]]
               ]"

acct = client.service_named("SoftLayer_Account")
items = acct.object_mask(BILLING_ITEM).getNextInvoiceTopLevelBillingItems

unless items.kind_of?(Array)
  items = [items]
end

pp items.empty? ? "No results" : items.first

On small accounts (say like 50 devices), this will return just fine. But on large accounts (1000+ devices), this will fail with a HTTP 500 error after about 40 seconds (well before the 120 second timeout). Bringing the timeout down to low values does cause it to abort with a predictable timeout error, so it seems like there's something happening on the SL side.

Other queries like fetching virtual guests with deep object masks will respect the long timeouts past 40 seconds, so this seems like it may be unique to the billing layer you have gating this data.

...unless of course I'm missing something obvious. :)

[cbreden]

For reference:

Wed Jan 20 13:56:42 PST 2016
/Users/cbreden/.rvm/rubies/ruby-1.9.3-p551/lib/ruby/1.9.1/xmlrpc/client.rb:549:in `do_rpc': HTTP-Error: 500 Internal Server Error (RuntimeError)
    from /Users/cbreden/.rvm/rubies/ruby-1.9.3-p551/lib/ruby/1.9.1/xmlrpc/client.rb:420:in `call2'
    from /Users/cbreden/.rvm/rubies/ruby-1.9.3-p551/lib/ruby/1.9.1/xmlrpc/client.rb:410:in `call'
    from /Users/cbreden/.rvm/gems/ruby-1.9.3-p551/gems/softlayer_api-3.0.2/lib/softlayer/Service.rb:268:in `call_softlayer_api_with_params'
    from /Users/cbreden/.rvm/gems/ruby-1.9.3-p551/gems/softlayer_api-3.0.2/lib/softlayer/APIParameterFilter.rb:194:in `method_missing'
    from bill_test.rb:39:in `<main>'
Wed Jan 20 13:57:16 PST 2016

[renier]

Have you tried limiting the result set to get it in chunks?