Decouple the current billing index from best block-number

[sameh-farouk]

Describe the bug

Currently, all network validators are required to have offchain worker enabled. This is because we depend on the best block number in retrieval contracts ids from the billing shards. Skipping subsets of contracts in the billing cycle can occur in cases such as:

• Offchain worker is not enabled on one or more validators
• Offchain workers are not triggered on a block import (this is on substrate, it is not guaranteed to trigger the hook every block).
• Local key store has multiple signing/aura keys (some validators were incorrectly set up).

My suggestion is to store the current index in the smart contract pallet storage and increment it upon every successful billing cycle.

By decoupling the current billing index from best block-number we would ensure that all contracts billing are handled as long as there is at least one validator have offcahin worker enabled on the network and configured correctly .

if this implemented, The only side effect for less offchain workers would be prelonged cycle (less workers = more time) but this would be better than no billing done at all.

This is primarily an improvement to enable better error tolerance, but we can also plan and proceed in parallel with a more comprehensive redesign that involve zos nodes, gateways, and other components responsible for triggering the billing instead of the offchain workers if needed.

[sameh-farouk]

I want to add that the motivation here beside better operation error tolerance, that I don't want to make a critical feature (contracts billing) depends on one option passed to the tfchain binary specially if we still plan to launch the validators program. As alternative maybe we could enforce enabling the offchain worker from the code.

[sameh-farouk]

This can be done using the approach suggested here https://github.com/threefoldtech/tfchain/issues/932#issuecomment-1954991279

[sameh-farouk]

Closing this now as the concerns here were handled as part of https://github.com/threefoldtech/tfchain/issues/932