Address Inaccuracies In Usage Statistics Counts

[aaronskiba]

Summary

The following identifies two key issues affecting the accuracy of usage statistics for users and plans. Both of these issues stem from how stats are currently aggregated by organisation.

Technical Details

1. Inaccurate Counts of Users and Plans Due to Users Switching Orgs

The following is some of the code used to populate entries within the stats table (this is true for both type = 'StatJoinedUser' and type = 'StatCreatedPlan' entries):

# app/services/org/create_joined_user_service.rb
  OrgDateRangeable.split_months_from_creation(org_obj) do |start_date, end_date|
     StatJoinedUser::CreateOrUpdate.do(
       start_date: start_date,
       end_date: end_date,
        org: org_obj
      )
   end

# lib/org_date_rangeable.rb
  def split_months_from_creation(org, &block)
    starts_at = org.created_at
    ends_at = starts_at.end_of_month
    enumerable = []

    until starts_at.future? || ends_at.future?
      yield(starts_at, ends_at) if block
      enumerable << { start_date: starts_at, end_date: ends_at }
      starts_at = starts_at.next_month.beginning_of_month
      ends_at = starts_at.end_of_month
    end

    enumerable
  end

split_months_from_creation iterates through all months, starting at the date that the org was created at. This implementation leaves us open to the following edge case:

1. A user creates an account (and chooses an org in the process)
2. The user creates one or more plans
3. org X is created at a later date
4. The user switches their account to org X

Because split_months_from_creation starts from org.created_at and both user.created_at < org.created_at and plan.created_at < org.created_at, neither the user nor their plans will be included within the corresponding usage statistics counts.

2. Duplicate Plan Counting When Selecting "All" Organisations

The following is some of the code used to populate type = 'StatCreatedPlan' entries within the stats table:

# app/models/stat_created_plan/create_or_update.rb
  def count_plans(start_date:, end_date:, org:, filtered:)
    Role.joins(:plan, :user)
        .administrator
        .merge(users(org))
        .merge(plans(start_date: start_date, end_date: end_date, filtered: filtered))
        .select(:plan_id)
        .distinct
        .count
  end

count_plans counts all distinct plans where a user belongs to the org and is an administrator of the plan. This implementation leaves us open to the following edge case:

1. User A belongs to Org P and is an administrator of Plan Y
2. User B belongs to Org Q and is an administrator of Plan Y

In this case, when the usage statistics is calculated for the number of plans across all orgs, Plan Y will be counted twice — once for Org P and once for Org Q.