timemory icon indicating copy to clipboard operation
timemory copied to clipboard

Rework storage to support fixed storage size and localized storage

Open jrmadsen opened this issue 4 years ago • 0 comments

  • currently tim::storage<T> implements tim::impl::storage<T, bool> where the bool is whether the component collects data or not
  • tim::impl::storage<T, bool> should be updated to:
    • tim::impl::storage<T, true> should be renamed to tim::impl::dynamic_value_storage<T>
    • tim::impl::storage<T, false> should be renamed to tim::impl::void_storage<T>
    • A new storage type should be introduced with a fixed storage size using a ring-buffer, i.e. tim::impl::fixed_value_storage<T>
      • This new storage type will enable more complex usage of timemory component during sampling since memory must be pre-allocated within a signal handler
      • For components which support sampling, may actually want tim::storage<T> to hold both a fixed_value_storage instance and a dynamic_value_storage instance.
  • Singleton stuff w.r.t. storage should be migrated into tim::storage<T> instead of being held by the impl instance
    • This will help enable storage to be localized (see below)

Localized Storage

Currently, there is no way to do call-graph tracking locally, i.e. all call-graph tracking updates the same singleton. Therefore, it becomes a bit convoluted to do things like what happens when the compiler instrumentation is applied to code instrumented with timemory, i.e. you have two separate storage instances: one from compiler instrumentation and one from the manual instrumentation. A new storage_manager (non-templated) should be created which handles the storage instances of components. The default implementation should essentially provide the global "singletons". Here is a theoretical mock-up:

namespace tim
{
class storage_manger
{
public:
    storage_manager() = default;
    storage_manager(bool _use_global) : m_use_global{ _use_global } {}

    template <typename Tp>
    auto get_storage_instance()
    {
          using storage_type = tim::storage<Tp>;

          // return global instance
          if(m_use_global) return storage_type::instance();

          // return local instance
          auto& _val = m_storage_map[std::type_index{ typeid(Tp) }];
          if(!_val) _val = std::make_unique<storage_type>();
          return dynamic_cast<storage_type*>(_val.get());
    };

    void finalize(const std::string& _data_prefix)
    {
        if(m_use_global) return;

        auto _save = tim::settings::output_prefix(); // this isn't ideal but currently how it's done
        tim::settings::output_prefix() = _prefix;
        m_storage_map.clear();  // will trigger output
        tim::settings::output_prefix() = _save;
    };

private:
    bool m_use_global = false;
    std::unordered_map<std::type_index, std::unique_ptr<tim::base::storage>> m_storage_map;
};

namespace quirk
{
// template parameter to bundler which will require passing storage_manager to constructor
struct localized_storage : concepts::quirk_type
{};
}
}

namespace quirk = tim::quirk;

int main()
{
    tim::storage_manager _local_storage{};

    using local_bundle_t = tim::component_tuple<wall_clock, quirk::localized_storage>;

    // local_bundle_t _bundle{ "main" }; // Error!
    local_bundle_t _bundle{ "main", &_local_storage }; // Correct

    // local_bundle_t _bundle{ "main", quirk::config<quirk::localized_storage>{} }; // Error!
    local_bundle_t _bundle{ "main", &_local_storage, quirk::config<quirk::localized_storage>{} }; // Correct

    // ... etc. ...

   // generate output for 
   _local_storage.finalize("mydata");

   tim::timemory_finalize();
}

jrmadsen avatar Mar 03 '21 22:03 jrmadsen