add profiler for data packs and processors
Is your feature request related to a problem? Please describe. Profilers could test whether our new data structure has improvements and measure the time from each processor.
Describe the solution you'd like Add profilers for data packs and processors and mimic specific usage scenarios (such as adding pos tag and tokenization). Add another profiler with pure python and try to do similar actions as the data pack profiler. Then, compare what part of the data pack takes the most time and what part has heavy overheads. Some examples are here.
Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.
Additional context Add any other context or screenshots about the feature request here.