Diffetto doesn't seems to be use-full tool:

[dhruvkaushal11]

trafficstars

During Trace1 Capture, Device can be on Compile State 1 and for Trace2 Capture, Compile State 2, So, not ideal to find Diff via this. Just based on 1 iteration count we can never get a stable number

[theapache64]

@dhruvkaushal11First of all, I apologize for the late reply; I must have missed the notification. Secondly, thank you for sharing your feedback here. Now, let's discuss... :)

During Trace1 Capture, Device can be on Compile State 1 and for Trace2 Capture, Compile State 2,

Are you asking about comparing two builds or just one build? If it's just one build, what do you think about performing the first iteration without tracing, so the following iterations (2 and 3) will not include this unreliable variable in the middle, right? Or if you are referring to something else, please provide more details.

[dhruvkaushal11]

First iteration doesn't represent a real world happy flow scenario for max users base, as new installs <<<< active installs.

Are you asking about comparing two builds or just one build?

I am talking about the same build. I get a different value. Same method which was taking previously let say 200ms next time shows 40ms.

I understand the issue is not with diffetto but building averaging multiple iteration + handling compile state will make the results more stable and concludable.

[theapache64]

@dhruvkaushal11 I'd like to understand this problem better. Let's discuss this with an example.

Let's say there's an app. It has a method X(). X() usually takes 100ms to finish execution. At some later point, somebody added a few more lines of code, which increased the execution time to 200ms. Let's consider these scenarios as before (100ms) and after (200ms), and let's assume we don't know what regressed and we have our before and after build. The tracing steps will be more like this:

1. Install before.apk
2. Open the app and perform the scenario you want to trace
3. Now, close the app and remove from recent apps.
4. Start tracing
5. Repeat step 2
6. Stop tracing
7. Install after.apk
8. Repeat steps 2-6

Now you have two traces. Where do you see the instability in the traces?

[theapache64]

Closing this issue due to lack of input. Feel free to reopen with the response for above @dhruvkaushal11