remix-project icon indicating copy to clipboard operation
remix-project copied to clipboard

Implement different tracking events

Open STetsing opened this issue 8 months ago • 0 comments

To better understand the adoption of AI feature, operations that are AI assisted must be tracked using Matamo

Here are some examples:

  • number of accepted AI suggestions (suggestion, insertion and completion) when compiling a contract
  • number of accepted AI error correction and successful compiled
  • in short every AI tools interacting (preceding tools execution or adjacent) with remix tools

for instance when compiling a contract, a score might be generated for

  • all "unchanged suggestions" the user accepted
  • all "remaining unchanged suggestions" the user accepted. word similarity can summarize this score, each word difference counts for 1. I.e. AI suggestion is function add(uint256 a, uint256 b) public return, when compiling the contract following changes have been made: function add(uint256 a, uint256 b) public pure returns(x). There are 2 changes, therefore the unnormalized score is then 6, and the normalized is 6/8 = 0.75

These metrics are meant to collected during a single user session

STetsing avatar Jun 05 '24 11:06 STetsing