ecosystem-contributions icon indicating copy to clipboard operation
ecosystem-contributions copied to clipboard

Published 20 hours ago • verified publisher icon ethereum-optimism

Foundation Mission Request: Analysis of S6 Growth Grants [DATA]

Open opjulian opened this issue 6 months ago • 9 comments

Foundation Mission Request - S6 Growth Grant Analysis

Proposed Foundation Mission Request: Analysis of S6 Growth Grants

S6 [Intent](https://gov.optimism.io/t/collective-intents-season-5/6883): Grow Application Developers on the Superchain

Proposal [Tier](https://gov.optimism.io/t/collective-trust-tiers/5877): Ember

Baseline grant amount: 10,000 OP

Should this Foundation Mission be fulfilled by one or multiple teams: One

OP Labs or Optimism Foundation Sponsor: Thomas (@thbialek)

Submit by: September 23rd

Selection by: October 21st

How will this Foundation Mission Request help accomplish the above Intent?

Growth grants are an essential building block of ecosystem development in the Collective, empowering projects to supercharge their growth campaigns, bootstrap impactful initiatives, and branch out into new opportunities. The proposed Foundation Mission Request for analyzing the performance of growth grants in Season 6 contributes to the Collective’s Intent #3 by growing application developers in a much more resource-efficient manner.

  1. Performance Evaluation: The retrospective analysis will assist the Foundation in fostering greater transparency in the evaluation of Season 6 growth grants by quantifying performance in a comparable, standardized, and publicly available fashion. Partnering with the community to progress toward a more data-informed performance evaluation framework for growth grants will provide valuable insights.
  2. Impact Measurement: Instead of having to rely on subjective guesswork or a mishmash of incoherent metrics, the Grants Council will be in a position to assess grant performance based on measurable metrics that capture progress toward the Collective’s Intents. This kick-off analysis will act as a steppingstone to a more meritocratic and extensive impact measurement system that is capable of widening the the lens beyond its initial scope.
  3. Resource Efficiency: A metrics-driven approach for calculating impact and measuring performance will separate signal from noise in the myriad of programs that have been buttressed by growth grants in Season 6. As a result, this will allow the Collective to unearth insights about how effectively S6 growth grants have been distributed among applicants while streamlining the grant allocation process in the future, ultimately equipping the Grants Council with better tools to identify high-impact projects.

Taken together, the three positive externalities outlined above will make the previously rather vague concepts of “impact” and “performance” in the context of growth grants more tangible. This will promote a much more deliberate performance assessment process for S6 grants — one that is geared toward harnessing the power of growth grants to grow application developers on the Superchain by standardizing their performance assessment.

What is required to execute this Foundation Mission Request?

Problem Statement

Due to the enormous wealth of talented builders working on important problems facing the Collective, a plethora of growth grants has been awarded to numerous initiatives with the goal of rallying the community to move the needle on our Intents.

However, despite the abundance of growth grants given to builders, there has been a notable scarcity of analyses measuring their effectiveness. Consequently, we are currently in the dark about how individual growth grants have performed, which growth grants have fared better than others, and which verticals have been the main beneficiaries of this resource stream.

In the past, the Collective has embarked on various isolated endeavors to study the performance of growth grants, resulting in fragmented deep dives devoid of a cohesive and scalable system. Thus, the absence of standardization has been the Achilles heel of these scattered efforts.

Against this backdrop, the desired outcome of this Mission Request is a detailed, metrics-driven assessment of all growth grants issued in Season 6.

Suggested Solution

In the quest for a systematic performance analysis of growth grants, multiple approaches have emerged over time. In this vein, applicants are free to synthesize and draw inspiration from previous attempts at applying a methodical approach to measuring the performance of growth grants, such as [this assessment](https://gov.optimism.io/t/may-2023-governance-call-op-rewards-analytics-update/5974) carried out by the data team at OP Labs.

In general, there are always tradeoffs between standardization and customization when conceiving a performance measurement system that takes many variables, such as industry vertical, incentivization period, and type of program, into consideration, but the goal is to strike a healthy balance between these two factors.

Although the scope of this Mission Request is limited to growth grants in Season 6, the analysis should already be designed with long-term scalability and a broader scope in mind. The success of this Mission Request will depend on how well it helps us answer the fundamental questions sketched out below.

Questions the analysis should try to answer:

  • Allocation Patterns: How (un)evenly have growth grants been allocated across projects, verticals, and types of incentive programs in Season 6? Where have been our blind spots in terms of areas we have under-allocated to in Season 6?
  • Utilization: What have been the most common patterns in terms of scope and purpose of the tokens deployed by the grantees?
  • Success Metrics: Is it possible to establish baseline values for the success metrics to benchmark against in future Seasons? Can we identify any notable differences for these metrics between grants, projects, and verticals?
  • Impact: How much impact has been generated by each growth grant? Can we observe a significant uplift in key metrics during the incentivization period compared to a control group?
  • Scalability: Can we design the analysis to be easily replicable for future Seasons? How can we structure the analysis be to be highly composable and to easily facilitate additional follow-up analyses?
  • Normalization: How can we normalize growth grant performance by the amount of OP tokens received and the length of the incentivization period? How can we make this normalized view comparable across projects and verticals?
  • Causality: Can we infer a causal relationship between the executed growth grants and changes in success metrics from this analysis?

Definition of Done

  • Analysis: The analysis should be based on the Foundation’s [suggested success metrics](https://gov.optimism.io/t/season-6-suggested-mission-requests/8107) and focus on all growth grants in Season 6. The findings of the analysis should be presented in a short written report that highlights the most important learnings.
  • Dashboard: The analyzed success metrics should be made publicly available in the form of a Dune dashboard to showcase the results in a transparent manner.

What milestones will help the Collective track progress towards completion of this Foundation Mission (RFP)?

  1. Specification: The grantee will specify and describe the methodology for the analysis in a structured document, detailing a step-by-step guide for carrying out the analysis.
  2. Feedback Session: The Foundation will set up a dedicated feedback session to discuss the proposed methodology and will approve the proposal.
  3. Implementation: The grantee will execute the agreed upon analysis. The deliverables should be captured in a written report, a Dune dashboard, and public GitHub repositories (if applicable). Overall, the results of the analysis should be easily digestible and actionable in order to pave the way for the Grants Council to incorporate gained insights into its performance evaluation process for growth grants.

How should Badgeholders measure impact upon completion of this Mission (RFP)?

  • Ideally, this analysis should get adopted by the Grants Council in Season 6 and form a crucial cornerstone of the success evaluation process for growth grants in future Seasons.
  • Owing to the built-in modularity and the general-purpose nature of the analysis, other OP Chains should adopt this framework for their own growth grant programs, driving more consistency across grant programs in the Superchain.
  • The grantee’s work should function as a tool for facilitating a more effective grant allocation process that ultimately should lead to the selection of more impactful growth grants in future Seasons.

Application Instructions

To apply for this Foundation Mission, please complete the form in the expandable section below and leave your response as a comment on this issue thread. Submissions will be open until [insert submit by date], at which time the Foundation will review all submissions and select [inset number of team eligible for selection] individual/team to complete the work defined here.

Application Instructions

To apply for this Foundation Mission, please complete the form in the expandable section below and leave your response as a comment on this issue thread. Submissions will be open until August 16th, at which time the Foundation will review all submissions and select 1 individual/team to complete the work defined here.

Submission Form

Copy the entire application below and leave a comment on this issue with your answers completed. A representative from the Optimism Foundation may reach out using the contact info provided to request more information as necessary.

Foundation Mission Application

Submission form

Please verify that you meet the qualifications for submitting at the above [Tier](https://gov.optimism.io/t/collective-trust-tiers/5877/2)

  • Project Lead: Please specify the best point of contact for your team
  • Contact info:
  • L2 recipient address:

What makes your Team best-suited to execute this Mission?

  • [...]
  • [...]

Please describe your proposed solution based on the above Solution Criteria (if applicable):

  • [...]
  • [...]

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each peice of work:

  • [...]
  • [...]

Please define the [critical milestone(s)](https://gov.optimism.io/t/grant-policies/5833) that should be used to determine whether you’ve executed on this proposal:

  • [...]
  • [...]

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

  • [...]
  • [...]

Grants are awarded in OP, locked for one year.

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

  • [ ] I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance.
  • [ ] I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant
  • [ ] I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the [Operating Manual](https://github.com/ethereum-optimism/OPerating-manual/blob/main/manual.md#valid-proposal-types)
  • [ ] I confirm that I have read and understand the [grant policies](https://gov.optimism.io/t/token-house-grant-policies/5833)
  • [ ] I understand that I will be expected to following the public grant reporting requirements outlined [here](https://gov.optimism.io/t/suggested-public-reporting-requirements-for-grantees/4176)

opjulian avatar Aug 23 '24 17:08 opjulian