IOG Catalyst Team: Success measurement

Catalyst Core technical infrastructure and Fund Administration services will continue to process an increasing volume of proposals submitted and votes cast. We estimate an additional 1,200 community-approved projects to receive funding over the next 12 months under the scope of proposed services.

Based on services provided by Catalyst Team to date, the outcomes of this proposal are projected to yield:

• 500-600 new technical development projects

• 300-400 community marketing activation projects, including education, expansion, and new Cardano user onboarding efforts

• 100-200 small-scale ‘Catalyst Open’ projects, where goals are centered on short-term, foundational R&D, and open-source development objectives

• 800+ newly completed projects based on the existing catalog of active projects and those funded between Fund10 and Fund13

• A minimum of five workshops per funding campaign held in collaboration with the community to review and refine key processes, such as:

  • Catalyst Standards: Defining minimum benchmarks for developers wishing to experiment on the Catalyst Continuous TestNET with solutions considered for wider implementation into the Catalyst voting system.

  • Catalyst Training: Proof-of-achievement reviews, Catalyst Continuous Testnet

  • Retrospective on F10 Community Review improvements: How recent changes have impacted workflows or results and any adjustments needed to be considered for Fund11 and beyond.

  • Scoping requirements for Catalyst ‘Categories’: Defining processes for scoping priorities for funding, budget allocations, and governance processes to decide on future directions for Catalyst community-led challenges or Categories.

To build a fund-by-fund picture of network health and growth, the Catalyst Team will measure impact against both performance and usage metrics to understand the vibrance of the ecosystem. The Catalyst Team has built and will continue to build and deploy various tools to aggregate, measure and analyze a number of data points in various categories. The Catalyst Team intends to continue to use such data to make improvements to its own processes and to continue to suggest improvements and enhancements for the Catalyst Program to the Cardano community. The categories of data captured and analyzed include the following:

Pre-Voting / Registration Phase (updated before voting starts):

• Mainnet Bridge and Snapshot:

  • Total voter registrations / staked ada

  • Voter registrations valid in active fund / staked ada

  • Voter registrations eligible to receive rewards in active fund / staked ada

These metrics help determine the trend of voter participation in Catalyst compared to previous funds: How many voters registered to participate; which of these registrations performed voting actions; and which performed enough votes to be eligible for a share of voter rewards.

• Voting Application:

  • Wallets connected

  • Support tickets submitted

  • Voting App specific feedback from end of fund user-survey

These metrics help determine if any bugs have been uncovered in production and the number of wallets interacting with the voting app.

• Idea submission and Community Review:

  • New community reviewer registrations

  • Fund-on-fund recurring community reviewer participation

  • Number of proposals submitted at draft deadline

  • Number of proposals withdrawn

  • Number of proposals to be voted on

  • Avg number of reviews carried out

    • LV0, LV1

  • Number of flagged reviews

    • By segmented classification of Catalyst scripts

    • That are included by LV2

    • That are discarded LV2

    • Avg number checked by LV2

These metrics help identify and understand the trends associated with the Community Review stage which in turn helps to forecast the relative costs to the treasury in future funding rounds.

• Voting Phase (updated when voting ends):

  • Wallets eligible to cast votes

  • Wallets casting at least one vote

  • Total votes cast

  • Max / avg votes per second

  • Avg votes cast per voter

  • Total active voters / stake

  • Avg voting stake per proposal

  • Avg voting stake per funded proposal

  • Avg voting stake per completed* proposal

These metrics help illustrate the growth trends in Catalyst network participation and related factors that affect overall decision-making and outcomes.

• Fund Campaign promotion:

  • External link clicks

  • Proposals opened in ideascale in app browser

  • Gitbook pages opened

  • Help / support links

  • New mailing list signups

  • New accounts created

• Catalyst Execution

  • Milestone proof-of-achievements verified

  • Ratio of funded-projects reporting on time each month

  • Ratio of projects reporting monthly project is ‘On Track/Not on Track’

  • Ratio of Projects completed versus:

    • Withdrawn / canceled projects

    • Active projects that have not met delivery schedule

These metrics illustrate the progress being made by funded projects and affect the health of outcomes achieved by the overall Catalyst funding programme.

All system health, performance, and usage metrics will be published at regular intervals at the end of each fund alongside the results of each voting stage. The data will also be used in the interim and end-of-year evaluation reports outlined in Milestone 5.