IOG Catalyst Team: Success measurement

In the short term, user acceptance testing will provide a baseline of initial qualitative and quantitative data to help refine discovery and development.

Over the course of delivery, the Catalyst Team will measure several indicators to assess the success of the Project including:

• Reducing the number of interfaces/interactions required to complete all actions required of voters, representatives, and proposers

• New users time needed for education and completing quality actions

• Types and number of new supporting information or community review data provided to voters about a given proposal

• Watchers, contributors, and forks on the open-source project repository

• Data produced from preprod voting events conducted on testnet

• Testnet users testing and providing feedback on latest features

• Community contributors collaborating on CIP changes

• New community wallets supporting latest changes

Feature-specific user acceptance testing will be conducted with at least 5 users, with 80% acceptance rate, collecting deep observational usage data in line with best practices for product design and testing research.

Following which, an open, public demonstrator and community acceptance test will be conducted, where at least 30 users will complete a series of test case scenarios in a live testnet fund cycle and provide feedback to gain further data for UX analyses before production-readiness.

In the long-term, once the final deliverables are ready for primetime, we expect to see a demonstrable increase in participation across the Catalyst fund lifecycle - leading to greater community input, in addition to better quality of community engagement, helping to improve Cardano community-funding proposals under consideration.

We will measure the following KPIs:

• End of Fund survey data on experience, trust, and fairness

• Registered and delegated wallets / stake

• Active voting wallets / stake

• Avg voting stake per (funded) proposal

• Total votes cast & avg per wallet

• # and types of useful supporting information and community review data provided to voters about a proposal

We assert that increasing the share of circulating ADA participating in Catalyst voting (either by registering or delegating) boosts the effectiveness of the system as a whole. Similarly, increasing the number of wallets and share of stake voting on every proposal boosts both quality and diversity of funding outcomes.

End-of-fund survey responses to qualify improvements to overall user-experience, trust in the system, and fairness of decision making will further indicate whether this proposal has delivered on its commitments to Catalyst users.