IOG Catalyst Team: Deliverables, outputs, and intended outcomes of each milestone
We will demonstrate all milestones by the following:
All code and documentation artifacts published to the Catalyst Github repository.
Green, automated testing Reports for all module deliverables for each milestone.
A test report completed by internal QA resources, which validates each of the listed features is present, and complete.
Milestone 1:
This milestone is foundational development infrastructure work. It is necessary to ensure that licenses are appropriately applied, and that the development work can proceed smoothly from a solid foundation.
The deliverable is an Apache2 repository for the Hermes and Athena applications, and includes:
integrated and automated results of unit testing of code.
integrated and automated publishing of documentation.
correct license files are present and referenced in the documentation.
documentation detailing how to build the application locally.
facilities for the public to comment or query on the work undertaken in the repository.
Initial implementations of each application will form the foundation for future work.
Milestone 2:
This milestone delivers essential utility API’s and the necessary integration to the Cardano Blockchain. Delivering these APIs allows experimentation and early validation with the initial implementations of the Athena Modules to refine the internal design of Hermes and Athena.
2.1 Utility APIs :These are general purpose, foundational APIs that the Athena code will need, and provide a solid foundation for building the Hermes Runtime.
2.2 Logging API functionality to::
Log messages from Hermes WASM Modules.
Specify Levels of logs (Debug, Info, Warn, Error).
Add Structured Data to Logs.
Display Logs with appropriate context with the running Hermes node.
Logging API outputs will include:
Documented WASI API Definition
Hermes Implementation (Rust Module)
Interface to WASM Runtime.
Automated Test Cases running in CI.
2.3 Time API functionality to:
Get current time (UTC).
Get configured local timezone.
2.4 Time API outputs will include:
Documented WASI API Definition
Hermes Implementation (Rust Module)
Interface to WASM Runtime.
Automated Test Cases running in CI
2.5 Chronological Event API (Cron) functionality to:
Define periodic Events.
Define one-shot Events.
Route Cron events to WASM Modules.
Cron event outputs will include:
Documented WASI API Definition
Hermes Implementation (Rust Module)
Interface to WASM Runtime.
Automated Test Cases running in CI.
2.6 Hashing API functionality to:
Generate a balke2b-256 hash.
Hashing API outputs will include:
Documented WASI API Definition
Hermes Implementation (Rust Module)
Interface to WASM Runtime.
Automated Test Cases running in CI.
2.7 Cryptography API functionality to:
Generate ed25519 Private Key.
Derive Public Key from ed25519 Private Key.
Sign binary data with ed25519 Private Key.
Check Signature with ed25519 Public Key.
2.8 WASI Bindings to allow WASI Modules to integrate correctly with Hermes.
Build a WASI Module which links to a Hermes module to provide the necessary low level WASI API implementation.
Provide stdin/stdout/stderr compatibility to WASM Modules running in Hermes through the Logging API.
2.9 Cardano Blockchain Integration
Provide a standalone Rust Crate with the following features:
Configurable upstream Node (Either Node to Node or Node to Client protocol).
Ability through an API to tell the follower where to follow from (Slot#, Genesis, Tip).
Ability through an API to retrieve an arbitrary Block from a node.
Validate each block is consecutive, based on known data for the previous block.
API call which can validate an arbitrary block.
Validate each block based on block structure for a given era.
Generate events for new blocks, invalid blocks, rollback events and upstream node communication errors.
API call which can iterate a block and return data for individual transactions from the block.
Has no persistence.
Does not execute plutus or validate ledger state, only that the Blockchain is intact and the blocks are properly formatted.
Automated Test Cases running in CI.
2.9.2 Hermes - Cardano Blockchain Integration API
Functionality to:
Specify the upstream node address.
Generate Hermes events for each blockchain event.
An interface to all the API calls supplied by the Follower Crate.
Hermes Cardano Blockchain Integration API outputs will include:
Documented WASM API Definition
Hermes Implementation (Rust Module)
Interface to WASM Runtime.
Automated Test Cases running in CI.
Milestone 3:
This milestone delivers the remaining APIs and Web3 integrations needed to implement and deliver Athena in later milestones.
3.1 Hermes - File Data Handler and API
Applications in Hermes will be composed of a number of files. Applications will also need access to sandboxed and secure generalized file data storage. This module delivers the file management APIs used both internally by Hermes, and the Applications running on Hermes themselves.
Functionality will:
Load Hermes Application Packages
Check Applications are properly signed by trusted keys.
Find metadata within the Application Package.
Find Data within the Application Package.
Create and Sign Application Packages from individual files.
Create and manage sandboxed file storage for each application.
Expose an API to the WASM Modules to Create/Delete/Read/Write/Update files.
Define the layout of a Hermes Application Package, and the required metadata, configuration, static data, and WASM modules they must contain.
The Hermes Application Package is based on an open format and is able to be supported by third party developers.
3.2 Hermes - HTTP Gateway
Hermes will deliver its UX via a built-in HTTP Gateway which will allow industry standard web development tools to deliver fully distributed UX. This is an internal module of the Hermes engine.
Functionality will:
Listen on a configurable port for HTTP connections from local Web clients.
Route traffic to Hermes packaged applications via their subdomains.
Properly respond to CORS requests.
For regex defined paths, generate events to validate authorization to access a particular resource.
Serve static data directly from the application package if it matches the requested path for a GET Method.
Direct all other requests to the Application Webasm as HTTP Events for dynamic processing.
Hermes HTTP Gateway outputs will include:
Documented WASI API Definition
Interface to WASM runtime.
Automated test cases running in continuous integration.
3.3 Hermes - Sandboxed Database Handler
A critical component is a functional SQL Database which many complex applications use. Hermes will incorporate a sandboxed database API which will allow applications to securely access a database defined by the application running on Hermes.
Functionality will:
Expose the low level SQLite API to the WASM runtime and WASM Modules.
Prevent WASM from escaping the confines of the database.
Prevent the Database from growing beyond its allocated maximum size.
Hermes Sandbox Database Handler outputs will include:
Documented WASI API Definition
Interface to WASM Runtime.
Automated Test Cases running in CI.
3.4 Hermes - LibP2P / IPFS Integration and Handler
The Hermes nodes will intercommunicate as peers using functionality provided by the LibP2P Library. It will also enable high level distributed functionality provided by IPFS.
Functionality will:
Retrieve Data and Files published to IPFS.
Pin that data so it can be shared and served to peers.
Enable generalized Peer to Peer use of distributed Pub/Sub and Key/Value networking.
Enable WASM modules to subscribe to updates on a Pub/Sub topic.
Enable Reading the Value from a Key distributed in a Peer to Peer DHT.
Validate Values for data published to either a Pub/Sub topic, or the DHT.
Allow WASM modules run by Hermes to prove data is valid, and evict misbehaving peers.
Generate Events so that WASM run by Hermes can be processed when it updates.
Hermes LibP2P / IPFS Integration and Handler outputs will include:
Documented WASI API Definition
Interface to WASM Runtime.
Automated Test Cases running in continuous integration.
Milestone 4:
This milestone focuses on finalizing the WASM execution engine that is being built throughout the other milestones. This is delivered in Milestone 4 because all subsidiary modules must be functionally complete before fully validating the implementation of this module. Furthermore, a meaningful application logic is needed to execute and ensure the event and API integration is correct and behaving as required.
While Athena modules will commence development in Milestone 2, it is not possible to deliver them until the Hermes application engine itself is complete. This milestone therefore brings together the final components of Hermes and the first components of Athena.
4.1 Hermes - Functionally complete
Functionality will:
Handle and orchestrate a pool of WASM executors that execute the packaged application’s business logic.
Load WASM modules for a packaged application.
Execute an init function provided by each WASM module.
Given an event to be delivered to a packaged application:
Get all the WASM Modules for the application that processes that event.
Schedule those modules on free WASM executors from the pool.
Clone the WASM memory state before execution.
Ensures a packaged applications configuration and context are maintained during the lifetime of a WASM Events execution.
Automated test cases running in continuous integration.
4.2 Athena - Registration Tracking:
This is a set of WASM Modules deployed via the Athena application package, and run by the Hermes Application Engine.
Functionality will:
Process transaction events from the Cardano Blockchain in real time.
Detect and validate project Catalyst voter registrations.
Store registration data in a local database.
Enable registrations to be queried through the Hermes HTTP gateway. (Rest API)
In the scope of this proposal, Athena only tracks Voter Registration, and does not manage representatives, or other registration types.
Handle rollbacks correctly, and ensure that all recorded registrations are correct as at the current block processed from the blockchain.
4.3 Athena - Voting Power Calculation
Read ledger state in real time from the Cardano Blockchain block data.
Calculate and record staked ADA balances for registered voters into the applications database.
Handle rollbacks correctly, and ensure that all recorded registrations are correct as at the current block processed from the blockchain.
Recover Cardano blockchain following from the last scanned state.
Enable staked ADA and Voting power to be queried through the Hermes HTTP gateway. (Rest API)
Milestone 5:
This is the final milestone and the last Hermes modules and Athena UX are delivered in this milestone.
5.1 Athena - Catalyst Event Distributed Database
Functionality will:
Subscribe to a Pub/Sub channel which alerts to updates to updates to Catalyst Voting Event Data.
Retrieve the latest Catalyst Event Data from IPFS, based on the update event received.
Update the data stored in the local database.
Serve a copy of the latest data to distributed peers.
Provide dynamic endpoints that allow Catalyst Event data to be queried to enable voting on the current Fund.
Validate the data has been authoritatively published by a trusted source.
Allow the Athena user to customize which source of event data they trust.
5.2 Athena - Catalyst Event Authoritative DB Update
Provide an API via the Hermes HTTP gateway to:
Sign data updates to be distributed to peers, to validate the source of the data.
Publish data to peers.
5.3 Athena - Catalyst Voting Module
Provide an API via the Hermes HTTP gateway to:
Accept votes for Proposals from local users.
Publish those votes securely over pub/sub to connected peers to be recorded by the Ballot Box.
Verify the vote has been stored in the Ballot Box and will be counted.
5.4 Athena - Catalyst Ballot Box
Via the Pub/Sub network, save all cast votes in the local DB.
Provide an API via the Hermes HTTP gateway to:
Retrieve Votes that are published
Authoritatively publish a receipt that the vote has been recorded securely.
Perform an encrypted tally on all recorded votes.
Reject votes cast outside the voting window.
5.5 Athena - Graphical UX
To demonstrate the capability of running fully decentralized UX, Athena will incorporate a fully functional voting interface in a browser which looks and feels like a Web2 served centralized application.
Functionality will:
Provide Static endpoints to deliver a fully functional voting UX in a local browser.
Implement dynamic endpoints as required to create templated data for use by the user interface.
Interact with the API’s supplied by the HTTP Gateway running in the local Hermes Engine.
Do not use any centralized Web2 resources, such as centralized APIs, CDNs or Telemetry.
Last updated