At its core, IOTA is a protocol for permissionless and trustworthy transmission on the Internet of Things (IoT), enabling a frictionless exchange of data and value (IOTA Tokens) between machines and people. Thus, anyone can secure data on the Tangle and make it verifiable by third parties at no cost. In the IoT, with its low-resource micro-devices, the amount of data (transactions) to be stored and passed through the Tangle via nodes is a major problem, because with IOTA nodes having limited storage space, the Tangle can fill the database quickly. To combat this problem, Nodes have a feature called “Local Snapshots” that allows them to delete old transactions and keep their local copy of the Tangle small.
However, for many business use cases, the data in transactions needs to be stored for long periods of time. For example, financial data must be stored for 10 years in some cases, and identity data must be retained for at least the life of the identity (in some cases, your identity must even be retained beyond your lifetime). For example, consider the case of a transaction that includes a Decentralized Identity (DID) document. A user shares this DID with a government to request a driver’s license, but when the government searches for the DID in the Tangle, it may no longer be available and the request could be denied.
To enable these use cases without burdening the nodes, the IOTA Foundation has developed a permanode called Chronicle; with Chronicle, anyone can store all transactions outside the Tangle in a separate distributed database and make them available as long as they are needed.
Chronicle is the official permanode solution of the IOTA Foundation. It allows to store all transactions reaching a node in a distributed database that is secure and scales well. Chronicle is used to store the Tangle’s unlimited data flow and make it queryable. In other words, it enables unlimited storage of the Tangle’s entire history and makes that data easily accessible.
Chronicle is a framework for building permanode services that receive transactions from an IOTA network and store them in a Scylla cluster. Originally written in Elixir, it has now been fully ported to Rust to support interoperability with other IOTA projects, such as Bee, and to provide a more secure programming environment.
Explanation: Elixir is a general-purpose, functional, concurrent programming language that runs on the Erlang virtual machine. It can be used to create distributed, fault-tolerant, soft real-time, as well as always-on programs. It provides a complex network infrastructure that can be extended by multiplex networks with different functions by building microservices for each layer that can communicate with public and private datasets under different policies.
Chronicle uses Scylla as its default storage solution because it offers the following key features:
- Fault tolerance: users can set a replication strategy to determine how data should be replicated to avoid a single point of failure.
- Data consistency: Users can set a consistency level to determine whether a read or write operation is successful.
- Fast and efficient data retrieval: Scylla uses LSM-based memory with high write throughput.
- Lifetime: Users can define the lifetime of their data
- Low cost of ownership: licenses (including free and enterprise licenses) and cost of ownership compare favorably with other solutions
Chronicle also provides tools for building permanent solutions on an efficient runtime environment based on tokio. With Chronicle you can:
Store IOTA transactions in real time, using one or more Scylla clusters.
Search for all stored transactions using an HTTP API
Extend your own application with custom crates and configurations.
To do this, Chronicle includes the following crates that you can use as tools to build your own permanode and extend its functionality:
- API: API app that allows you to access the database
- Broker: ZMQ broker app for subscribing to new and confirmed transactions on an IOTA node
- Common: Runtime code that handles app loading and termination
- CQL: Code to decode and encode Cassandra Query Language (CQL) commands to interact with a Scylla node
- Storage: Storage app for connecting to a Scylla node, storing and searching transactions on the Scylla node.
- Dashboard: This component is an application for managing and monitoring components such as adding or removing Scylla nodes without downtime or connecting to new IOTA nodes in Chronicle Broker.
Query as a service
Basic economics dictates that an economic incentive is necessary to provide any amount of resources to store other people’s data. Therefore, the IF has created a way to offer community members or businesses that Tangle data “query as a service”. This makes Chronicle currently unique. With a little additional development, all Node owners could earn money by charging IOTA tokens for access to historical Tangle data. By requesting a quorum of internal and external nodes, the absolute veracity of the data could be guaranteed.
Functionality can be further extended through multiplex networks by building microservices for second layer solutions that can communicate with public and private datasets under different policies.
A mere Permanode that stores all transactions will not work in a few years due to the constantly growing number of transactions. The required storage space would be exorbitantly high and would cost the operator a lot of money. The goal of a Selective Permanode is to store only the data that is relevant for the operator itself. For larger companies, this would be an enormous incentive to participate in the network with their own Selective Permanodes and at the same time store their own relevant transactions, for a self-determined period of time. In August 2019, well-known community member Olaf van Wijk was already awarded financial support from the Ecosystem Development Fund. These funds will support his research on a “Selective Permanode”. For this purpose Olaf also founded the startup “Aion”. Here you can find the latest status report.
Chronicle is an open source project and everyone is encouraged to contribute to get it closer to production readiness:
Last Updated on 26. March 2021