IOTA— How to stream your device data via IOTA (MAM) messages to the distributed immutable Tangle and visualize its content.
This article compares IOTA and uses its protocols for our specific business / use case and compares it to the selected IoT platform solutions.
I send measurement data to the Tangle (IOTA’s immutable database network) and explore how to visualize it for our scaled, hopefully successful environmental sensor business case. The questions on how expensive will be this implementation per device and what is in for the user. Bearing in mind that the current network and protocol is under constant development and that the future importance and advantages of IOTA maybe only fully unfold when moving from Internet based to mesh IoT network based architectures. Furthermore for our use-case in my opinion a combination of cloud based and IOTA based features would be more appropriate for the moment anyways.
When it comes to Blockchain or in this case Distributed Ledger Technologies (DLT) usually its cryptocurrencies which comes to everybody’s mind. For the moment we won’t need this feature. IOTA, beside enabling the exchange of value transactions (like other cryptocurrencies) has some interesting other features. As widely known IOTA enables no-fee based zero value immutable transactions. This service alone could be used to onboard all the data to IOTA’s network, but it is not organized (streamed) nor private. This is were a second level implementation Mask Authenticated Messaging (MAM) claims to introduces a secure and reliable streaming of data to this permissionless distributed ledger. The trick for enabling zero fee transactions lies with ending the separation of those who validate the ledger (called incentivized miners in other blockchain architectures) and those who perform transactions in the network. In IOTA you are both and if you like to add a transaction you need to validate two others before yours gets validated.
Sounds good. Lets try to implement the easiest solution for the M0-M3 milestones first and then explore the costs when scaling the use — case.
This article and the implementation of the use case mainly follows.
Certainly there is already an excellent open minded community which sets up many tutorials and Proof of Concept (PoC) cases available even though the IOTA Foundation is quite early to the party. I try to implement a solution based on the open source freely available protocol and point out where we would need to adapt when scaling up from our prototype.
Overview of implementation and results
The solution is based on
- Sending data with MAM. This is the MAM publisher program from our Raspberry Pi (RPi). MAM object creation and signing is done on device.
- Performing the tip selection (find randomly two transactions which needs validation called as a function getTransactionToConfirm (gTTA)) on a serviced Permanode for the cost calculation. Actually using the devnets full Node for the prototype in the implementation (which comes for free).
- Outsourcing Proof of Work (PoW) to the same full Node. (https://devnet.thetangle.org/nodes; https://powsrv.io/)
- Broadcasting it via the same full node which then gossips it to its neighbors to the entire network.
- Using MAM to fetch the data — MAM receiver — on a web application deployed on google app engine.
- Visualization via google charts (deployed on google app engine).
Most of the libraries of IOTA’s ecosystem are on github. Meaning we can use our package manager to quickly dive in.
As IOTA is permissionless we don’t need to create somewhere an account or get any other authorization to use the system. As it turns out the property “permissionless” is most exciting but as well most difficult to property to secure the network.
Overview of flow of data
A much more advanced and complete prototype of a real MAM use case has been implemented and written down by Erwin Rooijakkers.
He listed already some of the current drawbacks like
- Non available out of the box secure authorization of your MAM channels.
Will most likely been introduced in the upcoming MAM+ release
- Non availability of permanodes (services) which stores the immutable data forever. (Drawbacks of having an immutable encrypted storage for ever and its implication of future tech potentially exploiting it.)
There is a service now available in beta (prices are subject to change) the estimation in this article only reflect the current pricing table — February 2019 https://thetangle.business
- Currently relatively high computational / energy costs for the Proof of Work (PoW) (used as a spam protection for the network) necessary before attaching a message to the tangle.
Can be significantly optimized using Field Programmable Gate Arrays (FPGA)
- Computational effort for the tip selection. The algorithm is necessary for finding the best two tips to be confirmed before issuing the transactions.
The computational effort of PoW
Beside all the other tasks (Trytes to Byte conversion, signing, etc.) PoW holds the heaviest job for the device. Both show 83s to 90s for PoW on our RPi(not knowing which PoW implementation they used). Which would work for us as we just send every 3 minutes a new measurement.
The energy consumed for doing PoW
- Atis article derives approximately 55J (Joules) for the PoW which is 1.5*10^-5 kWh.
- Erwins article derives 0,00005kWh ~180J for one transaction for the PoW.
As far as I understood the computational time and with that the necessary energy consumed can be reduced to approximately 300ms at the moment using a FPGA instead of the RPi. Basically implementing the PoW routines with hardware gates to accelerate its computation.
Message size and TPS requirements
Assuming 1600 bytes per IOTA transaction (2673 trytes) and we need two transaction per MAM message. Further every device send every 3 minutes one MAM message (this is for our message 2 IOTA transactions).
Implementation #1 (currently possible)
- The Full Node is performing the PoW, gTTA and findTransactions (all heavy tasks outsourced — we need to trust the Node here).
- The Node just never performs snapshots (growth with the tangle network and all its transactions).
- MAM implementation without public key encryption (we just send the encryption key (side-key) e.g. by email).
This is my estimation based on thetangle.business service rates. Using getTransactionToApprove (gTTA) — the tip selection (0.3€ per 1000 executions) and then attachingToTanlge (PoW) (1€ per 100 executions). Further I have just assumed per user 100 fetches (findTransactions) from the MAM stream per day. We need for one MAM message (with the current payload and security) two transactions. Both transactions to be valid to be broadcast would need perform PoW. But we only need one gTTA for the MAM bundle .
Cost for the use case for implementation #1
This is a steep price and probably not worth to follow for our use case for now.
Implementation #2 (currently possible)
- Our RPi device is performing the PoW in ~90s / transactions. For our MAM message this would take ~180s . The energy costs would be with the device holder.
- The Node performs local snapshots after (let’s say 100 days) (data is lost after snapshots ) which means we lose older data. We just use for our cost estimation the permanode service from thetangle.business.
- MAM implementation with self-made key encrypted exchange (hopefully quantum proof).
Estimation of cost for Implementation #2
Estimation is based on thetangle.business rates for gTTA and findTransactions. The PoW is done on the RPi and tries to approximate only the portion of energy necessary for PoW ( knowing that the RPi has to run anyway full time). For one device it’s assumed to stay in the always free limits but only cost the user in additional energy costs.
I stay positive that prices can be further reduced when using optimized HW and SW K1. As pointed out earlier it is all beta and early.
When it comes to the soft points I experienced some difficulties to figure out which libraries are up to date and can be used. Discord chat and the community really helps here when stuck K3. I particular liked the move from the foundation to restrict the chat to more development related topics rather than having it’s all about submerged in price talks on the IOTA token. Just recently the documentation got a new facelift too. The development talks are even live to be followed in the chat now – thats very interesting! The functionality which is provided out of the box is all available or getting build in variety of programming languages — but there is no real graphical user interface based platform yet .
When it comes to our particular use-case (visualization of environmental sensor data) its all a bit to much drama to store every measurement on the tangle and visualize it form there. But when you imagine it becomes necessary (by law) for hospitals to measure contamination levels — an immutable ledger for the audit trail might get more interesting (cost and value wise).
Implementation #3 (possible future setup — omega)
- The device is performing PoW in a very efficient way (maybe even only very limited amount, less than required today). Main protection will be Network-bound Proof of Work (NBPoW).
Bandwidth is a scarce recourse and naturally will limit the ability of bad actors as the costs of reserve a dominant part of the bandwidth becomes too expensive.
- Permanodes store all necessary transactions.
Either distributed (not all transactions on one node) with Iota Controlled agenT (ICT) – incentivized via IOTA tokens. Or centrally via IRI permanode — icentivised via IOTA tokens. At least that is my limited understanding of it.
- MAM+ used with out of the box sidekey safely encrypted exchange.
In order to fully enjoy this architecture I am afraid we will need to wait some years.
Implementing the use case
As outlined earlier IOTA is getting actively developed. By joining the Discord chat you can even see the Devs exchange information for there PRs on github. As well the second level protocol MAM is under development towards MAM+.
That means that the library I am using below might be in parts already outdated when you read this.
Transmitting a message on a public MAM channel
On the RPi
[email protected]:~/tmmiot $
[email protected]:~/tmmiot/IOTA/tmmiot-IOTA-agent $
Before we are transmitting the sensor data lets shortly try to just send a message and understand what are the different options provided by MAM.