Distributed Supercomputer using Blockchain and Machine Learning - Ideas2IT

Distributed Supercomputer using Blockchain and Machine Learning

Share This
Our Solution

We built a Distributed Supercomputer where any computer on the internet running Windows, Mac or Linux could serve as a node. The nodes that are used for distributed computing are clustered using the Expectation-Maximization algorithm which buckets similar nodes into a cluster.

The assignment of a node for doing a computational task is done using a multi-arm bandit simulation. Financial settlements are done using Nebulas, a third-generation blockchain.

For managing jobs, three different schedulers were considered – Kubernetes, Nomad, and Docker Swarm. Finally, Nomad was chosen as it requires no external services for saving its state and stores it on the master node itself. Additionally, it is a single binary that acts both as the master node and the client.