Our Solution
We built a Distributed Supercomputer where any computer on the internet running Windows, Mac or Linux could serve as a node. The nodes that are used for distributed computing are clustered using the Expectation-Maximization algorithm which buckets similar nodes into a cluster.
The assignment of a node for doing a computational task is done using a multi-arm bandit simulation. Financial settlements are done using Nebulas, a third-generation blockchain.
For managing jobs, three different schedulers were considered – Kubernetes, Nomad, and Docker Swarm. Finally, Nomad was chosen as it requires no external services for saving its state and stores it on the master node itself. Additionally, it is a single binary that acts both as the master node and the client.