• 17 Posts
  • 28 Comments
Joined 3 years ago
cake
Cake day: July 31st, 2023

help-circle
























  • TLTR;

    Sparky is a distributed jobs framework that allows orchestration of remote tasks on cluster of nodes. It’s simple to set up and easy to use. This post is a brief overview of Sparky architecture design.

    Sparky targeted audience is:

    cloud providers to manage underlying multiple hosts infrastructure

    data scientists to process data in distributed manner (aka data pipelines)

    software engineers and devops doing any tasks with distributed nature in mind