DisPyTE - Distributing Python Task Environment

Dispyte is a tool that aims at distributing tasks across a heterogeneous network. It is written in pure Python. Thus, it can be simultaneously used on various operating systems. Dispyte has been successfully tested under GNU/Linux and MS Windows NT (2000 and XP). Because of its modular object oriented design it can be conveniently extended: allowing not only for numerous different applications, but also for integrating alternative communication mechanisms. Owing to its implicit fault tolerance and load balancing, it can be used as a powerful alternative for MPI or OpenMP.

Dispyte's core consists of collection of interfaces. Additionally, all common classes, for network communications or task and receiver administrations, are fully implemented. The user then has to derive his/her application-specific implementations from the provided interface, mainly in order to specify how tasks are ``produced''. In the current implementation the network communications are handled using remote calls. As its underlying network engine, Dispyte uses Twisted, an event-driven networking framework, also implemented in Python.

It also provides implementations for common classes. A Dispyte master is not only able to re-distribute (unsuccessful) tasks, but also to check for new available workers at run-time. Thus, allowing for a fairly fault-tolerant operation, as worker errors do not stop or interrupt the process. Another feature of Dispyte is its implicit load balance: If the master is waiting for unfinished tasks and discovers idle workers it re-sends these tasks to the vacant nodes. Once the master receives a task's result the calculation processes on all nodes that are redundantly working on the same task are stopped. These nodes are then available to work on new tasks.

- pure Python
- easy extensible
- works with heterogenous networks
- implicit load balance
- fail save network communication with Twisted
- runs on GNU/Linux and MS Windows NT (2000 and XP)