SyncFree - Large-scale computation without synchronisation
The goal of SyncFree is to enable large-scale distributed applications without global synchronisation, by building upon the recent concept of Conflict-free Replicated Data Types (CRDTs). CRDTs allow unsynchronised concurrent updates, yet ensure data consistency. This revolutionary approach maximises responsiveness and availability, enabling data to be located near its users, in decentralised clouds. SyncFree aims to enable extreme scale replication, by applying eventual consistency techniques to CDRTs and beyond CDRTs.
Global-scale applications, such as virtual wallets, advertising platforms, social networks, online games, or collaboration networks all require consistency across distributed data items. As networked users, objects, devices, and sensors proliferate, the consistency issue is increasingly acute for the software industry. Classical alternatives are unsatisfactory as they either as they rely on synchronisation to ensure strong consistency, or forfeit synchronisation and consistency altogether with ad-hoc eventual consistency. The former approach does not scale beyond a single data centre and is expensive. The latter is extremely difficult to understand, and remains error-prone, even for highly-skilled programmers.
Open Source software available on
CDRTs avoid both global synchronisation and the complexities of ad-hoc eventual consistency by leveraging some simple formal properties of CRDTs. CRDTs are designed so that unsynchronised concurrent updates do not conflict and have well-defined semantics. By combining CRDT objects from a standard library of proven datatypes (counters, sets, graphs, sequences, etc.), large-scale distributed programming is simpler and less error-prone. CRDTs are a practical and cost-effective approach. The SyncFree project develops both theoretical and practical understanding of large-scale synchronisation-free programming based on CRDTs. Project results are new industrial applications, new application architectures, large-scale evaluation of both, programming models and algorithms for large-scale applications, and advanced scientific understanding.
The SyncFree project advances both the theory and practice of large-scale application architectures, and especially of CRDTs and related mechanisms. As the SyncFree industrial partners already have large user bases and feel the need for increased scalability in their applications, the project aims to include an extreme-scale crowd-sourced experiment, pushing the scalability needs of real world applications. An open-source library of CRDTs, to be used in future scalable distributed applications will be made available, leaving a lasting and beneficial impact far beyond the end of the project. Beyond CRDTs the project explores global invariants in an extreme-scale environment to develop programming tools and patterns for extreme scale replication, and to experiment in vivo with extreme scale real applications.