Algebraic File Synchronization: Adequacy and Completeness

# Algebraic File Synchronization: Adequacy and Completeness

Elod Pal Csirmaz
elod@epcsirmaz.com
###### Abstract

With distributed computing and mobile applications, synchronizing diverging replicas of data structures is a more and more common problem. We use algebraic methods to reason about filesystem operations, and introduce a simplified definition of conflicting updates to filesystems. We also define algorithms for update detection and reconciliation and present rigorous proofs that they not only work as intended, but also cannot be improved on.

To achieve this, we introduce a novel, symmetric set of filesystem commands with higher information content, which removes edge cases and increases the predictive powers of our algebraic model. We also present a number of generally useful classes and properties of sequences of commands.

These results are often intuitive, but providing exact proofs for them is far from trivial. They contribute to our understanding of this special type of algebraic model, and toward building more complete algebras of filesystem trees and extending algebraic approaches to other data storage protocols. They also form a theoretical basis for specifying and guaranteeing the error-free operation of applications that implement an algebraic approach to synchronization.

Keywords: file synchronization, algebraic approach, distributed systems, reconciliation, data replication

MSC classes: 08A70, 68M07, 68M14, 68P05, 68P20

ACM classes: D.4.3, E.5, F.2.2, G.2

## 1 Introduction

Synchronization of data structures is a mechanism behind many services we take for granted today: accessing and editing calendar events, documents and spreadsheets on multiple devices, version control systems and geographically distributed internet and web services that guarantee a fast and responsive user experience.

In this paper we investigate a command-based approach to synchronizing filesystem trees. Presented with multiple copies (replicas) of the same filesystem that have been independently modified, we regard the aim of the synchronizer to modify these replicas further to make them as similar as possible. The synchronization algorithm we describe follows the two main steps described by Balasubramaniam and Pierce [2]: update detection, where we identify modifications that have been applied to the replicas since they diverged and represent them as sequences of commands; and reconciliation, where we identify commands that can be propagated to other replicas and do so. The remaining commands, which could not be propagated, represent conflicting updates; resolving these requires a separate system or human intervention.

In this paper we build on our previous work done with Prof. Norman Ramsey on an algebraic approach to synchronization [12], and add to the theoretical understanding of synchronization by providing rigorous proofs that the algorithms we describe work as intended and offer a solution in our framework that cannot be improved on.

A central problem of command-based synchronizers during both update detection and reconciliation is the ordering (scheduling) of commands. If update detection is based on comparing the original state of the filesystem to its new state, then we can easily collect a set of updates, but we need to order these in a way that they could be applied to the filesystem without causing an error (for example, a directory needs to be created before a file can be created under it). Similarly, during reconciliation when we consider all the commands that have been applied to the different replicas, we need to find a way of merging these and creating a global ordering that we can apply to the filesystems. In fact, an insight in [12] is that if the ordering changes the effect of two commands, then under certain circumstances they are not compatible and will give rise to conflicts. Accordingly, in this paper the commutativity of command pairs and their other properties play a special role.

In terms of the usual classification of synchronizers [14, 10, 13], our approach to reconciliation is therefore operation-based as we investigate sets of commands, while the update detection algorithm makes the full synchronizer similar to state-based synchronizers as its starting point is the states of the replicas themselves. Moreover, our ordering algorithms only depend on properties of commands such as commutativity or idempotence, and therefore they can be classified as semantic schedulers. This is in opposition to syntactic scheduling, which uses the origin and the original temporal order of updates, and which may give rise to otherwise resolvable conflicts [13]. In general, ordering plays a central role in operation-based synchronizers. We refer to an excellent survey by Saito and Shapiro of so-called optimistic replication algorithms [13]; and, as examples, to IceCube, where multiple orders are tested to find an acceptable one ([7], see also [8]), or Bayou, where reconciling updates happens by redoing them in a globally determined order [15].

We start by defining a model of filesystems and a set of commands that we will use to describe updates and modifications in Section 2. This is followed by investigating the properties and behaviors of command pairs in Section 3. Section 4 describes update detection, and it is in Section 4.1 that we describe how commands can be ordered and prove that under some simple conditions all valid orders have the same effect.

Section 5 defines our reconciliation and conflict detection algorithms. We then proceed to prove that the output of reconciliation is correct in the sense that it can be applied to the replicas without causing an error, and it is maximal inasmuch as no further updates can be applied under any circumstances. In order to be able to do this, we introduce the concept of conditional operation of sequences of commands, and show that it has a number of highly convenient properties. We finish by comparing our results to other research and outlining directions for further work on the introduced algebraic system.

The approach in the present paper offers an improvement over results presented in [12] in multiple ways. We introduce a new set of commands that is symmetric and captures more information about the updates. This has the effect that reasoning about commands becomes simpler as there are fewer edge cases, and more powerful as predictions made by the reasoning are more accurate due to the additional information content. In fact, the new command set not only simplifies our reconciliation algorithm, but also makes it maximal.

The previous work also lacked proofs that the update detection and reconciliation algorithms it presented work as intended, which we provide in the current paper. While the results are intuitive, providing rigorous proofs is far from trivial. During the process, we define a number of useful concepts and show their relationships and the properties they possess. In our view these construct a special algebraic model that is worthy of interest and of further research on its own.

## 2 Definitions

### 2.1 The Filesystem

We model a working filesystem using a function with a set of nodes (filesystem paths) as its domain, and a set of possible contents or values as its codomain. Alternatively, if the filesystem has encountered a problem and is no longer usable, we assign it the special value broken:

###### Definition 1 (Filesystem).
 Φ={N→V{broken}

In our model, serves as a namespace or “skeleton” for the filesystem. It contains all possible nodes, including the ones where the file system contains no file or directory, where in our model has a special value, .

The nodes form a disjoint union of rooted directed trees, which determines the following parent function, and the ancestor / descendant and incomparable relations. Tao et al. [14] describe a similar filesystem model, although they also model inodes and restrict the filesystem to just a single tree.

###### Definition 2 (parent).

The function returns the parent node of , or returns none if is the root of a tree.

###### Definition 3 (n≺m).

On the ancestor / descendant relation is the partial ordering determined by the parent function. We write if is the ancestor of , that is, for some integer . We write if or .

###### Definition 4 (n≀≀m).

We write , or and are incomparable if and ; that is, incomparable nodes are on different branches or on different trees.

Every working filesystem has a so-called tree property, which means that if the filesystem is not empty at a node, and the node has a parent, then there must be a directory at the parent node. To model this, in we select another special value to represent directories, as we assume that apart from their location in the filesystem, all directories are equal. We do this because, as Bill Zissimopoulos pointed out [16], we often do not want to consider metadata stored in directories (e.g. permission settings) during synchronization, as these are not generally understood well by users, and, if needed, conflict resolution on these settings can be easily automated. We will also see that this assumption makes it possible to define a maximal reconciliation algorithm.

Let therefore the value representing directories be , and let us define subsets of for each type of value: let be , let be , and let contain all remaining values that represent different file contents together with any metadata associated with them:

The tree property can then be defined as

###### Definition 5 (Tree property).
 ∀n∈N: Φ(n)≠⊥ ∧{parent}(n)≠{none} ⇒Φ({parent}(n))∈VD.

This means that as we move down from the root of a tree of nodes, the types of values we encounter in the filesystem can only change according to the transition diagram in Fig. 1, from directories () to files () to empty nodes ().

In this paper, and denote filesystems, and , and are nodes in . We use for an arbitrary element in , and by we mean a value of type in .

### 2.2 Commands on Filesystems

Next we define commands on filesystems. As described above, we aimed to select a set of commands that captures as much information about the operations as possible, and is also symmetric. We start by summarizing our reasons for doing so.

Let us consider what kind of information is usually encoded in filesystem operations. A minimal set of commands, based on the most frequent tools implemented by filesystems, may be , which creates a file or directory () at the empty node ; , which replaces the earlier value at with ; and , which deletes the value at —where and (but ). Regarding their output, that is, the state of the filesystem at after applying the command, we know that after or , , whereas after , will be . However, from [12] and [3] we know that a useful set of axioms will in some cases need to distinguish between, for example, s that result in directories () and ones that result in files (), and treat them as separate commands, as their behaviors are quite different when combined with other commands. Indeed, Bill Zissimopoulos’ work demonstrated [16] that extending this distinction to more commands ultimately simplifies the definition of conflicting commands, as our model will then able to predict the behavior of commands more precisely. In other words, encoding the type of the output (, or ) in the commands is definitely useful. At the same time, there is never any need to consider the exact output value of a command, as the success or failure of filesystem commands only depends on the types of values in the filesystem.

Notice, however, that the commands listed above also encode some information about their input, the state of the filesystem before the command is applied. In particular, requires that there are no files or directories at , while and require the opposite. This creates an arbitrary asymmetry where there is now more information available about their output than about their input. As, based on the above, we expect that encoding more information in the commands results in a model with greater predictive powers, and in order to resolve this asymmetry, we propose a set of commands that encode the type of the original state of as well. (Some real-life filesystem commands like do this already.)

###### Definition 6 (Regular commands).

Therefore we have four pieces of information related to each command, and represent them using quadruples that list their input type (), output type (), the node to which they are applied (), and the value they store at the node ():

 ⟨X,Y,n,vY⟩,

where .

We use , or to represent the different types, and omit the final value if it is determined by the output type (when it is or ). For example, will represent , and will represent .

A command can only succeed if the original value at node has a type that matches the input type of the command. If this is not the case, or if the resulting filesystem no longer has the tree property, then we say that the command breaks the filesystem, and assigns broken to it. Broken filesystems are considered equal, but not equal to any working filesystem.

So that we could reason about sequences of commands that break every filesystem, we introduce, in addition to the regular commands, the command that simply breaks every filesystem.

We note that a command or a sequence of commands is applied to a filesystem by prefixing the command or sequence to it, for example: , , or simply if is a sequence of commands.

###### Definition 7 (Effect of commands).

We use a replacement operator of the form to denote a filesystem derived from by replacing its value at with :

 Φ[v/n](m)={vif~{}m=nΦ(m)otherwise.

The effect of the commands is then as follows:

 BreakΦ={broken} ⟨X,Y,n,vY⟩Φ=⎧⎪ ⎪ ⎪⎨⎪ ⎪ ⎪⎩{broken}if~{}Φ={broken}{broken}if~{}Φ(n)∉VX{broken}if~{}Φ[vY/n]~{}violates the tree propertyΦ[vY/n]otherwise.

In general, when we describe multiple or unknown commands, we may substitute one or both of their types with variables, as we did with . In this context, if we write that , we mean that the input and output types of the commands are the same ( and ), while a full equality () implies that their nodes and output values are also the same.

For reasons also listed in [12], in this paper we will not consider a or command. Regarding the theoretical reasoning we aim to follow, this turns out to be useful because this would be the only command that affects filesystems at two nodes at once, therefore describing the dependencies for would call for a more complicated model. From a pragmatic perspective, this restriction does not mean that in an application implementing conflict resolution using the algorithm described here would not be able to handle renames by pre- and post-processing changes in the filesystem to discover them, which can enhance usability, but which (especially when a rename is combined with changes to the content) is a non-trivial problem in itself.

## 3 Investigating Command Pairs

So that we can describe the effects of commands in general, let us introduce some notation and note some observations. We already know that commands usually do not occur in isolation, and are applied to filesystems in time. Therefore we will investigate sequences of commands with a well-defined order.

###### Definition 8 (Sequences of commands: ∘ and λ).

We use “” to concatenate commands to form a sequence, or concatenate sequences to form a longer sequence: , or , with the meaning that the command on the left is applied first:

 α∘βΦ=β(αΦ).

Sequences of commands form a free semigroup, and, as usual, denotes its unit element, the empty sequence, which by definition leaves all filesystems unchanged.

The following two relations (which echo the ones defined in [12]) allow us to investigate sequences independently of filesystems. In the definitions, stand for arbitrary sequences of commands.

###### Definition 9 (≡).

means that and are equivalent, that is, they behave in the same way on all filesystems: . It is a reflexive, commutative and transitive relation, and clearly .

###### Definition 10 (⊑).

means that extends , that is, they behave in the same way on any filesystem does not break: . It is a reflexive and transitive relation, and we can see that , and .

Our aim is to derive information about the effects of sequences of commands independently of the actual filesystems. In order to make this possible, we investigate the smallest building blocks of sequences: pairs of commands that act on a filesystem directly one after the other. This approach is useful as there are a limited number of command pairs, because, as we argued above, we can disregard the exact output values of commands apart from their type, and we can also abstract the relationship between the nodes in the two commands to a finite number of cases.

We methodically investigated of all possibilities using a computer program111The program is accessible on-line at
https://github.com/csirmaz/AlgebraicSyncPaper/blob/master/p2/prove.py.
to determine which pairs of commands cause errors all the time, which can be simplified to one or no commands, and which commute and can be reversed without any change in their overall effect. Our basis for this investigation was the model of filesystems introduced in this paper. These properties of command pairs are crucial as they determine how a set of commands can be re-ordered to be applied to a filesystem during synchronization, and what commands will never be compatible. Below we list a number of statements derived using this method.

Pairs of commands in general have the form

 ⟨X,Y,n,vY⟩∘⟨Z,W,m,vW⟩

where , , and values are of the appropriate type: and .

###### Rule 1.

Commands on incomparable nodes commute: where .

###### Rule 2.

Commands on incomparable nodes do not break every filesystem: where .

###### Rule 3.

Commands on the same node break every filesystem if their types are incompatible: where .

###### Rule 4.

Commands on the same node simplify: where , and or .

###### Rule 5.

Commands on the same node simplify: where and and .

###### Rule 6.

Commands on distant relatives break all filesystems: and where and and and .

###### Definition 11 (Construction pair).

A pair of commands on nodes and is a construction pair if and the commands are one of the following:

 ⟨⊥,D,n∗,d⟩∘⟨⊥,F,n,f⟩ ⟨⊥,D,n∗,d⟩∘⟨⊥,D,n,d⟩ ⟨F,D,n∗,d⟩∘⟨⊥,F,n,f⟩ ⟨F,D,n∗,d⟩∘⟨⊥,D,n,d⟩
###### Rule 7.

All other commands on a child break every filesystem: where and and and the pair is not a construction pair.

###### Definition 12 (Destruction pair).

A pair of commands on nodes and is a destruction pair if and the commands are one of the following:

 ⟨F,⊥,n⟩∘⟨D,⊥,n∗⟩ ⟨F,⊥,n⟩∘⟨D,F,n∗,f⟩ ⟨D,⊥,n⟩∘⟨D,⊥,n∗⟩ ⟨D,⊥,n⟩∘⟨D,F,n∗,f⟩
###### Rule 8.

All other commands on a parent break every filesystem: where and and and the pair is not a destruction pair.

###### Definition 13 (Assertion command).

A command is an assertion command if for every filesystem it either breaks it or leaves it in the same state. In other words, assertion commands have the same input and output type, and there must also be only one possible value for their output. Accordingly, and are the only two assertion commands.

###### Rule 9.

An assertion command can be added on a descendant node:

 ⟨⊥,⊥,n⟩∘⟨X,Y,n∗,vY⟩≡⟨X,Y,n∗,vY⟩≡⟨X,Y,n∗,vY⟩∘⟨⊥,⊥,n⟩

where and . (Note that this and next rule is true because the assertion command is adjacent to the original command.)

###### Rule 10.

An assertion command can be added on an ancestor node:

 ⟨D,D,n∗⟩∘⟨X,Y,n,vY⟩≡⟨X,Y,n,vY⟩≡⟨X,Y,n,vY⟩∘⟨D,D,n∗⟩

where and .

###### Rule 11.

Assertion commands can be removed: where or .

We also define the concept of two commands being independent:

###### Definition 14 (A≀≀B: Independent commands, sequences and sets).

Two commands and are independent, and we write if they have the same effect in either order and do not break all filesystems:

 ⟨X,Y,n,vY⟩∘⟨Z,W,m,vW⟩≡⟨Z,W,n,vW⟩∘⟨X,Y,n,vY⟩≢Break.

For two sequences or unordered sets of commands and we write if for all in and all in , .

It is intentional that we use the same symbol for independent commands and sequences as for incomparable nodes. As we will see in the next Lemma, these two concepts are closely related.

###### Lemma 1.

Two different commands that are not assertion commands are independent iff the nodes they change are incomparable:

 ⟨X,Y,n,vY⟩≀≀⟨Z,W,m,vW⟩⇔n≀≀m

if and none of and is or .

###### Proof.

This is easy to show based on the output of the software that investigates the behavior of command pairs. However, this proposition can also be derived from the Rules already listed in the following way.

We proceed in two steps. First, we note that from Rules 1 and 2 we know that if , then and must necessarily be independent, and therefore cannot be the case.

Next, we prove that

is also impossible as it would lead to contradiction. As we know that , or .

If , then from Rule 3 we know and as otherwise the commands, in one order or the other, would break all filesystems and they could not be independent. The commands are not assertion commands, and so from Rule 5 we know that can only happen if and . Therefore which is only possible if both commands are and have the same output value. This, however, contradicts our assumption that they are different.

If or , then from Rule 6 we know that if they are not directly related, then breaks all filesystems, and they cannot be independent. From the construction and destruction pairs and Rules 7 and 8 we also see that even if they are directly related, either or breaks all filesystems, so they again cannot be independent. ∎

## 4 Update Detection

In a command-based reconciliation solution we assume that we have two sequences of commands and that have modified a single filesystem yielding two different replicas and that we need to reconcile. While it is conceivable that the sequences would be based on a record of all operations that modified the two filesystems, in most filesystem implementations such records are not implemented, and therefore we must construct suitable sequences by comparing and to their common ancestor, . This is called update detection.

A sequence that transforms into must contain at least one command on each node that changes between the two states of the filesystem. Also, it is easy to see that this is also sufficient, as there are commands available for any input and output value. Specifically, if at node , and , then we add the command to where and .

To be able to describe such sequences better, let us introduce the following properties:

###### Definition 15 (Minimal set or sequence).

A sequence or set of commands is minimal if it contains at most one command on each node.

###### Definition 16 (Simple set or sequence).

A sequence or set of commands is simple if it is minimal and it does not contain assertion commands.

This update detector therefore yields a simple set of commands because we only add a single command for each node, and we only add commands that are necessary, that is, there will be no assertion commands in the set.

The next step in generating the sequences is to order the commands collected. As this task is at the heart of reconciliation itself independently of update detection, we discuss it in the next section. Then, in Theorem 1, we prove that the resulting sequence returned by the update detector actually works without breaking the filesystem.

### 4.1 Ordering commands

We often encounter the case where we only have a set of commands without a specified order. As we have seen above, this can occur after the first stage of update detection, but, more importantly, it is actually the task of the reconciler to determine whether there is an order, and if yes, what order, in which updates from different replicas can be applied to a filesystem.

In this section we describe an algorithm with which all possible orderings of a simple set of commands can be generated, but before doing so, let us consider the following lemma.

###### Lemma 2.

Given a set of commands that does not contain assertion commands, and which can be applied to a filesystem in some order without breaking it, and which contains commands on and where , then the set must also contain a command on each node between and .

###### Proof.

Without loss of generality, we can assume that . We prove that under the given conditions, the set must contain a command on . Then, by reapplying this result, we know that the set must contain commands on every ancestor of up to .

Furthermore, we prove this proposition for sequences, not sets, as if all sequences must contain a command on , then so must a set because otherwise there would be no order in which it can be applied to any filesystem.

Let be a sequence that satisfies the conditions; we therefore know that it contains a command on and another command on . We use an indirect proof and assume that there are no commands on in . Next, we create a new sequence in which and are next to each other. If they are already next to each other in , there is nothing to do. Otherwise, consider the command next to in the direction where is. Let this command be . If is to the right, then looks like the following:

 A=⋯∘⟨X,Y,n,vY⟩∘⟨Q,R,m,vR⟩∘⋯∘⟨Z,W,n∗,vW⟩∘⋯

If , then swap and . Based on Rule 1 we know that the new sequence is equivalent to . Otherwise, we know as there are no commands on , and so from Rule 6 we get which contradicts our assumptions. (Note that cannot contains assertion commands.) By repeating this step we can therefore convert into where and are neighboring commands. However, then Rule 6 applies to the sequence and therefore which is again a contradiction. ∎

We can now define our algorithm to generate orders of sets of commands.

Let be a simple set of commands that we know can be applied to at least one filesystem in some order without breaking it. Then we can split into a maximum number of disjunct subsets (components) so that the nodes of two commands from two different subsets are incomparable. These will simply be the components of the subset of the node forest that contains the nodes affected by .

From Rule 1 we know that commands from different components can be applied in any order. Therefore once we have ordered the commands inside the components, we generate all permutations of the components themselves, including the ones in which commands from separate components mix and overlap (but where the possible orders determined for commands inside the components are respected).

Next, consider the ordering inside a component.

This is not a problem in the case of components containing a single command only. (Note that a file-to-file command () can only appear in a component of one.)

In each component that is larger than one, we know that all nodes are related, and from Lemma 2 we also know that it must contain commands on nodes in between the related nodes. Therefore the nodes form a rooted ordered tree, and the pairs of commands connected by the edges in the tree must be construction or destruction pairs. (Otherwise, there would be no ordering of the component that is not equivalent to and via Rule 1 there would be no ordering of the whole set, either.)

To order the commands in the component, select the command with the topmost node, so that there is no command on its parent node (if there is one). If the command removes content at this node (it is of the form ), then it must follow all other commands in the component, and all parent–child pairs in the whole component must be destruction pairs. Otherwise, the command with the topmost node needs to precede the other commands, and the component must be made of construction pairs. These follow from the patterns of construction and destruction pairs, which at no point can mix without causing any ordering to break all filesystems.

Then move the command with the topmost node to the beginning or the end of the sequence we are creating out of the component, and recursively start the whole algorithm again on the commands in the component that remain, potentially splitting them into sub-components. The result of this process is a set of possible orders for each component, which are then combined into a set of possible orders for the whole set of commands.

###### Definition 17 (→℘).

We use to denote the set of sequences that contain all commands in the simple set and represent all orders generated by this algorithm.

It follows from the algorithm that:

###### Corollary 1.

If contains commands on both and , then there is a where they are next to each other.

As a specific case, we can regard a simple sequence of commands as a set and determine its . Lemma 4 is an important property of simple sequences: namely, that all of their valid orderings are equivalent, and all equivalent sequences are mere reorderings. To prove this, first we prove the following:

###### Lemma 3.

If two simple sequences and are equivalent and do not break all filesystems (), then they must contain the same commands.

###### Proof.

We use an inverse proof. Let and be simple sequences such that , and be a filesystem that they do not break.

We also assume that they do not contain the same commands. Without loss of generality, we can assume that contains , and either contains a different command on , or no command on at all. As is simple, we know that is not an assertion command, and therefore and as it has type . If so, then if has no command on , then and and cannot be equivalent. Otherwise we know that as does not break either, and that and therefore as must be and only has one command on . However, then , which is a contradiction. ∎

###### Lemma 4.

For all simple sequences and ,

 T≡S⟺T∈→℘(S).
###### Proof.

First, as only has at most one command on each node, any sequence in either breaks a filesystem, or it leaves it in the same state as .

Second, based on the algorithm we also know that contains all possible orderings of as the only ordering between commands it prescribes are when the commands in the reverse order would break every filesystem.

If, for example, in a component or sub-component made up of construction pairs, the command on the topmost node, , was not before all other commands in the component as prescribed by the algorithm, then any resulting sequence would necessarily contain a where , potentially followed by commands on incomparable nodes, and then followed by . From Rule 1 we know we can convert this sequence to an equivalent sequence by swapping the commands on incomparable nodes with , which sequence would therefore have the commands on and next to each other. Then from Rules 6 and 8 we know the sequence would break all filesystems as a reversed construction pair is never a valid destruction pair.

We can prove that an order not generated by the algorithm in the case of a component made of destruction pairs breaks all filesystems in a similar way.

Also, from Rule 1 we know that the order between any other command pairs is not fixed in the sense that if one order works, so will the other.

From these we know that all sequences in are equivalent, and that any sequence of the same commands outside it necessarily breaks all filesystems. We also know as and therefore .

From Lemma 3 we know that if , then contains the same commands as , from which we see that . ∎

It follows from this that given a simple set, we can simply assume that it has an order. Accordingly, we will at times treat not only sequences as sets, but simple sets as sequences.

### 4.2 The Correctness of Update Detection

Based on Lemma 2 we can also show that the update detector does function as intended as:

###### Theorem 1.

A sequence of commands returned by the update detector when comparing the non-broken to the original will not break if it is applied to it.

###### Proof.

Let be the original filesystem, and be the filesystem after the changes we intend to detect. Let us assume that some mechanism recorded all changes that occurred to until it reached the state . Our set if commands is sufficient to record any change, as there are commands for every input and output type pair, therefore we know that the changes can be recorded as a sequence of commands where . Note that will not contain assertion commands, as they do not represent an actual change in the filesystem.

Therefore, Lemma 2 applies to and we know if it contains commands on and where , then it also contains commands on nodes in between; which is equivalent to saying that if changed at node and , then it had to change on the nodes in between.

If so, then, similarly to what happens in the ordering algorithm, we can separate into components of commands on related nodes, as commands from separate components freely commute yielding equivalent sequences. From the arguments above we also know that each component must be made up of only destruction or only construction pairs, but not a mixture of the two. This means that in each component, commands on parents either strictly precede or succeed child nodes (as otherwise would hold), from which it follows that once the components are separated in a new sequence , commands on the same nodes are next to each other in .

Applying Rules 4 and 5 we can convert into an in which multiple commands on the same nodes are simplified into a single command or an empty sequence, and therefore is simple and . It is easy to see that must contain the same commands as the output of the update detector, , as both are simple but reflect all changes between and . If so, then and based on Lemma 4 we have , and as , therefore . ∎

An important consequence of this proof is that any sequence of recorded updates that were applied to a filesystem can be converted to a simple sequence using the rules listed in Section 3 alone.

## 5 Reconciliation

If we start with two copies of the filesystem , and two different sequences are applied to the copies to yield and , then our aim is to define sequences of commands and so that and would be as close to each other as possible.

We work based on the assumption that to achieve this, we need to apply to the commands that have been applied to , and vice versa. As some commands may have been applied to both filesystems, our first approximation is and (in some suitable order). This, however, will break both filesystems if there have been incompatible updates in and . Our aim is therefore to provide an algorithm that selects the commands and so that and , and show that these are the longest sequences with this property, that is, adding any command left out from or will break the filesystems.

We assume that and are returned by an update detector, and so they are simple sequences. (Otherwise, they can be simplified to become simple sequences; see Theorem 1.) We also assume that and are not broken. With these assumptions, we can now define our reconciliation algorithm.

###### Definition 18 (Reconciliation).

We determine —which we apply to to reconcile it with —to be the largest subset of that is independent of , where and are simple sequences representing the updates applied to replicas and :

 AR|B={α| α∈A∖B ∧ ∀β∈B∖A:α≀≀β }.

Whether two commands are independent is easy to determine programmatically either based on Definition 14 and the rules listed in Section 3, or based on Lemma 1 in at most time, where is the number of commands. Then, to generate the sequence from the resulting set, we can order the commands using the algorithm described in Section 4.1 as any subset of is also simple.

can be obtained in a similar way by reversing and in the definition.

Next, we would now like to prove that and can be applied to the replicas, that is, without loss of generality,

 AR|BΦB≠{broken}.

So that we can formalize statements needed to prove this, let us introduce two relations that describe under what conditions sequences of commands work, that is, do not break a filesystem.

### 5.1 Conditional Operation and Inverse Sequences

###### Definition 19 (w(x)).

For a set of sequences means that work at the same time, that is,

 ∃Φ:A1Φ≠{broken}∧⋯∧AkΦ≠% {broken}.

As the sequences form a set, their order is irrelevant. We also know that .

###### Definition 20 (w(x|y)).

For two sets of sequences, means that all of work where all of work, that is,

 ∀Φ: B1Φ≠{broken}∧⋯∧BkΦ≠{broken} ⇒ A1Φ≠{broken}∧⋯∧AlΦ≠{broken}.

As above, the order of the sequences in the sets is not relevant.

###### Definition 21 (Sets of sequences).

For brevity, we will use calligraphic letters (e.g. and ) to denote sets of sequences, and use to denote the set of sequences .

It is easy to see that the following corollaries are true:

.

###### Corollary 3.

, that is, if a sequence works, its initial segment also works.

###### Corollary 4.

can be chained: if , then .

We also prove the following lemmas.

###### Rule 12.

The combination of independent commands works wherever the original commands work:

 ∀Φ: ⟨X,Y,n,vY⟩≀≀⟨Z,W,m,vW⟩ ∧⟨X,Y,n,vY⟩Φ≠{broken} ∧⟨Z,W,m,vW⟩Φ≠{broken} ⇒⟨X,Y,n,vY⟩∘⟨Z,W,m,vW⟩Φ≠{broken}.
###### Proof.

We name this proposition a Rule because to prove it, we must reach back to our filesystem model. We proceed in an indirect way and assume that . We know so it must be applying that breaks it. Applying a command can only result in a broken filesystem in three cases. First, if the filesystem was already broken, which cannot be the case here. Second, if the input type does not match the filesystem, but we know and so as based on Lemma 1, . Third, if the new filesystem violates the tree property. This again cannot be the case because we also know that and the tree property only depends on the types of the parent and children of , which therefore cannot be changed by . ∎

This result can be extended to sequences:

###### Lemma 5.

The combination of independent sequences works wherever the original sequences work:

 S≀≀T⇒w(S∘T|S,T).
###### Proof.

Assume that there is a filesystem so that and , but .

From Lemma 1 and Rule 1 we know that the commands in and pairwise commute, and so any sequence that contains the commands from and and preserve their original partial order is equivalent to on all filesystems.

Let the command that breaks in when applying be so that . It is still true that , and by definition , but . Also, from above we know that and so .

If we denote the first command in with , this means that , which we can combine with , and Rule 12 (using as the reference filesystem) to arrive at .

We can repeat this step for , the next command in , and from and arrive at . This can be repeated until is exhausted and we get , which is a contradiction. ∎

We also prove the following:

###### Lemma 6.

If and are minimal sequences, , and there are commands on node in both () and (), then the input types of these commands must match ().

###### Proof.

This result is similar to Lemma 3 and is easily shown using an indirect proof: if , then there is no filesystem that either or would not break, and consequently and cannot work on the same filesystem. ∎

We continue by defining inverse commands and sequences which allow us to move parts of sequences between the condition and consequence parts of .

###### Definition 22 (Inverse commands and sequences).

The inverse of command is where is an arbitrary value from . We write for the inverse of sequence , which consists of the inverses of the commands in in reverse order.

###### Definition 23 (≃).

For two filesystems we write iff at all nodes the types of their values are the same, or they are both broken.

From the definitions we can clearly see that

.

###### Lemma 7.

A common initial segment of a set of sequences can be moved to the other side of by inverting it:

 w(B∘A|C)⇒w(A|B−1∘C) w(A|B∘C)⇒w(B−1∘A|C)
###### Proof.

This is based on the fact that in our model, unless they break a filesystem, commands leave the type of the filesystem value unchanged or change one type into another, but never merge types. That is, sequences—as functions mapping filesystems to filesystems—are essentially bijections over type-equality () with the only “sink” being broken:

 ∀S,Φ,Θ: SΦ≠{broken}⇒ (SΦ≄SΘ⟺Φ≄Θ).

For the proposition

 w(B∘A|C)⇒w(A|B−1∘C)

the rest of the proof is illustrated by Fig. 2 where , the domain of , represents the set of filesystems the sequence does not break, or neither of the sequences in the set breaks.

We see that the set of filesystems the sequences in do not break intersect with the range of . As is a bijection between its domain and range, we can use to map this intersection back onto the domain of . As the domain of must be a subset of this projected intersection. If so, then we can use to map the domain of , which yields the domain of . As it is also a part of the domain of , we get .

The second part of the lemma can be proven in a similar way. ∎

###### Lemma 8.

The combination of sequences with a common head and independent tails continues to work under the same conditions:

 w(A∘B|S)∧w(A∘C|S)∧B≀≀C⇒w(A∘B∘C|S)
###### Proof.

Based on Lemma 7 we know and , and from Lemma 5 we know . Combining these using Corollary 4 we get . Finally, applying the second line of Lemma 7 yields , which proves our lemma as (Corollary 5). ∎

### 5.2 The Correctness of Reconciliation

We are now ready to prove that the proposed algorithm for reconciliation is correct, that is, applying its result is not going to break the replicas. We reformulate the original proposition, , based on and so we aim to prove that works wherever and work:

###### Theorem 2.

If are simple, then , where, to restate Definition 18,

 AR|B={α| α∈A∖B ∧ ∀β∈B∖A:α≀≀β }.

This is trivial unless , so we assume that it is true. Let us first investigate the part of and that is excluded from : their intersection.

###### Lemma 9.

Let and be two simple sequences so that . Then their intersection, in some suitable order, also works on any filesystem either of them works on: , and, by symmetry, .

###### Proof.

The difficulty is that commands in can occur anywhere in . Therefore, first we prove the following: if all commands are marked in that also appear in , then there is an in in which all marked commands are at the beginning.

We show that can be transformed via equivalences into for which it is true. Our proposition is that if the marked commands are not at the beginning, then the sequence contains an unmarked command followed by a marked command. We show that these can be swapped resulting in an equivalent sequence. Then, by repeating this process similarly to bubble sorting, we can generate a suitable .

Let us consider therefore the marked command preceded by an unmarked command in , and let the marked command be , and the preceding unmarked command be :

 A=⋯∘⟨Z,W,m,vW⟩∘⟨X,Y,n,vY⟩∘⋯

As is simple and , from Rules 6 to 8 we know that these commands can only be on incomparable nodes or form a construction or destruction pair. In the first case, swapping the commands results in a sequence equivalent to , and we show that the last two cases are impossible as they would lead to contradiction.

In these cases, either or . If has a command on , then from Corollary 1 we know that there is a in where it is next to (also in ), and from Lemma 4 we know that . When describing our ordering algorithm in Section 4.1 we saw that the command on the parent path determines whether a pair of commands is a construction or destruction pair, and that this, in turn, determines whether the command on the child path must precede or follow the other command as otherwise the sequence would break all filesystems. This argument holds for both and , and so the command on must be on the same side of the command on in both sequences.

If has no command on , then let be , but with an extra command added to it just before according to Rules 9 and 10, from which we know that . We also know that is still minimal (but no longer simple).

In either case, therefore, we have a minimal which has a command on just before and . Let this command on be .

 A=⋯∘⟨Z,W,m,vW⟩∘⟨X,Y,n,vY⟩∘⋯ B′=⋯∘⟨Q,R,m,vR⟩∘⟨X,Y,n,vY⟩∘⋯

As and so , from Lemma 6 we know that .

Going back to the construction and destruction pairs we see that the output type of the first command is always the same, and it is either or depending on the relationship between and . Therefore and as . If originally had a command on , this is a contradiction as was not marked, but we see it must also be in and therefore in . If had no command on , this is a contradiction because the command on in is an assertion command, so , but contained no assertion commands.

We now know that there is an in which commands in are at the beginning, and therefore from Corollary 3 we know that and by symmetry . ∎

We would like to prove that . From the above we know that we can move commands in to the beginning of and so this is equivalent to

 w((A\nobreak\nobreak∩\nobreak\nobreakB)∘(B\nobreak\nobreak∖\nobreak\nobreakA)∘AR|B|A,B).

As we already know that and that (as ), we can prove our theorem based on Lemma 8 if we can prove that

 w((A\nobreak\nobreak∩\nobreak\nobreakB)∘AR|B|A,B).

Using Lemma 7 and Corollary 5 we can rephrase this as

 w(AR|B|(A\nobreak\nobreak∩\nobreak\nobreakB