Fractional repetition codes with flexible repair from combinatorial designs

Fractional repetition codes with flexible repair from combinatorial designs

Oktay Olmez and Aditya Ramamoorthy This work was supported in part by the NSF under grants CCF-1320416, CCF-1149860, CCF-1116322 and DMS-1120597 and by TUBITAK project numbers 114F246 and 115F064. The material in this work has appeared in part at the 50th Annual Allerton Conference on Communication, Control and Computing, 2012, the 2013 International Symposium on Network Coding and the 2013 Asilomar Conference on Signals, Systems and Computers. Oktay Olmez (oolmez@ankara.edu.tr) is with the Department of Mathematics at Ankara University, Tandogan, Ankara, Turkey. Aditya Ramamoorthy (adityar@iastate.edu) is with the Department of Electrical and Computer Engineering at Iowa State University, Ames, IA 50011. Copyright (c) 2014 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org.
Abstract

Fractional repetition (FR) codes are a class of regenerating codes for distributed storage systems with an exact (table-based) repair process that is also uncoded, i.e., upon failure, a node is regenerated by simply downloading packets from the surviving nodes. In our work, we present constructions of FR codes based on Steiner systems and resolvable combinatorial designs such as affine geometries, Hadamard designs and mutually orthogonal Latin squares. The failure resilience of our codes can be varied in a simple manner. We construct codes with normalized repair bandwidth () strictly larger than one; these cannot be obtained trivially from codes with . Furthermore, we present the Kronecker product technique for generating new codes from existing ones and elaborate on their properties. FR codes with locality are those where the repair degree is smaller than the number of nodes contacted for reconstructing the stored file. For these codes we establish a tradeoff between the local repair property and failure resilience and construct codes that meet this tradeoff. Much of prior work only provided lower bounds on the FR code rate. In our work, for most of our constructions we determine the code rate for certain parameter ranges.

{keywords}

fractional repetition code, combinatorial design, Steiner systems, affine geometry, high girth, resolvable design, regenerating codes, local repair.

I Introduction

Large scale data storage systems that are employed in social networks, video streaming websites and cloud storage are becoming increasingly popular. In these systems, the integrity of the stored data and the speed of the data access needs to be maintained even in the presence of unreliable storage nodes. This issue is typically handled by introducing redundancy in the storage system, through the usage of replication and/or erasure coding. However, the large scale, distributed nature of the systems under consideration introduces another issue. Namely, if a given storage node fails, it need to be regenerated so that the new system continues to have the properties of the original system. It is of course desirable to perform this regeneration in a distributed manner and optimize performance metrics associated with the regeneration process. Firstly, one would like to ensure that the regeneration process be fast. For this purpose we would like to minimize the data that needs to be downloaded from the surviving nodes. Moreover, we would like the surviving nodes and the new node to perform very little (ideally no) computation, as this also induces a substantial delay in the regeneration process that is comparable to the download time (since nowadays, memory access bandwidth is comparable to network bandwidth [1]). In addition, the regeneration induces a workload on the surviving storage nodes and it is desirable to perform the regeneration by connecting to a small number of nodes. Connecting to a small set of nodes also reduces the overall energy consumption of the system.

(a)

(b)
Fig. 1: (a) A DSS with . Each node contains a subset of size of the packets from . First node for instance contains symbols that is . When there is no confusion we simply use the notation instead of the set notation . (b) The DSS is constructed by applying a -MDS code followed by the inner fractional repetition code shown in the figure. It is specified with parameters . When a node fails we contact the remaining four nodes and download two packets from each to repair the failed node. This DSS can be obtained from the DSS on the left by trivial -expansion, where .

In recent years, codes which are designed to satisfy the needs of data storage systems have been the subject of much investigation and there is extensive literature on this topic. Depending upon the specific metrics that are optimized there are different requirements that the distributed storage system needs to satisfy. However, broadly speaking, all systems have the following general characteristics. A distributed storage system (henceforth abbreviated to DSS) consists of storage nodes, each of which stores packets (we use symbols and packets interchangeably). A given user, also referred to as the data collector needs to have the ability to reconstruct the stored file by contacting any nodes; this is referred to as the maximum distance separability (MDS) property of the system. To ensure reliability in the system, the DSS also needs to repair a failed node. This is accomplished by contacting a set of surviving nodes and downloading packets from each of them for a total repair bandwidth of packets. Thus, the system has a repair degree of , normalized repair bandwidth and total repair bandwidth . The new DSS should continue to have the MDS property.

A simple technique for obtaining a DSS is to treat the file that needs to be stored as a set of symbols over a large enough finite field, generate encoded symbols by using an MDS code (such as a Reed-Solomon (RS) code) and then store each encoded symbol on a different storage node. It is well recognized that the drawback of this method is that upon failure of a given storage node, a large amount of data needs to be downloaded from the remaining storage nodes (equivalent to recreating the file). To address this issue, the technique of regenerating codes was developed in the work of Dimakis et al. [2]. In the framework of [2], the repair degree and the system needs to have the property that a failed node can be repaired from any set of surviving nodes. The principal idea of regenerating codes is to use subpacketization. In particular, one treats a given physical block as consisting of multiple symbols (unlike the MDS code that stores exactly one symbol in each node). Coding is now performed across the packets such that the file can be recovered by contacting a certain minimum number of nodes. In addition, one can regenerate a failed node by downloading appropriately coded data from the surviving nodes. The work of [2] identified a fundamental tradeoff between the amount of storage at each node and the amount of data downloaded for repairing a failed node under the mechanism of functional repair, where the new node is functionally equivalent to the failed node, though it may not be an exact copy of it. Two points on the curve deserve special mention and are arguably of the most interest from a practical perspective. The minimum bandwidth regenerating (MBR) point refers to the point where the repair bandwidth, is minimum. Likewise, the minimum storage regenerating (MSR) point refers to the point where the storage per node, is minimum.

In a different line of work, it has been argued that repair bandwidth is not the only metric for evaluating the repair process. It has been observed that the number of nodes that are contacted for purposes of repair is also an important metric that needs to be considered. The model of [2], which enforces repair from any set of surviving nodes requires to be at least . The notion of local repair was introduced in [3, 4, 5], and considers the design of DSS where . However, one only requires that there is some set of surviving nodes from which the repair can take place.

The majority of work in the design of codes for DSS considers coded repair where the surviving nodes and the new node need to compute linear combinations of the stored symbols for regeneration. It is well recognized that the read/write bandwidth of machines is comparable to the network bandwidth [1]. Thus, this process induces additional undesirable delays [6] in the repair process. The process can also be potentially memory intensive since the packets comprising the file are often very large (of the order of GB). Motivated by these issues, reference [7] considered the following variant of the DSS problem. The DSS needs to satisfy the property of exact and uncoded repair, i.e., the regenerating node needs to produce an exact copy of the failed node by simply downloading packets from the surviving nodes. This allows the entire system to work without requiring any computation at the surviving nodes. In addition, they considered systems that are resilient to multiple failures. However, the DSS only has the property that the repair can be conducted by contacting some set of nodes, i.e., unlike the original setup, repair is not guaranteed by contacting any set of nodes. This is reasonable as most practical systems operate via a table-based repair, where the new node is provided information on the set of surviving nodes that it needs to contact. The work of [7] proposed a construction whereby an outer MDS code is concatenated with an inner “fractional repetition” code that specifies the placement of the coded symbols on the storage nodes. The main challenge here is to design the inner fractional repetition (FR) code in a systematic manner.

In this work, we present several families of FR codes and analyze their properties. This paper is organized as follows. In Section II, we outline our precise problem formulation, elaborate on the related work in the literature and summarize the contributions of our work. We discuss our FR code constructions for the case when in Section III, and explain the Kronecker product technique in Section IV. The locally recoverable FR codes where are considered in Section V and Section VI outlines the conclusions and opportunities for future work.

Fig. 2: BIBD also known as the Fano plane. Nodes of the DSS, which can be obtained from the Fano plane, are listed on the right.

Ii Background, Related Work and Summary of Contributions

A DSS is specified by parameters where - number of storage nodes, - the minimum number of nodes to be contacted for recovering the file, - the number of nodes to be contacted in order to regenerate a failed node and - the storage capacity. In case of repair, the new node downloads packets from each surviving node, for a total of packets. Let denote the size of file being stored on the DSS. We consider the design of fractional repetition codes that are best explained by means of the following example [8] with .

Example 1

Consider a file of packets that needs to be stored on the DSS. We use a MDS code that outputs packets and . The coded packets are placed on storage nodes as shown in Fig. (a)a. This placement specifies the inner fractional repetition code. It can be observed that each is repeated times and the total number of symbols . Any user who contacts any nodes can recover the file (using the MDS property). Moreover, a failed node can be regenerated by downloading one packet each from the four surviving nodes, i.e., and , so that .

Thus, the approach uses an MDS code to encode a file consisting of a certain number of symbols. Let denote the number of encoded symbols. Copies of these symbols are placed on the nodes such that each symbol is repeated times and each node contains symbols. Moreover, if a given node fails, it can be exactly recovered by downloading packets from some set of surviving nodes, for a total repair bandwidth of . It is to be noted that in this case , i.e., these schemes operate at the MBR point. In the example above, , so that . One can also consider systems with in general. A simple way to do this is replicating the symbols in the storage system. The resultant DSS has the parameters with . However, in this work we show that there are infinite families of FR codes with which cannot be obtained this way. In Fig. (b)b we illustrate the DSS obtained by replicating the -DSS when .

Fig. 3: The figure shows a DSS where . A node can be repaired by contacting the other two nodes in the same column. The system is resilient up to node failures.

Before introducing the formal definition of a fractional repetition (FR) code we need the notion of -recoverability. Let denote the set .

Definition 1 (-recoverability)

Let and be subsets of . Let and consider with . We say that is -recoverable from if there exist for each such that and .

Definition 2 (FR Codes)

A fractional repetition (FR) code for a -DSS with repetition degree and normalized repair bandwidth ( and are positive integers) is a set of subsets of a symbol set with the following properties.

  • The cardinality of each is .

  • Each element of is contained in exactly sets in .

  • Let denote any sized subset of and . Each is -recoverable from some -sized subset of . Let be the maximum value of such that this property holds.

We provide the following example to illustrate that requirement (c) of Definition 2 plays an important role in our study.

Example 2

Consider the sets , and two different families of subsets of as shown below.

Both and satisfy the requirements (a) and (b) of Definition 2. However, note that . This implies that is not -recoverable from the set

So cannot be a fractional repetition code. In contrast, any failed set in is -recoverable and thus is a fractional repetition code with .

The value of is a measure of the resilience of the system to node failures, under the constraint of exact and uncoded repair. The file size is given by

and the code rate is defined as . We emphasize that depends on .

Note that the parameters of a FR code are such that . Thus, the code rate . Moreover as , the maximum rate of any FR code is at most . It is to be noted that the parameter also sets the code rate of the outer MDS code; it is exactly . For a FR code and an index set , we say that nodes for cover symbols if .

Fig. 4: A failed node can be recovered by contacting two nodes and downloading one packet from each. The code is resilient up to five failures and the file size is . The minimum distance is of the code is , since any four nodes can recover the file.

The work of [7], only considered FR codes with and , i.e., for recovery the new node would contact surviving nodes and download a single packet from each of them. For their codes, the requirement (c) in Definition 2 is satisfied and the system is resilient to failures, i.e., . It is to be noted that the requirement of is essential in the problem formulation considered in [2] since the systems require node recovery from any set of surviving nodes. In that setup if , it is easy to see that one can always specify a failed node and a set of nodes from which recovery is impossible. However, in the framework of [7], the recovery requirement is relaxed. Specifically, to recover from a failure, the new node contacts a specific set of nodes from which it regenerates the failed node. Thus, the recovery process is table-based and for each node we only need to guarantee the existence of one set of nodes from which recovery is possible. Thus, it becomes possible to have systems with . In fact, in Section V, of this paper, we present several constructions of FR codes where . In the literature, these are referred to as codes that allow for local repair.

For FR codes, the failure resilience and the code rate are two evaluation metrics and it is evident that there is a tradeoff between them. Indeed, if the outer MDS code does not add any redundancy, i.e., then would need to be chosen such that any nodes cover all the symbols and the code rate of the system would be exactly . However, in this case the DSS will be resilient to at most failures under any possible recovery procedure, i.e., even without any constraint on the repair. In contrast, if the outer code introduces nontrivial redundancy, the file size would be lower but it may be possible to reconstruct the DSS in the presence of more than failures. To see this, consider Example 1 where the outer MDS code has rate . Note that under exact and uncoded repair, this DSS is resilient to only one failure. However, the DSS can be reconstructed even in the presence of the failure of any two nodes, since any three surviving nodes cover at least nine symbols. Our proposed codes will also be evaluated in terms of their minimum distance which quantifies this tradeoff.

Definition 3 (Minimum Distance of a DSS)

The minimum distance of a DSS denoted is defined to be the size of the smallest subset of storage nodes whose failure guarantees that the file is not recoverable from the surviving nodes.

The Singleton bound on the minimum distance in this context can be found, e.g., in eq. (15) in reference [9].

Lemma 1 (Singleton Bound)

Consider a DSS with parameters with file size and minimum distance . Then,

It turns out that codes that have the local repair property, i.e., codes with suffer a penalty on the maximum possible minimum distance. This tradeoff was captured in the case of scalar (i.e., ) codes by [3] and by [4] in the case of vector (i.e., ) codes.

Lemma 2

Consider a DSS with parameters with file size and minimum distance . Then,

We note that if , we have so that the bound above reduces to the Singleton bound.

Observation 1

A given DSS meets the Singleton bound if . Similarly, a code meets the bound in Lemma 2 if .

It is to be noted that the bound in Lemma 2 holds for all possible local repair codes. In this work, we consider the added constraint that the repair takes place purely by download. Thus, for our constructions, the bound in Lemma 2 is in general loose. In Section V we derive a tighter upper bound on the minimum distance of codes where the repair process is local and operates purely by download.

At various points we will need to use the well-known inclusion-exclusion principle for computing the maximum file sizes that can be supported by our DSS. For the sake of completeness, we state the result here.

Theorem 1

[Inclusion-Exclusion principle] Consider sets . If , let . Then

(1)

It can also be shown that

(2)
Method Range of Comments
Steiner Systems with Steiner systems with are completely characterized and explicit constructions are known. Here we list the necessary and sufficient conditions for the cases
  • Steiner systems with exists for any .

  • Steiner systems with exists for any .

  • Steiner systems with exists for any .

For there are only finitely many exceptions where the existence of Steiner systems is unknown. For this we refer the reader to the tables provided in section 3 of the book [10].
Transposed Steiner Systems File size equals if the original Steiner system has a maximal arc.
  • There exist a Steiner system with and a maximal arc if .

  • There exist a Steiner system with and a maximal arc if is a prime power.

To our best knowledge results about the existence of maximal arcs in Steiner systems with higher values of are not known.
Grids for even , for odd The file size calculation can be done for any positive integer .
MOLS (Remark 4) File size equals if . Use the construction of two MOLS for order greater than 6 [11].
MOLS (Lemma 9)
is a prime. Use the construction of MOLS where the order is a prime power.
TABLE I: Constructions where and . Note that we can perform trivial -expansion to obtain higher .
Comments
is a prime.
is a prime.
is a prime. These designs are known as Unitals.
. These designs are known as Denniston designs.
TABLE II: Well-known infinite families of Steiner systems when . These can be found in [10].

Many of our constructions will result from combinatorial designs that we briefly introduce (a detailed description can be found in [12]).

Definition 4 (Combinatorial Design)

A combinatorial design (or, simply a design) is a pair where is a finite set of elements called “points” and is a collection of non-empty subsets of called “blocks”.

A prototypical example with several applications is the balanced incomplete block design (BIBD).

Definition 5 (Balanced Incomplete Block Design)

A balanced incomplete block design (BIBD) is a pair that forms a combinatorial design such that ; every element of is contained in exactly blocks and every -subset of is contained in exactly blocks.

Let denote the number of blocks. By using combinatorial double counting arguments it can be seen that for a BIBD, the following relations hold.

(3)
(4)

A BIBD can be used as the FR code in a DSS as long as -recoverability is guaranteed for an appropriate (there are several instances when ). These BIBDs include finite projective planes and affine planes. Table II contains a list of well-known families of Steiner systems. A BIBD is equivalent to a projective plane of order . Projective planes have interesting geometric properties that can be used in determining the corresponding file size. For instance, any two blocks of a BIBD share exactly one point and any two points are contained in exactly one block in a projective plane. The smallest example of a projective plane corresponding to is known as the Fano plane and is depicted in Fig. 2. For more information on the projective planes and affine planes we refer the Chapter 2 of [12].

One can use the Fano plane to design the inner FR code, by interpreting the points as symbols and the blocks as storage nodes. Suppose we first apply a -MDS to the file. Then we can place the coded symbols on the storage nodes as depicted in Fig. 2. Note that these storage nodes are obtained from the blocks. The obtained DSS has the property that any two nodes share exactly one symbol. Thus, using Theorem 1 contacting any three nodes recovers at least distinct symbols and hence the file. Furthermore, we can identify a set of three nodes whose intersection is empty, e.g., nodes 1, 2 and 4. Thus, the maximum file size this DSS can support is 6. An affine plane can be obtained by deleting one block and its all points from a projective plane. Hence, an affine plane of order is equivalent to a BIBD. Here any two points are contained in exactly one block. However, there are in general pairs of blocks that do not have any points in common. More generally, a FR code can be obtained from Steiner systems.

Definition 6 (Steiner Systems)

A Steiner system is a set of elements and a collection of subsets of of size called blocks such that any -subset of the symbol set appears exactly one of the blocks.

Steiner systems are examples of -designs. A FR code is a -design if every -subset of symbols is contained in exactly nodes. The concept of -designs can be viewed as a generalization of the concept of BIBDs. Naturally, a Steiner system is a -BIBD where

Thus, projective planes and affine planes are instances of Steiner systems. A given FR code can be put in one-to-one correspondence with an incidence matrix as explained below.

Definition 7 (Incidence Matrix of a FR Code)

An incidence matrix of a FR code where and is the binary matrix defined by

We shall sometimes refer to the FR code by simply referring to its incidence matrix . We will occasionally refer to the bipartite graph corresponding to the FR code as well. This is defined next.

Definition 8 (Bipartite graph of a FR Code)

For a FR code where and with incidence matrix , we define its bipartite graph as follows. We associate the storage nodes in with the vertices and the points in with the vertices so that and are disjoint. There exists an edge between and and if and only if .

Method Range of Comments
Affine Resolvable Designs The file size exceeds the trivial lower bound . The parallel classes need to be chosen in a careful manner. If , then we choose and if , we choose .
Hadamard Designs is not restricted to be a prime power.
TABLE III: Constructions where and . In many cases these construction parameters cannot be obtained by trivial -expansion.
Base Code Method Range of Comments
obtained via the transpose of a Steiner system with with maximal arc of size Kronecker product of with itself There exist a Steiner system with and a maximal arc if . In several cases, the codes obtained via Kronecker product cannot be obtained by trivial -expansion.
TABLE IV: Constructions obtained via Kronecker product, where .
Method Range of Comments
Use undirected graph , , degree and girth 1 Need and . Codes meet the minimum distance bound for locally recoverable codes. Since an -cage minimizes the number nodes in the system, we will have highest possible code rate for this particular construction. Here we list some of the well-known infinite families of -cage.
  • -cages are complete graphs on vertices.

  • -cages are complete bipartite graphs on vertices.

  • When is a prime power -cages can be obtained from incidence graphs of projective planes.

  • When is a prime power -cages and -cages can be obtained from incidence graphs of generalized polygons.

For more see the survey [13].
Use copies of FR code with parameters such that any nodes cover symbols. for . Parameters satisfy . Codes meet the minimum distance bound for locally recoverable codes with exact and uncoded repair (cf. Section V).
TABLE V: Constructions of local FR codes where .
Definition 9 (Transposed FR Code)

For a FR code with incidence matrix , the code specified by is called transposed FR code of and denoted by if the design obtained from is -recoverable for some .

Note that, in the transposed code, the roles of the storage nodes and the symbols are reversed. An infinite family of transposed codes can be obtained from Steiner systems with . In such Steiner systems any pair of symbols is contained in exactly one node which implies that any pair of nodes in the transposed design share exactly one symbol. This in turn means that the transposed design is 1-recoverable.

Incidence matrices with appropriate parameters can be combined via operations such as the Kronecker product to obtain new matrices (equivalently FR codes) with a new set of parameters. We use this technique extensively in the sequel to generate families of FR codes.

Definition 10 (Kronecker Product)

If is an -by- matrix and is a -by- matrix, then the Kronecker product is the -by- matrix

Let and be two incidence matrices of FR codes and with parameters and respectively. Let be the columns of and be the columns . A new FR code can be obtained from the old one by the following incidence matrix

We can find an appropriate permutation matrix such that the matrix is equal to the Kronecker product of and . Note that matrix reorders the columns of .

We can obtain a DSS by replicating the symbols of another DSS via the Kronecker product. In the subsequent discussion we will refer to this technique for obtaining codes with as trivial -expansion.

Definition 11

[Trivial -expansion] Let be incidence matrix of a FR code with parameters with . Let be the all-ones column vector. The FR code obtained from which has parameters is called a trivial -expansion of the code with .

In the remainder of this section, we discuss some illustrative examples of FR codes. Our first example is a code with that cannot be obtained by trivial -expansion.

Example 3 (A Non-trivial Code with )

Consider the DSS shown in Fig. 3. The ten symbols are obtained by using an outer MDS code followed by the FR code illustrated in Fig. 3. Note that the DSS can recover from a single node failure by downloading two packets each from two nodes in the same column; hence . Moreover, any two nodes share 0,1, or 2 symbols in common which implies that any two nodes recover at least symbols, thus . According to the Singleton bound . The system requires only two surviving nodes to recover the file thus the code is resilient up to 13 failures (since ) and thus meets the Singleton bound. However, this code (with ) cannot be arrived at simply by replication. To see this we note that if this were true, the original DSS with must correspond to a storage capacity of and have a number of symbols which is 5. However, this means that there can be at most distinct storage nodes of capacity two. Thus our design with cannot be obtained this way.

The idea underlying Example 3 can be formalized as follows.

Observation 2 (Non-trivial FR Codes with )

A FR code with parameters , and distinct storage nodes cannot be obtained from a trivial -expansion if .

Next, we demonstrate an example of a locally recoverable DSS, i.e., a system where that is constructed using the Kronecker product method.

Example 4 (Locally Recoverable Code Using Kronecker Product Technique)

Let be a FR code with and with incidence matrix . The code obtained from is presented in Fig. 4 where denotes the identity matrix. Suppose that the outer MDS code has parameters , so that . Consider contacting any of the four nodes depicted in Fig. 4. These nodes will fall into one of the three columns in the figure. So, there are three cases we need to examine.

  • Case (a): Two nodes can be chosen from one of the columns and one from each of the rest. The union of these nodes has a cardinality of .

  • Case (b): We first select two columns and two nodes within each column. In this case the size of the union is .

  • Case (c): Finally, we can select two columns and choose three nodes in one column and one node in the other column. In this case the cardinality of the union is .

Thus, it is evident that contacting any nodes will recover at least symbols. Note that a failed node can be recovered by contacting the remaining two nodes in its column by downloading one packet from each of them. Thus, . This implies that the code is locally recoverable. By applying a similar case analysis for the failure patterns we can conclude that the code is resilient to 5 failures and it meets the minimum distance bound in Lemma 2.

Ii-a Summary of Contributions

In this work we present several constructions of FR codes. The contributions of our work can be summarized as follows. We construct a large class of FR codes for from combinatorial structures such as grids, mutually orthogonal Latin squares (MOLS), resolvable designs and Hadamard designs. These were first presented in the literature in the conference version of the current manuscript [14]. While [7] presented constructions based on Steiner systems, our work presents a rigorous analysis of the file size of the corresponding DSS. The Kronecker product technique for generating new DSS from existing ones is also new [15]. Furthermore, our conference paper [16] was the first to present locally recoverable FR codes where .

Tables IV contain a description of the various constructions and the corresponding DSS parameter values that can be achieved by these constructions. We defer an in-depth discussion of these parameters to the respective sections. However, we highlight the key contributions of our work by referring to appropriate rows of Tables IV below. Specific details about the construction techniques can be found in the corresponding sections of the paper.

  • We construct a large class of FR codes based on resolvable designs [12] where the repetition degree () of the symbols can be varied in an easy manner (see Table I (rows 3 – 5) and Table III). The constructions of [7] lack this flexibility as they are mostly based on Steiner systems where the repetition degree is usually fixed by the construction.

  • We construct FR codes where , i.e., the new node downloads more than one packet from the surviving nodes. We emphasize that starting with a FR code with , it is trivially possible to arrive at a code with by trivial -expansion (cf. Definition 11). However, such a strategy only results in a limited range of system parameters that can be achieved. We present several codes (see Tables III and IV) that achieve certain parameter ranges that cannot be achieved in a trivial manner.

  • Determining the file size that can be supported by a given FR code turns out be challenging. Much of the literature in combinatorial designs only discusses the pairwise overlaps between the content of the different storage nodes. However, the file size depends on the union of all subsets of storage nodes of size . In this work we determine the file sizes for most of our constructions. In particular, we demonstrate a family of FR codes whose file size is strictly larger than a simple lower bound that is obtained by applying the inclusion-exclusion principle (see row 1, Table III). We also determine the file size for a large class of codes obtained from Steiner systems that were originally considered in [7] (see row 2, Table I). Several of our constructions are shown to meet the Singleton bound for specific file sizes, which demonstrates their optimality.

  • We present the Kronecker product as a technique for constructing new FR codes from existing ones (Table IV) and analyze the properties of codes thus obtained.

  • In this work, we propose a large family of locally recoverable FR codes where , i.e., the repair degree is strictly smaller than the number of nodes contacted for recovering the stored file. We derive an appropriate minimum distance bound for our class of codes that enjoy local, exact and uncoded repair, and demonstrate constructions that meet these bounds (Table V).

Ii-B Discussion of related work

The work of Dimakis et al. [2] initiated the work on regenerating codes, by demonstrating the tradeoff between the storage capacity of nodes and the repair bandwidth. Their work considered functional repair, where the new node is functionally equivalent to the failed node and demonstrated that random network coding suffices for achieving this tradeoff. Following this, several papers [17, 8, 18, 19, 20, 21, 22, 7, 14] considered the construction of exact repair regenerating codes, where the new node is an exact copy of the failed node. In most cases, these constructions either operate at the minimum storage regenerating (MSR) point [17, 21, 23, 22, 18] or the minimum bandwidth regenerating (MBR) point [17, 8, 24, 7, 14]. More recently, codes with local repair have been investigated where the metric for repair is the number of surviving nodes that are contacted for repair [3, 5, 4, 9, 25, 16].

Constructions of repair-by-transfer codes, where node repair is performed simply by downloading symbols from surviving nodes was first presented in the work of [24] where they constructed a repair-by-transfer MBR code with . Repair by transfer codes have also appeared in [26, 27]. The work of [7] also considered such codes (termed “exact and uncoded repair”) but with a repair degree that can be strictly smaller than . The repair operates by contacting a specific set of surviving nodes and is hence table based. Reference [7] introduced the system architecture whereby an MDS code is applied to a file consisting of symbols to obtain symbols. These symbols are then placed onto the storage nodes and this placement is referred to as the fractional repetition (FR) code. The codes in [7], were derived from Steiner systems. They provided lower and upper bounds on the corresponding file sizes. Following this, the work of [28] constructed FR codes from bipartite cages. These codes enjoy the property that the node storage capacity is much larger than the replication degree. For the given parameters they design codes with the smallest number of storage nodes. In [28], they used MOLS to construct bipartite cages and the codes thus obtained are different from ours. In our construction we obtain the storage nodes directly from the set of MOLS and also obtain net FR codes. Reference [29] presents necessary and sufficient conditions on the existence of a FR code with certain parameters; however, it does not consider the issue of determining the file size for a given .

The work of [30] presents several FR code constructions based on combinatorial structures including regular and biregular graphs, graphs with a given girth, transversal designs, projective planes and generalized polygons. They consider codes where and and show that the file size of their constructions meets the upper bound presented in [7] for . This work is closely related to the content of Section III of our work. Their construction of FR codes from transversal designs treats the blocks of the transversal design as symbols. Thus, it can be considered as working with the transpose of the incidence matrix corresponding to the original transversal design. Our FR codes in Section III are obtained from nets which can also be viewed as transposes of transversal designs. However, as discussed in Section III-C, the analysis of file size for our constructions cannot be obtained from the results in [30]. Our work differs in the sense that we present constructions with non-trivial values, Kronecker product constructions and local FR codes.

The problem of local repair for scalar codes () was first considered in [3]. This was extended to vector codes () in [9, 4]. References [9, 4] study the tradeoff between locality and minimum distance and corresponding code constructions. In [9], the authors presented constructions that use the repair-by-transfer MBR codes of [24] as individual components. Local codes were also studied in [31] where the design consists of an outer Gabidulin encoder followed by inner local MBR encoders. This work (see Construction III.1 in [31]) also provides examples of local FR codes by using -designs. However, the achievable parameters are limited as needs to be chosen to be at most and explicit constructions of -designs for large are largely unknown (when there are only finitely many known explicit constructions [10]). In Section V we focus on regenerating codes that allow a repair process in a local manner by simply downloading packets from the surviving nodes. We provide an upper bound for the minimum distance and constructions of codes which meet this bound. Our constructions use local FR codes instead of repair by transfer MBR codes. We also note that our codes are quite different from those that appear in [31, 9] and allow for a larger range of code parameters. Regenerating codes using -designs were also presented in [19]. The architecture of the codes consists of a layered erasure correction structure that ensures a simple decoding process. These codes are showed to be achieve performance better than time-sharing between MBR and MSR points.

Iii Construction of FR codes when

In this section we present the construction of FR codes where . As discussed in Example 2 it is possible that certain set systems do not satisfy the property of -recoverability and hence cannot be used to construct FR codes. However, there are a large class of combinatorial designs that can be used to construct FR codes. In particular, we present various constructions of FR codes that are derived from balanced incomplete block designs (BIBDs) and resolvable designs. Our constructions address several issues that exist with prior constructions in the literature. For instance, resolvable designs allow the repetition degree of the symbols in the FR code to be varied in a simple manner, a flexibility that prior constructions typically lack. We present a large class of codes that cannot be obtained via trivial -expansion.

Our first set of constructions are FR codes based on Steiner systems with (that are BIBDs) which have been previously considered in the literature [7]. However, to our best knowledge, prior work does not provide results on the file size of the constructions. In the discussion below, we present a certain class of Steiner systems for which we can determine the file size of the FR codes obtained from their transpose. To demonstrate the difficulty of determining the file size for a general Steiner system, we first discuss two non-isomorphic Steiner systems with the same parameter values that result in FR codes with different file sizes. This demonstrates that file size calculations for Steiner systems cannot be performed just based on the system parameters. Accordingly, we consider Steiner systems that have maximal arcs [32, 33]. It turns out that we can determine the file size of the corresponding transposed codes.

Iii-a FR codes from Steiner systems

We consider Steiner systems . Note that the repetition degree of any symbol is and any two distinct symbols are contained in exactly one node. Consider the FR code obtained from it and its transpose.

In general, it is a challenging task to find the file size for a given FR code. For codes obtained from Steiner systems and their transposes, lower bounds based on the inclusion-exclusion principle were presented in [7]. However, it is important to note that the file size depends critically on the structure of the Steiner system, i.e., two Steiner systems with the same parameters can have different file sizes. To see this, consider two non-isomorphic Steiner systems denoted and ; the nodes of these designs are provided in Tables VI and VII. These designs can also be found in [10].

Let be a subset of symbols of the design such that no -subset of is contained in a node. By checking all subsets of the symbol set one can observe that the maximum size of in and equals () and () respectively.

This observation results in different file sizes in the codes obtained from the transposes of and , denoted and respectively. In fact for , the design yields a code which has file size which matches the inclusion-exclusion lower bound given by . However, the design yields a code with file size which is strictly larger111This example corrects an error in Lemma 11 of [7]..

We now elaborate on the role of in the above example. Firstly, note that if is a Steiner system, then any two storage nodes in intersect in one symbol. Consider the corresponding transposed codes and , where the roles of symbols and nodes is now reversed. As for is of size , it implies that we can pick storage nodes in such that the intersection of any three storage nodes is empty (owing to the definition of ). Thus, upon applying the inclusion-exclusion principle, we obtain the file size to be .

In contrast, the maximum size of in is . Thus, for any set of storage nodes in there is at least one three-way intersection that is non-empty. Upon exhaustive enumeration, one can realize that the file size in this case is which is strictly higher than .

The notion of the set introduced above can be formalized in terms of a maximal arc in Steiner systems. For Steiner systems that possess a maximal arc, we can therefore determine the file size. In addition, prior results in [32, 33], demonstrate that such maximal arcs exist in a large class of Steiner systems. In the discussion below, we make these arguments in a formal manner.

TABLE VI: Nodes of the Steiner system
TABLE VII: Nodes of the Steiner system
Definition 12 (-arc)

Let be a design. A subset with is called an -arc if for each node either or holds.

The definition of -arc implies that any three symbols from are not contained in any node in . The largest set with this property is called a maximal arc of the design [34]. It turns out that we can determine the file size for FR codes obtained from transposes of Steiner systems with nontrivial maximal arcs.

TABLE VIII: Nodes of the Steiner system
TABLE IX: Transposed code obtained from the Steiner system

For a maximal arc , consider a symbol . In this case there are pairs of symbols such that . Since is a Steiner system and is a maximal arc there are distinct nodes in where each of these pairs occurs. Now, the repetition degree of the system is . Thus, there are nodes which contain the symbol but no other symbol from . Based on our assumption, each node in is such that either or . Thus, it has to be the case that .

Lemma 3

Let be a FR code derived from a Steiner system with , such that it has a maximal arc of size . Then, the transposed FR code is such that its code rate is for .

Proof:  In the transposed code , consider any subset of nodes of size , where . As any two symbols in the original code occur in exactly one node of it holds that two nodes and in are such that . In addition, the storage capacity of the nodes in is equal to .

Using the inclusion-exclusion principle (cf. Theorem 1), we observe that these nodes cover at least symbols in . Now we pick a set of nodes in that correspond to a subset of the maximal arc in . Based on the argument above, it is clear that any two of these nodes intersect in exactly one symbol and any of the nodes have an empty intersection if . It follows that the union of these nodes has exactly symbols. The result follows.

Next we provide an explicit example. Let be the FR code obtained from a Steiner system .

Example 5 (File size of FR code obtained from the transpose of Steiner System )

The nodes in are specified in Table VIII and the nodes of the transposed code are specified in Table IX.

Since the maximal arc should be a set of with cardinality , we can choose the symbols greedily and construct the set as a maximal arc for this Steiner system

According to Lemma 3, the file size for for can be determined by just considering the nodes

as these correspond to the symbols of in . For these values of , the file size of the code is . Moreover, it is optimal with respect to Singleton bound for (cf. Observation 1).

Remark 1 (Steiner Systems with )

It is known that several Steiner systems possess maximal arcs. Here we provide the known results for small values of .

  • (Maximal arcs in Steiner systems with ) By Skolem’s construction [35] we have for all and . Moreover, for all and there exists a Steiner system with at least one maximal arc [32].

  • (Maximal arcs in Steiner systems with ) It is known [12] that Steiner systems with exist if and only if and

    Furthermore, if is a prime power, then there exists an Steiner system with a maximal arc of size [33].

To our best knowledge, there are no other general results about the existence of maximal arcs in Steiner systems with higher values of .

Iii-B FR codes from resolvable designs

A major drawback of FR codes obtained from Steiner systems is that the repetition degree of the symbols is quite inflexible. In particular, it is not possible to vary the repetition degree and hence the failure resilience of the DSS in an easy way. To address this issue, we now introduce FR codes that are derived from resolvable designs.

A design is said to be resolvable if we can divide the blocks in into equal-sized partitions such that (a) each partition contains all the symbols in , and (b) the blocks in a given partition have no symbols in common. Under certain conditions, these designs also allow for -recoverability. A FR code obtained from such a design is called a resolvable FR code and is naturally resilient to any failure pattern that ensures that at least one partition is left intact. In the discussion below, we introduce the notion of a net FR code (a subclass of resolvable FR codes) that ensures -recoverability.

Under this overall framework, we construct several families of net FR codes that allow us to vary the repetition degree in an easy manner. We demonstrate that there exist net FR codes with that cannot be derived by trivial -expansion. Furthermore, we answer an open question of [7] by demonstrating a FR code that cannot be constructed from Steiner systems. We also provide explicit calculations of the file size for certain ranges of . The overall structure of this subsection is as follows. We first introduce our construction, show that it results in a net FR code and then calculate its file size.

Definition 13 (Resolvable FR Code)

Let where be a FR code. A subset is said to be a parallel class if for and with we have and . A partition of into parallel classes is called a resolution. If there exists at least one resolution then the code is called a resolvable FR code.

For a resolvable FR code, we call two storage nodes parallel if they belong to the same parallel class and non-parallel otherwise. The properties of a resolvable FR code are best illustrated by means of the following example.

Example 6

Consider a DSS with parameters and . Suppose that we arrange the symbols in in a array shown below.

Let the rows and the columns of form the nodes in the FR code (see Fig. 5), thus . It is evident that there are two parallel classes in , (corresponding to rows) and (corresponding to columns). As , this code can tolerate one failure.

By our construction it is evident that for and , we have . Using this we can compute the file size when , as follows. Let with . Then, the number of distinct symbols in a set of nodes from is

where nodes are from and