Repair Duality with Locally Repairable and Locally Regenerating Codes

Repair Duality with Locally Repairable and Locally Regenerating Codes

Danilo Gligoroski1, Katina Kralevska1, Rune E. Jensen2, and Per Simonsen3
Email: danilog@ntnu.no, katinak@ntnu.no, runeerle@idi.ntnu.no, per.simonsen@memoscale.com
1Department of Information Security and Communication Technology, NTNU, Norwegian University of Science and Technology 2Department of Computer Science, NTNU, Norwegian University of Science and Technology 3MemoScale AS, Norway
Abstract

We construct an explicit family of locally repairable and locally regenerating codes whose existence was proven in a recent work by Kamath et al. about codes with local regeneration but no explicit construction was given. This explicit family of codes is based on HashTag codes. HashTag codes are recently defined vector codes with different vector length (also called a sub-packetization level) that achieve the optimal repair bandwidth of MSR codes or near-optimal repair bandwidth depending on the sub-packetization level. We applied the technique of parity-splitting code construction. We show that the lower bound on the size of the finite field for the presented explicit code constructions can be lower than the one given in the work of Kamath et al. Finally, we discuss the importance of having two ways for node repair with locally regenerating HashTag codes: repair only with local parity nodes or repair with both local and global parity nodes. To the best of the authors’ knowledge, this is the first work where this duality in repair process is discussed. We give a practical example and experimental results in Hadoop where we show the benefits of having this repair duality.

Keywords: Vector codes, Repair bandwidth, Repair locality, Exact repair, Parity-splitting, Hadoop.

I Introduction

The repair efficiency of erasure coding in distributed storage systems is measured with two main metrics: the amount of transferred data during a repair process (repair bandwidth) and the number of accessed nodes in a repair process (repair locality). Two types of erasure codes that are optimal with respect to these two metrics have emerged: Regenerating Codes (RCs) [1] and Locally Repairable Codes (LRCs) [2, 3, 4].

Minimum Storage Regenerating (MSR) codes are an important class of RCs that minimize the amount of data stored per node and the repair bandwidth. Explicit constructions of MSR codes can be found in [5, 6]. LRCs relax the maximum distance separable (MDS) requirement in order to minimize the number of nodes accessed during a repair. Studies on implementation and performance evaluation of LRCs can be found in [7, 8, 9, 10]. Combining the benefits of RCs and LRCs in one storage system can bring huge savings in practical implementations. For instance, repair bandwidth savings by RCs are important when repairing huge amounts of data, while a fast recovery and an access to small number of nodes enabled by LRCs are desirable for repair of frequently accessed data.

Several works present code constructions that combine the benefits of RCs and LRCs [11, 12, 13]. Rawat et al. in [11] and Kamath et al. in [12] have independently investigated codes with locality in the context of vector codes, and they call them locally repairable codes with local minimum storage regeneration (MSR-LRCs) and codes with local regeneration, respectively. Rawat et al. [11] provided an explicit construction, based on Gabidulin maximum rank-distance codes, of vector linear codes with all-symbol locality for the case when the local codes are MSR codes. However, the complexity of these codes increases exponentially with the number of nodes due to the two-stage encoding. In [12], Kamath et al. gave an existential proof without presenting an explicit construction. Another direction of combining RCs and LRCs is to use repair locality for selecting the accessed nodes in a RC [13], while an interpretation of LRCs as exact RCs was presented in [14]. Two different erasure codes, product and LRC codes, are used to optimize for recovery performance and reduce the storage overhead in [15].

Our Contribution: We construct an explicit family of locally repairable and locally regenerating codes whose existence was proven in a recent work by Kamath et al. [12] about codes with local regeneration. In that work, an existential proof was given, but no explicit construction was given. Our explicit family of codes is based on HashTag codes [5, 6]. HashTag codes are MDS vector codes with different vector length (also called a sub-packetization level) that achieve the optimal repair bandwidth of MSR codes or near-optimal repair bandwidth depending on the sub-packetization level. We apply the technique of parity-splitting of HashTag codes in order to construct codes with locality in which the local codes are regenerating codes and which hence, enjoy both advantages of locally repairable codes as well as regenerating codes.

We also show (although just with a concrete example) that the lower bound on the size of the finite field where these codes are constructed, given in the work by Kamath et al. [12], can be lower. The presented explicit code construction has a practical significance in distributed storage systems as it provides system designers with greater flexibility in terms of selecting various system and code parameters due to the flexibility of HashTag code constructions.

Last but not least, we discuss the repair duality and its importance. Repair duality is a situation of having two ways to repair a node: to repair it only with local parity nodes or repair it with both local and global parity nodes. To the best of the authors’ knowledge, this is the first work that discusses the repair duality and how it can be applied based on concrete system and code parameters.

The paper is organized as follows. Section II presents mathematical preliminaries. In Section III, we describe a framework for explicit constructions of locally repairable and locally regenerating codes. The repair process is analyzed in Section IV where we explain the repair duality. Experimental results of measurements in Hadoop are given in Section V. Conclusions are summarized in Section VI.

Ii Mathematical Preliminaries and Related Work

Inspired by the work of Gopalan et al. about locally repairable codes [2], Kamath et al. extended and generalized the concept of locality in [12]. In this paper, we use notation that is mostly influenced (and adapted) from those two papers.

Definition 1 ([12])

A -linear vector code of block length is a code having a symbol alphabet for some , i.e.

and satisfies the additional property that for given and ,

also belongs to where is a scalar multiplication of the vector .

Throughout the paper, we refer to the vectors as vector symbols or nodes. Working with systematic codes, it holds that for the systematic nodes for and for the parity nodes for . For every vector code there is an associated scalar linear code over of length . Accordingly, the dimension of the associated scalar code is . For a convenient notation, the generator matrix of size of the scalar code is such that each of the consecutive columns corresponds to one code symbol , and they are considered as thick columns . For a subset we say that it is an information set for if the restriction of to the set of thick columns with indexes lying in has a full rank, i.e. .

The minimum cardinality of an information set is referred as quasi-dimension of the vector code . As the vector code is -linear, the minimum distance of is equal to the minimum Hamming weight of a non-zero codeword in . Finally, a vector code of block length , scalar dimension , minimum distance , vector-length parameter and quasi-dimension is shortly denoted with . While in the general definition of vector codes in [12] the quasi-dimension does not necessarily divide the dimension of the associated scalar, for much simpler and convenient description of the codes in this paper we take that , i.e. . In that case the erasure and the Singleton bounds are given by:

(1)

In [1], Dimakis et al. studied the repair problem in a distributed storage system where a file of symbols from a finite field is stored across nodes, each node stores symbols. They introduced the metric repair bandwidth , and proved that the repair bandwidth of a MDS code is lower bounded by

(2)

where is the number of accessed available nodes (helpers).

Lemma 1 ([1])

The repair bandwidth of a MDS code is minimized for . MSR codes achieve the lower bound of the repair bandwidth equal to

(3)

A MSR code has the maximum possible distance in addition to minimizing the repair bandwidth, but it has the worst possible locality.

Corollary 1

The locality of a MSR code is equal to .

Any vector code is MDS if and only if its generator matrix can be represented in the form , where the parity matrix

(4)

possesses the property that every square block submatrix of is invertible. The entries are square sub-matrices of size , and a block submatrix is composed by different entries of .

In order to analyze codes with local regeneration, Kamath et al. introduced a new family of vector codes called uniform rank-accumulation (URA) codes in [12]. They showed that exact-repair MSR codes belong to the class of URA codes.

Definition 2

[12, Def. 2] Let be a vector code with a generator matrix . The code is said to have information locality if there exists a set of punctured codes of with respective supports such that

  • and

If we put in Def.2, then we get the definition of information locality introduced by Gopalan et al. [2]. They derived the upper bound for the minimum distance of a code with information locality for as

(5)

A general upper bound was derived in [12] as

(6)

Huang et al. showed the existence of Pyramid codes that achieve the minimum distance given in (5) when the field size is big enough [7]. Finally, based on the work by Gopalan et al. [2] and Pyramid codes by Huang et al. [7], Kamath et al. proposed a construction of codes with local regeneration based on a parity-splitting strategy in [12].

Iii Codes with Local Regeneration from HashTag Codes by Parity-Splitting

In [5, 6], a new class of vector MDS codes called HashTag codes is defined. HashTag codes achieve the lower bound of the repair bandwidth given in (3) for , while they have near-optimal repair bandwidth for small sub-packetization levels. HashTag codes are of a great practical importance due to their properties: flexible sub-packetization level, small repair bandwidth, and optimized number of I/O operations. We briefly give the basic definition of HashTag codes before we construct codes with local regeneration from them by using the framework of parity-splitting discussed in [12].

Definition 3

A HashTag linear code is a vector systematic code defined over an alphabet for some . It encodes a vector , where for , to a codeword where the systematic parts for and the parity parts for are computed by the linear expressions that have a general form as follows:

(7)

where and the index pair is defined in the -th row of the index array . The index arrays are defined as follows:

where the values of the indexes are determined by a scheduling algorithm that guarantees the code is MDS, i.e. the entire information can be recovered from any out of the vectors .

One scheduling algorithm for Def. 3 is defined in [5, 6].

Example 1

The linear expressions for the parity parts for a HashTag code with are given here. The way how we obtain them is explained in Section 4.1 in [6]. We give one set of coefficients for equation (7) from the finite field with irreducible polynomial . This code achieves the lower bound of repair bandwidth in (3), i.e. the repair bandwidth is for repair of any systematic node.

+ + + + +
+ + + + +
+ + + + +
+ + + + +
+ + + + +
+ + + + +
+ + + + +
+ + + + +
+ + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
+ + + + + + +
Fig. 1: There are parity nodes from a systematic MDS code with a sub-packetization level . The parity splitting technique generates from every parity node local parity nodes and renames the parity nodes as global parity nodes .

We adapt the parity-splitting code construction for designing codes with local regeneration described in [12] for the specifics of HashTag codes. The construction is described in Construction 1. For simplifying the description, we take some of the parameters to have specific relations, although it is possible to define a similar construction with general values of the parameters. Namely, we take that and . We also take that the parameters for the information locality are such that and .

Construction 1

An input to the construction is a HashTag MDS code with a sub-packetization level . This input comes with the associated linear parity equations (7), i.e. with the associated systematic generator matrix . The MDS code can be, but it does not necessarily have to be a MSR code. Another input is the information locality that the constructed code with local regeneration will have. The following steps are performed:

Step 1.

Split systematic nodes into disjunctive subsets , where every set has nodes. While this splitting can be arbitrary, take the canonical splitting where , , , .

Step 2.

Split each of the linear equations for the first parity expressions (7) into sub-summands where the variables in each equation correspond to the elements from the disjunctive subsets.

Step 3.

Associate the obtained sub-summands to new local parity nodes.

Step 4.

Rename the remaining parity nodes that were not split in Step 1 - Step 3 as new global parity nodes.

Step 5.

Obtain a new systematic generator matrix from the local and global parity nodes.

Step 6.

Return as a generator matrix of a vector code with information locality .

A graphical presentation of the parity-splitting procedure is given in Fig. 1.

Theorem 1

If the used MDS HashTag code in Construction 1 is MSR, then the obtained code with information locality is a MSR-Local code, where

(8)

Proof: (Sketch) Since in Construction 1 we took that and , it means that the scalar dimension of the code is for some integer . Then the proof continues basically as a technical adaptation of the proof of Theorem 5.5 that Kamath et al. gave for the pyramid-like MSR-Local codes constructed with the parity-splitting strategy in [12].       

Note that if , then HashTag codes are sub-optimal in terms of the repair bandwidth. Consequently, the produced codes with Construction 1 are locally repairable, but they are not MSR-Local codes.

Example 2

Let us split the MSR code given in Example 1 into a code with local regeneration and with information locality . In Step 1 we split 6 systematic nodes into disjunctive subsets and . According to Step 2 of Construction 1, the first global parity in Example 1 is split into two local parities and as follows:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
The remaining two global parities are kept as they are given in Example 1, they are only renamed as and . The overall code is a code or with the terminology from [8] it is a code.

Example 3

Let us split the same MSR code now with parameters . In Step 1 we split 6 systematic nodes into disjunctive subsets , and . In Step 2 of Construction 1, the first global parity is split into three local parities: , and as follows:

+ + + + + + + + + + + + + + + + + + + + + + + + + + +
The remaining two global parities are kept as they are given in Example 1, but they are just renamed as and . The overall code is a code or with the terminology from [8] it is a code.

There are two interesting aspects of Theorem 1 that should be emphasized: 1. We give an explicit construction of an MSR-Local code (note that in [12] the construction is existential), and 2. Examples 2 and 3 show that the size of the finite field can be slightly lower than the size proposed in [12]. Namely, the MSR HashTag code used in our example is defined over , while the lower bound in [12] suggests the field size to be bigger than . We consider this as a minor contribution and an indication that a deeper theoretical analysis can further lower the field size bound given in [12].

Iv Repair Duality

Theorem 2

Let be a MSR HashTag code with . Further, let be a code with local regeneration and with information locality obtained by Construction 1. If we denote with the minimum repair bandwidth for single systematic node repair with , then

(9)

Proof:W hen repairing one systematic node, we can always treat local nodes as virtual global nodes from which they have been constructed by splitting. Then with the use of other global nodes we have a situation of repairing one systematic node in the original MSR code for which the repair bandwidth is . On the other hand, if we use the MSR-Local code, then we have the following situation. There are systematic nodes in the MSR-Local code, and the total length of the MSR-Local code is . The file size for the MSR-Local code is decreased by a factor , i.e. it is . If we apply the MSR repair bandwidth for these values we get:

Theorem 2 is one of the main contributions of this work: It emphasizes the repair duality for repairing one systematic node: by the local and global parity nodes or only by the local parity nodes. We want to emphasize the practical importance of Theorem 2. Namely, in practical implementations regardless of the theoretical value of , the number of I/O operations and the access time for the contacted parts can be either crucial or insignificant. In those cases an intelligent repair strategy implemented in the distributed storage system can decide which repair procedure should be used: the one with global parity nodes or the one with the local parity nodes. We illustrate this by the following example.

Example 4

Let us consider the MSR HashTag code given in Example 1 and its corresponding local variant from Construction 1 with information locality given in Example 2. That means that the code with local regeneration has 6 systematic nodes, 2 local and 2 global parity nodes.

Let us analyze the number of reads when we recover one unavailable systematic node. If we recover with the local nodes, then we have to perform 3 sequential reads, reading the whole data in a contiguous manner from 3 nodes. If we repair the unavailable data with the help of both local and global parity nodes, it reduces to the case of recovery with a MSR code, where the number of sequential reads is between 8 and 24 (average 16 reads) but the amount of transferred data is equivalent to 2.67 nodes.

More concretely, let us assume that we want to recover the node .

  1. For a recovery only with the local parity , 3 sequential reads of , and are performed.

  2. For a recovery with the local and global parities:

    1. First, read , and from , and , and from and , and from to recover , and .

    2. Additionally, read , and from and , and from and , and from .

    3. Then, read ,