Optimizing relinearization in circuits for homomorphic encryption
Fully homomorphic encryption (FHE) allows an untrusted party to evaluate arithmetic circuits, i.e., perform additions and multiplications on encrypted data, without having the decryption key.
One of the most efficient class of FHE schemes include BGV/FV schemes, which are based on the hardness of the RLWE problem. They share some common features: ciphertext sizes grow after each homomorphic multiplication; multiplication is much more costly than addition, and the cost of homomorphic multiplication scales linearly with the input ciphertext sizes. Furthermore, there is a special relinearization operation that reduce the size of a ciphertext, and the cost of relinearization is on the same order of magnitude as homomorpic multiplication. This motivates us to define a discrete optimization problem, which is to decide where (and how much) in a given circuit to relinearize, in order to minimize the total computational cost.
In this paper, we formally define the relinearize problem. We prove that the problem is NP-hard. In addition, in the special case where each vertex has at most one outgoing edge, we give a polynomial-time algorithm.
Fully homomorphic encryption (FHE) is an encryption technique which allows any untrusted party to evaluate functions on encrypted data without the decryption key. As a typical application, FHE allows a client to outsource computation to an untrusted cloud. It has generated interest in fields such as health and finance, due to the need to analyze sensitive data without having access to the data itself. Since Gentry introduced the first FHE scheme in 2009, there has been a line of work that proposed new FHE schemes with improved efficiency, among which two of the most widely used schemes are [BGV14] and its scale-invariant counterpart [FV12]. Implementations of these schemes include [HS14], [CLP], and [AMBG16]. There has been numerous work that design applications based on these schemes. Some of them ([GBDL16, BCIV17]) evaluate machine learning models on encrypted data. Others use FHE to design secure protocols such as private information retrieval [MBFK16] and private set intersection [CLR17].
Unfortunately, in these schemes homomorphic operations are still several-orders of magnitude slower than performing the same operation on plaintexts. Therefore, any optimization in the computation time has great interest. In order to use FHE to evaluate a function, one first needs to express the function as an arithmetic circuit. The circuit is represented as a direct acyclic graph with each vertex being either an input, an output, or an arithmetic operation such as multiplication and addition. In both schemes mentioned above, a fresh ciphertext is a pair of polynomials. When a homomorphic multiplication is performed, the length of the output ciphertext grows. More precisely, if we denote the length of a ciphertext by , then . The length of the result of a homomorphic addition is the maximum length of the two operands, i.e., .
Rouhgly speaking, the computational cost to perform a homomorphic multiplication scales linearly with its input lengths. In both schemes, we can model the amount of work it takes to perform a homomorphic multiplication between two ciphertexts and by
where is some scheme-dependent constant. In FHE, there is also a squaring operation, which takes as input an encryption of and returns an encryption of . It has the same cost 111Actually, the cost of squaring is a constant factor smaller than multiplying with itself. For simplicity, we will assume that the costs are equal. This simplification does not invalidate the results. and length effect as multiplication, but only takes one input.
Homomorphic additions, on the other hand, takes much less time to perform compared to multiplication. Hence in this work we will assume that additions are “free”. For the same reason, we will adopt the common notation from the FHE literature, and denote by depth of a circuit by the largest number of multiplication vertices contained in a path.
Note that it is undesirable to let the ciphertext sizes grow, since it will increase both the computational cost and the storage burden. To control the ciphertext sizes, both schemes support a special operation called Relinearization. Effectively, relinearizing a ciphertext means reducing its length, while keeping the underlying message the same. We can use this operation to reduce the length of a ciphertext to any integer between two and its original length. The cost of relinearization scales linearly with the reduction in ciphertext length. In other words, there exists a constant such that reducing the ciphertext lengths by takes units of work.
Suppose we are given an arithmetic circuit to perform on encrypted inputs. It is now an optimization problem to decide where and how much to relinearize, in order to minimize the total amount of work, consisting of multiplication cost and relinearization cost. Previous works employ the simple strategy of relinearizing after every multiplication/squaring. In this way, the multiplication costs are kept minimal. However, this strategy is not always optimal, as we will demonstrate in Section 2.1.
In Section 2, we will formally describe the problem and show why this simple strategy can be sub-optimal. In Section 3, we prove that the relinearize problem is NP-hard by reducing from the knapsack problem. Finally, in Section 4, we restrict to the special case where each vertex in the circuit has at most one outgoing edge, and give a polynomial time algorithm.
1.2 Related work
The work [CDS15] is an effort to find a good circuit representation of a function, in order to minimize the total computation time.
Bootstrapping is an operation that refreshes the so-called noise in FHE ciphertexts. It is an essential yet expensive operation. The two papers [LP13] and [BLMZ17] aim at minimizing the total number of bootstrapping operations in a circuit, while keeping the noise from overflowing in order to ensure the final result is correct. In their work, the authors implicitly assume the relinearization is done after every multiplication. Similarly, we will make a simplifying assumption that the boostrapping time is a constant, so that it does not factor into our optimizatoin problem. It will be interesting to combine these works in order to achieve an overall optimization that targets both operations.
Acknowledgement The author thanks Rebecca Hoberg and Mohit Singh for helpful discussions in preparing this work. We modify the usual definition of the arithmetic circuits to include the squaring operations in FHE.
2 Problem Description
To formally describe our problem, we need to properly define circuits used in FHE applications.
a (squaring-enabled) arithmetic circuit is a directed acyclic graph , where there are three kinds of vertices: input vertices has indegree 0 and outdegree 1; output vertices has indegree and outdegree 0; add/multiply operation vertices have indegree 2 and outdegree 1; finally, square operation vertices have indegree and outdegree both equal to 1.
We will define the relinearize problem as an integer programming problem on arithmetic circuits. For every vertex , we maintain an integer variable (the final length of vertex during homomorphic evaluation of ), and an integer variable , which indicates the amount of relinearization at . We will denote the two parents of a vertex by and . If is a squaring vertex, then we set . We denote addition vertices by and multiplication/square vertices by . To resolve ambiguity, we make the convention that if a vertex has two distinct parents, then it is understood as a multiplication; otherwise it is a squaring.
Then the relinearize problem on is
2.1 An example
To demonstrate the non-trivality of the relinearize problem, we consider the following circuit:
First, we apply the simple strategy and relinearize at every multiplication vertex. Then the total cost is equal to . Alternatively, we can choose to only relinearize the vertex . Then the multiplication cost increases to , while the relinearization cost is , so the total cost is . Comparing this with the previous cost, we see that as long as , the simple strategy is not optimal.
3 NP-hardness of the Relinearize Problem
We prove a polynomial reduction from the knapsack problem to the relinearize problem, which establishes that the latter problem is NP-hard. First we recall the definition of knapsack problem.
Given positive integers , and . The (0-1) knapsack problem is:
For our convenience, we make some modifications to the setting of the relinearize problem. We change the inputs lengths from two to one, and we modify the equation to . One can check that under this modificaiton, the length of every vertex is smaller by one. Hence the modified problem is equivalent to the original problem.
To prepare for the main theorem, we make some convenient definitions.
A circuit is of type if it consists of one input vertex, one output vertex, and multiplication/squaring vertices, such that if the first non-input vertex length is reduced from 2 to 1, then the length of the output vertex reduces by .
Figure 1 is an example of .
For all integers , there exists a circuit of type which has at most vertices. Moreover, the cost to evaluate this circuit is bounded above by .
If is a power of 2, we can realize by a circuit that does consecutive squarings. The total cost of executing the circuit is . In general, we can start by building the circuit . Then for every nonzero bit in the binary representation of , we need to add a multiplication vertex. Since there are at most bits, we know the number of vertices is at most .
As for the evaluation cost, note that each vertex in the circuit has length bounded above by , hence evaluating it has cost bounded by . The claim follows because there are at most vertices. ∎
Next we describe some simple ways to construct new circuits from old ones.
(1) The addition/multiplication of two circuits. Take two circuits and with unique output vertices and . Then (resp. ) is the circuit that is the union of and , plus an extra addition (resp. multiplication) vertex that has and as parents. See Figure 2 for an example.
(2) The concatenation of two circuits. Let be two circuits such that the number of output vertices of is equal to the number of inputs of . Then we simply “feed” the outputs of to inputs of . We denote the resulting circuit by . See Figure 3 for an example.
(3) The -repeat of a circuit along a subset of vertices. Let be a circuit and let be vertices of . Let be a positive integer. Then we keep the vertices and all their ancestors, and copy the rest of the circuit times. The resulting circuit is denoted by . See Figure 4 for an example.
(4) The gluing of two circuits along a subset of vertices. Let and be two circuits and be subsets of their vertices, such that the subgraph of consisting of ancestors of (including vertices in ) is isomorphic to the corresponding subgraph in . Then the gluing of and along is the circuit that contains the common subgraph and the disjoint union of the rest of the two graphs. We denote the new circuit by when and the isomorphism is clear from context. See Figure 5 for an example. Note that (3) is a special case of (4).
Now we are ready to state our main theorem. Consider a knapsack problem with parameters and .
There exists a circuit , and integers such that
(1) has vertices.
(3) There exists a set of vertices in , such that if the length is the length of in an optimal solution to the relinearize problem on . Then is an optimal solution to
Hence is an optimal solution to the original knapsack problem.
Since our proof is long, we will break it into several parts. First, let be positive integers whose values will be determined later. We define a circuit
Here , and , where is the first non-input vertex in the circuit . In particular, with no relinearization the length of is equal to 2. Consider the relinearize problem on the circuit and let be the new lengths of . Without loss of generality, we assume that for all (if , then any optimal solution of the knapsack problem always have , and we can reduce the dimension of the problem by one).
and . Then for any optimal solution to the relinearize problem on , the only vertices that could have nonzero relinearization are the .
By Lemma 1, the total cost of evaluating a circuit of type is bounded by , hence relinearizing any single vertex in this circuit has benefit bounded by . The situation is similar for . Note that relinearizing verteices in could reduce the length of vertices in , but the benefit is still bounded above by . For the same reason, the benefit of relinearizing any vertex in any of the copies of is bounded by . Since , this completes the proof. ∎
Suppose . Then for any optimal solution to the relinearization problem on we must have
Here again we recall that denote the length of in an optimal solution.
Suppose the claim is false. Then there exists such that . We relinearize the vertex , which reduces the length of the final output in each copy of by , and the length of the output vertex of
is reduced by . Since , the length of the input vertex in each is reduced by at least one, and the cost reduction from each is at least . Hence we the benefit we collect from relinearizing is at least , whereas the cost is . Since we assumed , we know relinearizing the vertex reduces the total cost. This is a contradiction, since we started with an optimal solution. ∎
Now we can starting proving Theorem 1.
(of Theorem 1) Let . We take , , and . It is easy to see that are of size polynomial in . One can verify that and . Thus, by Lemma 3, we have if are the new length of in any optimal solution to the relinearization problem on . This means we have the correct constraint. However, the costs are wrong: the total cost of evaluating the circuit is given by
Note that the coefficient before is equal to , and we want to modify this coefficient to . First, note that
Let . We claim that there exists a circuit of such that relinearizing its first non-input vertex reduces the total multiplication cost by . We omit the details of construction of since it is similar to that of . In particular, the can be constructed with at most vertices. We then let
and set . Since , one can see that in any optimal solution of the relinearize problem on , the vertices in have zero relinearization. Thus, the relinearize problem on is equivalent to
which is equivalent to
This proves part (3) of Theorem 1. Part (1) is clear since the number of vertices in is bounded by . Hence it is logarithm in the parameters and linear in the number of variables . For (2), note that we set , so it suffices to prove it for . By construction, is also bounded by a polynomial in . This completes the proof. ∎
The relinearize problem is NP-hard.
4 An Simple Case
Assume we are in the situation where each non-input vertex in the circuit has two inputs and at most one output. In this case, we have a polynomial time algorithm for the relinearize problem. For a vertex , define to be the minimal cost to compute the circuit up to vertex , so that the new length of is .
Recall that and denote the parents of . If is a multiplicative vertex, we have
If is an addition vertex, we have
Here it is important that the vertices all only have a single output, since otherwise and might have a common ancestor, in which case relinearizing this ancestor might benefit both of them.
Suppose . Then in the above formulae, it suffices to take the minimum over range .
For the input vertices, the lengths is at most . For any non-input vertex , we prove inductively that its length cannot exceed its number of ancestors. The length is at most , and by inductive hypothesis, both and are at most their number of ancestors (or plus one if it happens to be an input vertex). That is, . Here denote the number of ancestors for , respectively. ∎
Now our algorithm proceeds as follows. We traverse the vertices. At each vertex, we compute for values of , and each computation requires operations. Thus the total running time is . Finally, the optimal cost is given by , where is the output node of the graph .
5 Conclusion and Future Work
Fully homomorphic encryption evaluates boolean circuits, and relinearization is a standard technique to reduce the ciphertext sizes after evaluation. In this paper, we consider the goal of optimizing where and how much to perform the relinearization operation in any given circuit, in order to minimize the total computational cost. We formalized it as a discrete optimization problem, and proved that the problem is NP-hard. In the special case where every node has at most one ouptut node, we give a polynomial time algorithm.
For future directions, it is of interest to design fast approximate algorithms for the relinearization problem. Also, one can aim at optimizing specific circuits that appear in the literature for applications of FHE. Examples include components of the AES encryption/decryption circuit and machine learning models such as logistic regression or neural network.
- [AMBG16] Carlos Aguilar-Melchor, Joris Barrier, Serge Guelton, Adrien Guinet, Marc-Olivier Killijian, and Tancrede Lepoint. Nfllib: Ntt-based fast lattice library. In Cryptographers’ Track at the RSA Conference, pages 341–356. Springer, 2016.
- [BCIV17] Joppe W Bos, Wouter Castryck, Ilia Iliashenko, and Frederik Vercauteren. Privacy-friendly forecasting for the smart grid using homomorphic encryption and the group method of data handling. In International Conference on Cryptology in Africa, pages 184–201. Springer, 2017.
- [BGV14] Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. (Leveled) fully homomorphic encryption without bootstrapping. ACM Transactions on Computation Theory (TOCT), 6(3):13, 2014.
- [BLMZ17] Fabrice Benhamouda, Tancrède Lepoint, Claire Mathieu, and Hang Zhou. Optimization of bootstrapping in circuits. In Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 2423–2433. SIAM, 2017.
- [CDS15] Sergiu Carpov, Paul Dubrulle, and Renaud Sirdey. Armadillo: a compilation chain for privacy preserving applications. In Proceedings of the 3rd International Workshop on Security in Cloud Computing, pages 13–19. ACM, 2015.
- [CLP] Hao Chen, Kim Laine, and Rachel Player. Simple encrypted arithmetic library-SEAL v2.
- [CLR17] Hao Chen, Kim Laine, and Peter Rindal. Fast private set intersection from homomorphic encryption. IACR Cryptology ePrint Archive, 2017:299, 2017.
- [FV12] Junfeng Fan and Frederik Vercauteren. Somewhat practical fully homomorphic encryption. IACR Cryptology ePrint Archive, 2012:144, 2012.
- [GBDL16] Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In International Conference on Machine Learning, pages 201–210, 2016.
- [HS14] Shai Halevi and Victor Shoup. Algorithms in helib. In International Cryptology Conference, pages 554–571. Springer, 2014.
- [LP13] Tancrède Lepoint and Pascal Paillier. On the minimal number of bootstrappings in homomorphic circuits. In International Conference on Financial Cryptography and Data Security, pages 189–200. Springer, 2013.
- [MBFK16] Carlos Aguilar Melchor, Joris Barrier, Laurent Fousse, and Marc-Olivier Killijian. Xpir: Private information retrieval for everyone. Proceedings on Privacy Enhancing Technologies, 2016:155–174, 2016.