The Synthesis and Analysis of Stochastic Switching Circuits
Stochastic switching circuits are relay circuits that consist of stochastic switches called pswitches. The study of stochastic switching circuits has widespread applications in many fields of computer science, neuroscience, and biochemistry. In this paper, we discuss several properties of stochastic switching circuits, including robustness, expressibility, and probability approximation.
First, we study the robustness, namely, the effect caused by introducing an error of size to each pswitch in a stochastic circuit. We analyze two constructions and prove that simple series-parallel circuits are robust to small error perturbations, while general series-parallel circuits are not. Specifically, the total error introduced by perturbations of size less than is bounded by a constant multiple of in a simple series-parallel circuit, independent of the size of the circuit.
Next, we study the expressibility of stochastic switching circuits: Given an integer and a pswitch set , can we synthesize any rational probability with denominator (for arbitrary ) with a simple series-parallel stochastic switching circuit? We generalize previous results and prove that when is a multiple of or , the answer is yes. We also show that when is a prime number larger than , the answer is no.
Probability approximation is studied for a general case of an arbitrary pswitch set . In this case, we propose an algorithm based on local optimization to approximate any desired probability. The analysis reveals that the approximation error of a switching circuit decreases exponentially with an increasing circuit size.
In his master’s thesis of 1938, Claude Shannon demonstrated how Boolean algebra can be used to synthesize and simplify relay circuits, establishing the foundation of modern digital circuit design [Shannon1938]. Later, deterministic switches were replaced with probabilistic switches to make stochastic switching circuits, which were studied in [Wilhelm2008]. There are a few features of stochastic switching circuits that make them very similar to neural systems. First, randomness is inherent in neural systems and it may play a crucial role in thinking and reasoning. Switching (and relaying) technique provides us a natural way of manipulating this randomness. Second, in a switching system, each switch can be treated as either a memory element or a control element for computing. This might enable creating an intelligent system where storage and computing are highly integrated. In this paper, we study stochastic switching circuits from a basic starting point with focusing on probability synthesis. We consider two-terminal stochastic switching circuits, where each probabilistic switch, or pswitch, is closed with some probability chosen from a finite set of rational numbers, called a pswitch set. By selecting pswitches with different probabilities and composing them in appropriate ways, we can realize a variety of different closure probabilities.
Formally, for a two-terminal stochastic switching circuit , the probabilities of pswitches are taken from a fixed pswitch set , and all these pswitches are open or closed independently. We use to denote the probability that the two terminals of are connected, and call the closure probability of . Given a pswitch set , a probability can be realized if and only if there exists a circuit such that . Based on the ways of composing pswitches, we have series-parallel (sp) circuits and non-series-parallel (non-sp) circuits. An sp circuit consists of either a single pswitch or two sp circuits connected in series or parallel, see the circuit in Fig. 1(a) and 1(b) as examples. The circuit in Fig. 1(c) is a non-sp circuit. A special type of sp circuits is called simple-series-parallel (ssp) circuits. An ssp circuit is either a single pswitch, or is built by taking an ssp circuit and adding another pswitch in either series or parallel. For example, the circuit in Fig. 1(a) is an ssp circuit but the one in Fig. 1(b) is not.
In this paper, we first study the robustness of different stochastic switching circuits in the presence of small error perturbations. We assume that the probabilities of individual pswitches are taken from a fixed pswitch set with a given error allowance of ; that is, the error probabilities of the pswitches are bounded by . We show that ssp circuits are robust to small error perturbations, but the error probability of a general sp circuit may be amplified by adding additional pswitches. These results might help us understand why local errors do not accumulate in a natural system, and how to enhance the robustness of a system when designing a circuit.
Next, we study the problem of synthesizing desired probabilities with stochastic switching circuits. We mainly focus on ssp circuits due to their robustness against small error perturbations. Two main questions are addressed: (1) Expressibility: Given the pswitch set , where is an integer, what kind of probabilities can be realized using stochastic switching circuits? And how many pswitches are sufficient to realize them? (2) Approximation: Given an arbitrary pswitch set , how can we construct a stochastic switching circuit using as a few as possible pswitches, to get a good approximation of the desired probabilities?
The study of probability synthesis based on stochastic switching circuits has widespread applications. Recently, people found that DNA molecules can be constructed that closely approximate the dynamic behavior of arbitrary systems of coupled chemical reactions [Soloveichik10], which leads to the field of molecular computing [Cook09]. In such systems, the quantities of molecules involved in a reaction are often surprisingly small, and the exact sequence of reactions is determined by chance [Fett2007]. Stochastic switching circuits provide a simple and powerful tool to manipulate stochasticity in molecular systems. Comparing with combinational logic circuits, stochastic switching circuits are easier to implement using molecular reactions. Another type of applications is probabilistic electrical systems without sophisticated computing components. In such systems, stochastic switching circuits have many advantages in generating desired probabilities, including its constructive simplicity, robustness, and low cost.
The remainder of this paper is organized as follows: Section II describes related work and introduces some existing results on stochastic switching circuits. In Section III, we analyze the robustness of different kinds of stochastic switching circuits. Then we discuss the expressibility of stochastic switching circuits in Section LABEL:section_scenario1 and probability approximation in Section LABEL:section_scenario2, followed by the conclusion in Section LABEL:switch_section_conclusion.
Ii Related Works and Preliminaries
There are a number of studies related to the problem of generating desired distributions from the algorithmic perspective. This problem dates back to von Neumann [Neumann1951], who considered of simulating an unbiased coin using a biased coin with unknown probability. Later, Elias [Elias1972] improved this algorithm such that the expected number of unbiased random bits generated per coin toss is asymptotically equal to the entropy of the biased coin. On the other hand, people have considered the case that the probability distribution of the tossed coin is known. Knuth and Yao [Knuth1976] have given a procedure to generate an arbitrary probability using an unbiased coin. Han and Hoshi [Han1997] have demonstrated how to generate an arbitrary probability using a general -sided biased coin. All these works aim to efficiently convert one distribution to another. However, they require computing models and may not be applicable for some simple or distributed electrical/molecular systems.
There are a number of studies focusing on synthesizing a simple physical device to generate desired probabilities. Gill [Gill62][Gill63] discussed the problem of generating rational probabilities using a sequential state machine. Motivated by neural computation, Jeavons et al. provided an algorithm to generate binary sequences with probability from a set of stochastic binary sequences with probabilities in [Jeavons94]. Their method can be implemented using the concept of linear feedback shift registers. Recently, inspired by PCMOS technology [Chakrapani2007], Qian et al. considered the synthesis of decimal probabilities using combinational logic [Qian2011]. They have considered three different scenarios, depending on whether the given probabilities can be duplicated, and whether there is freedom to choose the probabilities. In contact to the foregoing contributions, we consider the properties and probability synthesis of stochastic switching circuits. Our approach is orthogonal and complementary to that of Qian and Riedel, which is based on combinational logic. Generally, each switching circuit can be equivalently expressed by a combinational logic circuit. All the constructive methods of stochastic switching circuits in this paper can be directly applied to probabilistic combinational logic circuits.
In the rest of this section, we introduce the original work that started the study on stochastic switching circuits (Wilhelm and Bruck [Wilhelm2008]). Similar to resistor circuits [MacMahon1892], connecting one terminal of a switching circuit (where ) to one terminal of a circuit (where ) places them in series. The resulting circuit is closed if and only if both of and are closed, so the probability of the resulting circuit is
Connecting both terminals of and together places the circuits in parallel. The resulting circuit is closed if and only if either or is closed, so the probability of the resulting circuit is
Based on these rules, we can calculate the probability of any given ssp or sp circuit. For example, the probability of the circuit in Fig. 1(a) is
and the probability of the circuit in Fig. 1(b) is
Let us consider the non-sp circuit in Fig. 1(c). In this circuit, we call the pswitch in the middle a ‘bridge’. If the bridge is closed, the circuit has a closure probability of . If the bridge is open, the circuit has a closure probability of . Since the bridge is closed with probability , the overall probability of the circuit is
An important and interesting question is that if is uniform, i.e., for some , what kind of probabilities can be realized using stochastic switching circuits? In [Wilhelm2008], Wilhelm and Bruck proposed an optimal algorithm (called B-Algorithm) to realize all rational probabilities of the form with , using an ssp circuit when . In their algorithm, at most pswitches are used, which is optimal. They also proved that given the pswitch set , all rational probabilities with can be realized by an ssp circuit with at most pswitches; given the pswitch set , all rational probabilities with can be realized by an ssp circuit with at most pswitches.
Wilhelm and Bruck also demonstrated the concept of duality in sp circuits. The dual of a single pswitch of probability appearing in series is the corresponding pswitch of probability appearing in parallel. Similarly, the dual of a pswitch of probability appearing in parallel is a pswitch of probability appearing in series. For example, in Fig. 2, the circuit in (b) is the dual of the circuit in (a), and vice versa. It can be proved that dual circuits satisfy the following relation:
Theorem 1 (Duality Theorem [Wilhelm2008]).
For a stochastic series-parallel circuit and its dual , we have
where is the probability of circuit and is the probability of circuit .
In this section, we analyze the robustness of different kinds of stochastic switching circuits, where the probabilities of individual pswitches are taken from a fixed pswitch set, but given an error allowance of ; i.e., the error probabilities of the pswitches are bounded by . For a stochastic circuit with multiple pswitches, the error probability of the circuit is the absolute difference between the probability that the circuit is closed when error probabilities of pswitches are included, and the probability that the circuit is closed when error probabilities are omitted. We show that ssp circuits are robust to small error perturbations, but the error probability of a general sp circuit may be amplified with additional pswitches.
Iii-a Robustness of ssp Circuits
Here, we analyze the susceptibility of ssp circuits to small error perturbations in individual pswitches. Based on our assumption, instead of assigning a pswitch a probability of , the pswitch may be assigned a probability between and , where is a fixed error allowance.
Theorem 2 (Robustness of ssp circuits).
Given a pswitch set S, if the error probability of each pswitch is bounded by , then the total error probability of an ssp circuit is bounded by
We induct on the number of pswitches. If we have just one pswitch, the result is trivial. Suppose the result holds for pswitches, and note that for an ssp circuit with pswitches, the last pswitch will either be added in series or in parallel with the first pswitches. By the induction hypothesis, the circuit constructed from the first pswitches has probability of being closed, where is the error probability introduced by the first pswitches and . The st pswitch has probability of being closed, where and .
If the st pswitch is added in series, see Fig. 3(a), then the new circuit (with errors) has probability
of being closed. Without considering the error probability of each pswitch, the probability of the new circuit is . Hence, the overall error probability of the circuit is . By the triangle inequality and the induction hypothesis,
completing the induction.
Similarly, if the st pswitch is added in parallel, see Fig. 3(b), then the new circuit (with errors) has probability
of being closed. Without considering the error probability of each pswitch, the probability that the circuit is closed is . Hence, the overall error probability of the circuit with pswitches is . Again using the induction hypothesis and the triangle inequality, we have
This completes the proof. ∎
The theorem above implies that ssp circuits are robust to small error perturbations: no matter how big the circuit is, the error probability of an ssp circuit will be well bounded by a constant times . Let us consider a case that . In this case, the overall error probability of any ssp circuit is bounded by if each pswitch is given an error allowance of .
Iii-B Robustness of sp Circuits
We have proved that for a given pswitch set , the overall error probability of an ssp circuit is well bounded. We want to know whether this property holds for all sp circuits. Unfortunately, we show that as the number of pswitches increases, the overall error probability of an sp circuit may also increase. In this subsection, we will give the upper bound and lower bound for the error probabilities of sp circuits.
Theorem 3 (Lower bound for sp circuits).
Given a pswitch set , if the error probability of each pswitch is (where ), then there exists an sp circuit of size with overall error probability .