F-BLEAU: Fast Black-box Leakage Estimation

F-BLEAU: Fast Black-box Leakage Estimation

Giovanni Cherubin EPFL
   Konstantinos Chatzikokolakis University of Athens
   Catuscia Palamidessi INRIA, École Polytechnique

We consider the problem of measuring how much a system reveals about its secret inputs. We work under the black-box setting: we assume no prior knowledge of the system’s internals, and we run the system for choices of secrets and measure its leakage from the respective outputs. Our goal is to estimate the Bayes risk, from which one can derive some of the most popular leakage measures (e.g., min-entropy, additive, and multiplicative leakage).

The state-of-the-art method for estimating these leakage measures is the frequentist paradigm, which approximates the system’s internals by looking at the frequencies of its inputs and outputs. Unfortunately, this does not scale for systems with large output spaces, where it would require too many input-output examples. Consequently, it also cannot be applied to systems with continuous outputs (e.g., time side channels, network traffic).

In this paper, we exploit an analogy between Machine Learning (ML) and black-box leakage estimation to show that the Bayes risk of a system can be estimated by using a class of ML methods: the universally consistent learning rules; these rules can exploit patterns in the input-output examples to improve the estimates’ convergence, while retaining formal optimality guarantees. We focus on a set of them, the nearest neighbor rules; we show that they significantly reduce the number of black-box queries required for a precise estimation whenever nearby outputs tend to be produced by the same secret; furthermore, some of them can tackle systems with continuous outputs. We illustrate the applicability of these techniques on both synthetic and real-world data, and we compare them with the state-of-the-art tool, leakiEst, which is based on the frequentist approach.

I Introduction

\adl@mkpreaml\@addtopreamble\@arstrut\@preamble Dataset frequentist NN -NN
Random 100 secrets, 100 obs. 10 070 10 070 10 070
Geometric () 100 secrets, 10K obs. 35 016 333 458
Geometric () 100 secrets, 10K obs. 152 904 152 698 68 058
Geometric () 10K secrets, 1K obs. 95 500 94 204 107 707
Multimodal Geometric () 100 secrets, 10K obs. 44 715 568 754
Spiky (contrived example) 2 secrets, 10K obs. 22 908 29 863 62 325
Planar Geometric Gowalla checkins in San Francisco area X X 19 948
Laplacian " N/A X 19 961
Blahut-Arimoto " 1 285 1 170 1 343
TABLE I: Number of examples required for convergence of the estimates. “X” means an estimate did not converge.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description