Conditioning of three-dimensional
generative adversarial networks
for pore and reservoir-scale models
Geostatistical modeling of petrophysical properties is a key step in modern integrated oil and gas reservoir studies. Recently, generative adversarial networks (GAN) have been shown to be a successful method for generating unconditional simulations of pore- and reservoir-scale models. This contribution leverages the differentiable nature of neural networks to extend GANs to the conditional simulation of three-dimensional pore- and reservoir-scale models. Based on the previous work of Yeh et al. (2016), we use a content loss to constrain to the conditioning data and a perceptual loss obtained from the evaluation of the GAN discriminator network. The technique is tested on the generation of three-dimensional micro-CT images of a Ketton limestone constrained by two-dimensional cross-sections, and on the simulation of the Maules Creek alluvial aquifer constrained by one-dimensional sections. Our results show that GANs represent a powerful method for sampling conditioned pore and reservoir samples for stochastic reservoir evaluation workflows.111We have released our code for conditioning at http://github.com/LukasMosser/geogan
Conditioning of three-dimensional
generative adversarial networks
for pore and reservoir-scale models
Lukas J. Mosser Department of Earth Science and Engineering Imperial College London email@example.com Olivier Dubrule Department of Earth Science and Engineering Imperial College London firstname.lastname@example.org Martin J. Blunt Department of Earth Science and Engineering Imperial College London email@example.com
The spatial distribution of rock properties within a reservoir can have a critical impact on hydrocarbon recovery. In recent years a number of geostatistical methods have been developed to generate earth models given sparse information.
Different approaches exist to model the distribution geological facies and petrophysical properties. Variograms quantify geological and petrophysical variations using so-called two-point statistics (Matheron, 1975; Pyrcz and Deutsch, 2014). Anisotropic behavior is incorporated by introducing orientation-dependent variograms. Truncated Gaussian simulation obtains facies models by truncation of variogram-based models of Gaussian fields (Armstrong et al., 2011). In contrast, multiple-point statistical (MPS) methods (Strebelle, 2002) evaluate the dependency of the facies occurrence at a given location based on statistics available on multi-point templates. These statistics are provided by training images, that represent the conceptual geological knowledge and act as a discrete prior on the geological understanding of the subsurface reservoir.
Recent methods such as direct sampling (Mariethoz et al., 2010) have led to significant reduction in the computational overhead of MPS methods, allowing rapid sampling of three-dimensional reservoir models. Object-based methods populate model domains with predefined parameterized geometric representations of geobodies. This allows realistic representation of geological features, but the conditioning to well data is challenging when the size of geological objects is large compared to the well spacing.
At the reservoir scale (meters to 10s of kilometers) we have no knowledge of the true subsurface distribution of reservoir properties, except at discrete well locations; on the other hand, at the scale of individual pores of the reservoir rock, direct imaging methods such as micro-computed tomography allow images of the pore-grain structure to be made (Blunt et al., 2013; Berg et al., 2017). These images are often limited in size. Where large spatial domains are required for e.g. upscaling tasks, statistical models enable statistical and physical representations of the pore-grain structure. Due to the abundance of two-dimensional thin sections compared to three-dimensional CT measurements, models are often conditioned to match existing two-dimensional images.
Generative adversarial networks (GANs) represent a recent parametric approach developed by Goodfellow et al. (2014) to generate realistic samples, given a set of training images. Recently Mosser et al. (2017) have shown that GANs are able to generate very realistic stochastic representations of pore-scale structures. Chan and Elsheikh (2017) have shown that GANs are able to produce parametric geological representations. Laloy et al. (2017) incorporate GANs in a Markov Chain Monte-Carlo approach to create representations conditional to dynamic hydraulic data. Laloy et al. (2017) use conditional MPS simulations as a training set for variational autoencoders to sample conditioned geological representations. Yeh et al. (2016) propose a combined ”content + perceptual” loss approach and leverage the differentiable and parametric nature of the deep neural networks used to condition GAN simulations to pre-existing data.
Generative adversarial networks (GANs) (Goodfellow et al., 2014; Goodfellow, 2017) are a recent methodology developed in deep learning that allows modeling and sampling from a data distribution represented by a set of training examples. GANs consist of two differentiable functions; a generator that maps samples obtained from a multivariate standardized normal distribution to an image and a discriminator that takes on the role of a classifier to distinguish between simulations created by the generator and the training images. Both networks are trained in an alternating two-step procedure that optimizes a min-max objective function:
In this setting, the two functions have distinct and opposing objectives: the discriminator’s goal is to distinguish between real training images and samples obtained from the generator, whereas the generator tries to create samples that the discriminator falsely classifies as being a sample of the set of training images. Due to this two-player game between the generator and discriminator the training of GANs is inherently unstable. Using the Wasserstein distance as a surrogate objective function (Arjovsky et al., 2017) has proven to be a successful way of stabilizing the GAN training process. This contribution uses Wasserstein-GANs combined with a single-sided gradient penalty to train the generator-discriminator pairing (Petzka et al., 2017; Gulrajani et al., 2017).
Generative adversarial networks were trained on the three-dimensional Maules Creek reservoir dataset (Mariethoz and Caers, 2014) and on a gray-scale micro-CT image of a Ketton limestone. The Ketton dataset serves as a pore-scale example for model conditioning (Menke et al., 2017). An overview of the image processing performed on the Ketton dataset can be found in Mosser et al. (2017).
The generator and discriminator of each GAN are represented by a deep convolutional neural network (DCGAN) (Radford et al., 2015; Jetchev et al., 2016). Due to the differentiable nature of the deep neural network we can perform gradient-based optimization of the latent vector with respect to an objective function on the output of the GAN generator .
To constrain models to given conditioning data, the so-called content loss, we use a masked mean-squared error between the generator output and the conditioning data , where the mask limits the computation of the error to the location of the conditioning data only.
For binary indicator models, the objective function is the masked cross-entropy between the binary indicator at the conditioning location and the GANs output. Gray-level maps are thresholded at the 0.5 level leading to a binary indicator model sampled from the GAN.
Yeh et al. (2016) showed that only minimizing the content loss does not lead to visual realistic results and therefore introduced a so-called perceptual loss that is given by the discriminator’s evaluation of the generator’s output .
This perceptual loss evaluates the similarity of patterns observed on the generated samples and the training set. The perceptual loss is therefore added to the content loss in our optimization procedure and weighted by a user defined factor .
For continuous gray-scale images we stop optimization when the content loss is less than 1e-3, whereas for binary indicator models we use a unit accuracy i.e. perfect matching of the indicator variables at the conditioning locations after thresholding as the convergence criteria.
Optimization of the latent vector with regards to the loss is performed using stochastic gradient descent on the latent variable until the convergence criteria are met.
We evaluate the ability of GANs to generate conditional samples by conditioning the trained GAN networks to lower-dimensional data. Conditioning the three-dimensional output of the Ketton generator network to two-dimensional micro-CT data is performed using orthogonal intersections centered at the origin (Okabe and Blunt, 2004). The location of the orthogonal conditioning planes can be seen (black) below. Although the conditioning data is lower-dimensional, this has a spatial influence on the resulting realizations in the third dimension (Figure 1).
For the Maules Creek dataset we condition to a single well in the center of the domains shown in Figure 2. Conditioning of 1024 Maules Creek three-dimensional simulations was performed on a single GPU in 8 hours. We present the mean and standard deviation of the ensemble of realizations in Figure 2b-c. An elliptical influence of the conditioning data is observed. Good variation in the samples is shown by the high variance of the conditioned model ensembles. Each conditioned sample matches the indicator data at the well exactly.
Generative adversarial networks are a new powerful machine learning approach for generating three-dimensional simulations of porous media at the reservoir and pore scale. One pore-scale and one reservoir-scale case study have shown that conditional simulation can also be performed, constrained by lower-dimensional well data or cross-sections at the pore scale.
We thank H. Menke for providing the Ketton micro-CT image dataset as well as G. Mariethoz and J. Caers for sharing the Maules Creek training image. O. Dubrule thanks Total S.A. for seconding him as visiting professor at Imperial College London
- Arjovsky et al. (2017) Arjovsky, M., Chintala, S. and Bottou, L.  Wasserstein GAN. arXiv preprint arXiv:1701.07875.
- Armstrong et al. (2011) Armstrong, M., Galli, A., Beucher, H., Loc’h, G., Renard, D., Doligez, B., Eschard, R. and Geffroy, F.  Plurigaussian simulations in geosciences. Springer Science & Business Media.
- Berg et al. (2017) Berg, C.F., Lopez, O. and Berland, H.  Industrial applications of digital rock technology. Journal of Petroleum Science and Engineering, 157, 131–147.
- Blunt et al. (2013) Blunt, M.J., Bijeljic, B., Dong, H., Gharbi, O., Iglauer, S., Mostaghimi, P., Paluszny, A. and Pentland, C.  Pore-scale imaging and modelling. Advances in Water Resources, 51, 197–216.
- Chan and Elsheikh (2017) Chan, S. and Elsheikh, A.H.  Parametrization and Generation of Geological Models with Generative Adversarial Networks. arXiv preprint arXiv:1708.01810.
- Goodfellow (2017) Goodfellow, I.  NIPS 2016 Tutorial: Generative Adversarial Networks. arXiv preprint arXiv:1701.00160.
- Goodfellow et al. (2014) Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y.  Generative adversarial nets. In: Advances in Neural Information Processing Systems. 2672–2680.
- Gulrajani et al. (2017) Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. and Courville, A.  Improved training of wasserstein gans. arXiv preprint arXiv:1704.00028.
- Jetchev et al. (2016) Jetchev, N., Bergmann, U. and Vollgraf, R.  Texture Synthesis with Spatial Generative Adversarial Networks. arXiv preprint arXiv:1611.08207.
- Laloy et al. (2017) Laloy, E., Hérault, R., Jacques, D. and Linde, N.  Efficient training-image based geostatistical simulation and inversion using a spatial generative adversarial neural network. ArXiv e-prints.
- Laloy et al. (2017) Laloy, E., Hérault, R., Lee, J., Jacques, D. and Linde, N.  Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network. Advances in Water Resources, 110, 387 – 405.
- Mariethoz and Caers (2014) Mariethoz, G. and Caers, J.  Multiple-point geostatistics: stochastic modeling with training images. John Wiley & Sons.
- Mariethoz et al. (2010) Mariethoz, G., Renard, P. and Straubhaar, J.  The Direct Sampling method to perform multiple-point geostatistical simulations. Water Resources Research, 46(11), 11.
- Matheron (1975) Matheron, G.  Random sets and integral geometry. Wiley series in probability and mathematical statistics: Probability and mathematical statistics. Wiley.
- Menke et al. (2017) Menke, H., Bijeljic, B. and Blunt, M.  Dynamic reservoir-condition microtomography of reactive transport in complex carbonates: Effect of initial pore structure and initial brine pH. Geochimica et Cosmochimica Acta, 204, 267–285.
- Mosser et al. (2017) Mosser, L., Dubrule, O. and Blunt, M.J.  Reconstruction of three-dimensional porous media using generative adversarial neural networks. Phys. Rev. E, 96, 043309.
- Mosser et al. (2017) Mosser, L., Dubrule, O. and Blunt, M.J.  Stochastic reconstruction of an oolitic limestone by generative adversarial networks. arXiv preprint arXiv:1712.02854.
- Okabe and Blunt (2004) Okabe, H. and Blunt, M.J.  Prediction of permeability for porous media reconstructed using multiple-point statistics. Phys. Rev. E, 70, 066135.
- Petzka et al. (2017) Petzka, H., Fischer, A. and Lukovnicov, D.  On the regularization of Wasserstein GANs. arXiv preprint arXiv:1709.08894.
- Pyrcz and Deutsch (2014) Pyrcz, M.J. and Deutsch, C.V.  Geostatistical reservoir modeling. Oxford university press.
- Radford et al. (2015) Radford, A., Metz, L. and Chintala, S.  Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434.
- Strebelle (2002) Strebelle, S.  Conditional simulation of complex geological structures using multiple-point statistics. Mathematical Geology, 34(1), 1–21.
- Yeh et al. (2016) Yeh, R.A., Chen, C., Yian Lim, T., Schwing, A.G., Hasegawa-Johnson, M. and Do, M.N.  Semantic Image Inpainting with Deep Generative Models. arXiv preprint arXiv:1607.07539.