Browse Prior Art Database

Fast Simulation with Multiple Multi-Layer Neural Networks

IP.com Disclosure Number: IPCOM000247603D
Publication Date: 2016-Sep-19
Document File: 5 page(s) / 147K

Publishing Venue

The IP.com Prior Art Database

Abstract

For certain simple models, a fast simulation can be achieved through a neural network approach. In addition, the derivatives to model parameters for an inversion application can be quickly obtained. It has been proved that a two-layer neural network is in general sufficient to simulate any relationship. However, over-estimation or under-estimation could be a problem for real applications. To solve an under-estimation problem, more neurons in a hidden layer of a two-layer neural network can be added until it is going to be over-estimated. The increase of the hidden-layer neurons requires larger memory and more computations and sometime limited by computer resources. Also, there is no simple criterion to choose the best number of hidden neurons. To solve an over-estimation problem, verification and test data sets are needed for the network training process. The goodness will depend on how those data sets are selected and quite often the training stopped with unsatisfied accuracy. In the invention, the proposed new approach is with multiple multi-layer neural networks. In the preferred embodiment, two two-layer neural networks are used to build a complicated relationship between the response of a device and a model. The basic procedures are: 1) train the response data using a two-layer neural network with fewer hidden neurons to catch the main features between data and the model parameters; 2) calculate the residuals between the data and simulated responses with the trained small neural network; 3) train the residuals with a new neural network with more hidden neurons. Then, the response of the device can be obtained by summing the results from the two neural networks. The over-fitting on the second neural network training has less effect since it is just a small part of the total response. If the over-fitting is still a problem, the number of hidden neurons can be reduced and use a third neuron network for the new residuals. Based on this approach, the accuracy can always be improved to a required level.

This text was extracted from a Microsoft Word document.
At least one non-text object (such as an image or picture) has been suppressed.
This is the abbreviated version, containing approximately 51% of the total text.

Title:  Fast Simulation with Multiple Multi-Layer Neural Networks

Abstract:  For certain simple models, a fast simulation can be achieved through a neural network approach. In addition, the derivatives to model parameters for an inversion application can be quickly obtained. It has been proved that a two-layer neural network is in general sufficient to simulate any relationship. However, over-estimation or under-estimation could be a problem for real applications. To solve an under-estimation problem, more neurons in a hidden layer of a two-layer neural network can be added until it is going to be over-estimated. The increase of the hidden-layer neurons requires larger memory and more computations and sometime limited by computer resources. Also, there is no simple criterion to choose the best number of hidden neurons. To solve an over-estimation problem, verification and test data sets are needed for the network training process. The goodness will depend on how those data sets are selected and quite often the training stopped with unsatisfied accuracy.

In the invention, the proposed new approach is with multiple multi-layer neural networks. In the preferred embodiment, two two-layer neural networks are used to build a complicated relationship between the response of a device and a model. The basic procedures are: 1) train the response data using a two-layer neural network with fewer hidden neurons to catch the main features between data and the model parameters; 2) calculate the residuals between the data and simulated responses with the trained small neural network; 3) train the residuals with a new neural network with more hidden neurons. Then, the response of the device can be obtained by summing the results from the two neural networks. The over-fitting on the second...