Disclosure of simulation application design with the use of shared memory and data compression
Original Publication Date: 2004-Jan-08
Included in the Prior Art Database: 2004-Jan-08
This disclosure is a design technique for general simulation system which refers large amount of data like Monte Carlo simulation. The model system infrastructure of the simulation system which this article describes is IBM pSeries and AIX 4.3.3 based. And IBM LoadLeveler is used as a middleware to dispatch the simulation JOBs, each JOB is related to each interest rate scenario, to the simulation execution nodes ( 6 nodes ). The simulation JOB consists of multiple execution processes on the simulation execution nodes and each process accesses the simulation database on the DB nodes, executing SELECT statements concurrently to get interest rate, account data, agreement data, etc. The CPU usage of the simulation execution nodes stays at lower level because the performance bottleneck was workload concentration of the DB nodes. Consequently, the 1000 interest rate scenarios of Monte Carlo simulation took more than 24 hours even if all 6 nodes worked. In general, tuning of DBMS, gigabit networking, etc. could be considered in such case. However, to accelerate performance without upgrading hardware is required in some cases. Moreover, although it was possible to improve processing efficiency to some extent by increasing the number of simulation processes, there was a limit in increasing the efficiency because of resource restrictions ( the number of simultaneous SQL transactions, large memory usage, etc. ) of DB node.