Asynchronous persistent proxy for synchronous communication
Publication Date: 2017-Jul-31
The IP.com Prior Art Database
This publication will provide a solution how to perform Database Recovery over long distances or within high latency environment with no data loss. Currently this is a big inhibitor for many database customers.
Problem description: Many applications need synchronous communication in order to achieve certain business function. Example might be database instance that need to communicate synchronously to other database instance at different location in order to ensure that all written data is stored in both locations so that in case of failure data is not lost (this is also known as Disaster Recovery scenario). In these cases the processing on primary side is delayed until secondary side can confirm that data was persisted and only after this confirmation the write operation is reported as successful. Typical inhibitor is network latency that directly contributes to processing time as database need to wait for confirmation from secondary location. Very high latency can significantly extend processing steps and cause heavy performance issues. This is decreasing physical distance between two locations as network latency is dependent on length of optical fiber channels. This publication is introducing a concept of "Synchronous to asynchronous traffic converter" (referred as "converter") - specialized application able to receive synchronous streams from one or more data sources and able to deliver them to relevant destinations in asynchronous way. Today it is extremely difficult to achieve Disaster Recovery (DR) for existing databases over long distance (for example another continent) with no data loss (Recovery Point Objective (RPO) = 0). Typical solution today is to tolerate certain data loss (RPO > 0) by using asynchronous methods (where asynchronous nature will prevent impact on performance as primary application does not need to wait for confirmation) or to perform DR over short distance (where response from secondary application will be quick due to low network latency).
How does a database respond to SQL query that is modifying data
Example of Synchronous Disaster Recovery (no data loss but delay in processing = performance impact)
1. SQL query that modified data (insert, update, delete) is committed and need to be persisted 2. Primary database will send data in synchronous way towards the secondary database 3. Data will arrive to secondary database
1. SQL query that modified data (insert, update,
delete) is committed and need to be persisted
2. Data is persisted in local persistence
3. Persistence will confirm that data was successfully
4. Database will confirm to application that SQL
query was successfully committed
4. Data is persisted in persistence of secondary database 5. Persistence of secondary database will confirm that data was successfully stored 6. Confirmation is sent towards the primary database 7. Confirmation is received 8. After confirmation is received data is persisted in local persistence 9. Persistence will confirm that data was successfully stored 10. Database will confirm to application that SQL query was successfully committed Example of Asynchronous Disaster Recovery (with data loss but no delay in pr...