This is best explained with a simple example:

An application needs to fetch 63, 000, very well compressible (ratio 1:50), records over a 28.8kBit line. The total data to be transferred is around 81 mega bytes (85,509,224 bytes). This operation takes in NexusDB default mode (without batch and compression) 31,121 seconds which is about 2 records per second. Slow you say? Let's see.

The results are direct results of network lag time (latency) and maximum data transferred per second. Look at above example again: 28 kbit lines have typical ping times of around 150-200 ms. Fetching 63,000 records takes (at least) 126,000 messages sent over the network which results in 18,900 seconds minimum time for the transfer.

The other number to look at is the data amount. If we take into account the 81 mega bytes of data transferred over line (that can transfer a theoretical maximum of 3.6 Kbytes) we would need minimum time of 23195 seconds for transferring the data.

Now what can batch mode and compression do here? Let's use a BlockReadSize of 512 Kbytes and maximum ZIP compression to transfer the same data. Using NexusDB the data can be fetched in just 540 seconds. That's 116 records per second, which is over 50 times faster than in default mode. Why?

First we reduce the number of messages to 164 which gives us a minimum time of 24.6 seconds. For the data transfer we can compress the data to 1,706,639 bytes (ratio 1:50) with maximum zip compression in 512kByte blocks. This will then results in 462 seconds theoretical minimum.

Looking at the theoretical numbers the actual benchmarked NexusDB numbers look pretty good, taking network and message overhead into account.

Home | Site Contents | Documentation | NexusDB Manual V4 | Architecture and Concepts | NexusDB Concepts | Transports | Selecting the correct Transport