NexusDB

With this format there is no longer a single (or multiple) files per table, instead there is a single file (-set, more about that later) per database which is at the same time the current state of the database and a full transaction log.

When a transaction is comitted, no changes are made to any information already in the file, instead the changed blocks are appended to the file. The only exception to this is a field in the header of the file which references he file offset where information about the last fully committed transaction can be found.

When a new database is created it starts as a single (numbered) file. It is possible to instruct the server engine at any time between 2 transactions to start a new file. From that moment on no changes at all will be written to older files anymore (but they still need to be present).

It is also possible to instruct the server to create a checkpoint. This will result in the addition of a new file to the set which contains copies of the most current version of all blocks of all tables in the database. After the creation of a checkpoint, older files can be removed. The database remains fully usable while a checkpoint is created.

This format has a number of advantages:

• All transaction are always "failsafe" without a reduction in performance
• DDL statements (or their procedual equivalent, table creation, deletion, restructure) can be part of a transaction (and mixed with data manipulation).
• It is safe at any point in time to do a filesystem level backup or copy of a database which is actively in use.
• Incremental backups are easy to support by instructing the server to start a new file in the numbered file-set.
• It is possible to access any older database state (by starting a special read-only transaction and specifying a date/time, this is natrally limited if older files have been removed after creating a checkpoint).
• Performance of transaction commits is improved as they result in a single sequential write instead of random write access to many positions in (possibly) multiple files.
• One-way replication becomes trivial to implement.

There is one disadvantage, the resulting file set will naturally be larger then the current table files as it contains the complete transaction history of the database instead of just the most current state. But it is possible to remove older history information at any time by creating a checkpoint and then deleting (or archiving) older files. Given current drive sizes this should not be a significant issue.

We hope to have a EEP binaries for you girls and guys to test ready soon.