#1  
Old 16th October 2020, 02:56 AM
RichardLight RichardLight is offline
Junior Member
 
Join Date: Nov 2011
Posts: 22
Default When are edits saved to nx1 file?

Hi,

One of our users has observed that edits are not saved to disc immediately. The nx1 file holding the table in question only has its date/time updated when the user closes the client program completely. This is with a local NexusDb 4.5 server, built into the client.

Presumably the edited and new records are held in memory. They ask if this exposes them to any risk of data loss, should there be for example a power cut.

Is there a 'flush to disc' command or option? I don't see anything obvious.

Thanks,

Richard
  #2  
Old 16th October 2020, 03:21 AM
Thorsten Engler [NDA]
 
Posts: n/a
Default Re: When are edits saved to nx1 file?

RichardLight wrote:

>
> Hi,
>
> One of our users has observed that edits are not saved to disc
> immediately. The nx1 file holding the table in question only has its
> date/time updated when the user closes the client program completely.
> This is with a local NexusDb 4.5 server, built into the client.
>
> Presumably the edited and new records are held in memory. They ask if
> this exposes them to any risk of data loss, should there be for example
> a power cut.
>
> Is there a 'flush to disc' command or option? I don't see anything
> obvious.


One of your users has observed that he doesn't know how file access works under
windows.

The file system will not update the meta data of the file, which includes the
last written date/time, until the file handle is closed.

That does NOT mean that the contents of the file has not been written before.

The modified date/time is no indication of when changes are written. It tells
you when a file handle which was used to write to a file was last closed.

All changes are being written (as in, the WriteFile API is being called) at the
time that a transaction is committed.

While there is a "flush" command, and NexusDB uses it, please be aware that all
modern HDDs and SSDs ignore it in hardware (makes benchmarks look much better).

So while NexusDB always writes all changes instantly when a transaction is
committed and will NOT perform any write caching inside the server, there is a
time window between when the WriteFile API call returns and when the data is
actually guaranteed to be written all the way to disk and save from a power
failure.

Depending on the exact hardware and software configuration, this time window is
generally from fractions of a second to maybe a few seconds. (Or longer if
there have been a lot of writes and windows is condfigured to perform agressive
write caching, but even with active write caching, windows will always write
out any outstanding data as fast as it can, this only affects when the
WriteFile API call returns.)

So in short, all changes will always be written to disk as fast as possible
when a transaction is committed, there is no danger that any data remains
unwritten inside NexusDB depending on anything after the Commit call.

All file operations without an explicit transaction are internally executed in
an implicit transaction, so there is no difference in behaviour.
  #3  
Old 16th October 2020, 03:30 AM
Rodrigo G?mez
 
Posts: n/a
Default Re: When are edits saved to nx1 file?

Hello Richard,

I don't recall the specifics, but I think that what you are seeing is
the normal behavior. It might have to do more with the OS caching the
files than Nexus itself, but of that I am not sure.

What you can do is enable the "Force Fail Safe" option in the nxServer
options. This, IIRC, will make sure everything is flushed to disk when
it has to, which avoids the case you are thinking of. The downside is
that everything is slower, although I don't have a number for that as I
have never used that option. It might be that with newer flash drives,
NVMe and so on, it is not as a problem as it was before, but you will
have to see how does it work in or impacts your environment. It is a
server side option, so you don't have to change anything in your code
unless you are using an embedded server or a custom one that doesn't
expose that option.

Uhm... when checking the options on the nxServer administration page the
help says about Force Fail Safe:

"Force Failsafe makes the server work in failsafe mode only. Please note
that this slows down the server significantly, but in certain
circumstances and environment, data safety and consistency is essential.
If the option is enabled the server engine writes a journal for every
transaction and only if the transaction is successfully completed and
the journal committed the actual transaction is considered successful.
Otherwise the engine automatically applies the action set in the
Failsafe mode setting."

I'm not sure then if what I said makes sense, maybe my understading of
how it worked is incorrect. But what I do understand is that this makes
data corruption of the tables much more unlikely (not that it is a
common problem, to begin with) even if the mechanism is not what I thought.

Hopefully Thorsten can shed some light in this subject.

HTH,

--
Rodrigo G?mez [NDX]
M?xico, GMT-6
  #4  
Old 16th October 2020, 03:37 AM
Rodrigo Gómez
 
Posts: n/a
Default Re: When are edits saved to nx1 file?

Thanks for the reply Thorsten. I had answered with some incorrect ideas
but after seeing your reply cancelled the message.

Now, I wonder, could you share some light about the Force Fail Safe
mechanism? In what cases could it make sense to use it, I mean, in what
possible scenarios would it make a difference to have that enabled or not?

Thanks!

--
Rodrigo Gómez [NDX]
México, GMT-6
  #5  
Old 17th October 2020, 01:05 AM
David Marcus David Marcus is offline
Member
 
Join Date: Oct 2011
Posts: 93
Default

Quote:
Originally Posted by Rodrigo Gómez View Post
Now, I wonder, could you share some light about the Force Fail Safe mechanism? In what cases could it make sense to use it, I mean, in what possible scenarios would it make a difference to have that enabled or not?
I always have FailSafe on. I suppose if my app was too slow and I had a way of dealing with corruption in the database, I'd turn it off. In my day job, I work for InterSystems, and our database product has a similar feature which is on by default and strongly recommended.


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Something interesting to do with table edits via SQL Will Owyong [NDX] nexusdb.public.discussions 0 29th March 2011 05:17 PM
Optimizing edits with Nx 2. Rudy Poseika nexusdb.public.support 1 7th September 2007 03:32 AM
Affect of Batch Mode when have mixed append & edits Jesse nexusdb.public.support 0 21st April 2007 02:23 PM
Data not saved Jan-Erik Johansen nexusdb.public.support 13 4th August 2004 10:22 PM
RC6 EM, blob file options are not saved Roberto Nicchi nexusdb.public.support 2 30th August 2003 04:29 PM


All times are GMT +11. The time now is 04:37 AM.


Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.