Atomic File Transactions, Part 1
by Jonathan Amsterdam11/07/2001
One of the many powerful features that transaction-processing systems provide is atomicity. An activity is atomic if it either happens in its entirety, or does not happen at all. Atomicity is crucial for writing correct software in many applications; for example, a bank's software may implement a transfer from account A to account B as a withdrawal from A followed by a deposit to B. If the first action happens, then the second had better happen as well.
Databases provide atomicity for data stored within them, but filesystems are not atomic with respect to their files. This two part article series explains how to achieve atomicity for standard filesystem actions. The code for such a system is implemented in the Java package com.astrel.io.atomic, available here. In part one, I will explain transactions and atomicity in more detail. I'll then discuss how to start using this package.
It's not hard to find programs that could benefit from atomic filesystem operations. Installers are a prime example: they do a lot of filesystem manipulation, and if there is an error or they crash, one would like the filesystem back the way it was. Word processors, spreadsheets and other standard "office" programs write, delete and rename files. Even business software that primarily accesses a database may use the filesystem to store non-critical data like user preferences. Crashes during these operations, or sequences of operations, can potentially corrupt the data. Finally, if you don't require powerful search capabilities, using flat files may be faster than dealing with a database. All of these programs will (and do) work without atomicity, but all would be more robust with it.
You may be under the impression that filesystems already provide atomicity. After all, when you delete a file, either the delete succeeds and the file is gone, or it fails and the file is still there. That is correct: modern filesystems make sure file deletion and other basic operations, like creation and renaming, are atomic; in other words, they prevent themselves from being corrupted (for what else would you call a filesystem in which a file was half deleted and half not?). But they do not ensure that file writes are atomic: once you overwrite some data in a file, you can't get it back. They also do not provide for the atomicity of sequences of actions: you can delete atomically and rename atomically, but you can't do a delete followed by a rename atomically.
It is possible to build a transactional filesystem from the ground up -- that is, from low-level filesystem operations like block reads and writes. In fact, databases do just that. But here I will describe how to make an existing filesystem atomic.
If you are concerned about the performance of atomic filesystem operations, your concern is well-placed; atomicity comes at a cost. You will have to determine whether the increase in robustness is worth the slowdown in your application. In many cases, you can do the I/O in a background thread, mitigating the performance impact.
The ACID Test
In standard parlance, a transaction is a sequence of operations with four properties, called the ACID properties, for Atomicity, Consistency, Isolation, and Durability.
Atomicity means that a transaction can end in only one of two ways: either successfully, in which case all its effects take place, or unsuccessfully, in which case it has no effect -- for all intents and purposes, the transaction never happened.
Consistency just means that a transaction is written correctly, so that when it completes successfully, the system is in a consistent state.
Isolation means that the transaction appears to execute completely alone, even if, in fact, other transactions are running simultaneously. Isolation is important, and can be enforced automatically, but my package does not attempt to do so (atomicity alone is hard enough). You can use Java's thread synchronization features to enforce isolation within a single virtual machine, and you can use the File.createNewFile method to implement a file-locking protocol that will enforce isolation across virtual machines. For the rest of this article, we'll assume that isolation has been done for us: that simultaneously running transactions never access the same file.
Durability means that a successful transaction's changes are permanent. In practice, "permanent" means "on disk." Since our transactions involve the filesystem, durability comes at no extra charge.
Since there's no standard term for something that has atomicity and durability but not isolation, I will use the word "transaction," but will occasionally write "atomic transaction" to indicate that these are not full-blooded transactions.
Using Transactions
Almost everyone who's programmed for a database knows how to use transactions:
- You begin a transaction.
- You do a bunch of things as part of the transaction -- "under" the transaction, we will say.
- You attempt to commit the transaction, locking in your changes.
The attempted commit may succeed or fail. If it succeeds, then the transaction is committed, and its effects have taken place for good. If the commit fails, the transaction mechanism guarantees that you'll be back at the beginning: none of the transaction's effects will have occurred, or all those that have occurred will be undone.
|
Related Reading
|
There is another way to end a transaction besides committing it: you can abort it, or (synonymously but more politely) "roll it back." You may choose to do this because one of the actions you attempted under the transaction didn't work the way you'd hoped, or perhaps you got new information that made the transaction's effects unnecessary or ill-advised. In any case, rolling back a transaction undoes all its effects, returning the system to the initial state.
In the event of a crash during a transaction, the transaction will be rolled back automatically when the transaction-processing system restarts. This is called recovery.
Working with the Atomic Transaction Package
Now that you understand the basics of transactions, you are ready to use the com.astrel.io.atomic package. Users of the package need to understand only two non-exception classes: TransactionManager and Transaction.
TransactionManager
A TransactionManager (TM) represents a set of transactions that are part of the same program. You begin by constructing a TransactionManager, giving the constructor a directory where the TM will hold all of the files it needs in order to provide atomicity:
TransactionManager tm = new
TransactionManager("backup");The TM must wholly control that directory; it is a grievous error for other programs, or even other TMs, to use the same directory. Using a relative pathname for the directory, as shown here, is wise only if you know your program will always be started from the same directory. Otherwise, you should use an absolute or user-relative pathname.
Your program can use multiple TMs, each with its own directory, and each TM can support multiple simultaneous transactions. But it is up to you to ensure that no two simultaneous transactions access the same file -- the package does not provide isolation.
The TM constructor also handles recovery: if there was a crash the last time you ran your program, then by the time the constructor has returned, any affected transactions have been rolled back.
Once you have a TM, you invoke its beginTransaction method to get a Transaction:
Transaction t = tm.beginTransaction();
We'll see how to use the Transaction object in a moment. The other major method of TransactionManager is close. Call close when you are done with the TM. It will roll back any active transactions and close all open files.
Pages: 1, 2 |
