| |||||||||
|
Adding delta-level replication to MIT Kerberos 5
This patch allows large sites, such as our UMICH.EDU realm, the ability to keep multiple MIT Kerberos 5 KDCs up to date without the need to regularly send the entire database to each replica (slave server). The problem we found in our initial work with the MIT Kerberos 5 code was that it took a long time for our over 200,000 entry (and growing!) database to be rebuilt on the slave server after receiving the cononical dump file from the kprop process. Using the propagation code in our environment would mean that the replicas could be many hours behind the master server.
To get around this, we changed the code in the internal routines that modify the Kerberos database on the master KDC machine so that they write a single "dump-format" record to a delta file when an entry is added to, modified, or deleted from, the Kerberos database. A new process, krep, running on the master server, monitors this file. When it notices a new entry in the file, it sends the single dump-format entry to each of the slave servers. The slaves run a new process, krepd, which communicate with krep. They modify the database on the slave server just as it has been modified on the master.
We invented new dump records to account for entry deletion since the normal propagation code does not need to handle this situation. Although we chose not to implment policies in our realm, the policy database entries are also replicated.
A (possibly unfortunate) choice made when this code was written was the use of pthreads. This choice was made because it seemed like the best way to handle the problem. It was written with the knowledge that the Kerberos libraries are not thread-safe. However, there is only one thread in each process that deals with the Kerberos libraries.
The krep process starts a thread for each slave server in the realm. This thread maintains a TCP connection with its assigned slave server. It is responsible for sending the "dump" records to the slave. There is an additional thread started by krep which monitors the delta file and the progress of each slave thread. When it notices that all the slaves have successfully received all the entries in the delta file, it truncates the file so that it does not grow unbounded.
The krepd process has two threads. One thread is responsible for communication with the krep process running on the master server. It receives "dump" records and writes them to a local file. The other thread is responsible for processing the records in the local file and truncating it when all the records have been successfully processed.
A side-effect of this patch to the KDC code is that it requires that a krep process be running on the master server to truncate the delta file, even if there are no slave servers present in the realm.
This code has been running in production in the UMICH.EDU realm since December of 1998. We currently run our production KDCs on AIX and Solaris platforms. The code has also been tested on the Linux platform in another realm on campus.
Patches to add delta-level replication support to MIT Kerberos 5:
Patches for MIT version 1.4.2:
- gzipped file (44K)
- unzipped file (177K) Updated 11/16/2005
- patch -p1 < replication-patch-1.4.2
- cd src
- util/reconf
- ./configure [with your options]
Note that UMICH and UMICH_REPLICATION will be #defined by default because of changes to configure.in in the patch.
- make
(This is actually a bit more than just the replication changes, but the replication changes are all #ifdef'd with UMICH_REPLICATION.)
To build the patched source you must recreate the configure files:
Patches for MIT version 1.3.4:
- gzipped file (80K) Updated 07/19/2004
- Includes extra locking around use of Kerberos Library to eliminate "Cryptosystem internal error while encoding message block" errors.
(This is actually a bit more than just the replication changes, but the replication changes are all #ifdef'd with UMICH_REPLICATION.)
Patches for MIT version 1.3.1:
- gzipped file (84K) Updated 03/24/2004
(This is actually a bit more than just the replication changes, but the replication changes are all #ifdef'd with UMICH_REPLICATION.)
Patches for MIT version 1.2.5:
- gzipped file (70K) Updated 03/27/2003
(This is actually a bit more than just the replication changes, but the replication changes are all #ifdef'd with UMICH_REPLICATION.)
Patches for MIT version 1.2.1:
- gzipped file (70K)
(This is actually a bit more than just the replication changes, but the replication changes are all #ifdef'd with UMICH_REPLICATION.)
Patches for MIT version 1.0.5
Comments? Suggestions? Send them to: iaa@umich.edu
| Referrals |
 |
|
 |
