How time flies? It only seemed like yesterday that I installed CentOS 5.4 at a customer site. This week I needed to update rsync from 2.6.8 to what ever the latest version is. So I typed:

yum update rsync

and got the following message.

Loaded plugins: fastestmirror
rpmdb: unable to join the environment
error: db4 error(11) from dbenv->open: Resource temporarily unavailable
error: cannot open Packages index using db3 - Resource temporarily unavailable (11)
error: cannot open Packages database in /var/lib/rpm
Traceback (most recent call last):
File “/usr/bin/yum", line 29, in ?
yummain.user_main(sys.argv[1:], exit_code=True)
File “/usr/share/yum-cli/yummain.py", line 309, in user_main
errcode = main(args)
File “/usr/share/yum-cli/yummain.py", line 157, in main
base.getOptionsConfig(args)
File “/usr/share/yum-cli/cli.py", line 187, in getOptionsConfig
self.conf
File “/usr/lib/python2.4/site-packages/yum/__init__.py", line 664, in <lambda>
conf = property(fget=lambda self: self._getConfig(),
File “/usr/lib/python2.4/site-packages/yum/__init__.py", line 239, in _getConfig
self._conf = config.readMainConfig(startupconf)
File “/usr/lib/python2.4/site-packages/yum/config.py", line 804, in readMainConfig
yumvars[’releasever’] = _getsysver(startupconf.installroot, startupconf.distroverpkg)
File “/usr/lib/python2.4/site-packages/yum/config.py", line 877, in _getsysver
idx = ts.dbMatch(’provides’, distroverpkg)
TypeError: rpmdb open failed

I also tried:

rpm -qa

which resulted in:

rpmdb: unable to join the environment
error: db4 error(11) from dbenv->open: Resource temporarily unavailable
error: cannot open Packages index using db3 - Resource temporarily unavailable (11)
error: cannot open Packages database in /var/lib/rpm

Even though the message says Resource temporarily unavailable, it was in fact always unavailable!
Hunted around and found this article which helped a lot.

The problem is caused by an empty database index file in the RPM repository, probably caused by corruption. In my case it was because most of the index files were missing.

[root@host tmp]# ls -l /var/lib/rpm
total 39656
-rw-r–r– 1 root root 5107712 Nov 11 11:58 Basenames
-rw-r–r– 1 root root 12288 Oct 5 2010 Conflictname
-rw-r–r– 1 root root 0 Dec 12 11:42 __db.001
-rw-r–r– 1 root root 1036288 Nov 11 11:58 Dirnames
-rw-r–r– 1 root root 5259264 Nov 11 11:58 Filemd5s
-rw-r–r– 1 root root 24576 Nov 11 11:58 Group
-rw-r–r– 1 root root 16384 Nov 11 11:58 Installtid
-rw-r–r– 1 root root 40960 Nov 11 11:58 Name
-rw-r–r– 1 root root 30818304 Nov 11 11:58 Packages
-rw-r–r– 1 root root 331776 Nov 11 11:58 Providename
-rw-r–r– 1 root root 102400 Nov 11 11:58 Provideversion
-rw-r–r– 1 root root 12288 Oct 5 2010 Pubkeys
-rw-r–r– 1 root root 413696 Nov 11 11:58 Requirename
-rw-r–r– 1 root root 208896 Nov 11 11:58 Requireversion
-rw-r–r– 1 root root 81920 Nov 11 11:58 Sha1header
-rw-r–r– 1 root root 45056 Nov 11 11:58 Sigmd5
-rw-r–r– 1 root root 12288 Feb 16 2010 Triggername

So all I needed to do was remove all the index files and rebuild them.

rm -rf /var/lib/rpm/__db*
rpm -rebuilddb

The RPM repository now looks like:

-rw-r–r– 1 root root 2768896 Jan 6 09:49 Basenames
-rw-r–r– 1 root root 12288 Oct 5 2010 Conflictname
-rw-r–r– 1 root root 0 Jan 6 09:49 __db.000
-rw-r–r– 1 root root 24576 Jan 6 09:49 __db.001
-rw-r–r– 1 root root 1318912 Jan 6 09:49 __db.002
-rw-r–r– 1 root root 450560 Jan 6 09:49 __db.003
-rw-r–r– 1 root root 978944 Jan 6 09:49 Dirnames
-rw-r–r– 1 root root 5267456 Jan 6 09:49 Filemd5s
-rw-r–r– 1 root root 24576 Jan 6 09:49 Group
-rw-r–r– 1 root root 16384 Jan 6 09:49 Installtid
-rw-r–r– 1 root root 45056 Jan 6 09:49 Name
-rw-r–r– 1 root root 25427968 Jan 6 09:49 Packages
-rw-r–r– 1 root root 335872 Jan 6 09:49 Providename
-rw-r–r– 1 root root 98304 Jan 6 09:49 Provideversion
-rw-r–r– 1 root root 12288 Oct 5 2010 Pubkeys
-rw-r–r– 1 root root 409600 Jan 6 09:49 Requirename
-rw-r–r– 1 root root 196608 Jan 6 09:49 Requireversion
-rw-r–r– 1 root root 81920 Jan 6 09:49 Sha1header
-rw-r–r– 1 root root 45056 Jan 6 09:49 Sigmd5
-rw-r–r– 1 root root 12288 Feb 16 2010 Triggername

I re-issued the update command:

yum update rsync

and it successfully installed rsync 3.0.6-4.el5_7.1.

Interestingly, CentOS 5.4 has been deprecated and so this version of rsync was for CentOS 5.7, but it didn’t need to install any dependencies.