SUSE Conversations


LJDT: RPM Package Verification and Repair



By: ab

April 3, 2009 4:05 pm

Reads:891

Comments:0

Rating:0

This entry is part 1 of 10 in the series LJDT - Linux Just Does That

As everybody who uses a computer in a multi-user environment eventually finds out permissions on files and directories sometimes get in your way. This is typically a good thing since they also get in the way of things like viruses and malware and other users on the system trying to get your stuff or break your computer. Windows users, well, you haven’t perhaps dealt with this much in life since windows is a bit of a red light district (and I mean that in a bad way) as you probably also already know.

So when permissions become torn up on your precious application files because of a bad set of instructions, a typo, or some other glitch in software (it happens, even with Linux), what is a user to do? The first thing that comes to mind may be to have a backup that you can restore, but that implies several things which I do not want to imply, but will list them for grins.

First, it implies you know that you should backup, so essentially you are an IT person. Second, it implies you actually took a backup, which implies you don’t really exist. Third, it implies you took a backup recently, and now we’re pushing the limits of quantum theory. Fourth, it implies your backup knows about permissions (not that hard to believe) and fifth that it wasn’t lost in the same operation that messed up your permissions.

This article is primarily made for when you mess up application files permissions, like those that actually run the software (OpenOffice, Pidgin, Thunderbird, Firefox, etc.). These permissions are set to typically be world-usable but not world-writable. Most of the time (with SUSE) they are installed via RPMs and are owned by ‘root’ and while ‘root’ can tinker with the actual contents of the files most regular users can only access the files (execute, read) which keeps them safe from your own bad decisions. In those RARE cases where you become ‘root’ and are following directions on how to configure some random package and you mistype with a ‘chmod’ or ‘chown’ command you can quickly make your system unusable just like you can with any bad command in any OS. A few options are available to help us in these cases. The RPM software has been around for quite a while and provides options out of the box… yes, without third-party software, Linux Just Does That.

So let’s dive in. For those of you not inclined to use the command line, welcome to computers. Not just microsoft computers, but real computers. If you hadn’t been tinkering as ‘root’ in the first place this might not be a problem now but the only way to efficiently troubleshoot any system (including windows or Mac, in my opinion) is from the command line. I’ll explain why in another article someday, but for now I’ll just be very clear in what needs to be done.

The ‘rpm’ command is the interface to the RPM database which holds information about files/directories and tons of metadata about your application and other files. The command has a lot of options to use, but a few (when learned) provide the bulk of everything necessary (use ‘man’ pages for the rest when needed of course).

A few of the options you should know include -q (query), -V (verify), -i (install), -U (upgrade), -F (freshen), and two new ones that I just learned, –setperms –setugids (set permissions and set ownership, respectively). So how do you use these?

First, the -q option: this is the basic starting point for RPM if you already have your system installed. It basically lets you query the RPM database (store of information about installed stuff) or a package on your machine for information. This option is combined with other flags to get more data and in this it can be confusing. For example there is a -i parameter that lets you install packages, but if you use -qi it gets information about whatever you are querying. I’ll provide some examples for clarification.

rpm -qa   #Query All, as in everything in your RPM database.  By default this dumps package names to the screen
ab@mybox:~/Desktop> rpm -qa | head -3
terminfo-base-5.6-90.55
libwnck-lang-2.24.1-2.22
libgnome-lang-2.24.1-2.28

Notice that I selected three results and they happened to be just three packages from my system retrieved in who-knows-what order. The last one looks to be related to Gnome, which makes sense since I use it primarily. If I select that one specifically and use the -qi (query information) options I get the following:

ab@mybox:~/Desktop> rpm -qi libgnome-lang
Name        : libgnome-lang                Relocations: (not relocatable)
Version     : 2.24.1                            Vendor: SUSE LINUX Products GmbH, Nuernberg, Germany
Release     : 2.28                          Build Date: Wed 25 Feb 2009 03:41:23 AM MST
Install Date: Fri 06 Mar 2009 11:37:04 PM MST      Build Host: baur
Group       : System/Localization           Source RPM: libgnome-2.24.1-2.28.src.rpm
Size        : 2565938                          License: GPL v2 or later; LGPL v2.1 or later
Signature   : RSA/8, Wed 25 Feb 2009 03:41:54 AM MST, Key ID e3a5c360307e3d54
Packager    : http://bugs.opensuse.org
URL         : http://www.gnome.org/
Summary     : Languages for package libgnome
Description :
Provides translations to the package libgnome
Distribution: SUSE Linux Enterprise 11

Okay, now this is getting interesting. I just found that my box is SLE(D) 11 and this is a ligbnome-lang package version 2.24.1, built on some box named ‘baur’ from my vendor (SUSE) in Germany (where SUSE is from). I see its size, a signature (so I know it’s really what it says it is), a license, and all kinds of good information. Let’s go a step further and I can query all of the files that the RPM database knows about that came from this package. The ‘-ql’ option lets me List all files from this package:

rpm -ql libgnome-lang |head -3

/usr/share/locale/am/LC_MESSAGES/libgnome-2.0.mo
/usr/share/locale/ar/LC_MESSAGES/libgnome-2.0.mo
/usr/share/locale/as/LC_MESSAGES/libgnome-2.0.mo

Okay, now I can see three (of however-man) files that were laid down by this package. Where are we going with all of this? Well the RPM database can be (and often is) told to keep track of metadata about these files, such as ownerships, permissions, checksums, etc. Let’s try the -V (verify) option with the command:

rpm -V libgnome-lang

After a short delay I’m right back to my prompt with no output. This means that nothing was found to be different. You could add a lower-case ‘v’ to that in order to get more-verbose output and see a bunch of lines like this if desired, or you could just `echo $?` and see the return code from the command was zero (0) which typically means everything is good regardless of the command used:

........    /usr/share/locale/am/LC_MESSAGES/libgnome-2.0.mo                                                   
........    /usr/share/locale/ar/LC_MESSAGES/libgnome-2.0.mo                                                   
........    /usr/share/locale/as/LC_MESSAGES/libgnome-2.0.mo                                                   
........    /usr/share/locale/az/LC_MESSAGES/libgnome-2.0.mo

Notice the leading dots? They would be replaced by characters indicating a problem if there was a discrepancy detected. For example configuration files are often changed by users or the system or something and they therefore often show up as having a different size or md5 checksum from the original, but this is expected…. they’re configuration files. As an example I know something has modified my /etc/profile file incorrectly (no, you shouldn’t be modifying this directly… create your own user-level scripts in ~/.profile or else create your own system-wide scripts in the /etc/profile.d directory) so let’s see which package owns it:

rpm -qf /etc/profile
aaa_base-11-6.3

The result is that aaa_base owns this file. If I were to get a new patch to aaa_base tomorrow my custom changes would be lost when the files were updated. Let’s run the Verify option against aaa_base:

rpm -V aaa_base
.......T  c /etc/csh.login
S.5....T  c /etc/inittab
S.5....T  c /etc/mailcap
S.5....T  c /etc/mime.types
S.5....T  c /etc/profile

So here we can see that the Size, the md5sum, and the Timestamp are changed on several files, and the Timestamp is changed alone on /etc/csh.login. How were they changed? Who knows, but I know where they originated and now I know that something in them may be wrong. If I were troubleshooting an issue and found that some important files, perhaps libraries or executables (binaries) were changed then I would be very suspicious. Configuration files change all the time, but binary files should not unless the package changes (typically).

Since permissions were how I led into all of this let’s pretend that those are wrong. I’ll demo that by hacking my permissions badly on purpose:

chmod 777 /etc/profile

rpm -V aaa_base
.......T  c /etc/csh.login
S.5....T  c /etc/inittab
S.5....T  c /etc/mailcap
S.5....T  c /etc/mime.types
SM5....T  c /etc/profile

Notice that now my Mode (permissions) are wrong. Currently they are set to any average user can come in here and add anything they want to my default profile, which could include a bad command like ‘rm’ with recursive and force flags (nuking my hard drive quickly the next time I login as ‘root’ for some bad reason). I need to fix this but I just don’t know what the permissions used to be (I really don’t… I should have backed this up…. if my demo fails then this article is going to be a really costly one to me). The options to set these back to what the RPM database knows are –setperms and –setugids. In my example I have the wrong permissions on this one file so I’ll have RPM set all of the permissions for the files in that package back for me:

rpm --setperms aaa_base
chmod: cannot access `/var/adm/fillup-templates/gshadow.aaa_base': No such file or directory

Now this is a good example for two reasons. First, my default bash prompt tells me the return code for my previous command and it is ’0′ so all is well, but the warning above is interesting. Verify your own with `echo $?` before you do anything else and you should get a ’0′ back if everything went properly. What does that message above about gshadow.aaa_base mean? Well apparently that file (/var/adm/fillup-templates/gshadow.aaa_base) exists per this package, but it couldn’t be found. That’s not really related to my initial problem, though, so let’s run the Verify again:

root@mybox:~# rpm -V aaa_base
.......T  c /etc/csh.login
S.5....T  c /etc/inittab
S.5....T  c /etc/mailcap
S.5....T  c /etc/mime.types
S.5....T  c /etc/profile

Look at that… the ‘M’ (mode) is now gone because it matches what the RPM database has recorded for it. So doing this for any packages that I know I have broken lets me put permissions back to normal. We saw that there is a –setugids option that fixes ownership so let’s break that next:

chown ab:wheel /etc/profile

root@mybox:~# rpm -V aaa_base
.......T  c /etc/csh.login
S.5....T  c /etc/inittab
S.5....T  c /etc/mailcap
S.5....T  c /etc/mime.types
S.5..UGT  c /etc/profile

So now we see that the User and Group ownership options are wrong for my /etc/profile file. It’s owned by my user (‘ab’) and the ‘wheel’ group. According to my assumption I should be able to run one command to fix this nicely for me:

rpm --setugids aaa_base
chown: cannot access `/var/adm/fillup-templates/gshadow.aaa_base': No such file or directory
chgrp: cannot access `/var/adm/fillup-templates/gshadow.aaa_base': No such file or directory

root@mybox:~# rpm -V aaa_base
.......T  c /etc/csh.login
S.5....T  c /etc/inittab
S.5....T  c /etc/mailcap
S.5....T  c /etc/mime.types
S.5....T  c /etc/profile

root@mybox:~# ll /etc/profile
-rw-r--r-- 1 root root 9663 Mar 25 21:28 /etc/profile

Voila, the ownerships are back to ‘root’ (user) and ‘root’ (group) and the Verify comes back clean(er) again.

So what does this all really mean? It’s possible, because Linux Just Does That, to fix permissions, ownerships, and other things recorded by the RPM database (and stored there from the package that was used for the installation) out of the box. The system keeps track of these for you (among who knows how many other things) so it can verify the integrity of applications that came in this way. Now there are a few caveats to note. I believe (though have not confirmed) that the builder of the RPM package must specify which of the options (Size, Mode, md5, User-owner, Group-owner, Timestamp, etc.) are saved by the package when the package is actually built. This is a developer thing that you as an end user don’t need to worry about since I’ve seen very few packages NOT store all of these options on all files that were within the package but keep it in mind. If I were to create a package and didn’t save these data as I built the package then the Verify option of RPM wouldn’t be as useful to you. Also, as we mentioned, configuration files change size/contents all the time since that is their purpose.

Also it is possible (and regularly happens) that files are not owned by a specific RPM even though that RPM may have created them. This usually applies to configuration files and not more-useful (in this case) files like libraries and other binaries or application files. As an end user I create documents all the time as well that obviously don’t come from any given RPM. /etc/passwd is a critical file that fits this profile on my system; it is not owned by any package but it had better be there and had better have the right permissions. As a configuration file it always changes from time to time so keep this in mind.

Finally even applications can be shipped without an RPM package. For example I use Thunderbird regularly and I just extract it to an ‘apps’ directory under my user’s home directory so it is only accessible by my user. Since it didn’t come with an RPM there is no way to do the tests above (any of them) for this package. That’s my own prerogative and it’s something to be aware of since many great applications may not come in RPM format. Anything you get via the SUSE repositories (or similar for other RPM-based distributions) should be fine since they are based around the concepts of packages.

That leads to another topic: what about other distributions and operating systems? RPM has had this functionality for years but there are others… Debian and Ubuntu use .deb packages. Gentoo uses a tool called ‘emerge’ to build its software. Solaris uses PKG commands (pkginfo, for example) and its own type of packages. There are a lot of package options out there and this is all fairly common stuff for package management but I do not know the details for other systems. Fedora (another RPM-based distribution) obviously has it, and a site I found doing research on this led me to some the pkgchk command on Solaris: http://www.cyberciti.biz/tips/reset-rhel-centos-fedora-package-file-permission.html. A bit more use of Google revealed ‘debsums’ which appears to be available for Debian-based distributions (Debian, Ubuntu, Backtrack 4, etc.).

So what can we learn from all of this? Well, a few things come to mind. First, this is troubleshooting and fixing issues that you know are broken. I wouldn’t typically recommend running something like the following to magically go and reset permissions on all files on your system that come from packages. Will it hurt things? Maybe not, but it’s not a good idea (just like anything as ‘root’ without being very careful and understanding the consequences is not a good idea):

rpm -qa | xargs rpm --setugids --setperms   #No seriously, don't run this unless you're ready to rebuild your box.

With that said we can use these skills to fix things. When you try to SSH to a machine you just mangled unknowingly and find you cannot login because the SSH server is implementing some decent security you can fix files using the RPM command. You can also reinstall software of course but sometimes that is not the best option; using a chisel is always best even if there is a jackhammer or sledgehammer nearby assuming the chisel does the job (feel free to ask any archaeologist).

While the information here is perhaps a bit complex it’s meant to illustrate a point more than to help just anybody start messing with permissions. Troubleshooting is a skill. Tracing a symptom to a cause is a skill. Hopefully with a bit of the information here and some time and effort a system that you otherwise think is lost can be recovered and restored to full health. Reinstalling software shouldn’t be the norm… it should be the exception.

Another point I find myself making over and over is to check other references. I mentioned the ‘man’ pages above; in case you are not aware there is a command in Linux/Unix called ‘man’ that accesses a built-in manual of data (there is also ‘info’ that gives a different manual). If I wanted to know more about the ‘rpm’ command I would type the following on any machine with the ‘rpm’ command installed and could likely get more than my heart’s content about the command, its options, its development history, and whatever else the developers threw in there:

man rpm

‘man’ itself is a command on the most (if not all) machines. Another option to find out more about a given command is to append ‘–help’ as its first parameter:

rpm --help

The following section results from that command being executed and this parameter (or a variant of it) works for every command I have used:

ab@mybox:~/Desktop> rpm --help                                
Usage: rpm [OPTION...]                                                       
  --quiet                                                                    

Query options (with -q or --query):
  -c, --configfiles                list all configuration files
  -d, --docfiles                   list all documentation files
  --dump                           dump basic file information 
  -l, --list                       list files in package       
  -P, --patches                    list patches or patched files 
  --queryformat=QUERYFORMAT        use the following query format
  -s, --state                      display the states of the listed files
  -a, --all                        query/verify all packages             
  -f, --file                       query/verify package(s) owning file   
  -g, --group                      query/verify package(s) in group      
  -p, --package                    query/verify a package file           
  -W, --ftswalk                    query/verify package(s) from TOP file tree
                                   walk                                      
  --pkgid                          query/verify package(s) with package      
                                   identifier                                
  --hdrid                          query/verify package(s) with header       
                                   identifier                                
  --fileid                         query/verify package(s) with file identifier
  --specfile                       query a spec file                           
  --triggeredby                    query the package(s) triggered by the       
                                   package                                     
  --whatrequires                   query/verify the package(s) which require a 
                                   dependency                                  
  --whatprovides                   query/verify the package(s) which provide a 
                                   dependency                                  
  --nomanifest                     do not process non-package files as         
                                   manifests                                   

Verify options (with -V or --verify):
  --nomd5                          don't verify MD5 digest of files
  --nofiles                        don't verify files in package   
  --nodeps                         don't verify package dependencies
  --noscript                       don't execute verify script(s)   
  -a, --all                        query/verify all packages        
  -f, --file                       query/verify package(s) owning file
  -g, --group                      query/verify package(s) in group   
  -p, --package                    query/verify a package file        
  -W, --ftswalk                    query/verify package(s) from TOP file tree
                                   walk                                      
  --pkgid                          query/verify package(s) with package      
                                   identifier                                
  --hdrid                          query/verify package(s) with header       
                                   identifier                                
  --fileid                         query/verify package(s) with file identifier
  --specfile                       query a spec file                           
  --triggeredby                    query the package(s) triggered by the       
                                   package                                     
  --whatrequires                   query/verify the package(s) which require a 
                                   dependency                                  
  --whatprovides                   query/verify the package(s) which provide a 
                                   dependency                                  
  --nomanifest                     do not process non-package files as         
                                   manifests                                   

File tree walk options (with --ftswalk):
  --comfollow                      FTS_COMFOLLOW: follow command line symlinks
  --logical                        FTS_LOGICAL: logical walk                  
  --nochdir                        FTS_NOCHDIR: don't change directories      
  --nostat                         FTS_NOSTAT: don't get stat info            
  --physical                       FTS_PHYSICAL: physical walk                
  --seedot                         FTS_SEEDOT: return dot and dot-dot         
  --xdev                           FTS_XDEV: don't cross devices              
  --whiteout                       FTS_WHITEOUT: return whiteout information  

Signature options:
  --addsign                        sign package(s) (identical to --resign)
  -K, --checksig                   verify package signature(s)            
  --delsign                        delete package signatures              
  --import                         import an armored public key           
  --resign                         sign package(s) (identical to --addsign)
  --nodigest                       don't verify package digest(s)          
  --nosignature                    don't verify package signature(s)       

Database options:
  --initdb                         initialize database
  --rebuilddb                      rebuild database inverted lists from
                                   installed package headers           

Install/Upgrade/Erase options:
  --aid                            add suggested packages to transaction
  --allfiles                       install all files, even configurations
                                   which might otherwise be skipped      
  --allmatches                     remove all packages which match <package>
                                   (normally an error is generated if       
                                   <package> specified multiple packages)   
  --badreloc                       relocate files in non-relocatable package
  -e, --erase=<package>+           erase (uninstall) package                
  --excludedocs                    do not install documentation             
  --excludepath=<path>             skip files with leading component <path> 
  --fileconflicts                  detect file conflicts between packages   
  --force                          short hand for --replacepkgs --replacefiles
  -F, --freshen=<packagefile>+     upgrade package(s) if already installed    
  -h, --hash                       print hash marks as package installs (good 
                                   with -v)                                   
  --ignorearch                     don't verify package architecture          
  --ignoreos                       don't verify package operating system      
  --ignoresize                     don't check disk space before installing   
  -i, --install                    install package(s)                         
  --justdb                         update the database, but do not modify the 
                                   filesystem                                 
  --nodeps                         do not verify package dependencies         
  --nomd5                          don't verify MD5 digest of files           
  --nocontexts                     don't install file security contexts       
  --noorder                        do not reorder package installation to     
                                   satisfy dependencies                       
  --nosuggest                      do not suggest missing dependency          
                                   resolution(s)                              
  --noscripts                      do not execute package scriptlet(s)        
  --notriggers                     do not execute any scriptlet(s) triggered  
                                   by this package                            
  --oldpackage                     upgrade to an old version of the package   
                                   (--force on upgrades does this             
                                   automatically)                             
  --percent                        print percentages as package installs      
  --prefix=<dir>                   relocate the package to <dir>, if          
                                   relocatable                                
  --relocate=<old>=<new>           relocate files from path <old> to <new>    
  --repackage                      save erased package files by repackaging   
  --replacefiles                   ignore file conflicts between packages     
  --replacepkgs                    reinstall if the package is already present
  --test                           don't install, but tell if it would work or
                                   not                                        
  -U, --upgrade=<packagefile>+     upgrade package(s)                         

Common options for all rpm modes and executables:
  -D, --define='MACRO EXPR'        define MACRO with value EXPR
  -E, --eval='EXPR'                print macro expansion of EXPR
  --macros=<FILE:...>              read <FILE:...> instead of default file(s)
  --nodigest                       don't verify package digest(s)            
  --nosignature                    don't verify package signature(s)         
  --rcfile=<FILE:...>              read <FILE:...> instead of default file(s)
  -r, --root=ROOT                  use ROOT as top level directory (default: 
                                   "/")                                      
  --querytags                      display known query tags                  
  --showrc                         display final rpmrc and macro configuration
  --quiet                          provide less detailed output
  -v, --verbose                    provide more detailed output
  --version                        print the version of rpm being used

Options implemented via popt alias/exec:
  --scripts                        list install/erase scriptlets from
                                   package(s)
  --setperms                       set permissions of files in a package
  --setugids                       set user/group ownership of files in a
                                   package
  --conflicts                      list capabilities this package conflicts
                                   with
  --obsoletes                      list other packages removed by installing
                                   this package
  --provides                       list capabilities that this package provides
  --requires                       list capabilities required by package(s)
  --suggests                       list capabilities this package suggests
  --recommends                     list capabilities this package recommends
  --enhances                       list capabilities this package enhances
  --supplements                    list capabilities this package supplements
  --basedon                        list packages this patch-rpm is based on
  --info                           list descriptive information from package(s)
  --changelog                      list change logs for this package
  --xml                            list metadata in xml
  --triggers                       list trigger scriptlets from package(s)
  --last                           list package(s) by install time, most
                                   recent first
  --dupes                          list duplicated packages
  --filesbypkg                     list all files from each package
  --fileclass                      list file names with classes
  --filecolor                      list file names with colors
  --filecontext                    list file names with security context from
                                   header
  --fscontext                      list file names with security context from
                                   file system
  --recontext                      list file names with security context from
                                   policy RE
  --fileprovide                    list file names with provides
  --filerequire                    list file names with requires
  --buildpolicy=<policy>           set buildroot <policy> (e.g. compress man
                                   pages)
  --with=<option>                  enable configure <option> for build
  --without=<option>               disable configure <option> for build

Help options:
  -?, --help                       Show this help message
  --usage                          Display brief usage message

The resources for most commands are plentiful in the computing world. Forums, IRC, product-specific e-mail lists, etc. all focus on helping with learning, and most of it is free.

Good luck.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
Series NavigationLJDT: iSCSI for Shared Storage >>

Tags:
Categories: Appliances, Technical Solutions

Disclaimer: As with everything else at SUSE Conversations, this content is definitely not supported by SUSE (so don't even think of calling Support if you try something and it blows up).  It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.

Comment

RSS