SUSE Conversations


Advanced Update Server Mirroring including OES

variia

By: variia

August 15, 2008 1:12 pm

Reads:246

Comments:2

Rating:0

Related articles about this which I think is fair to share here as a starter:

This article is based on those above with additional notes for SUSE Linux Enterprise Desktop, SUSE Linux Enterprise Server, SP1, SP2 and OES2.

There are still questions and misunderstanding around this on http://forums.novell.com hence in this complete guide I’d like to show how we do all of this and perhaps give you a detailed explanation on the related components.

Ingredients:

“yup” software (just a shell script really) which can be found here:

or alternatively on the SDK (Software Development Kit) media (my preference) but that version will be slightly behind although works fine.

You will also need a bit of a storage, we currently mirror (without src or debug packages):
SLED10-SP1 x86_64/i586
SLES10-SP1 x86_64/i586
SLED10-SP2 x86_64/i586
SLES10-SP2 x86_64/i586
OES2 x86_64/i586
SDK-SP1 x86_64/i586
SDK-SP2 x86_64/i586

The storage we consumed (updates only) so far is 44GB just to give you an idea about the requirements. Remember that your subscription needs to match what you want to mirror. You may not be able to achieve everything presented here simply because you don’t have subscription for certain products. It makes sense to do out of this guide just what you subscribed for.

You need to think about how you want to share the packages. We chose “ftp” simply because it’s efficient, fast and doesn’t require much configuration although out of the scope of this document.

Hint: we use vsftpd product mainly with defaults, anonymous access enabled and write access is disabled.
FYI: my yup server is a SLES (XEN) virtual machine with 256MB of RAM with 1 core and it’s perfectly adequate for this purpose.

Setting up the ORIGINAL MEDIA as installation source:

Updates aren’t enough, they are just the packages which got “updated” since the product release therefore it’s important to configure the original installation sources (media) as well. I downloaded all DVD ISOs from http://download.novell.com and copied them to my yup server:

root@geeko:~ # ls -l /home/ftp/iso | awk '{print $8}'
OES2-i386-CD1.iso
OES2-x86_64-CD1.iso
SLE-10-SP1-SDK-DVD-i386-GM-DVD1.iso
SLE-10-SP1-SDK-DVD-x86_64-GM-DVD1.iso
SLE-10-SP2-SDK-DVD-i386-GM-DVD1.iso
SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso
SLED-10-SP1-x86_64-DVD1.iso
SLED-10-SP1-x86-DVD1.iso
SLED-10-SP2-x86_64-DVD1.iso
SLED-10-SP2-x86-DVD1.iso
SLES-10-SP1-DVD-i386-DVD1.iso
SLES-10-SP1-DVD-x86_64-DVD1.iso
SLES-10-SP2-DVD-i386-GM-DVD1.iso
SLES-10-SP2-DVD-x86_64-GM-DVD1.iso

This way if anybody needs the ISOs they can retrieve it but that’s not the reason why we copied them to the yup server. We will mount them and create “installation sources” for all versions shown above. Making the original media available this way will give you great benefits including an ultimate network installation source, you will never need the “disk in the drive”.

You need to create the directories first (mount points):

root@geeko:~ # mkdir -p /home/ftp/install/{SLE10-SDK-SP1,SLED10-SP1,SLES10-SP1,SLE10-SDK-SP2,SLED10-SP2,SLES10-SP2,OES2}

And the architecture dependent directories underneath them:

root@geeko:~ # mkdir -p /home/ftp/install/SLE10-SDK-SP1/{sles-10-i586,sles-10-x86_64}
root@geeko:~ # mkdir -p /home/ftp/install/SLED10-SP1/{sled-10-i586,sled-10-x86_64}
root@geeko:~ # mkdir -p /home/ftp/install/SLES10-SP1/{sles-10-i586,sles-10-x86_64}
root@geeko:~ # mkdir -p /home/ftp/install/SLE10-SDK-SP2/{sles-10-i586,sles-10-x86_64}
root@geeko:~ # mkdir -p /home/ftp/install/SLED10-SP2/{sled-10-i586,sled-10-x86_64}
root@geeko:~ # mkdir -p /home/ftp/install/SLES10-SP2/{sles-10-i586,sles-10-x86_64}
root@geeko:~ # mkdir -p /home/ftp/install/OES2/{sles-10-i586,sles-10-x86_64}

Note: OES2 is just an addition (ADD-ON product) on top of SLES10-SP1 at this time of writing hence it’s still necessary to host SP1 even if it’s now considered obsolete.

Now we need to mount the ISOs underneath these directories. It’s easier to add them to your fstab so at next boot they will be mounted automatically. Note: you will need to have enough loop devices under /dev to do this, on SLES10-SP2 it shouldn’t be a problem:

root@geeko:~ # cat /etc/fstab
-snip-
# SP1
/home/ftp/iso/SLED-10-SP1-x86-DVD1.iso /home/ftp/install/SLED10-SP1/sled-10-i586 iso9660 loop 0 0
/home/ftp/iso/SLED-10-SP1-x86_64-DVD1.iso /home/ftp/install/SLED10-SP1/sled-10-x86_64 iso9660 loop 0 0
/home/ftp/iso/SLES-10-SP1-DVD-i386-DVD1.iso /home/ftp/install/SLES10-SP1/sles-10-i586 iso9660 loop 0 0
/home/ftp/iso/SLES-10-SP1-DVD-x86_64-DVD1.iso /home/ftp/install/SLES10-SP1/sles-10-x86_64 iso9660 loop 0 0
/home/ftp/iso/SLE-10-SP1-SDK-DVD-i386-GM-DVD1.iso /home/ftp/install/SLE10-SDK-SP1/sles-10-i586 iso9660 loop 0 0
/home/ftp/iso/SLE-10-SP1-SDK-DVD-x86_64-GM-DVD1.iso /home/ftp/install/SLE10-SDK-SP1/sles-10-x86_64 iso9660 loop 0 0

# SP2...
/home/ftp/iso/SLED-10-SP2-x86-DVD1.iso /home/ftp/install/SLED10-SP2/sled-10-i586 iso9660 loop 0 0
/home/ftp/iso/SLED-10-SP2-x86_64-DVD1.iso /home/ftp/install/SLED10-SP2/sled-10-x86_64 iso9660 loop 0 0
/home/ftp/iso/SLES-10-SP2-DVD-i386-GM-DVD1.iso /home/ftp/install/SLES10-SP2/sles-10-i586 iso9660 loop 0 0
/home/ftp/iso/SLES-10-SP2-DVD-x86_64-GM-DVD1.iso /home/ftp/install/SLES10-SP2/sles-10-x86_64 iso9660 loop 0 0
/home/ftp/iso/SLE-10-SP2-SDK-DVD-i386-GM-DVD1.iso /home/ftp/install/SLE10-SDK-SP2/sles-10-i586 iso9660 loop 0 0
/home/ftp/iso/SLE-10-SP2-SDK-DVD-x86_64-GM-DVD1.iso /home/ftp/install/SLE10-SDK-SP2/sles-10-x86_64 iso9660 loop 0 0

# OES2...
/home/ftp/iso/OES2-i386-CD1.iso /home/ftp/install/OES2/sles-10-i586 iso9660 loop 0 0
/home/ftp/iso/OES2-x86_64-CD1.iso /home/ftp/install/OES2/sles-10-x86_64 iso9660 loop 0 0
-snip-

Double check these please, I might have made typos… ;) If it’s all correct mount them:

root@geeko:~ # mount -a

Adding the ORIGINAL MEDIA to your systems as network installation sources:

Great, now you have all these version/releases being available as installation sources, you just need to configure your machines to use them. It’s of course based on an assumption that your ftp side is all sorted and the ftp home directory (top level) is /home/ftp but it can really be anything.

How you update your system is up to you. We found ZENworks (novell-zmd service and “rug”) to be bad. They chew up the CPU often, it doesn’t wake up sometimes, generate zombie processes, etc. therefore we disabled it and use solely zypper (libzypp). YaST also uses zypper backends hence what you configure for zypper will be available for YaST also. It’s the best doing it from the command line because it’s fast and efficient. You need to do this on all your servers/clients and of course just for the products you want. Here it is an example for SLED10-SP2-i586:

root@sled10sp2:~ # zypper sa -t YaST ftp://yupserver.domain.co.nz/install/SLED10-SP2/sled-10-i586 LOCAL-YUP-INSTALL-SOURCE-SLED10-SP2-32bit

The concept is simple: you tell what type of the repository you are adding (mounted ISOs can only be YaST) the ftp URI followed by a meaningful name you can refer to later on. Further info:

root@geeko:~ # man zypper

Setting up UPDATE repositories as network installation sources:

Now we need to configure yup to retreive updates for all these products/versions. We need a directory first to hold the updates:

root@geeko:~ # mkdir /home/ftp/update

This will be the top level for updates but for this we don’t need to create anything underneath, yup will create all necessary subdirectories in very similar fashion as we did for the installation above.

I used the official rpm from SDK source (already mounted remember), it’s SLES maintained and works perfectly:

root@geeko:~ # rpm -ivh /home/ftp/install/SLE10-SDK-SP2/sles-10-i586/suse/noarch/yup-232-2.2.noarch.rpm

The configuration files are here:

/etc/sysconfig/yup
/etc/sysconfig/yup2zlm (out of the scope of this document, it’s for delivering patches internally via Zenworks)

For delivering patches to SLES/SLED the /etc/sysconfig/yup is perfectly adequate but our problem is that we want OES2 which brings another problem into our scenario: OES2 doesn’t have any patch levels available so I cannot mirror SLED, SLES (have different patchlevels now) and OES (doesn’t) with one configuration file. We need to create 2 configuration files for our standard SLEx and OES products. It has to be in different directory though:

root@geeko:~ # mkdir /etc/sysconfig/yup.d
root@geeko:~ # cp /etc/sysconfig/yup /etc/sysconfig/yup.d/yup-sle
root@geeko:~ # cp /etc/sysconfig/yup /etc/sysconfig/yup.d/yup-oes

Looking at the header of the /usr/sbin/yup script it tells me that if there’s any configuration file in /etc/sysconfig/yup.d those will be used and the one /etc/sysconfig/yup will be ignored.

Let’s see my configuration for SLES10/SLED10 products first:

root@geeko:~ # grep . /etc/sysconfig/yup.d/yup-sle | grep -v \#
YUP_CURL=""
YUP_CURLPARAMS="--proxytunnel -x myproxy.domain.co.nz:80 --limit-rate 200K"
YUP_DEST_DIR="/home/ftp/update"
YUP_ID="xxxxxxx"
YUP_PASS="xxxxxxxxxx"
YUP_SERVER="nu.novell.com"
YUP_ID_SLED=""
YUP_PASS_SLED=""
YUP_SERVER_SLED=""
YUP_ID_SLES=""
YUP_PASS_SLES=""
YUP_SERVER_SLES=""
YUP_ARCH="i586 x86_64"
YUP_PRODUCTS="SLES10 SLED10"
YUP_DEBUGINFO="no"
YUP_SDK="yes"
YUP_SRCRPM="no"
YOU_NCC_ACCOUNT=""
YOU_NCC_PASSWORD=""
YOU_SERVER="you.novell.com"
YUP_SUBVERSIONS="SP1 SP2"
YUP_DRYRUN="no"
YUP_HARDLINK="yes"
YUP_PATSIZE="+20000"
YUP_MTIME="-8"
YUP_SP_SUBCHANS="-Updates"

It’s fairly straight forward although I have some extra options for curl because I’m behind firewall and wanted to avoid saturating my bandwidth hence the download limit. One ID/PASS was enough for both products, I enabled hardlinks to save some space and I definitely enabled “-Updates” only. “-Online” includes the patches going from one release to another for instance SP1->SP2. Mirroring these are wasting space on your disk, wasting bandwidth, etc. I’ll show later on how to do this in a smart way.

Now let’s see my configuration for OES (fairly similar, minor changes really):

root@geeko:~ # grep . /etc/sysconfig/yup.d/yup-sle | grep -v \#
YUP_CURL=""
YUP_CURLPARAMS="--proxytunnel -x myproxy.domain.co.nz:80 --limit-rate 200K"
YUP_DEST_DIR="/home/ftp/update"
YUP_ID="xxxxxxx"
YUP_PASS="xxxxxxxxxx"
YUP_SERVER="nu.novell.com"
YUP_ID_SLED=""
YUP_PASS_SLED=""
YUP_SERVER_SLED=""
YUP_ID_SLES=""
YUP_PASS_SLES=""
YUP_SERVER_SLES=""
YUP_ARCH="i586 x86_64"
YUP_PRODUCTS="OES2"
YUP_DEBUGINFO="no"
YUP_SDK="no"
YUP_SRCRPM="no"
YOU_NCC_ACCOUNT=""
YOU_NCC_PASSWORD=""
YOU_SERVER="you.novell.com"
YUP_SUBVERSIONS=""
YUP_DRYRUN="no"
YUP_HARDLINK="yes"
YUP_PATSIZE="+20000"
YUP_MTIME="-8"
YUP_SP_SUBCHANS="-Updates"

Note: I found the comments in the configuration files a little bogus. Sometimes it shows that you need to put your setting within “” (double quotes), separate with space or comma, etc. sometimes doesn’t. I had extensive experiment with this and I can confirm that it’s the only working way as shown above. yup will not display error or anything, you will just find a messed up directory structure perhaps missing products, etc.

Once you have done your confguration with your favorite editor you can start downloading the patches:

root@geeko:~ # /usr/sbin/yup

In the meantime it’s worthwhile to set up a daily cron job to do this automatically:

root@geeko:~ # vi /etc/cron.d/yup-update
05 04 * * * root /usr/sbin/yup

Adding the UPDATE repositories to your systems as network installation sources:

The syntax is almost the same as before with a little difference, update repositories are YUM type. The same example for SLED10-SP2-i586 will look like this:

root@sled10sp2:~ # zypper sa -t YUM ftp://yupserver.domain.co.nz/update/SLED10-SP2-Updates/sled-10-i586 LOCAL-YUP-UPDATE-SOURCE-SLED10-SP2-32bit

For the correct directory path you need to browse your server after the yup mirror process finished the first run and retrieved all your patches, created all necessary directories, etc.

Setting up the ONLINE UPGRADE repositories as network installation sources:

This solution works only for SLEx10 SP1 and above, please do not use it for GA or any other release. We need to set up a set of repositories to support network based upgrades going from SLEx10 SP1 -> SP2. The benefit of this is that you minimize the downtime, you upgrade while the system is running then do a quick reboot immediately at completion.

The background of this was posted by Novell already:
How to update to SLES/SLED 10 SP2

The document is incorrect I believe, it works for SLED also and we have some extra tricks to do due to our source is ISO mount where we cannot really create directories within. I still believe that mounting an ISO is cleaner, better then copying everything to the disk especially if you need to host the ISOs anyway but it’s your call.

We need to create some more directories (if you excluded -Online from the yup configuration you will have no problems with these):

root@geeko:~ # mkdir -p /home/ftp/update/SLED10-SP2-Online/{sled-10-i586,sled-10-x86_64}/patches
root@geeko:~ # mkdir -p /home/ftp/update/SLES10-SP2-Online/{sles-10-i586,sles-10-x86_64}/patches

You may have noticed by now that we have a similar directories within the original media sources. We need to “map” these locations and kind of “convert” them into our online upgrade repository what we will do with soft links. I prefer relative paths but you could you absolute paths too:

root@geeko:~ # cd /home/ftp/update/SLED10-SP2-Online/sled-10-i586/patches
root@geeko:/home/ftp/update/SLED10-SP2-Online/sled-10-i586/patches # ln -s ../../../../install/SLED10-SP2/sled-10-i586/suse .
root@geeko:/home/ftp/update/SLED10-SP2-Online/sled-10-i586/patches # ln -s ../../../../install/SLED10-SP2/sled-10-i586/patches/repodata .

root@geeko:/home/ftp/update/SLED10-SP2-Online/sled-10-i586/patches # cd ../../sled-10-x86_64/patches
root@geeko:/home/ftp/update/SLED10-SP2-Online/sled-10-x86_64/patches # ln -s ../../../../install/SLED10-SP2/sled-10-x86_64/suse .
root@geeko:/home/ftp/update/SLED10-SP2-Online/sled-10-x86_64/patches # ln -s ../../../../install/SLED10-SP2/sled-10-x86_64/patches/repodata .

root@geeko:/home/ftp/update/SLED10-SP2-Online/sled-10-x86_64/patches # cd ../../../SLES10-SP2-Online/sles-10-i586/patches
root@geeko:/home/ftp/update/SLES10-SP2-Online/sles-10-i586/patches # ln -s ../../../../install/SLES10-SP2/sles-10-i586/suse .
root@geeko:/home/ftp/update/SLES10-SP2-Online/sles-10-i586/patches # ln -s ../../../../install/SLES10-SP2/sles-10-i586/patches/repodata .

root@geeko:/home/ftp/update/SLES10-SP2-Online/sles-10-i586/patches # cd ../../sles-10-x86_64/patches
root@geeko:/home/ftp/update/SLES10-SP2-Online/sles-10-x86_64/patches # ln -s ../../../../install/SLES10-SP2/sles-10-x86_64/suse .
root@geeko:/home/ftp/update/SLES10-SP2-Online/sles-10-x86_64/patches # ln -s ../../../../install/SLES10-SP2/sles-10-x86_64/patches/repodata .

It looks a bit cryptic, unfortunately that’s the way it is. Please double check the paths, I’m doing this from memory hence it’s quite likely that I remembered wrong or made typo. ;)

Adding the UPGRADE repositories to your systems as network installation sources:

The concept is similar with minor changes. The upgrade repository is YUM type and the path is slightly different. Because it only makes sense to add upgrade repository to SP1 system here it is the same example for SLED10-SP1-i586:

root@sled10sp1:~ # zypper sa -t YUM ftp://yupserver.domain.co.nz/update/SLED10-SP2-Online/sled-10-i586/patches LOCAL-YUP-UPGRADE-SOURCE-SLED10-SP2-32bit

Once you finished with the upgrade process you must remove the upgrade repository and configure the update ones as presented in this guide earlier. You can do this either from the command line or from YaST, it’s your choice again.

As a final word I do suggest everybody to read all linked articles published in this guide as well just to have a better vision and understanding about this technology.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Tags: ,
Categories: Open Enterprise Server on SLES, SUSE Linux Enterprise Server, Technical Solutions

Disclaimer: As with everything else at SUSE Conversations, this content is definitely not supported by SUSE (so don't even think of calling Support if you try something and it blows up).  It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.

2 Comments

  1. By:mwfolso

    Hi:

    Question – I have yup up and running fine but would also like to download patches for SLE10 SDK SP2

    Currently I am downloading patches for SLES10 and SLED10 with this line in my /etc/sysconfig/yup file

    YUP_PRODUCTS=”SLES10 SLED10″

    to get the patches for SDK I added —

    YUP_PRODUCTS=”SLES10 SLED10 SLE10-SDK”
    and got the line in the output when I run yup:

    Warning: SLE SLE10-SDK unknown. Proceeding anyways …

    Any clue what the syntax is to download the SDK?

    Thanks -

    Michael

  2. By:variia

    Hi,

    That’s wrong! Remove your addition. You need to say :
    YUP_SDK=”yes”
    in the same config file to download patches for SDK.

    Please read the the comments in that file carefully.

    Regards,
    Ivan

Comment

RSS