— LiveStream

Applying an upgrade patch on AIX the right way means following a disciplined, recoverable workflow: open a vendor ticket, take full backups, commit existing filesets, clone to an alternate disk, apply the Technology Level or Service Pack, then verify with oslevel -s. This guide walks through every step in the correct order, explains why each one matters, and adds the rollback safety net that turns a risky maintenance window into a routine one.

AIX patching is unforgiving if you skip the preparation. A botched AIX upgrade patch can leave a logical partition (LPAR) unbootable, and unlike a quick package rollback on Linux, recovery on AIX depends entirely on backups and alternate-disk clones you created before you started. The procedure below is built around that reality.

The problem: patching production AIX without a safety net

On IBM Power servers running AIX, fixes ship as filesets bundled into Technology Levels (TL) and Service Packs (SP). A Service Pack is a cumulative collection of fixes on top of a Technology Level; a Technology Level is a larger, periodic update that can change kernel behavior, device drivers, and library versions.

Because a TL/SP touches core OS components, the danger is not the install itself but the lack of a reliable fallback. If the system fails to reboot after the change and you have no mksysb image and no alternate-disk clone, you are facing a full rebuild. The fix is process discipline: every patch is preceded by a recoverable state and applied to a test LPAR first.

Before you start: prerequisites and planning

Sort these out before the maintenance window opens. Rushing the prep is where most outages begin.

• Open a support case with IBM for the change. Record the case number so you can escalate quickly if the patch misbehaves, and so the activity is auditable.
• Obtain the fix legitimately. Download the Technology Level or Service Pack from IBM Fix Central (the modern, recommended path) rather than ordering physical CD media. Fix Central lets you select your exact source and target OS levels and bundles the correct dependencies. Physical media is legacy and slow.
• Stage the fileset images on a known-good distribution or NIM (Network Installation Management) server. In the source procedure this host was referred to internally as the staging server; the principle is the same: one trusted copy that every target pulls from.
• Schedule and communicate. Email the project stakeholders (project manager, technical architect, ETL, DBA, QA) at least a day in advance with the outage window. Surprise outages cause incidents, not patches.
• Check disk and free space. Confirm you have a spare disk for the alternate-disk clone and adequate free space in /usr, /tmp, and /var. Run df -g and lsvg rootvg to confirm.

Transfer integrity matters

When you copy the patch from the staging host to the target system (over NFS, or with scp if NFS is not available), verify the files arrived intact. A truncated fileset will fail mid-install. Run a checksum on the staging copy and again on the target, then compare:

1. On the staging server: cksum /staging/aix_patch/*
2. Copy the directory: scp -r /staging/aix_patch root@target:/tmp/aix_patch
3. On the target: cksum /tmp/aix_patch/* and confirm every value matches.

If even one checksum differs, re-copy that file before going any further.

Step-by-step: how to apply the AIX upgrade patch

This is the core AIX patching procedure. Test it on a non-production LPAR first, then repeat on production. The sequence below intentionally commits existing filesets and clones the disk before applying the new fix, so you always have a clean rollback target.

1. Pick and patch a test LPAR first

Choose a non-production LPAR (a test or QA partition) that mirrors production as closely as possible. Apply and validate the full procedure there. Only after it boots cleanly and applications pass smoke tests do you schedule the production change.

2. Take backups you can actually restore from

Two independent backups protect you: a bootable OS image and your enterprise backup.

1. Create a bootable system backup with mksysb. To write it to a file in a filesystem with enough space: mksysb -i /backups/$(hostname)_mksysb_$(date +%Y%m%d). To write to tape: mksysb -i /dev/rmt0.
2. Confirm the last TSM (IBM Spectrum Protect / Tivoli Storage Manager) incremental backup completed successfully. Check the scheduler log or run dsmc query backup -subdir=yes "/" to spot-check.
3. Run a fresh TSM incremental to capture any changes since the last scheduled run: dsmc incremental. This closes the gap between the nightly backup and your maintenance window.

3. Warn logged-in users

Broadcast the outage to everyone currently on the box with the wall command so interactive sessions are not caught mid-work:

1. wall "This host will be DOWN for OS maintenance until the published end time. Please save your work and log off now. A confirmation email will follow when the system is back. Thank you."

4. Quiesce applications and file systems

Cleanly stop every application, database, and middleware stack in the correct shutdown order. Then unmount any NFS mounts so a patch reboot does not hang waiting on a remote server:

1. Stop applications using their own scripts (do not just kill processes).
2. Unmount NFS: umount -t nfs (or unmount each mount point individually). Verify nothing is mounted with mount | grep nfs.

5. Record the current OS level

Capture the starting point so you can prove what changed and confirm success later:

1. oslevel -s — shows the full level string in the form 7200-05-04-2220 (Version-TL-SP-Build).
2. instfix -i | grep ML or oslevel -r — shows the recommended maintenance/Technology Level.

6. Commit the currently applied filesets

AIX installs filesets in two states. An APPLIED fileset keeps the previous version on disk so it can be rejected (rolled back). A COMMITTED fileset removes the saved previous version, freeing space and finalizing prior updates. You commit existing applied filesets before a new patch so the alternate-disk clone you take next is clean and the new fix has room.

1. Preview the commit (the -p flag means "preview only, change nothing"): installp -p -c all.
2. Review the output for failures or dependency warnings. Resolve them before proceeding.
3. Commit for real: installp -c all.

7. Reboot to a clean baseline

Reboot so the committed state is fully active and the system is in a known-good condition before you clone it: shutdown -Fr. After it comes back, confirm services and the OS level are unchanged.

8. Clone rootvg to an alternate disk (your rollback)

This is the single most important safety step. alt_disk_copy makes a bootable clone of rootvg on a spare disk. You apply the patch to the running system; if anything goes wrong, you simply set the bootlist back to the untouched clone and reboot to the pre-patch state in minutes.

1. Identify a free disk: lspv (look for a disk showing None as its volume group).
2. Clone rootvg to it, for example to hdisk1: alt_disk_copy -B -d hdisk1. The -B skips changing the bootlist so your current disk still boots by default.
3. Confirm the clone: lspv should now show altinst_rootvg on the target disk.

9. Apply the upgrade patch

Now install the Technology Level or Service Pack. Always preview first, then apply. Using smitty update_all is the supported menu-driven path; the command-line equivalent points install_all_updates or installp at your fileset directory:

1. Preview: install_all_updates -d /tmp/aix_patch -p (or installp -p -agXY -d /tmp/aix_patch all).
2. Apply (note: apply, not commit — leave the new fix in APPLIED state so it can be rejected): install_all_updates -d /tmp/aix_patch.
3. Watch for any fileset that reports a FAILED result in the summary at the end of the output.

Leaving the new patch APPLIED (not committed) means you can run installp -r all to reject it if a problem surfaces before you commit, in addition to the alternate-disk fallback.

10. Reboot and verify

Reboot to activate the new kernel and drivers, then confirm the level moved:

1. shutdown -Fr
2. After boot: oslevel -s — the TL/SP/Build portion should now match the patch you applied.
3. instfix -i -k <APAR> to confirm a specific fix landed, or lppchk -v to verify fileset consistency.

11. Restart applications and remount file systems

Bring the environment back up in the reverse of the shutdown order:

1. Remount NFS: mount -a (or each mount point), then verify with mount.
2. Start databases, middleware, and applications.
3. Run application smoke tests and confirm with the DBA/QA that everything is healthy.

12. Close out the change

Update the server install log and the change/incident ticket with the before/after oslevel -s values, then email the project manager confirming the upgrade is complete and the system is available. Documentation is part of the patch, not an afterthought.

Common pitfalls when patching AIX

These are the mistakes that turn a routine AIX upgrade patch into an outage:

• Committing the NEW patch immediately. Commit old filesets before patching, but leave the freshly applied fix in APPLIED state so installp -r all can reject it if needed.
• Skipping the alternate-disk clone. Without alt_disk_copy, your only fallback is a full mksysb restore, which is far slower. Always clone first.
• Running out of space. A patch can fail halfway if /usr or /tmp fills. Check df -g and expand filesystems with chfs -a size=+1G /usr beforehand.
• Leaving NFS mounted. A dead NFS server can hang the reboot. Unmount before, remount after.
• Patching production first. Always validate on a test LPAR. A driver regression should be discovered in QA, not in production.
• Trusting an unverified copy. Always cksum the fileset images after transfer; a corrupt fileset is a leading cause of failed installs.
• Mismatched source level. Service Packs assume a specific Technology Level. Confirm the patch on Fix Central matches your current oslevel -r, or apply the prerequisite TL first.

Verification: confirming the patch succeeded

Do not declare success on the reboot alone. Run this short verification pass:

If something is wrong, you have two graceful exits: installp -r all to reject the just-applied filesets, or set the bootlist to altinst_rootvg with bootlist -m normal hdisk1 and reboot to the pre-patch clone. Once you are fully satisfied after a soak period, you can commit the new patch with installp -c all to reclaim the saved-version space.

Modern context: NIM, multibos, and Live Update

The procedure above is the dependable command-line baseline and works on every supported AIX release. On current AIX (7.2 and 7.3), a few modern tools make patching smoother:

• NIM centralizes fileset repositories and lets you push updates and alt_disk operations across many LPARs from one master.
• multibos creates a second bootable instance of the Base Operating System on the same rootvg, an alternative rollback mechanism to alt_disk_copy.
• AIX Live Update (7.2 and later) can apply certain kernel fixes without a reboot, eliminating the outage window for qualifying Service Packs.

Note that AIX 5.3, 6.1, and 7.1 are past end of support; if you are still running them, plan a migration to AIX 7.2 or 7.3, but the installp, alt_disk_copy, and oslevel workflow remains the same.

Key Takeaways

• Open an IBM case and pull the exact Technology Level or Service Pack from IBM Fix Central, then verify the transfer with cksum.
• Back up twice: a bootable mksysb image plus a fresh TSM incremental, before touching anything.
• Commit existing filesets and reboot to a clean baseline, then clone rootvg with alt_disk_copy for instant rollback.
• Apply (do not commit) the new patch, reboot, and confirm with oslevel -s and lppchk -v.
• Always rehearse on a test LPAR first, and keep the new fix in APPLIED state until a soak period proves it stable.

Frequently Asked Questions

What is the difference between applying and committing a fileset in AIX?

An applied fileset keeps the previous version saved on disk, so you can roll back with installp -r all. A committed fileset deletes that saved version, finalizing the update and reclaiming space but removing the easy rollback. Apply new patches; commit only after they are proven stable.

How do I check the AIX patch level after an upgrade?

Run oslevel -s for the full Version-TL-SP-Build string (for example 7200-05-04-2220), oslevel -r for the Technology Level, and instfix -i -k <APAR> to confirm a specific fix is present. Use lppchk -v to validate overall fileset consistency.

How do I roll back a failed AIX patch?

If the patch is still in APPLIED state, run installp -r all to reject it. For a deeper failure, set the bootlist back to your alternate-disk clone (bootlist -m normal hdisk1) and reboot to the pre-patch state. As a last resort, restore from the mksysb image you took beforehand.

Should I patch production AIX directly?

No. Always validate the full procedure on a test or QA LPAR that mirrors production, confirm a clean reboot and passing application smoke tests, then schedule the production change with backups and an alternate-disk clone in place.

For more sysadmin walkthroughs and hands-on IT tutorials, subscribe to our YouTube channel @explorenystream.