DevOps · K8s · Volleyball · Travel  •  DevOps · K8s · Volleyball · Travel  •  DevOps · K8s · Volleyball · Travel
Explore NY Stream

— LiveStream

How To Configure Distributed Shell on AIX

Distributed Shell (dsh) on AIX lets an administrator run a single command across many servers at once over SSH, instead of logging into each host one at a time. This guide shows how to configure dsh on AIX end to end: confirm the package is installed, set up passwordless SSH, define your node list with the right environment variables, and run real commands against multiple hosts with clean, readable output.

What Distributed Shell (dsh) on AIX Does

The Distributed Shell command, dsh, is part of IBM's Cluster Systems Management (CSM) toolset on AIX. It takes one command, fans it out to a list of remote hosts in parallel, and collects the output back on the host where you launched it. For routine fleet tasks like checking whether a user account exists, verifying a filesystem, or confirming a patch level across dozens of servers, dsh turns an hour of manual logins into a single line.

Under the hood, dsh does not talk to hosts directly. It shells out to a remote command program you choose, the modern and only safe choice being SSH. That means your SSH trust and your node list do the heavy lifting, and dsh simply orchestrates them.

A note on currency: CSM (which ships csm.dsh) is an older, end-of-life IBM product. The configuration below is accurate for environments that still run CSM, and it is still widely found on legacy AIX estates. On modern systems you will more likely use the Distributed Systems Management (DSM) filesets bundled with PowerHA / RSCT, which provide the same dsh and dcp commands and behave almost identically. If you are building something new from scratch, agentless tools such as Ansible or ClusterShell (clush) are the current best practice. The mental model in this guide carries over to all of them.

Before You Begin: Requirements

Two things must be true before dsh will work. Get these right first and the rest is configuration.

  • Passwordless SSH from the source host to every target. The host that issues the dsh command must be able to SSH into each managed host without being prompted for a password or passphrase. This is done with an SSH key exchange (public-key authentication).
  • The dsh package installed on the source host. Only the host launching commands needs the dsh tooling; the targets just need a reachable SSH daemon.
  • A shell you control. This guide assumes the Korn shell (ksh), the AIX default, so configuration goes in .profile or .kshrc.

Confirm the dsh package is installed

Check that the CSM Distributed Shell fileset is present on the source host:

  1. Run the fileset query and filter for dsh:

    lslpp -L | grep dsh

  2. You should see the csm.dsh fileset listed, for example:

    csm.dsh 1.7.0.10 C F Cluster Systems Management Dsh

If nothing is returned, the fileset is not installed. Install it from your AIX media or software repository (CSM filesets), or, on a DSM-based system, confirm the dsm.dsh fileset instead. The command name remains dsh either way.

Set up the SSH key exchange

If you have not already established passwordless SSH, do it now. Generate a key pair on the source host and push the public key to each target's authorized_keys file:

  1. Generate a key pair (press Enter to accept the default path and an empty passphrase for unattended use):

    ssh-keygen -t rsa -b 4096

  2. Copy the public key to a target host:

    ssh-copy-id user@targethost

    If ssh-copy-id is unavailable on your AIX build, append the key manually:

    cat ~/.ssh/id_rsa.pub | ssh user@targethost 'mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'

  3. Verify the trust works without a prompt:

    ssh user@targethost hostname

The command must return the remote hostname without asking for a password. If it prompts, dsh will hang or fail across the fleet, so fix this before going further.

How to Configure dsh: The Environment Variables

The behaviour of Distributed Shell on AIX is driven almost entirely by three environment variables. Setting them correctly is the core of the configuration.

VariablePurpose
DSH_NODE_RSHThe remote command program dsh uses to reach each host. Set this to the full path of SSH so that all fan-out happens over an encrypted channel.
DSH_NODE_LISTThe path to a plain-text file containing the hostnames you want to manage, one per line. Used whenever you do not name hosts explicitly on the command line.
DSH_NODE_OPTSOptional. Options passed through to the remote command. Setting it to -q runs ssh -q, which suppresses SSH banner and warning messages for cleaner output.

Build a node list file

Create a simple text file with one hostname per line. This file is what DSH_NODE_LIST points at. You can maintain your own; many sites keep a shared, curated host list. Example contents of /home/admin/hostlist.txt:

wimcsm
impvcs01
irdqa05

Keep the list accurate. A stale or wrong hostname produces noisy errors and slows the whole run while SSH waits to time out.

Add the variables to your shell profile

Log on to the host that will issue dsh commands, then edit your .profile or .kshrc so the settings persist across logins. Add these lines, adjusting the list path to your own file:

  1. Open your profile in an editor:

    vi ~/.kshrc

  2. Add and export the three variables:

    export DSH_NODE_RSH=/usr/bin/ssh
    export DSH_NODE_LIST=/home/admin/hostlist.txt
    export DSH_NODE_OPTS="-q"

  3. Save the file, then log out and back in (or source it) so the variables take effect:

    . ~/.kshrc

  4. Confirm the values are set in your session:

    echo $DSH_NODE_RSH $DSH_NODE_LIST $DSH_NODE_OPTS

Using export is important: a variable that is set but not exported will not be visible to dsh as a child process. The source material's original lines omitted export, which is a common cause of "it works in my prompt but dsh ignores it" confusion.

Using dsh: Real Command Examples

With the package installed, SSH trust in place, and the variables exported, you can start running commands across hosts. Note that the command is lowercase dsh in every case.

Scenario 1: Target specific hosts with -w

To check whether the user account gliu06 exists on three named hosts (wimcsm, impvcs01, and irdqa05), pass them with the -w option:

dsh -w wimcsm,impvcs01,irdqa05 'lsuser gliu06'

Key points about -w:

  • Hostnames are comma-separated with no spaces between them.
  • -w overrides DSH_NODE_LIST for that single run, so it is perfect for ad-hoc checks against a handful of machines.
  • Quote the remote command (here 'lsuser gliu06') so the local shell passes it through intact instead of trying to interpret it.

Scenario 2: Run against your whole node list

When you run dsh with no host options, it reads every hostname from the file in DSH_NODE_LIST and runs the command on all of them:

dsh 'lsuser grproy'

On a large fleet, this can flood your terminal. If the user does not exist on most hosts, every one of them returns a "user does not exist" error, burying the few useful lines. The next section cleans that up.

Scenario 3: Filter and format the output

Two small techniques make fleet-wide output readable. First, redirect standard error to discard the noise. In the shell, file descriptor 1 is stdout and 2 is stderr, so 2>/dev/null throws away error messages while keeping real results:

dsh 'lsuser grproy' 2>/dev/null

Better still, pipe the result through cut to keep only the first field. Here -d ":" tells cut to split on the colon delimiter and -f1 prints the first column:

dsh 'lsuser grproy' 2>/dev/null | cut -d ":" -f1

The result is a tidy list of hostname-prefixed matches, with the per-host noise stripped away. The dsh output format is hostname: result, so the leading column tells you exactly which servers responded.

Common Pitfalls and How to Avoid Them

Most dsh problems come down to SSH, environment, or syntax. Watch for these:

  • SSH still prompts for a password. Then dsh will appear to hang. Re-test the key exchange with ssh targethost hostname and fix permissions: the remote ~/.ssh should be 700 and authorized_keys should be 600.
  • Variables set but not exported. Always use export. A bare DSH_NODE_RSH=/usr/bin/ssh in your profile is visible to your shell but not to dsh.
  • A space after -w. Write -w host1,host2 (no space inside the list). Spaces between hostnames break the list parsing.
  • Unquoted remote commands. Wildcards, pipes, and redirections meant for the remote host must be quoted, or the local shell consumes them first. Compare dsh 'ps -ef | grep db2' (pipe runs remotely) with dsh ps -ef | grep db2 (pipe runs locally on collected output) — both are valid but do different things.
  • Stale node list. A decommissioned host in DSH_NODE_LIST makes every run wait for an SSH timeout. Keep the file current.
  • Mixed case in the command. It is dsh, not Dsh or DSH. AIX commands are case-sensitive.
  • Fan-out load. Running heavy commands across hundreds of hosts at once can spike load. Many dsh builds support a fan-out limit (for example -f) to cap concurrent sessions; use it on big estates.

Verification: Confirm dsh Works

Before trusting dsh with real work, prove the plumbing end to end:

  1. Check the package: lslpp -L | grep dsh returns the csm.dsh (or dsm.dsh) fileset.
  2. Check the variables: echo $DSH_NODE_RSH $DSH_NODE_LIST $DSH_NODE_OPTS shows all three values in a fresh login session.
  3. Check SSH trust: ssh <target> hostname returns the remote name with no prompt.
  4. Run a harmless fan-out: dsh -w host1,host2 'hostname' returns one line per host, each prefixed with the host it came from.
  5. Run against the list: dsh 'date' 2>/dev/null returns the current date from every host in your node file.

If every step passes, your Distributed Shell on AIX setup is correct and ready for production use.

Key Takeaways

  • dsh runs one command across many AIX hosts in parallel over SSH, collecting all output back on the source host.
  • The setup rests on three pillars: the csm.dsh (or dsm.dsh) fileset, passwordless SSH key trust, and the three DSH_NODE_* environment variables.
  • Always export DSH_NODE_RSH=/usr/bin/ssh, DSH_NODE_LIST, and DSH_NODE_OPTS="-q" in .profile or .kshrc.
  • Use -w host1,host2 (no spaces) for ad-hoc targets; clean noisy output with 2>/dev/null and cut -d ":" -f1.
  • CSM is end of life; the same dsh ships in DSM, and Ansible or ClusterShell are the modern choices for new fleets.

Frequently Asked Questions

What is dsh on AIX used for?

dsh (Distributed Shell) lets an AIX administrator execute the same command on multiple servers simultaneously from one host. It is used for fleet-wide checks and changes such as verifying user accounts, confirming patch levels, restarting services, or auditing filesystems without logging into each machine individually.

How do I tell dsh to use SSH instead of rsh?

Set the DSH_NODE_RSH environment variable to the full path of the SSH binary and export it: export DSH_NODE_RSH=/usr/bin/ssh. Once exported in your shell profile, every dsh invocation reaches remote hosts over SSH rather than the insecure legacy rsh.

Why does dsh keep asking for a password?

Because passwordless SSH is not set up correctly between the source and target hosts. Generate a key with ssh-keygen, copy the public key into each target's ~/.ssh/authorized_keys, and fix permissions (700 on ~/.ssh, 600 on authorized_keys). Confirm with ssh <target> hostname, which must return the name with no prompt.

Is CSM dsh still supported, and what replaces it?

CSM is end of life. The same dsh and dcp commands live on in IBM's Distributed Systems Management (DSM) filesets shipped with RSCT/PowerHA, so existing scripts keep working. For brand-new automation, agentless tools like Ansible or ClusterShell (clush) are the recommended modern equivalents.

For more AIX and system administration walkthroughs, subscribe to @explorenystream on YouTube.