Solaris Installation Procedure

From CLONWiki
Jump to navigation Jump to search

This sections describes Solaris 10 update 6 (10/08) installation procedure for CLON cluster (Sparc-, Xeon- and Opteron-based machines).

IMPORTANT: if reinstalling system, save following directories: /etc, /var, /opt/sfw/etc, /ftfpboot, /root, /opt/realport. Also save image of 'll /' and 'df -kh' to make it easy to restore the same structure.

You will need 5 CDs 'Solaris 10 Software' (or one DVD) and one DVD 'Software Companion'. Download CDs or DVD image from http://www.sun.com. If you do not have DVD-ROM, you can mount iso image of the Companion after system itself is installed (see instructions below).

If you already have Solaris installed, type init 0 as root, and on prompt type boot cdrom. You can change boot sequence changing environment variable boot-device (setting it to cdrom disk will inforce corresponding boot sequence). For new Sun server you may want to minimize power-on tests, to do it on boot prompt type setenv diag-level min (default is usually max).

Boot from CD1. Several questions will be asked, for example for clondaq1: 

 hostname: clondaq1
 IP address: 129.57.167.107
 mask: 255.255.255.0 (will be written to /etc/netmasks)
 route: 129.57.167.99 (others: 129.57.68.100, 129.57.69.1, 129.57.160.1, will be written to /etc/defaultrouter)
 services: choose DNS
 domain name: jlab.org
 DNS servers: 129.57.167.5, 129.57.167.14, 129.57.32.100, 129.57.32.101
 search domain: jlab.org, acc.jlab.org (will be written to /etc/resolv.conf)

If setting secondary ports, do not specify default router for them.

On screen Enabling Remote Services choose No, it will enable ssh service only (no ftp, no finger etc).

Choose Initial installation, then Custom installation.

Partition disk: if chouse ZFS, whole disk will be rpool, in that case set swap area to 8192MB and select Put /var on a separate dataset. If chouse UFS, if only one disk exist, set 12GB for 'swap', 40GB for system '/', mount the rest as '/space' (clon05 and others with smaller disks: 4096MB swap, 15GB /). If 2 disks exists and there is no intension to use second one, make it /space and use whole first disk for / and 12G swap. Sun has its own disk name scheme, for example c1t0d0 means ... .

Follow instructions on screen and finish installation.

NOTE: 'format' command shows all disks.

After installation is finished do following (use VI editor, /opt/sfw/bin/emacs is not installed yet, it will come from 'Software Companion' CD; you may install that software now if want 'emacs'): 

 mkdir /root (or 'ln' to file server; normally we have it locally); copy .cshrc file from another machine's /root directory, it must looks like following:

   #
   # .cshrc for root on clon cluster solaris machines
   #
   setenv PATH ${PATH}:/opt/SUNWspro/bin
   setenv PATH ${PATH}:/opt/sfw/bin
   setenv PATH ${PATH}:/usr/sfw/bin
   setenv PATH ${PATH}:/usr/ccs/bin
   setenv PATH ${PATH}:/usr/sadm/admin/bin
   setenv PATH ${PATH}:/usr/local/bin
   setenv PATH ${PATH}:/usr/openwin/bin
   #
   setenv LD_LIBRARY_PATH /opt/SUNWspro/lib
   setenv LD_LIBRARY_PATH /opt/sfw/lib:$LD_LIBRARY_PATH
   setenv LD_LIBRARY_PATH /usr/sfw/lib:$LD_LIBRARY_PATH
   setenv LD_LIBRARY_PATH /usr/local/lib:$LD_LIBRARY_PATH
   #
   set prompt="%m:%C> "
   #
   alias ll "ls -al"
   alias xt "xterm -sb -sl 5000 -ls"
   #
   #temporary!!!
   setenv CODA /usr/local/clas/release/current/coda
   #



 in file /etc/passwd: change root home directory from '/' to '/root' and shell from '/sbin/sh' to '/bin/tcsh'
 in file /etc/default/login: comment out CONSOLE=/dev/console
 in file /etc/ssh/sshd_config: PermitRootLogin?: 'no' - change to 'yes'
 run svcadm restart /network/ssh (to show all services, type svcs)

From now on 'ssh root@clonxt2' is possible from remote host.

Install software from CD 'Software Companion'. Use it's README and follow instructions. If you want to mount iso image use loopback file driver lofi, for example: 

 lofiadm -a /root/sol-10-u6-companion-ga.iso /dev/lofi/1
 mount -F hsfs -o ro /dev/lofi/1 /mnt

Image will be mounted as /mnt. To unmount do following: 

 umount /mnt
 lofiadm -d /dev/lofi/1

To configure NIS do following:

1. Create /etc/defaultdomain with one line containing 'CCCHP'

2. Add to /etc/hosts all hosts to be used in 'ypinit -c' command (see below), normally: 

129.57.167.5     clon00     clon00.jlab.org
129.57.167.14    clon10     clon10.jlab.org
129.57.33.22     ccs10      ccs10.jlab.org

3. Execute command domainname CCCHP (or reboot, but not necessary)

4. Execute following: 

ypinit -c
next host to add: clon00
clon10
ccs10 (for EEL building put it on first place)
svcadm enable /network/nis/client
ypwhich - shows NIS server (must show first one in the list above)
ypcat passwd | grep <username> - shows id/gid etc information

Edit '/etc/nsswitch.conf': 

   passwd: compat
   group: files nis
   hosts: files dns (done by default if 'dns' was clicked during installation)
   netgroup: nis
   automount: files nis

Fix accounts (can be used to add new account !):

1. /etc/passwd: add clasrun (for clon00/clon10 add also clasboot, clascron, clasmail), add last 3 lines

2. /etc/shadow: add clasrun, etc, add last line

3. Add new groups (onliners etc): /etc/group contails group name, group id and list of group members, while /etc/passwd contains <group id ??? NO !!!>

Set mounts if it was not done during installation. Example of /etc/vfstab file lines: 

   /dev/dsk/c1t1d0s6   /dev/rdsk/c1t1d0s6   /work      ufs   2   yes   -
   /dev/dsk/c1t2d0s2   /dev/rdsk/c1t2d0s2   /scratch   ufs   2   yes   -

Set automounts:

1. /etc/auto_master contents: 

   /-      /etc/auto_direct
   /-      auto_direct.ep
   /u      auto.u.ep
   /w      auto.w.ep

2. /etc/auto_direct contents: 

   # auto_direct (Solaris, clon10-specific)
   /home                           -rw,bg  clonfs2:/vol/vol0/home
   /apps                           -rw,bg  clonfs2:/vol/vol0/apps/SunOS_sun4u
   /usr/local/clas         -rw,bg  clonfs2:/vol/vol0/local/clas
   /usr/local/downloads    -rw,bg  clonfs2:/vol/vol0/local/downloads
   /usr/local/bin          -rw,bg  clonfs2:/vol/vol0/local/SunOS_sun4u/bin
   /usr/local/etc          -rw,bg  clonfs2:/vol/vol0/local/SunOS_sun4u/etc
   /usr/local/include      -rw,bg  clonfs2:/vol/vol0/local/SunOS_sun4u/include
   /usr/local/lib          -rw,bg  clonfs2:/vol/vol0/local/SunOS_sun4u/lib
   /usr/local/man          -rw,bg  clonfs2:/vol/vol0/local/SunOS_sun4u/man
   /usr/local/sbin         -rw,bg  clonfs2:/vol/vol0/local/SunOS_sun4u/sbin
   /usr/local/share        -rw,bg  clonfs2:/vol/vol0/local/SunOS_sun4u/share
   /usr/local/src          -rw,bg  clonfs2:/vol/vol0/local/SunOS_sun4u/src
   #
   # for openssl untill installation from source will works
   /usr/local/ssl          -rw,bg  clonfs2:/vol/vol0/local/SunOS_sun4u/ssl
   #
   # for diman (see $CLON_PARMS/discr/archive /pretrigger/archive etc)
   /usr/clas/archives      -rw,bg  clonfs2:/archives
   #
   # work/scratch disks from clon00
   ##/work                   -rw,bg  clon00:/work
   ##/scratch                -rw,bg  clon00:/scratch
   /data                   -rw,bg  clon10:/data
   /raidold                -rw,bg  clon10:/raidold
   #
   # clon00/clon10 specific
   /mnt/admin/clonfs2      -rw,bg  clonfs2:/
   #

3. Restart automount service: 

 svcadm restart /system/filesystem/autofs

Reboot machine: 

 reboot, or 'shutdown -g0 -i6 -y', or 'init 6' (to power down use 'init 5')
 reboot -- -r (after new hardware installation: '-r' force reconfigure, '-s' force single user mode; 'exit' from single user will continue to multiuser mode)

Install SunStudio: goto /usr/local/src/SunStudio12 directory and run installer. If directory does not exist, create it, copy tar file from /usr/local/downloads into it and untar.

Enable X11 forwarding: uncommented and properly set 2 following lines in /etc/ssh/ssh_config file: 

Host *
ForwardX11 yes

Configure SSH so it will create .Xauthority file in /tmp directory instead of $HOME directory.


Run updatemanager, subscribe and install patches (service plan # NK20211311). If machine does not have direct access to the web (for example clon20 on subnet 86), configure proxy from Update Manager's login page (proxy server: jprox.jlab.org, port:8080). More information about jlab proxy can be found at https://wiki.jlab.org/cc/external/wiki/index.php/Proxy_For_Special_Networks.

BUGs found

File /usr/include/sys/regset.h seems have a bug, see fix in http://clonweb/wiki/clondocs/Docs/regset.h (search for 'sergey').

Background information

If need to change IP address and keep the same machine name, edit files /etc/hosts, /etc/defaultrouter, /etc/netmasks, /etc/inet/ipnodes and maybe other files in /etc/inet directory. Files in /etc/net/*/hosts may need to be changed but we are not using them. If want to change machine name, change also files /etc/hostname.* and /etc/nodename. Run "ypinit -c" and set appropriate servers if new IP name is on different subnet and different set of NIS servers must be used. Reboot the machine.

Multiple ethernet ports setting: clon machines can be connected to the several subnets; for example clonxt2: 

129.57.167.108 - clonxt2 (iprb0) - goto /etc/hosts
129.57.68.23 - clonxt2-daq1 (e1000g0) - goto /etc/hosts
129.57.69.201 - clonxt2-daq2 (e1000g1) - goto /etc/hosts

To configure do following:

1. Add all interfaces to the /etc/hosts file

2. Create /etc/hostname.e1000g0 file containing clonxt2-daq1, etc for every extra port

3. Edit file /etc/netmasks adding new line for every new port, for example: 

 129.57.68.0    255.255.255.0

4. Reboot machine, or run commands: 

 ifconfig e1000g1 plumb
 ifconfig e1000g1 129.57.68.23 netmask 255.255.255.0 up


To enable/disable services running under svcs just use svcadm enable/disable command. For legacy services rename corresponding script, for example to disable snmp service rename it's startup script: 

 cd /etc/rc3.d
 mv S82initsma hide.S82initsma


If you ansvered "No" on screen Enabling Remote Services, the number of services will not be enabled. One of them is required if machine will be used as host to download VxWorks to ROCs. To start it type command: 

svcadm enable network/shell:default

To configure Xserver for multiple displays do following commands 

mkdir /etc/dt
mkdir /etc/dt/config
cd /etc/dt/config
cp /usr/dt/config/Xservers .

and replace one line in the end of Xservers file with approptiate, for example on clon03: 

###   :0        Local local_uid@console root /usr/X11/bin/Xserver :0 -nobanner
  :0   Local local_uid@console root /usr/X11/bin/Xserver :0 -nobanner -dev /dev/fb0 bottom -dev /dev/fb2 top -dev /dev/fb3 left

see X server for more datails.

Useful diagnostic commands: 

prtdiag -v

Proceed to Solaris Customization on CLON Cluster.


Installation from network

Create Install Server

Normally Install Server is created on clon00. Login into clon00 as root.

Create directoiesy to contain iso images: 

mkdir -p /scratch/SolarisSparc
mkdir -p /scratch/Solarisx86
mkdir -p /scratch/SolarisCompanion

Copy/download Solaris 10 ()Sparc and x86) and Software Companion dvd iso images to the /scratch disk.

Mount Sparc Solaris iso: 

lofiadm -a /scratch/sol-10-u6-ga1-sparc-dvd.iso /dev/lofi/1
mount -F hsfs -o ro /dev/lofi/1 /mnt

Copy the dvd image: 

cd /mnt/Solaris_10/Tools
./setup_install_server /scratch/SolarisSparc

Unmount iso image: 

cd
umount /mnt
lofiadm -d /dev/lofi/1

Repeat the same for x86 Solaris and Software Companion: 

lofiadm -a /scratch/sol-10-u6-ga1-x86-dvd.iso /dev/lofi/1
mount -F hsfs -o ro /dev/lofi/1 /mnt
cd /mnt/Solaris_10/Tools
./setup_install_server /scratch/Solarisx86
cd
umount /mnt
lofiadm -d /dev/lofi/1

lofiadm -a /scratch/sol-10-u6-companion-ga.iso /dev/lofi/1
mount -F hsfs -o ro /dev/lofi/1 /mnt
cd /mnt/Solaris_10/Tools
./setup_install_server /scratch/SolarisCompanion
cd
umount /mnt
lofiadm -d /dev/lofi/1

To share corresponding directories, following lines must be in /etc/dfs/dfstab file: 

# for Solaris installation over network; if do that, comment out /scratch !!!
share -F nfs -o ro,anon=0 -d "install server (Sparc)" /scratch/SolarisSparc
share -F nfs -o ro,anon=0 -d "install server (x86)" /scratch/Solarisx86

If /scratch directory was already exported, comment it out. Type commands: 

unshareall
shareall

NFS server can be restarted as well if necessary (usually not): 

svcadm restart network/nfs/server

Add clients to the install server, for example to add clon10 do following (to get MAC address ssh as root and type ifconfig -a): 

cd /scratch/SolarisSparc/Solaris_10/Tools
./add_install_client -e 0:14:4f:90:4:d4 clon10 sun4u

NOTE: clon10's MAC address will be placed into file /etc/ethers, that file will be created if does not exist. You can place MAC address into that file manually, then you do not need to type -e xx:xx:xx:xx:xx:xx".

NOTE: clon10 must be in /etc/hosts file

NOTE: add_install_client command produce useful output, read it carefully

Other machines (as on 29-Dec-2008): 

./add_install_client -e 0:3:ba:16:1c:47 clon01 sun4u
./add_install_client -e 0:3:ba:1d:9b:c0 clon02 sun4u
./add_install_client -e 0:3:ba:5b:76:30 clon03 sun4u
./add_install_client -e 0:3:ba:1d:a7:8f clon05 sun4u
./add_install_client -e 0:3:ba:1d:a6:6  clon06 sun4u


On client machine, type following on system OK prompt: 

boot net




OLD:
Volume manager automatically mounts the cdrom 
Change directories to the Tools director y
# CD /cdrom/sol_8_sparc_2/Solaris_8/Tools
Copy the image from the CD to the install directory.
# ./add_to_install_server /var/Solaris8
Retrieved from "https://clonwiki0.jlab.org/wiki/index.php?title=Solaris_Installation_Procedure&oldid=3801"