Linux Installation Procedure

Dell servers hardware trick(s)

• if looks dead, remove the power cords, then held in the on button for 30 seconds; then plug it back in and it may boots up fine - known Dell feature

RHEL7 (64 bit)

Installed using DVD provided by Computer Center, currently RHEL7.3. Requires MAC address to be registered with Computer Center (using jnet.jlab.org as 'boiarino') because it is network-based installation. On DELL servers, hold F2 after power on and go to 'Device Settings' to see mac address. Remember machine memory size as well, will needed when create partitions, find it in 'System BIOS -> Memory Settings'.

NOTE: all disks will be reformatted in a process, all information will be lost !

Dialog:

• boot: <enter>

• Please enter the hostname of this system: <hostname> <enter>

• What type of system are you building? (D)esktop: <enter>

• Is this a CUE level 1 or CUE level 2 System build (1/2): 2 <enter>

• Please enter the System Admin's CUE username: boiarino <enter>

• Do you want to use the default partitioning scheme (y/n)? n <enter>

• Are you sure you want to use your own partitioning scheme (y/n)? y <enter>

Choose 'Standard Partition' scheme and create following partitions: /boot - 1GB, swap - double memory size, / - the rest of space.

Installation will proceed. When prompted, remove installation disk and reboot machine if not done automatically.

After reboot patching process will be started and it will take a while, do not interrupt it, otherwise system will not be installed correctly.

When machine reboots for the last time, ask Paul Letta to set usual hallb root password. From now on remote root access is possible. Alternatively, user boiarino can execute most of actions using sudo, like sudo emacs for protected files. sudo reboot etc.

Go to /boot/grub2 and modify file grub.cfg, Linux kernel boot options, adding options net.ifnames=0 biosdevname=0, so your boot string has following section:

..... rhgb net.ifnames=0 biosdevname=0 quiet .....

so whole line looks something like this:

linux16 /vmlinuz-3.10.0-327.18.2.el7.x86_64 root=UUID=b5d1b5ad-e3f6-4db4-8972-aa9c1a91d116 ro vconsole.keymap=us crashkernel=auto  vconsole.font=latarcyrheb-sun16 rhgb net.ifnames=0 biosdevname=0 quiet LANG=en_US.UTF-8

Reboot machine, ethernet ports must be named eth0, eth1 etc.

To update manually, use commands yum list updates and yum update.

NOTE: add machine to the appropriate puppet group, it will do all following automatically (hopefully).

Edit the /etc/yum.conf file and add the following line to force installation of the 32bit and 64bit libraries:

multilib_policy=all

Install following packages:

yum install xterm mesa-dri-drivers (mesa-dri-drivers not in PUPPET)

##yum install ncurses-devel

##yum install libstdc++ (provides libstdc++.so.6)

yum install telnet (some people need it to check ports)

yum install rsh (for rlogin) (not in PUPPET)

yum install readline-devel (needed by EPICS)

yum install net-snmp-utils (for roc_reboot)

yum install tigervnc-server tigervnc

yum install xpdf nload

yum install mc (midnight commander) (not in PUPPET)

yum install mariadb-devel (mysqlclient etc)

##yum install jlabca ( needed by 'git clone ...' )

##yum install screen (terminal duplication)

yum install libxml2-devel glibc-devel qt-devel (Wesley's request)

yum install python-alembic (for RCDB) (not in PUPPET)

yum install openssl-devel apr-devel (for ActiveMQ) (not in PUPPET)

(fonts for EPICS EDMs)
yum install xorg-x11-fonts-75dpi xorg-x11-fonts-ISO8859-1-75dpi bitmap-miscfixed-fonts
yum install bitmap-console-fonts bitmap-fixed-fonts bitmap-lucida-typewriter-fonts
ln -s  /usr/share/X11/fonts/100dpi /etc/X11/fontpath.d/xorg-x11-fonts-100dpi:pri=60
ln -s  /usr/share/X11/fonts/75dpi /etc/X11/fontpath.d/xorg-x11-fonts-75dpi:pri=50
ln -s  /usr/share/X11/fonts/misc /etc/X11/fontpath.d/xorg-x11-fonts-misc:pri=40
ln -s  /usr/share/fonts/bitmap /etc/X11/fontpath.d/bitmap-fonts

Do following to be able to compile and run 32-bit apps on 64-bit machine:

yum install libXpm-devel libXaw-devel motif-devel glibc-devel ncurses-devel readline-devel (for compatibility with 32bit applications) (not in PUPPET)
#cd /usr/lib
#ln -s libXpm.so.4.11.0 libXpm.so
#ln -s libX11.so.6.3.0 libX11.so 
#ln -s libXt.so.6.0.0 libXt.so 
#ln -s libXext.so.6.4.0 libXext.so

Modify file /etc/passwd adding following:

hpsrun:x:8796:9998:hpsrun Account:/home/hpsrun:/bin/tcsh
clasrun:x:2508:9998:Online DAQ:/home/clasrun:/bin/tcsh
clasioc:x:6008:9998:CLAS IOCs:/home/clasioc:/bin/tcsh
clas12run:x:6007:146:clas12run Account:/home/clas12run:/bin/tcsh
clascron:x:3888:146:CLAS Cron Jobs:/home/clascron:/bin/tcsh
clasboot:x:6000:146:Boot Scripts Only:/home/clasboot:/bin/tcsh
clasmail:x:6003:146:CLAS Automated Mail:/home/clasmail:/bin/tcsh
epics:x:5001:146:EPICS Account:/home/epics:/bin/tcsh
nagios:x:6246:9997:Nagios:/home/nagios:/bin/tcsh
+@clon_cluster::::::
+sshd::::::
+::::::/bin/false

Note: user-related lines must be removed, like

+boiarino::::::

as well as

+::::::

Note: assumed that following line is in already:

+@ccc::::::

Note: if following line exist, it must be placed in the end, or removed:

+::::::/bin/false

Modify file /etc/group adding following:

onliners::9998:heyes,hujl,rwm,timmer,wolin,taylorw,cuevas,abbottd,abulafia,anciant,anghi,aswin,auger,avakian,banta,barrow,battagli,bjlee,bogdan,brooksw,bruhwel,boiarino,buckle,burchesk,burin,bzh,cetina,chen,cole,cords,corvi,costy,davidl,costy,davidl,dcarter,dcurry,dennisl,deppman,dhlee,doughty,drago,dytman,dugger,dugger,elton,fedro,ficenec,fklein,france,freyberg,gholden,gilfoyle,golovach,gurjyan,haddock,heddle,jenkins,jhardie,jmorris,kellyk,klusman,kmkim,kossov,lcsmith,longhi,manak,marchand,marki,mastersj,mjohnson,mosbackr,mueller,muguira,murphyl,necaise,odonnell,opper,parkkj,pasyuk,phkim,piot,riccardi,richardt,ripani,rossi,saw,sergpozd,serov,sholt,staylor,taiuti,tang,tedeschi,vineyard,vlassov,vvsap,weygand,wooyoung,burchesk,clasrun,giovanet,clasboot,clascoda,clasroot,clasmail,clascron,trigger,epics,southern,carstens,ccutter,danagu,heyes,huertas,jacobsg,kliv,lachniet,mestayer,primex,hovanes,nerses,sytnik,baltzell,carman
clon::383:clascoda,clascron,clasmail
nagios::9997:nagios

Modify file /etc/shadow using an example from another RHEL7 clon machine (clonioc2 etc). Do chmod 600 shadow before editing and chmod 000 shadow after.

Make sure file /etc/ssh/sshd_config has following line uncommented (must be pushed in by CC security scripts):

##PermitUserEnvironment yes

NOT NEEDED, DONE IN THE END OF FILE ALREADY !!! Uncomment following in file /etc/ssh/ssh_config:

##Host *
##ForwardX11 yes

Add line 'export NO_AT_BRIDGE=1' to the file '/etc/environment' to suppress annoying messages like '** (emacs:38714): WARNING **: Couldn't register with accessibility bus: Did not receive a reply.'

If big (>2GB) partition to be formated, do following (for example /dev/sdb):

parted /dev/sdb
(parted) print
(parted) mklabel gpt
(parted) print
mkpart primary 0GB 25855.2GB
(parted) print
(parted) quit

mkfs.xfs /dev/sdb1

then create /data and mount it using /etc/fstab:

/dev/sdb1                                 /data                   xfs     defaults        1 2

NOTE: sometimes machine can loose NIS connection, then ssh to it becomes intermittent; if it happens ssh as root (may have to try several times) and run command

ypwhich

to shows NIS server. If it returns nothing check file /etc/resolv.conf, it must have following (as on July 2019):

search jlab.org acc.jlab.org
nameserver 129.57.90.255
nameserver 129.57.32.101

If file is empty or does not contain right info, fix it. After that run command

ypbind

It should fix a problem. Run ypwhich to make sure.

Proceed to the Linux Customization on CLON Cluster.

RHEL6 (64 bit)

Installed using CD provided by Computer Center. Requires MAC address to be registered with Computer Center (using jnet.jlab.org as 'boiarino') because it is network-based installation.

NOTE: all disks will be reformatted in a process, all information will be lost !

Dialog:

• boot: <enter>

• ...[D]esktop: [use 's' for server, 'd' for desktop] <enter>

• root password: enter root password

• admin user: boiarino

Installation will proceed. When prompted, remove installation disk and reboot machine if not done automatically. After reboot patching process will be started and it will take a while, do not interrupt it, otherwise system will not be installed correctly.

To make emacs work, install

yum install libotf

Edit the /etc/yum.conf file and add the following line to force installation of teh 32bit and 64bit libraries:

multilib_policy=all

Install following packages:

yum install openmotif-devel xinetd mysql-devel cmake libXpm-devel net-snmp-utils
yum install mesa-libGL-devel mesa-libGLU-devel mesa-libGLw-devel (for OpenGL, needed by ROOT)
yum install libXaw-devel (for xterm)
yum install expat-devel libgcj-devel (for evio/et)

yum install compat-libtermcap (for xterm)
ln -s /lib/libtermcap.so.2 /usr/lib/libtermcap.so
###ln -s /lib64/libtermcap.so.2 /usr/lib64/libtermcap.so

Install following fonts for EPICS EDMs:

yum install xorg-x11-fonts-75dpi xorg-x11-fonts-ISO8859-1-75dpi bitmap-miscfixed-fonts
yum install bitmap-console-fonts bitmap-fixed-fonts bitmap-lucida-typewriter-fonts
ln -s  /usr/share/X11/fonts/100dpi /etc/X11/fontpath.d/xorg-x11-fonts-100dpi:pri=60
ln -s  /usr/share/X11/fonts/75dpi /etc/X11/fontpath.d/xorg-x11-fonts-75dpi:pri=50
ln -s  /usr/share/X11/fonts/misc /etc/X11/fontpath.d/xorg-x11-fonts-misc:pri=40
ln -s  /usr/share/fonts/bitmap /etc/X11/fontpath.d/bitmap-fonts

NOTE: Useful yum commands:

yum whatprovides libstdc++.so.5 - show which package contains libstdc++.so.5

RHEL5

Installed using CD provided by Computer Center. Requires MAC address to be registered with Computer Center (email to helpdesk@jlab.org) because it is network-based installation.

Dialog:

• boot: <enter>

• ...[D]esktop: Server (it is ignored, will do Desktop anyway ...)

• root password: enter root password

• system user: <enter>

• manually partition disk: 100MB /boot, 2048 etc MB swap, the rest /.

CFENGINE: there are procesess restoring some system config files (in particular /etc/ssh/sshd_config) in according to the Computer Center - maintained templete. To shut them down: (1) comment out cfengine-related stuff in root cronjobs; (2) do /etc/init.d/cfengine stop; (3) do chkconfig cfengine off. Ask CC (Sherman White, Paul Letta) to fix templete.

Run /usr/bin/system-config-authentication and set NIS servers list as clon00,clon10,nis1.

Add following in the end of /etc/passwd:

clasrun:x:2508:9998:Online DAQ:/home/clasrun:/bin/tcsh
clas12run:x:6007:146:clas12run Account:/home/clas12run:/bin/tcsh
clascron:x:6005:146:CLAS Cron Jobs:/home/clascron:/bin/tcsh
clasboot:x:6000:146:Boot Scripts Only:/home/clasboot:/bin/tcsh
clasmail:x:6003:146:CLAS Automated Mail:/home/clasmail:/bin/tcsh
epics:x:5001:146:EPICS Account:/home/epics:/bin/tcsh
nagios:x:6246:9997:Nagios:/home/nagios:/bin/tcsh
+@clon_cluster::::::
+@ccc::::::

Add following in the end of /etc/group:

onliners::9998:heyes,hujl,rwm,timmer,wolin,taylorw,cuevas,abbottd,abulafia,anciant,anghi,aswin,auger,avakian,banta,barrow,battagli,bjlee,bogdan,brooksw,bruhwel,boiarino,buckle,burchesk,burin,bzh,cetina,chen,cole,cords,corvi,costy,davidl,costy,davidl,dcarter,dcurry,dennisl,deppman,dhlee,doughty,drago,dytman,dugger,dugger,elton,fedro,ficenec,fklein,france,freyberg,gholden,gilfoyle,golovach,gurjyan,haddock,heddle,jenkins,jhardie,jmorris,kellyk,klusman,kmkim,kossov,lcsmith,longhi,manak,marchand,marki,mastersj,mjohnson,mosbackr,mueller,muguira,murphyl,necaise,odonnell,opper,parkkj,pasyuk,phkim,piot,riccardi,richardt,ripani,rossi,saw,sergpozd,serov,sholt,staylor,taiuti,tang,tedeschi,vineyard,vlassov,vvsap,weygand,wooyoung,burchesk,clasrun,giovanet,clasboot,clascoda,clasroot,clasmail,clascron,trigger,epics,southern,carstens,ccutter,danagu,heyes,huertas,jacobsg,kliv,lachniet,mestayer,primex,hovanes,nerses,sytnik
clon::383:clascoda,clascron,clasmail
nagios::9997:nagios

Create /etc/shadow files with entries for users clasrun, epics and nagios (copy from another machine).

Install updates: normally all updates should be installed by now, it can be done manually by commands

yum list updates
yum update

Configure SSH.

Install missing packages:

yum install openmotif-devel xinetd mysql-devel cmake

Some additional packages on request (for EPICS):

yum install mc lapack lapack-devel blas blas-devel giflib-devel

Proceed to the Linux Customization on CLON Cluster.

PROBLEMS:

1. /usr/lib/libg2c.so link is missing

FIXED ??? 2. /usr/lib/libXm.so link is missing

3. Created /etc/shadow file for clasrun, epics and nagios

RHEL4

Get RHEL4 CDs from Paul Letta.

Connect terminal, keyboard and mouse.

Boot from CD1. Follow instructions. When asked, choose following options:

• Choose 'Manually partition with Disk Druid', set /boot to 100MB, then / to 40GB, then swap to 8192MB or at least double memory size. The rest of disk can be set as /space.

• Network: choose manually, type clonpc2.jlab.org; click 'EDIT', deactivate 'Configure using DHCP',

type IP address and netmask (255.255.255.0), click Ok. Type Gateway (for example 129.57.167.99) and 3 DNS servers (129.57.167.5,129.57.167.14,129.57.32.100).

• Choose 'No firewall'

• Disable 'SELinux'

• Choose 'Customize software packages to be installed', click 'Everything'.

You will be asked to install CD2, CD3 and CD4, then CD1 again. Then you will be asked to click 'Reboot'. First remove CD, then click 'Reboot'. You will be asked few more questions. When asked to activate service, choose 'Tell me why I need to register ..' and click 'Next'. Choose 'I can not complete registration ..' and click 'Next'. You will see 'System user' screen, click 'Use network login..'. Choose 'Enable NIS support' and click 'Configure NIS'. Type NIS Domain: 'CCCHP' and NIS Server: clon00.jlab.org. Finish installation process.

• fix files /etc/passwd, /etc/shadow and /etc/group adding clasrun etc (see examples on another machines).

NOTE: if machine has multiple network ports, Linux may not assign them properly to specified IP names and addresses. If problem occures, open network GUI and delete all existing records. After that, click New and pick hadrware port you want to configure from the list of available ports (tricky part is to know which RJ-45 connector corresponds to which port ..). Configure and activate it, make sure it works. Repeat for other ports. Results will be stored in directory /etc/sysconfig/network-scripts, file names will be ifcfg-eth0, ifcfg-eth1 etc. If modifying those files manually, restart network by /etc/init.d/network restart. Finally, set routing, for example for clonmon command route -n shows following:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
129.57.160.0    0.0.0.0         255.255.255.0   U     0      0        0 eth1
129.57.68.0     0.0.0.0         255.255.255.0   U     0      0        0 eth3
129.57.69.0     0.0.0.0         255.255.255.0   U     0      0        0 dev5937
129.57.64.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth3
0.0.0.0         129.57.68.100   0.0.0.0         UG    0      0        0 eth3

Run following commands:

/sbin/route add -net 129.57.160.0 netmask 255.255.255.0 gw 129.57.160.5 dev eth1
/sbin/route add -net 129.57.68.0 netmask 255.255.255.0 gw 129.57.68.26 dev eth3
/sbin/route add -net 129.57.69.0 netmask 255.255.255.0 gw 129.57.69.206 dev dev5937
/sbin/route add -net 129.57.64.0 netmask 255.255.255.0 gw 129.57.64.57 dev eth0

Now route -n shows following:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
129.57.160.0    129.57.160.5    255.255.255.0   UG    0      0        0 eth1
129.57.160.0    0.0.0.0         255.255.255.0   U     0      0        0 eth1
129.57.68.0     129.57.68.26    255.255.255.0   UG    0      0        0 eth3
129.57.68.0     0.0.0.0         255.255.255.0   U     0      0        0 eth3
129.57.69.0     129.57.69.206   255.255.255.0   UG    0      0        0 dev5937
129.57.69.0     0.0.0.0         255.255.255.0   U     0      0        0 dev5937
129.57.64.0     129.57.64.57    255.255.255.0   UG    0      0        0 eth0
129.57.64.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth3
0.0.0.0         129.57.68.100   0.0.0.0         UG    0      0        0 eth3

In that example automount will do configured using clonmon-daq1 interface, because default clonmon interface is on external subnet. To make that routing table permanent (retore it after reboot) I tried to add four files with following contents to the /etc/sysconfig/network-scripts directory:

route-eth1:
  GATEWAY0=129.57.160.5
  NETMASK0=255.255.255.0
  ADDRESS0=129.57.160.0
route-eth3:
  GATEWAY0=129.57.68.26
  NETMASK0=255.255.255.0
  ADDRESS0=129.57.68.0
route-dev5937:
  GATEWAY0=129.57.69.206
  NETMASK0=255.255.255.0
  ADDRESS0=129.57.69.0
route-eth0:
  GATEWAY0=129.57.64.57
  NETMASK0=255.255.255.0
  ADDRESS0=129.57.64.0

It did not worked. Then I added four /sbin/route add .. lines to /etc/rc.local script, and it worked.

From now on you can use remote 'ssh' connection.

• To configure your system to use the patch server, issue the following commands as root:

# mount jlabsite:/site /mnt
# /mnt/CC/linux/satellite-connect --patchall

The script will prompt you for an activation key, but if you don't know what this is, just hit ENTER and it will use a sensible default. Depending on how many patches need to be applied to your system to bring it up to the current level, this script may take a very long time to run. When it finishes, you should reboot your machine to make the new patches take effect.

To install updates later run up2date -uf as root.

Configure SSH.

• Proceed with Linux Customization on CLON Cluster if applicable.

To boot Linux in single user mode (no graphics), add single to the end of boot line during boot process.

background info

Add startup script

Create script in /etc/init.d/ directory (use existing scripts as example). Must have a line like

# chkconfig:	- 91 09

which specifies start and stop order. Run commands

chkconfig --add <scriptname>
chkconfig --level 3 <scriptname> on

for every level you want that script to be executed (usualy 3 and/or 4 and/or 5). Check if it os done using command

chkconfig --list | grep <scriptname>

xxx

You can choose to download the RHEL operating system directly from the Red Hat Web site and not receive a media kit from HP containing the software. This scenario occurs when the media option AJR is deselected from the order. (The media kit option is always selected by default. )

It is necessary to register at the Red Hat Web site prior to download. Registration requires the activation code that each customer receives from HP when ordering either RHEL.

You can download RHEL for each available platform as four CD ISO images or one DVD ISO image.

Use the following procedure to download RHEL and create the installation media:

  1.

     Register and log in at the following Red Hat Web site:

     www.redhat.com/rhn
  2.

     Click on channels and then select Red Hat Enterprise Linux ES (v.4 for 64–bit Intel Itanium).
  3.

     Select Downloads and then download the ISO images of install, compatibility and source disks

     Files with ia64 in the name are for the Itanium platform.
  4.

     Create the CDs or a DVD from the ISO images.

IMPORTANT: Create the CDs or DVD using the contents of the ISO images. Do not create the CDs or DVD by burning the .iso files themselves to CD or DVD. For example, if you are using K3b to burn a CD, click Tools>CD>Burn CD Image, select the .iso image, then click Burn.

Bootup scripts for example for level 5 execution (multiuser with X):

cp <script-file> /etc/init.d/
chmod +x /etc/init.d/<script-file>
ln -s /etc/init.d/<scriptfile> /etc/rc.d/rc5.d/S50<scriptfile>
ln -s /etc/init.d/<scriptfile> /etc/rc.d/rc5.d/K50<scriptfile>

Changing hostname/IP address

Changing hostname on RHEL
1. Change the HOSTNAME line in /etc/sysconfig/network
2. Change the hostname (FQDN and alias) in /etc/hosts
3. Run '/bin/hostname new_hostname' for the hostname change to take effect immediately.
4. Run '/sbin/service syslog restart' for syslog to log using the new hostname.
A reboot is not required to change the system hostname.

IP address: You have to change the IP address entry in
/etc/sysconfig/network-scripts/ifcfg-eth0.