GE XVB601

From CLONWiki
Jump to navigation Jump to search

Company site: GE Intelligent Platforms

Board page on company site: XVB601

Customization for the CLON cluster

List of boards

  • [01] 00:20:38:04:23:5E - 129.57.68.89 - croctest2

Customizing kernel installed on the FLASH

  • /usr/bin/system-config-network
  • fix /etc/passwd, /etc/shadow, /etc/group
  • fix /etc/nsswitch.conf
  • fix /etc/yp.conf (add lines domain CCCHP server clon00.jlab.org and ypserver clon10.jlab.org)
  • fix /etc/sysconfig/network (add line NISDOMAIN=CCCHP)
  • /etc/init.d/ypbind start
  • chkconfig ypbind on


  • /etc/init.d/autofs restart
  • fix symbolic links to /apps, /home, /usr/local etc


Trying netboot following Bryan's instructions, using clonpc3

  • yum install system-config-netboot
  • Change line (/etc/xinetd.d/tftp):
disable                 = yes
To
disable                 = no
  • service xinetd restart
  • Add these entries to /etc/exports
/diskless/x86_64/Centos5.5/root/     129.57.167.0/255.255.255.0(ro,sync,no_root_squash)
/diskless/x86_64/Centos5.5/snapshot/ 129.57.167.0/255.255.255.0(rw,sync,no_root_squash)
Where the 129.57.167.0/255.255.255.0 only allows the 167 subnet to access these mounts.
  • Reload the NFS configuration or start NFS, if it's not started
service nfs reload
  • Make sure NFS is started at boot
/sbin/chkconfig --level 345 nfs on
  • Enable remote syslog (optional)
Change syslog argument in /etc/sysconfig/syslog from:
SYSLOGD_OPTIONS="-m 0"
To
SYSLOGD_OPTIONS="-m 0 -r"

Restart syslogd
service syslog restart

DHCP Server Setup On the machine hosting the DHCP server.

  • Add an entry in /etc/dhcpd.conf:
 host dafarm44
              {
              filename "linux-install/pxelinux.0"; # File location relative to /tftpboot/
              next-server 129.57.29.38;            # TFTP server
              hardware ethernet 00:20:38:03:D8:52; # Client MAC
              fixed-address 129.57.29.54;          # Client IP
              }
  • Restart dhcpd
service dhcpd restart

Configure Client OS

Here, we configure the OS that each and every Diskless Client will enjoy.

Setup of Diskless Kernel Run system-config-netboot (on TFTP server): 

system-config-netboot
  • In "First Time Druid" window. Click on "Diskless"
  1. Click "Forward" on first page
  2. Enter Name and Description.
  3. Enter IP address of TFTP server and the "root" directory exported via NFS
  4. Select a kernel
  5. Click on Apply.
This will take a minute or two to compile a boot kernel for the diskless client.

Setup of chroot environment There are lots of specifics here for setup on the JLab CUE and Data Acquisition Network (daqfs). This enviroment will give you the capability to do system updates/upgrades for all diskless clients at the same time. 

chroot /diskless/x86_64/Centos5.5/root
mount /proc /proc -t proc
mkdir /u 
mkdir /group
mkdir /daqfs
mv /home /home.old
ln -s /u/apps /apps
ln -s /u/site /site
ln -s /daqfs/home /home

Setup of diskless system configurations

Which NFS directories to mount

  • Add entry to /etc/fstab
phecda:/raid    /daqfs  nfs     rw,soft,intr 0 0
  • Mount daqfs
mount /daqfs

Which users/groups to allow access

  • Add entries to /etc/passwd
+@da::::::
+@ccc::::::
+::::::/bin/false

A proxy for yum (may be optional)

  • Add entry to /etc/yum.conf
proxy=http://jprox:8080

domain servers for ypbind

  • Add entries to /etc/yp.conf
domain CCCHP server nis2.jlab.org
domain CCCHP server nis3.jlab.org
  • Make sure ypbind started at boot
/sbin/chkconfig --level 345 nfs on

NTP setup (optional... ntp must be installed)

  • Add entries to /etc/ntp.conf
server 127.0.0.1 # local clock
fudge 127.0.0.1 stratum 10
server ntp1.jlab.org
server ntp2.jlab.org
driftfile /etc/ntp/drift
broadcastdelay 0.008
authenticate no

Some additional JLab CUE directories to mount

  • Add entries to /etc/auto.master
/group          /etc/auto.group --timeout=0
/u              auto.u.bb       --DOSNAME=Linux-RHEL5

Additional files, not included in the standard diskless filesystem

  • Add a new file /diskless/x86_64/Centos5.5/snapshot/files.custom. Entry:
/tmp/
/etc/minirc.dfl

Configure a NEW diskless client This operation must be peformed to add a new Diskless Client.

Configuration on the tftp/NFS server

  • In system-config-netboot "Network Installation and Diskless Environment" window:
  1. Click on "New"
  2. Enter hostname
  3. Select OS
  4. Enter "CCCHP" in Enable NISDOMAIN box.
  5. Enter options in "Edit Extra Kernel Boot Options"
    vga=0x305 acpi=force
  6. Enter hostname as snapshot name. Make sure "generate" is check-marked
  7. Click "OK"

Configuration on the Diskless Client

  • PXEboot must be enabled and should be the primary boot option. This is set in the BIOS.

Other useful packages to install Later on... I installed some other useful packages using the chroot environment. Here they are:

  • xorg-x11-xauth - Required to run X-applications remotely
  • ntp - Synchronize system clock with a central server
To Execute ntpdate on server startup:
  • Add -x to OPTIONS in
/etc/sysconfig/ntpd
  • Make sure it starts on boot
/sbin/chkconfig --level 345 nfs on

Network-based booting

The page is geared towards the setup of Diskless nodes for Intel-based VME Controllers, but may be useful for similar other applications. Much of what is here was derived from the Redhat Enterprise Linux 3 Documentation.

This examples assumes:

  • The DHCP server (phecda: 129.57.29.104) has been installed and works. Here we're using the dhcp 3.0.1 package distributed by RHEL4.
  • The TFTP and NFS server (mizar: 129.57.29.100) is running RHEL5.5 x86_64.
  • The Diskless Client (dafarm44: 129.57.29.54) will run Centos 5.5 x86_64.

There's nothing stopping you from having the DHCP server on the same machine as that servering TFTP and NFS. You can obviously have more than one diskless client.

Once the first four steps have been completed, new clients can be configured individually with step 5.

Client Setup of OS in VirtualBox

Here, we've downloaded the latest and greatest Centos 5.5 x86_64 ISO and have set up a VirtualBox Client to use it to boot. We will not attempt to describe the VirtualBox installation or Setup of the ISO to boot as the VirtualBox Guest.

The Centos 5.5 installation proceeds as normal installations go, but we opt to do a minimal installation (excluding KDE and/or GNOME components) since we're running on a VME controller. Other packages will be installed after first boot.

First Boot

At first boot (after installation), a UI will appear and provide you with some options.:

  • Disable firewall and SELinux. SELinux setup can be double checked in the file:
/etc/selinux/config
  • Disable sendmail, and other services not required for a VME controller

Update packages and transfer the OS to Server

Login as root

  • Update all installed packages:
yum update
  • Install the following packages, with:
yum install package
busybox-anaconda
gcc
kernel-devel
gdb
valgrind
cvs
subversion
compat-libstdc++-33
redhat-rpm-config
  • Copy the Client OS to the tftp/NFS Server.
    • On The Server:
mkdir -p /diskless/x86_64/Centos5.5/root
In The VirtualBox Guest:
rsync -Pav -e ssh --exclude='/proc/*' --exclude='/sys/*' / mizar:/diskless/x86_64/Centos5.5/root/

You are now done with the Centos5.5 Guest installed in the VirtualBox. It can be closed down now.

tftp and NFS Server Setup

Follow these steps on the machine serving out tftp and NFS:

Install system-config-netboot

(This should automatically install the tftp server) 

yum install system-config-netboot

Enable the tftp server to start at boot

  • Change line (/etc/xinetd.d/tftp):
disable                 = yes
To
disable                 = no
  • Restart xinetd
service xinetd restart

Configure NFS

  • Add these entries to /etc/exports
/diskless/x86_64/Centos5.5/root/     129.57.29.0/255.255.255.0(ro,sync,no_root_squash)
/diskless/x86_64/Centos5.5/snapshot/ 129.57.29.0/255.255.255.0(rw,sync,no_root_squash)
Where the 129.57.29.0/255.255.255.0 only allows the 29 subnet to access these mounts.
  • Reload the NFS configuration or start NFS, if it's not started
service nfs reload
  • Make sure NFS is started at boot
/sbin/chkconfig --level 345 nfs on

Enable remote syslog (optional)

  • Change syslog argument in /etc/sysconfig/syslog from:
SYSLOGD_OPTIONS="-m 0"
To
SYSLOGD_OPTIONS="-m 0 -r"
  • Restart syslogd
service syslog restart

DHCP Server Setup

On the machine hosting the DHCP server.

  • Add an entry in /etc/dhcpd.conf:
 host dafarm44
              {
              filename "linux-install/pxelinux.0"; # File location relative to /tftpboot/
              next-server 129.57.29.38;            # TFTP server
              hardware ethernet 00:20:38:03:D8:52; # Client MAC
              fixed-address 129.57.29.54;          # Client IP
              }
  • Restart dhcpd
service dhcpd restart

Configure Client OS

Here, we configure the OS that each and every Diskless Client will enjoy.

Setup of Diskless Kernel

Run system-config-netboot (on TFTP server): 

system-config-netboot
  • In "First Time Druid" window. Click on "Diskless"
  1. Click "Forward" on first page
  2. Enter Name and Description.
  3. Enter IP address of TFTP server and the "root" directory exported via NFS
  4. Select a kernel
  5. Click on Apply.
This will take a minute or two to compile a boot kernel for the diskless client.

Setup of chroot environment

There are lots of specifics here for setup on the JLab CUE and Data Acquisition Network (daqfs). This enviroment will give you the capability to do system updates/upgrades for all diskless clients at the same time. 

chroot /diskless/x86_64/Centos5.5/root
mount /proc /proc -t proc
mkdir /u 
mkdir /group
mkdir /daqfs
mv /home /home.old
ln -s /u/apps /apps
ln -s /u/site /site
ln -s /daqfs/home /home

Setup of diskless system configurations

Which NFS directories to mount

  • Add entry to /etc/fstab
phecda:/raid    /daqfs  nfs     rw,soft,intr 0 0
  • Mount daqfs
mount /daqfs

Which users/groups to allow access

  • Add entries to /etc/passwd
+@da::::::
+@ccc::::::
+::::::/bin/false

A proxy for yum (may be optional)

  • Add entry to /etc/yum.conf
proxy=http://jprox:8080

domain servers for ypbind

  • Add entries to /etc/yp.conf
domain CCCHP server nis2.jlab.org
domain CCCHP server nis3.jlab.org
  • Make sure ypbind started at boot
/sbin/chkconfig --level 345 nfs on

NTP setup (optional... ntp must be installed)

  • Add entries to /etc/ntp.conf
server 127.0.0.1 # local clock
fudge 127.0.0.1 stratum 10
server ntp1.jlab.org
server ntp2.jlab.org
driftfile /etc/ntp/drift
broadcastdelay 0.008
authenticate no

Some additional JLab CUE directories to mount

  • Add entries to /etc/auto.master
/group          /etc/auto.group --timeout=0
/u              auto.u.bb       --DOSNAME=Linux-RHEL5

Additional files, not included in the standard diskless filesystem

  • Add a new file /diskless/x86_64/Centos5.5/snapshot/files.custom. Entry:
/tmp/
/etc/minirc.dfl

Configure a NEW diskless client

This operation must be peformed to add a new Diskless Client.

Configuration on the tftp/NFS server

  • In system-config-netboot "Network Installation and Diskless Environment" window:
  1. Click on "New"
  2. Enter hostname
  3. Select OS
  4. Enter "CCCHP" in Enable NISDOMAIN box.
  5. Enter options in "Edit Extra Kernel Boot Options"
    vga=0x305 acpi=force
  6. Enter hostname as snapshot name. Make sure "generate" is check-marked
  7. Click "OK"

Configuration on the Diskless Client

  • PXEboot must be enabled and should be the primary boot option. This is set in the BIOS.

Other useful packages to install

Later on... I installed some other useful packages using the chroot environment. Here they are:

  • xorg-x11-xauth - Required to run X-applications remotely
  • ntp - Synchronize system clock with a central server
To Execute ntpdate on server startup:
  • Add -x to OPTIONS in
/etc/sysconfig/ntpd
  • Make sure it starts on boot
/sbin/chkconfig --level 345 nfs on
Retrieved from "https://clonwiki0.jlab.org/wiki/index.php?title=GE_XVB601&oldid=5177"