Linux Boot Server

Setting of the Linux server to be used for PXE and EFI boot of the VME controllers

Login to the server as root. Copy two files to the root directory and untar them:

cd /
cp /usr/downloads/tftpboot.tar.gz_clondaq15 tftpboot.tar.gz
cp /usr/downloads/diskless.tar.gz_clondaq15 diskless.tar.gz
gunzip tftpboot.tar.gz
gunzip diskless.tar.gz
tar xvf tftpboot.tar
tar xvf diskless.tar
rm tftpboot.tar diskless.tar

Make sure correct snapshot location in root area /diskless/CentOS7/x86_64/root/etc/sysconfig/readonly-root:

CLIENTSTATE=192.168.10.1:/diskless/CentOS7/x86_64/snapshot

In /diskless/CentOS7/x86_64/root area, /etc/ssh/sshd_config must have following setting (to allow remote ssh):

#UsePAM yes

it will be propagated to the controller's snapshot area on the first boot when corresponding snapshot/<vme_controller_name> directory will be created. Not sure about other settings:

HostbasedAuthentication yes
#HostbasedAuthentication no

IgnoreRhosts no
#IgnoreRhosts yes

PasswordAuthentication yes
#PasswordAuthentication yes
PasswordAuthentication yes

#UsePrivilegeSeparation sandbox		# Default for new installations.
UsePrivilegeSeparation sandbox		# Default for new installations.

To add or remove VME controllers, modify following files:

/var/named/10.168.192.in-addr.arpa.db
/var/named/clontest.com
/etc/dhcp/dhcpd.conf

restart corresponding services:

systemctl restart dhcpd
systemctl restart named

and check services status:

systemctl status dhcpd
systemctl status named

To set gateway, add

net.ipv4.ip_forward=1

to /etc/sysctl.conf and execute

sysctl -p

Configure NFS

Configure DHCP server

Configure DNS server

Configure Tftp and Tftpboot

Configure Iptables

Do not forget (is it for controllers ?):

chattr +i /etc/resolv.conf

yum install ypbind (????)

old gefvme removal

yum remove kmod-gefvme gefvme-library

If failed on 'gefvme-library', do

rpm -e --noscripts gefvme-library

If file /etc/modules-load.d/gefvme.conf was not removed by above commands, remove it manually (if working from chroot ... then everything is done in root area, but gefvme.conf may survive in snapshot area, in that case remove it on controller and not in chroot).

File /etc/modules-load.d/vme.conf with contents

cmem_rcc
jvme
vme_vivo
vme_ca91cx42
vme_tsi148
vme

may not be needed (if modules already in kernel ?).

new jvme (not sure if module installation is needed, maybe in kernel already ?)

On VME controller in BIOS, make sure VME memry size is set to 512M

Project git link: https://code.jlab.org/fedaq/drivers/jvme/-/tree/release-3.0

On vme controller as boiarino, copy jvme-release-3.0.tar to $CODA/src and untar it.

Create two environment scripts:

jvme_bash:

#!/bin/bash
export LINUXVME=${CODA}/src/jvme-release-3.0/linuxvme
export LINUXVME_INC=${LINUXVME}/include
export LINUXVME_LIB=${LINUXVME}/Linux_`uname -m`_vme/lib
export LINUXVME_BIN=${LINUXVME}/Linux_`uname -m`_vme/bin
export LD_LIBRARY_PATH=${LINUXVME_LIB}:${LD_LIBRARY_PATH}
export KERNELRELEASE=3.10.0-1062.9.1.el7.x86_64

jvme_tcsh:

#!/bin/tcsh
setenv LINUXVME ${CODA}/src/jvme-release-3.0/linuxvme
setenv LINUXVME_INC ${LINUXVME}/include
setenv LINUXVME_LIB ${LINUXVME}/Linux_`uname -m`_vme/lib
setenv LINUXVME_BIN ${LINUXVME}/Linux_`uname -m`_vme/bin
setenv LD_LIBRARY_PATH ${LINUXVME_LIB}:${LD_LIBRARY_PATH}
setenv KERNELRELEASE 3.10.0-1062.9.1.el7.x86_64

Run source jvme_tcsh. Go inside jvme-release-3.0. In two files CMakeLists.txt and src/CMakeLists.txt, change

set(libpath Linux-${CMAKE_SYSTEM_PROCESSOR}/lib)
set(libpath Linux-${CMAKE_SYSTEM_PROCESSOR}/bin)

to

set(libpath Linux_${CMAKE_SYSTEM_PROCESSOR}_vme/lib)
set(libpath Linux_${CMAKE_SYSTEM_PROCESSOR}_vme/bin)

Type cmake -B build -S . -DCMAKE_INSTALL_PREFIX=$LINUXVME

Fix Makefile in kernel_driver and three it's subdirectories, it must have following in the beginning:

KVERSION := $(KERNELRELEASE)
ifeq ($(origin KERNELRELEASE), undefined)
KVERSION := $(shell uname -r)
endif

In directory jvme-release-3.0, type make and make install,

Do cd kernel_driver and make (do NOT do make install).

On the server, do

mount -o bind /usr/clas12 /diskless/CentOS7/x86_64/root/usr/clas12
mount -o bind /usr/local /diskless/CentOS7/x86_64/root/usr/local
mount -o bind /home /diskless/CentOS7/x86_64/root/home
chroot /diskless/CentOS7/x86_64/root
cd /usr/clas12/release/2.0.0/coda/src
source jvme_bash
cd jvme-release-3.0/kernel_driver
make install

Still on server, add two files to /etc/udev/rules.d directory:

99-cmem.rules:

KERNEL=="cmem_rcc", MODE="0666"

99-vme.rules:

KERNEL=="bus/vme/ctl", MODE="0666"
KERNEL=="bus/vme/m_a16", MODE="0666"
KERNEL=="bus/vme/m_a24", MODE="0666"
KERNEL=="bus/vme/m_a32", MODE="0666"
KERNEL=="bus/vme/m_crcsr", MODE="0666"
KERNEL=="bus/vme/s_a32", MODE="0666"
KERNEL=="bus/vme/s_rsvd1", MODE="0666"
KERNEL=="bus/vme/s_rsvd2", MODE="0666"
KERNEL=="bus/vme/s_rsvd3", MODE="0666"

Reboot controller, check if everything is good.

NOTE: if changing something in kernel module(s), it is not needed to reboot every time after make/make install is done in jvme-release-3.0/kernel_driver directory. Just run ./load_driver.sh as root on controller, and all modules will be reloaded.

NOTE: to add /et to snapshot area: on server, create /et directories in both root and snapshot areas, then add line '/et' to /etc/statetab file in root area, then reboot controller.

Enable systemd log persistency (remember all reboots, not only last one)

Run emacs /etc/systemd/journald.conf, set

Storage=persistent

Do following:

mkdir /var/log/journal
systemd-tmpfiles --create --prefix /var/log/journal
systemctl restart systemd-journald

yum

On server where vme is loading from (as root; /zzz will be needed for mysqltcl installation below):

mkdir /diskless/CentOS7/x86_64/root/zzz
#mount -o bind /usr/local/src /diskless/CentOS7/x86_64/root/zzz
mount -o bind /usr/local /diskless/CentOS7/x86_64/root/usr/local
mount -o bind /usr/clas12 /diskless/CentOS7/x86_64/root/usr/clas12
chroot /diskless/CentOS7/x86_64/root

Add multilib_policy=all to /etc/yum.conf.

Add (and remove the rest ?) following to /etc/yum.repos.d/CentOS-Base.repo:

[base]
name=CentOS-$releasever - Base
baseurl=http://archive.kernel.org/centos-vault/centos/$releasever/os/$basearch/

Clean yum database:

rm /var/lib/rpm/__db.*

Install following using yum:

yum install motif-devel tcl-devel tk-devel libXpm-devel apr-devel libXaw-devel ncurses-devel 

Install following for dbedit:

yum install tix itcl itk

Install remaining tcl stuff from /usr/local/src, mounted as /zzz above:

cd /zzz/mysqltcl-3.052
make install
ln -s /usr/lib/mysqltcl-3.052 /usr/lib64/tcl8.5/mysqltcl-3.052

Login from console may not work because of file /etc/securetty permissions, it must be 644.

THere is a service PAM, it may prevent login from console if some required services did not start. To work around, comment out some lines in /etc/pam.d/system-auth-ac file:

#%PAM-1.0
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
#auth        required      pam_env.so
auth        sufficient    pam_unix.so nullok try_first_pass
#auth        requisite     pam_succeed_if.so uid >= 1000 quiet_success
#auth        required      pam_deny.so

#account     required      pam_unix.so
account     sufficient    pam_localuser.so
account     sufficient    pam_succeed_if.so uid < 1000 quiet
#account     required      pam_permit.so

password    requisite     pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type=
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
#password    required      pam_deny.so

session     optional      pam_keyinit.so revoke
#session     required      pam_limits.so
-session     optional      pam_systemd.so
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
#session     required      pam_unix.so