Linux Boot Server: Difference between revisions

From CLONWiki
Jump to navigation Jump to search
Boiarino (talk | contribs)
No edit summary
Boiarino (talk | contribs)
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
'''Setting of the Linux server to be used for PXE and EFI boot of the VME controllers'''
== '''VTP CentOS boot''' ==
 
'''UBOOT (example for gem0vtp):'''
 
Zynq> printenv
autoload=no
baudrate=115200
bootcmd=run tftpboot
bootdelay=3
bootro=setenv bootargs ${bootargs} ro
bootrw=setenv bootargs ${bootargs} rw
devicetree_image=devicetree.dtb
devicetree_load_address=0x2000000
ethact=ethernet@e000b000
ethaddr=ce:ba:f0:03:00:d1
extargs=setenv bootargs ${bootargs} console=ttyPS0,115200 ip=:::::eth0:dhcp uio_pdrv_genirq.of_id=my-uio rootwait cma=384MB
fdt_high=0x20000000
fdtcontroladdr=3ffa9e30
initrd_high=0x20000000
kernel_image=uImage-4.4.0-xilinx-00004-g8d932a
kernel_load_address=0x2080000
kernel_path=linux-install/ArchLinux-armv71
load_devicetree=if tftpboot ${devicetree_load_address} ${tftppath}/${devicetree_image}; then echo OK; else echo FAILED; reset; fi;
load_dhcp=if dhcp; then echo OK; else echo FAILED; reset; fi;
load_kernel=if tftpboot ${kernel_load_address} ${tftppath}/${kernel_image}; then echo OK; else echo FAILED; reset; fi;
modeboot=sdboot
netmask=255.255.255.0
nfsargs=setenv bootargs nfsroot=${rootserverip}:${rootpath},vers=3
rootpath=/vol/diskless/CentOS7/armv7/root
rootserverip=129.57.167.152
serverip=129.57.167.4
stderr=serial@e0000000
stdin=serial@e0000000
stdout=serial@e0000000
tftp_boot_string=echo TFTPing Image to RAM
tftpboot=run load_dhcp tftp_boot_string nfsargs extargs bootro load_kernel load_devicetree && bootm ${kernel_load_address} - ${devicetree_load_address}
tftpboot_rw=dhcp && run tftp_boot_string nfsargs extargs bootrw load_kernel load_devicetree && bootm ${kernel_load_address} - ${devicetree_load_address}
tftppath=linux-install/CentOS7-armv7-Diskless
Environment size: 1603/131068 bytes
Zynq>
 
'''tftpboot server settings'''
 
/tftpboot/linux-install/CentOS7-armv7-Diskless/uImage-4.4.0-xilinx-00004-g8d932a
                                                uImage-4.4.0-xilinx-00003-gdf4da5
                                                zynq-jlab-vtp.dtb_fixed_uio
                                                devicetree.dtb -> zynq-jlab-vtp.dtb_fixed_uio
 
By default, root filesystem will be read-only. To make it 'rw', stop booting process on serial connection, type ''run tftpboot_rw'' and push <enter>. After that you can ssh as root and run ''yum'' etc.
 
== '''Setting of the Linux server to be used for PXE and EFI boot of the VME controllers''' ==


Login to the server as root. Copy two files to the root directory and untar them:
Login to the server as root. Copy two files to the root directory and untar them:
Line 58: Line 109:
Configure [[DHCP server]]
Configure [[DHCP server]]


Configure [[DNS server]]
Configure [[DNS server]] ('''NOTE:''' resolv.conf is different for local server !)


Configure [[Tftp]] and [[Tftpboot]]
Configure [[Tftp]] and [[Tftpboot]]
Line 71: Line 122:




'''Configuring snapshot area'''


The list of files and directories which suppose to be in snapshot area (which has RW permissions) should be set in config file ''/diskless/.../root/etc/statetab''. In particular, ''/et'' has to be added there, and also directory ''/diskless/.../root/et'' should be created.


'''Reboot server, make sure everything is running as expected.'''


== '''Configuring in chroot''' ==


Since root area in VME controllers will be read-only, you cannot ssh to controllers as root and change anything there, only snapshot area can be modified. To make changes in root area, in particular to configure and run ''yum'', ''chroot' command should be used. Login to the boot server as root and do following:


 
  mount -o bind /home /diskless/CentOS7/x86_64/root/home
'''old gefvme removal'''
  mount -o bind /usr/local /diskless/CentOS7/x86_64/root/usr/local
 
yum remove kmod-gefvme gefvme-library
 
If failed on 'gefvme-library', do
  rpm -e --noscripts gefvme-library
 
If file ''/etc/modules-load.d/gefvme.conf'' was not removed by above commands, remove it manually (if working from ''chroot ...'' then everything is done in ''root'' area, but ''gefvme.conf'' may survive in ''snapshot'' area, in that case remove it on controller and not in ''chroot'').
 
File ''/etc/modules-load.d/vme.conf'' with contents
cmem_rcc
jvme
vme_vivo
vme_ca91cx42
vme_tsi148
vme
may not be needed (if modules already in kernel ?).
 
'''new jvme (not sure if module installation is needed, maybe in kernel already ?) '''
 
'''On VME controller in BIOS, make sure VME memry size is set to 512M'''
 
Project git link: https://code.jlab.org/fedaq/drivers/jvme/-/tree/release-3.0
 
On vme controller as ''boiarino'', copy ''jvme-release-3.0.tar'' to ''$CODA/src'' and  untar it.
 
Create two environment scripts:
 
jvme_bash:
#!/bin/bash
export LINUXVME=${CODA}/src/jvme-release-3.0/linuxvme
export LINUXVME_INC=${LINUXVME}/include
export LINUXVME_LIB=${LINUXVME}/Linux_`uname -m`_vme/lib
export LINUXVME_BIN=${LINUXVME}/Linux_`uname -m`_vme/bin
export LD_LIBRARY_PATH=${LINUXVME_LIB}:${LD_LIBRARY_PATH}
export KERNELRELEASE=3.10.0-1062.9.1.el7.x86_64
 
jvme_tcsh:
#!/bin/tcsh
  setenv LINUXVME ${CODA}/src/jvme-release-3.0/linuxvme
setenv LINUXVME_INC ${LINUXVME}/include
setenv LINUXVME_LIB ${LINUXVME}/Linux_`uname -m`_vme/lib
setenv LINUXVME_BIN ${LINUXVME}/Linux_`uname -m`_vme/bin
setenv LD_LIBRARY_PATH ${LINUXVME_LIB}:${LD_LIBRARY_PATH}
setenv KERNELRELEASE 3.10.0-1062.9.1.el7.x86_64
 
Run ''source jvme_tcsh''.
Go inside ''jvme-release-3.0''.
In two files ''CMakeLists.txt'' and ''src/CMakeLists.txt'', change
set(libpath Linux-${CMAKE_SYSTEM_PROCESSOR}/lib)
set(libpath Linux-${CMAKE_SYSTEM_PROCESSOR}/bin)
to
set(libpath Linux_${CMAKE_SYSTEM_PROCESSOR}_vme/lib)
set(libpath Linux_${CMAKE_SYSTEM_PROCESSOR}_vme/bin)
 
Type ''cmake -B build -S . -DCMAKE_INSTALL_PREFIX=$LINUXVME''
 
Fix Makefile in ''kernel_driver'' and three it's subdirectories, it must have following in the beginning:
KVERSION := $(KERNELRELEASE)
ifeq ($(origin KERNELRELEASE), undefined)
KVERSION := $(shell uname -r)
endif
 
In directory ''jvme-release-3.0'', type ''make'' and ''make install'',
 
Do ''cd kernel_driver'' and ''make'' (do NOT do ''make install'').
 
On the server, do
 
  mount -o bind /usr/clas12 /diskless/CentOS7/x86_64/root/usr/clas12
  mount -o bind /usr/clas12 /diskless/CentOS7/x86_64/root/usr/clas12
mount -o bind /usr/local /diskless/CentOS7/x86_64/root/usr/local
mount -o bind /home /diskless/CentOS7/x86_64/root/home
  chroot /diskless/CentOS7/x86_64/root
  chroot /diskless/CentOS7/x86_64/root
cd /usr/clas12/release/2.0.0/coda/src
source jvme_bash
cd jvme-release-3.0/kernel_driver
make install
Still on server, add two files to ''/etc/udev/rules.d'' directory:
99-cmem.rules:
KERNEL=="cmem_rcc", MODE="0666"


99-vme.rules:
After that you will see all root directories as if you login to VME controller, and modifications can be made.
KERNEL=="bus/vme/ctl", MODE="0666"
KERNEL=="bus/vme/m_a16", MODE="0666"
KERNEL=="bus/vme/m_a24", MODE="0666"
KERNEL=="bus/vme/m_a32", MODE="0666"
KERNEL=="bus/vme/m_crcsr", MODE="0666"
KERNEL=="bus/vme/s_a32", MODE="0666"
KERNEL=="bus/vme/s_rsvd1", MODE="0666"
KERNEL=="bus/vme/s_rsvd2", MODE="0666"
KERNEL=="bus/vme/s_rsvd3", MODE="0666"


Reboot controller, check if everything is good.
'''Configure yum'''
 
'''NOTE''': if changing something in kernel module(s), it is not needed to reboot every time after make/make install is done in ''jvme-release-3.0/kernel_driver'' directory. Just run ''./load_driver.sh'' as root on controller, and all modules will be reloaded.
 
 
'''NOTE''': to add ''/et'' to snapshot area: on server, create /et directories in both root and snapshot areas, then add line '/et' to /etc/statetab file in root area, then reboot controller.
 
 
 
 
'''Enable systemd log persistency (remember all reboots, not only last one)'''
 
Run ''emacs /etc/systemd/journald.conf'', set
 
Storage=persistent
 
Do following:
 
mkdir /var/log/journal
systemd-tmpfiles --create --prefix /var/log/journal
systemctl restart systemd-journald
 
'''yum'''
 
On server where vme is loading from (as root; ''/zzz'' will be needed for mysqltcl installation below):
 
mkdir /diskless/CentOS7/x86_64/root/zzz
#mount -o bind /usr/local/src /diskless/CentOS7/x86_64/root/zzz
mount -o bind /usr/local /diskless/CentOS7/x86_64/root/usr/local
mount -o bind /usr/clas12 /diskless/CentOS7/x86_64/root/usr/clas12
chroot /diskless/CentOS7/x86_64/root


Add ''multilib_policy=all'' to ''/etc/yum.conf''.
Add ''multilib_policy=all'' to ''/etc/yum.conf''.
Line 217: Line 155:
Install following using yum:
Install following using yum:


  yum install motif-devel tcl-devel tk-devel libXpm-devel apr-devel libXaw-devel ncurses-devel  
  yum install motif-devel tcl-devel tk-devel libXpm-devel apr-devel libXaw-devel ncurses-devel libconfig-devel


Install following for ''dbedit'':
Install following for ''dbedit'':
Line 231: Line 169:




'''NOTE''': Login from console may not work because of file ''/etc/securetty'' permissions, it must be 644.


 
'''NOTE''': THere is a service PAM, it may prevent login from console if some required services did not start. To work around, comment out some lines in ''/etc/pam.d/system-auth-ac'' file:
 
 
 
 
 
 
 
 
Login from console may not work because of file ''/etc/securetty'' permissions, it must be 644.
 
 
THere is a service PAM, it may prevent login from console if some required services did not start. To work around, comment out some lines in ''/etc/pam.d/system-auth-ac'' file:


  #%PAM-1.0
  #%PAM-1.0
Line 268: Line 195:
  session    [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
  session    [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
  #session    required      pam_unix.so
  #session    required      pam_unix.so
'''After Linux boot server is ready and operational, VME controllers can be booted. On the first boot, individual ''/diskless/.../snapshot/<controller name>'' directory will be created for each controller. For VME controller specific information see [[VME]]'''

Latest revision as of 13:18, 16 September 2025

VTP CentOS boot

UBOOT (example for gem0vtp):

Zynq> printenv
autoload=no
baudrate=115200
bootcmd=run tftpboot
bootdelay=3
bootro=setenv bootargs ${bootargs} ro
bootrw=setenv bootargs ${bootargs} rw
devicetree_image=devicetree.dtb
devicetree_load_address=0x2000000
ethact=ethernet@e000b000
ethaddr=ce:ba:f0:03:00:d1
extargs=setenv bootargs ${bootargs} console=ttyPS0,115200 ip=:::::eth0:dhcp uio_pdrv_genirq.of_id=my-uio rootwait cma=384MB
fdt_high=0x20000000
fdtcontroladdr=3ffa9e30
initrd_high=0x20000000
kernel_image=uImage-4.4.0-xilinx-00004-g8d932a
kernel_load_address=0x2080000
kernel_path=linux-install/ArchLinux-armv71
load_devicetree=if tftpboot ${devicetree_load_address} ${tftppath}/${devicetree_image}; then echo OK; else echo FAILED; reset; fi;
load_dhcp=if dhcp; then echo OK; else echo FAILED; reset; fi;
load_kernel=if tftpboot ${kernel_load_address} ${tftppath}/${kernel_image}; then echo OK; else echo FAILED; reset; fi;
modeboot=sdboot
netmask=255.255.255.0
nfsargs=setenv bootargs nfsroot=${rootserverip}:${rootpath},vers=3
rootpath=/vol/diskless/CentOS7/armv7/root
rootserverip=129.57.167.152
serverip=129.57.167.4
stderr=serial@e0000000
stdin=serial@e0000000
stdout=serial@e0000000
tftp_boot_string=echo TFTPing Image to RAM
tftpboot=run load_dhcp tftp_boot_string nfsargs extargs bootro load_kernel load_devicetree && bootm ${kernel_load_address} - ${devicetree_load_address}
tftpboot_rw=dhcp && run tftp_boot_string nfsargs extargs bootrw load_kernel load_devicetree && bootm ${kernel_load_address} - ${devicetree_load_address}
tftppath=linux-install/CentOS7-armv7-Diskless

Environment size: 1603/131068 bytes
Zynq>

tftpboot server settings

/tftpboot/linux-install/CentOS7-armv7-Diskless/uImage-4.4.0-xilinx-00004-g8d932a
                                               uImage-4.4.0-xilinx-00003-gdf4da5
                                               zynq-jlab-vtp.dtb_fixed_uio
                                               devicetree.dtb -> zynq-jlab-vtp.dtb_fixed_uio

By default, root filesystem will be read-only. To make it 'rw', stop booting process on serial connection, type run tftpboot_rw and push <enter>. After that you can ssh as root and run yum etc.

Setting of the Linux server to be used for PXE and EFI boot of the VME controllers

Login to the server as root. Copy two files to the root directory and untar them:

cd /
cp /usr/downloads/tftpboot.tar.gz_clondaq15 tftpboot.tar.gz
cp /usr/downloads/diskless.tar.gz_clondaq15 diskless.tar.gz
gunzip tftpboot.tar.gz
gunzip diskless.tar.gz
tar xvf tftpboot.tar
tar xvf diskless.tar
rm tftpboot.tar diskless.tar


Make sure correct snapshot location in root area /diskless/CentOS7/x86_64/root/etc/sysconfig/readonly-root:

CLIENTSTATE=192.168.10.1:/diskless/CentOS7/x86_64/snapshot

In /diskless/CentOS7/x86_64/root area, /etc/ssh/sshd_config must have following setting (to allow remote ssh):

#UsePAM yes

it will be propagated to the controller's snapshot area on the first boot when corresponding snapshot/<vme_controller_name> directory will be created. Not sure about other settings:

HostbasedAuthentication yes
#HostbasedAuthentication no
IgnoreRhosts no
#IgnoreRhosts yes
PasswordAuthentication yes
#PasswordAuthentication yes
PasswordAuthentication yes
#UsePrivilegeSeparation sandbox		# Default for new installations.
UsePrivilegeSeparation sandbox		# Default for new installations.


To add or remove VME controllers, modify following files:

/var/named/10.168.192.in-addr.arpa.db
/var/named/clontest.com
/etc/dhcp/dhcpd.conf

restart corresponding services:

systemctl restart dhcpd
systemctl restart named

and check services status:

systemctl status dhcpd
systemctl status named

To set gateway, add

net.ipv4.ip_forward=1

to /etc/sysctl.conf and execute

sysctl -p

Configure NFS

Configure DHCP server

Configure DNS server (NOTE: resolv.conf is different for local server !)

Configure Tftp and Tftpboot

Configure Iptables

Do not forget (is it for controllers ?):

chattr +i /etc/resolv.conf


yum install ypbind (????)


Configuring snapshot area

The list of files and directories which suppose to be in snapshot area (which has RW permissions) should be set in config file /diskless/.../root/etc/statetab. In particular, /et has to be added there, and also directory /diskless/.../root/et should be created.

Reboot server, make sure everything is running as expected.

Configuring in chroot

Since root area in VME controllers will be read-only, you cannot ssh to controllers as root and change anything there, only snapshot area can be modified. To make changes in root area, in particular to configure and run yum, chroot' command should be used. Login to the boot server as root and do following:

mount -o bind /home /diskless/CentOS7/x86_64/root/home
mount -o bind /usr/local /diskless/CentOS7/x86_64/root/usr/local
mount -o bind /usr/clas12 /diskless/CentOS7/x86_64/root/usr/clas12
chroot /diskless/CentOS7/x86_64/root

After that you will see all root directories as if you login to VME controller, and modifications can be made.

Configure yum

Add multilib_policy=all to /etc/yum.conf.

Add (and remove the rest ?) following to /etc/yum.repos.d/CentOS-Base.repo:

[base]
name=CentOS-$releasever - Base
baseurl=http://archive.kernel.org/centos-vault/centos/$releasever/os/$basearch/

Clean yum database:

rm /var/lib/rpm/__db.*

Install following using yum:

yum install motif-devel tcl-devel tk-devel libXpm-devel apr-devel libXaw-devel ncurses-devel libconfig-devel

Install following for dbedit:

yum install tix itcl itk

Install remaining tcl stuff from /usr/local/src, mounted as /zzz above:

cd /zzz/mysqltcl-3.052
make install
ln -s /usr/lib/mysqltcl-3.052 /usr/lib64/tcl8.5/mysqltcl-3.052


NOTE: Login from console may not work because of file /etc/securetty permissions, it must be 644.

NOTE: THere is a service PAM, it may prevent login from console if some required services did not start. To work around, comment out some lines in /etc/pam.d/system-auth-ac file:

#%PAM-1.0
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
#auth        required      pam_env.so
auth        sufficient    pam_unix.so nullok try_first_pass
#auth        requisite     pam_succeed_if.so uid >= 1000 quiet_success
#auth        required      pam_deny.so

#account     required      pam_unix.so
account     sufficient    pam_localuser.so
account     sufficient    pam_succeed_if.so uid < 1000 quiet
#account     required      pam_permit.so

password    requisite     pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type=
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
#password    required      pam_deny.so

session     optional      pam_keyinit.so revoke
#session     required      pam_limits.so
-session     optional      pam_systemd.so
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
#session     required      pam_unix.so




After Linux boot server is ready and operational, VME controllers can be booted. On the first boot, individual /diskless/.../snapshot/<controller name> directory will be created for each controller. For VME controller specific information see VME