ATCA

From CLONWiki
Jump to navigation Jump to search

ATCA electronics for HPS run designed in SLAC

  • packages needed:
yum install cppzmq-devel
  • serial connection to atca1 crate: connect atca's serial port to computer USB using appropriate adapter, and use minicom:
minicom -D /dev/ttyUSB0 (set 115200 8N1)
root/HpsRoot
# reboot <- will recycle atca crate power

#ifconfig eth0 down
#ifconfig eth0 up 129.57.86.147
  • atca crate is connected to the power strip, following command will recycle power:
roc_reboot atca1
  • ethernet connection to atca1 crate:
ssh -o RSAMinSize=1024 -o HostKeyAlgorithms=+ssh-rsa root@atca1
  • serial connection to COB blade (port labeled 'console', need usb-micro cable; NOTE: some cables we tried were not good, not sure why
minicom -D /dev/ttyUSB0 (set 115200 8N1)
  • old (nfs-based) bootloader:
zynq-uboot> printenv
autoload=no
baudrate=115200
bootcmd=run $modeboot
bootdefs=console=ttyPS0,115200 rw rootwait earlyprintk
bootdelay=3
devicetree_image=devicetree.dtb
ethact=Gem.e000b000
ethaddr=08:00:56:00:46:0d
kernel_image=uImage
loadbit=0
modeboot=nfsboot
nfsargs=setenv bootargs ${bootdefs} root=/dev/nfs nfsroot=${serverip}:${rootpath} ip=dhcp
nfsboot=dhcp && run nfsload && run nfsargs && bsir && run nfsfpga && bsic && bootm 0x3000000 - 0x2A00000
nfsfpga=nfs 0x1000000 ${rootpath}/boot/fpga.bit && fpga loadb 0 0x1000000 ${filesize}
nfsload=nfs 0x3000000 ${rootpath}/boot/${kernel_image} && nfs 0x2A00000 ${rootpath}/boot/${devicetree_image}
phycfg=1
ramdisk_devicetree=devicetree_ramdisk.dtb
ramdisk_image=uramdisk.image.gz
ramdisk_kernel=uImage
ramdisk_rootfs=/dev/ram
rtems_image=urtems.elf
sd_rootfs=/dev/mmcblk0p3
sdboot_linux=setenv modeboot sdboot_linux && echo Copying Linux from SD to RAM... && mmcinfo && fatload mmc 0:7 0x3000000 ${kernel_image} && fatload mmc 0:7 0x2A00000 ${devicetree_image} && bootm 0x3000000 - 0x2A00000
sdboot_rdisk=setenv modeboot sdboot_ramdisk && echo Copying Linux from SD to RAM... && mmcinfo && fatload mmc 0:7 0x3000000 ${ramdisk_kernel} && fatload mmc 0:7 0x2A00000 ${ramdisk_devicetree} && fatload mmc 0:7 0x2000000 ${ramdisk_image} && bootm 0x3000000 0x2000000 0x2A00000
sdboot_rtems=echo Copying RTEMS from SD to RAM... && mmcinfo && fatload mmc 0:5 0x3000000 ${rtems_image} && bootm 0x3000000
stderr=serial
stdin=serial
stdout=serial

Environment size: 1527/131068 bytes
zynq-uboot>
  • new (tftp-based) bootloader:
zynq-uboot> printenv
autoload=no
baudrate=115200
bootcmd=run $modeboot
bootdefs=console=ttyPS0,115200 rw rootwait earlyprintk
bootdelay=3
devicetree_image=devicetree.dtb
dnsip=129.57.90.255
enable-tftp=enp16s0f0
ethact=Gem.e000b000
ethaddr=08:00:56:00:46:0d
fileaddr=2a00000
filesize=2a1d
gatewayip=129.57.86.1
hostname=dtm1
ipaddr=129.57.86.180
kernel_image=uImage
kernel_path=slac-install
loadbit=0
modeboot=tftp_boot
netmask=255.255.255.0
nfsargs=setenv bootargs ${bootdefs} root=/dev/nfs nfsroot=${nfsip}:${nfspath}/${rootpath},vers=3,proto=tcp ip=dhcp
nfsfpga=tftpboot 0x1000000 ${kernel_path}/boot/fpga.bit && fpga loadb 0 0x1000000 ${filesize}
nfsip=129.57.167.152
nfspath=vol/diskless/ArchLinux/zynq_slac
phycfg=1
ramdisk_devicetree=devicetree_ramdisk.dtb
ramdisk_image=uramdisk.image.gz
ramdisk_kernel=uImage
ramdisk_rootfs=/dev/ram
rootpath=slac-install
rtems_image=urtems.elf
sd_rootfs=/dev/mmcblk0p3
sdboot_linux=setenv modeboot sdboot_linux && echo Copying Linux from SD to RAM... && mmcinfo && fatload mmc 0:7 0x3000000 ${kernel_image} && fatload mmc 0:7 0x2A00000 ${devicetree_image} && bootm 0x3000000 - 0x2A00000
sdboot_rdisk=setenv modeboot sdboot_ramdisk && echo Copying Linux from SD to RAM... && mmcinfo && fatload mmc 0:7 0x3000000 ${ramdisk_kernel} && fatload mmc 0:7 0x2A00000 ${ramdisk_devicetree} && fatload mmc 0:7 0x2000000 ${ramdisk_image} && bootm 0x3000000 0x2000000 0x2A00000
sdboot_rtems=echo Copying RTEMS from SD to RAM... && mmcinfo && fatload mmc 0:5 0x3000000 ${rtems_image} && bootm 0x3000000
serverip=129.57.167.4
stderr=serial
stdin=serial
stdout=serial
tftp_boot=dhcp && run tftp_load && run nfsargs && bsir && run nfsfpga && bsic && bootm 0x3000000 - 0x2A00000
tftp_load=tftpboot 0x3000000 ${kernel_path}/boot/${kernel_image} && tftpboot 0x2A00000 ${kernel_path}/boot/${devicetree_image}

Environment size: 1893/131068 bytes
zynq-uboot>


  • DTMs/DPMs nfs mount

Clonfarm1 machine is used as file server for DTMs/DPMs, because they are not supporting newer nfs versions required by modern file servers like clonfs1. Make sure nfs server is set and running on clonfarm1. File /etc/exports suppose to contain following lines: 

/data/hps/slac_svt/daq 129.57.0.0/16(rw,sync,no_subtree_check,no_root_squash)
#/data/hps/slac_svt/cache 129.57.0.0/16(rw,sync,no_subtree_check,no_root_squash)
/data/hps/slac_svt/server 129.57.0.0/16(rw,sync,no_subtree_check,no_root_squash)
/data/hps/slac_svt/diskless 129.57.0.0/16(rw,insecure,async,no_subtree_check,no_root_squash)

Run following to make sure, restart if needed: 

systemctl status rpcbind
systemctl status nfs-server


  • DTMs/DPMs boot

They are dhcp-boot devices, all settings are done by jlab computer center (Carl Bolitho). After boot, check it by logging in, for example ssh root@dtm1 with pass root (be patient, may take some seconds to login).


  • some useful scripts in $CLAS/slac_svt/svtdaq/daqV2/rceScripts/:
reboot_cobs.sh - crate power recycle
shutdown_cobs.sh - turn power off (can be turned back on by reboot_cobs.sh)
rem_udp_server.sh - ... (have to run it)
rem_control_server.sh - start GUI server
start_gui.sh - start gui

EEL building test setup

machine used: clondaq10 with TIPCIe card to connect atca blades, and svt3 as master if clondaq10 is slave
run as 'hpsrun', pass the same as clasrun
EXPID=hpsrun, SESSION=clastest10
coda config: clondaq10_er, svt3_clondaq10_er (rol1=hpssvt1_master.so or hpssvt1_slave.so, rol2=rol2.so)
trigger files: HPS/EEL2021/hps_v1_noThr_TOPRCE_TIMaster.trg, HPS/EEL2021/hps_v1_noThr_TOPRCE_VMEMaster.trg
data files monitoring: $CODA/src/bosio/main/evio_hpssvthist.c, kumac hps.kumac

Cable TIPCIe-ATCA blade pinout ('new' TIPCIe): 

signal        tipcie(40-pin)       atca(26-pin)
-------------------------------------------------
syncreset      1/2           --->      3/4
trigger        3/4           --->      5/6
busy          17/18          <---     11/12
clock         39/40          --->      7/8

Login and use of the test setup at SLAC

Tunneling to and starting VNC server at SLAC: 

ssh -L 5905:localhost:1234 jlabl1
ssh -L 1234:localhost:5905 rdsrv117.slac.stanford.edu
vncserver :5 -geometry 1800x1000 -localhost -nolisten tcp
top

vncserver -kill :5


To have write access on develpc: 

ssh clasrun@develpc
sudo tcsh


Running DAQ: 

clasrun account can get root access with:
sudo bash

ssh clasrun@slac1
killall runcontrol rcServer rocs (cleanup just in case)
runcontrol -rocs -log
click on "rocs" tab on the right-top of the screen to observe built-in xterms
click "Configure", pick configuration "slac", processes have to be started in xterms
click "Download" and pick trigger file "slac1.trg"
click "Prestart"
click "Go"
  • trigger file "slac1.trg" is located in default directory

$CLON_PARMS/trigger/. It contains electronics settings overloading defaults from $CLON_PARMS/ti, $CLON_PARMS/fadc250/ etc. In particular 

  TI_RANDOM_TRIGGER 1 3

enables TI internal random pulser with prescale 2**3, with prescale 3 event rate will be about 50kHz.


Rogue GUI: 

ssh clasrun@develpc
bash
cd /u1/hps/server/heavy-photon-daq/software
conda activate rogue-hps-dev-rth
python scripts/SvtCodaRun.py


Reboot RCE: 

source /u1/hps/setup_env.sh
cob_rce_reset dev-crate1-sm/2

RCE boot status: 

cob_dump -rce dev-crate1-sm


Log files if SLAC guys running daq: 

/home/clasrun/Xterm.log.slac1.2019.06.07.14.44.29.pE87M0
Retrieved from "https://clonwiki0.jlab.org/wiki/index.php?title=ATCA&oldid=8787"