pvDatabase being added 2 or 3 times! /afs/slac/g/lcls/tools/lib/linux-x86 /afs/slac/package/matlab/2012a/extern/lib/glnx86 /afs/slac/g/lcls/epics/base/base-cpp-R4-4-0/pvDatabaseCPP/lib/linux-x86 /afs/slac/g/lcls/epics/base/base-cpp-R4-4-0/pvDatabaseCPP/lib/linux-x86 python support not Physics Programmers Guide -------------------------- Plan. Convert to html and read, rote. - don't yet fix for dev /prod consistency. * Restructure into data analsysis (APIs) followed by programming (CVS etc) Add configuration for analysis from "dev" configuration for prog from dev configuration for analysis from mac configuration for prog from mac * Upgrade desc of CVS to Configure for prod and dev. Add cvs from within matlab. >> checkout('matlab/toolbox/') Error using checkout (line 37) No source control system specified. Set in preferences. * Add Prerequisites * Laptop + epics + epics v4 client side + VPN + wget/curl physics data files. + openAFS aklog, unlog/aklog over network switch. + cvs for development from laptop (see setup_meme_hostdef_mac, setup_meme). Issues of EPICS Setup --------------------- * epicsSetup uses "/afs/slac/" not, correctly, /afs/slac.stanford.edu/" as the AFS root, hence assuming that clients has enabled the AFS short name. But that's not a requirement of AFS. All references to /afs/slac/ should be expanded to /afs/slac.stanford.edu/. * epicsSetup includes things that are absolutely noting to do with epics, like WWW_ROOT, LCLS_DATA etc. -> Move all non-epics related things out of epicsSetup to, for instance, infrastructureSetup.bash, so that epicsSetup relates only to EPICS filesystem things. * epicsSetup calls envSet.bash, so it does both filesystem and also runtime configuration!! That means we can't use one infrastructure setup for all EPICS run accelerators, and then simply change the runtime and network config different machines. -> move envSet.nash out of epicsSetup.bash, make both called separately by, eg ENVS_lcls.bash ENVS_LCLS.bash lcls_hostDefaults_{prod,dev} epicsSetup.bash envSet.bash dataSetup.bash * $WWW_ROOT/comp/unix/package/epics This dir was last edited 7 years ago. And it shouldn't be assigned in epicsSetip.bash anyway, nothing to do with epics runtime. * # temporary set EPICS_HOST_ARCH=linux-x86 during the transition to 64 bit #export EPICS_HOST_ARCH=`$EPICS_BASE_RELEASE/startup/EpicsHostArch` export EPICS_HOST_ARCH=linux-x86 ok, as long as it's temporary. * There are lots of constructs like this: if [ -z `echo $LD_LIBRARY_PATH | grep ${EPICS_PVCPP}/pvAccessCPP/lib/${EPICS_HOST_ARCH}` ]; then export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${EPICS_PVCPP}/pvAccessCPP/lib/${EPICS_HOST_ARCH} fi These are quite error prone and lengthy. May be better to use some path tools (amny on the web, and shouldn't we use pathmunge (to merge the paths) with filepath expansion (to check for existence)? Also, in some places a check is made for whether the location is alredy in the PATH, and in some places a chck is made on whether the location exists on the filesystem, but seldom both. Shouldn't we do both everywhere, or at least be consistent. If we used pathmunge we could check for existence in the filesystem in there. * On prod LD_LIBRARY_PATH has non-existent item /usr/X11R6/lib, and also empty entry [physics@lcls-srv01 ~/greg]$ echo -n $LD_LIBRARY_PATH | xargs -d: stat -c %n stat: cannot stat `/usr/X11R6/lib': No such file or directory * On Prod LD_LIBRARY_PATH Oracle client side is added 6 times (!!) See printenv LD_LIBRARY_PATH | tr : '\n' That's 5 times too many times. * On dev LD_LIBRARY_PATH has a couple of non-existent items: $ echo -n ${LD_LIBRARY_PATH} | xargs -d: stat -c %n >1 /dev/null stat: cannot stat `/usr/X11R6/lib': No such file or directory stat: cannot stat `/afs/slac/g/lcls/package/python/tcltk/lib': No such file or directory * Test and debug made v. hard by non-use of source PATH: It's difficult to test changes to scripts without releasing them because it's diffcult to get the caller to call our version rather than prod - so v. difficult to test in context of normal execution. That's because fully-qualified file-paths are used. Better to use PATH facility for sourced scripts as well as executable scripts. * Both envSet.bash and envSet_dev.bash include epics runtime for BOTH prod and dev!! Presumably envSet.bash should be all and only prod envSet_dev.bash should be all and only dev * There is both a file envSet_dev.bash, in CVS, *and* symlink called envSet_dev.bash pointing from envSet.bash in dev $EPICS_SETUP!!! So any action "cvs co epics/setup" then has to be followed by delete envSet.bash and replace by a symlink! That's just nuts. ls -alsF ${EPICS_SETUP} 1 lrwxr-xr-x 1 jingchen cd 15 Nov 6 2013 envSet.bash -> envSet_dev.bash ACTION: Properly put dev setup in envSet.bash. Drive root path defaults from a default file. * tools/pthon/pythonSetup.bash only sets up for prod, not dev! * matlab setup sets up for prod but not dev, so physicists can't use dev for analysis Matlab setup * matlabSetup.bash sets the MATLABPATH!!! That's f*cking nuts! It's complicated in matlabSetup.bash, and it means users of dev or own matlab in office's can't reuse the matlab path! ACTION: Move all MATLABPATH out of matlabSetup.bash and into startup.m where it should have bloody been in the first place. Present Issues of Matlab and Physics Env Generally -------------------------------------------------- * /afs/slac/g/lcls/matlab/toolbox is completely out-of-date w.r.t. cvs update. As of 18-Aug-2015, 273 files in /afs/slac/g/lcls/matlab/toolbox needed cvs udpate: cvs status -l | awk '/Stat/&&!/Up-to/' | wc -l 273 So, we have to decide, is office physics going to be properly supported. If so, got to periodically check. * Squid server should permit access to Mathworks Matlab help and certain other pages, to control room OPI web clients. mathworks.com tinyurl.com * DEV / PROD / PROD ON DEV. Make rigorous definitions, and hence service level agreements, for DEV, and PROD ON DEV, as well as PROD. Up until now for LCLS, only “prod" has really been defined and has a service level agreement, even if it’s an informal one. It hasn’t been clear what "dev" really is. For instance, people say “it's for software development", but it’s unclear whether it is an infrastructure test bed - and does that include Oracle, logging, screens, matlab, python, etc.; whether it’s also an analysis platform for physicists in offices, whether model should be the development model, whether PV names should be development PV names, or prod, or both. Is it somewhere we test new versions of OS, matlab, python, Oracle, logging, etc. If so, how is the staging managed in a consistent way, so I know if I start matlab I’m starting a new version under test, or not. In the future, let’s have a ubiquitous dev environment. So it would go without saying that what is done for the prod control system is done automatically for dev too. And it must be clear if you’re a physicist or operator, how one can properly use dev machines for analysis. It should be clear without thinking, what you can absolutely expect to work (and what must you not expect to work).