Centos7 Desktops

The 64bit Centos7 desktops are intended to be a combination of a standard Linux desktop, interactive node, grid UI and low priority batch node.

The desktop should be labelled with a model number and identifying DIC number. Be sure to quote both if you are having any problems.

gamma.ph.liv.ac.uk, phi.ph.liv.ac.uk and hepcuda1.ph.liv.ac.uk are available for use with Centos7 HEP build.

Logging in

Use your normal HEP linux login. The default desktop environment is GNOME 3. You can choose a different environment by clicking on the cog icon after entering your username (before entering the password). We do not mandate any particular environment and all should run the full range of software, but GNOME tends to have better compatibility with some services particularly printing. For an interface more similar to that in SL6 choose the GNOME Classic or Cinnamon interfaces.

Desktops can be accessed via SSH from any system on-site (there is no external SSH access for any system other than gateway), and via VNC if you create a VNC session on the desktop. Please do not set up sessions on other people's desktops unless agreed beforehand.

Accessing VNC sessions

VNC sessions running on the desktop can be accessed on and off site. You will need to set up your session with a session number of 10 or lower (eg vncserver :1). On site you can connect your VNC viewer directly to this session eg
  • vncviewer hep140:1
Some sessions crash when using certain applications (eg ROOT with graphical windows), this is usually mitigated by passing the "-bs" switch when setting up the session eg
  • vncserver hep140:1 -bs
Off site you will need to tunnel through gateway. You can set this up manually but in SL6 or later this is built in, using eg
  • vncviewer -via gateway.ph.liv.ac.uk hep140:1

Storage

There is 230GB of temporary storage under /tmp. The standard $HEPTMP variable points to a directory under this. Please do not store files permanently in this area, use your main user area ( $HOME) or hepstore. Files under /tmp are not guaranteed to be kept after a reboot or to be on reliable storage and are cleaned up after 30days by default. Desktops are also intended to be stateless and easily replaced in case of faults.

Be careful with external drives. If they are left connected when a system is upgraded it could be reformatted and the data wiped. Always remove external drives if the system is left unattended or to be rebooted.

Trash

If you delete files using the graphical interface the files will be sent to Trash, where they can be retrieved later or permanently deleted. Be aware that files in trash still count towards your quota. Also on volumes other than your user area files sent to Trash will not be visible in the Trash folder. They will be stored in VOLUME/.Trash-uid eg for a user with uid 1234 on hepstore files will be placed in /hepstore/.Trash-1234. As this is not obvious and is awkard to clean up we recommend graphical file tools aren't used for file operations on bulk data storage areas ie hepstore, bundle, scratch etc.

WISP cloud storage

Centos7 supports the latest Owncloud synchronisation clients meaning you can synchronise files in your user area with the WISP cloud storage system. See the WispStorageGuide for more information on how to set this up.

Email

Pine is now known as 'alpine'.

Mozilla Thunderbird is installed.

Webmail is available at https://hep.ph.liv.ac.uk/webmail/ .

Grid UI

The desktops (and all other standard SL6 and Centos7 systems eg batch nodes) are configured with a grid UI by default, you do not need to configure anything to enable it. Ensure you have a valid proxy before submitting any batch jobs that will access grid resources. Some deprecated tools (eg lcg-cp) are missing, please use the gfal-* equivalents. Please report any grid tool you can't find an alternative for.

The Centos 7 UI has caused problems with ATLAS configuration in the past (particularly rucio) but we believe it has been fixed. If you experience problems let us know on helpdesk.

Batch Jobs

The Centos 7 desktops and nodes uses a new batch system, Slurm. Access isn't automatic, if you are denied access to submit jobs please email helpdesk to be added.

There is lots of documentation on the Slurm website.

To make the transition easier Slurm provides wrapper scripts to give similar utilities to the old PBS batch system, eg qsub, qstat, pbsnodes. PBS-style job scripts can be used with these wrappers, usually with no modifications. There are some subtle differences in the way the commands work compared to the old system (eg the way stdout/stderr files are handled) so be sure to check your scripts first.

Slurm has native commands for submitting jobs and querying the system (most tools give instructions with the --help switch or man pages). The equivalent of the PBS 'queue' is the Slurm 'partition'.
Slurm Command ActionSorted ascending Notes PBS Equivalent
scancel Cancel job(s) Give a list of jobids qdel
sview Graphical view of jobs and nodes    
sacct

Job accounting details

eg CPU, RAM consumption

-j to show specific job

-a to show all jobs

 
sinfo List partitions or nodes -N for node format, -l for more info qstat -Q
squeue List queued jobs and their state -l for long form output qstat
sshare List user usage and job priority

-a to show all users

Values updated every 5mins

 
srun Start an interactive job or jobstep Usually called within a jobscript qsub
sbatch Submit a batch script job

-p to specify partition (queue)

-t HH:MM:SS to request walltime

-c N to request N CPUs

--mem=N(KMG) to request N RAM

qsub
smap Text view of jobs and nodes -i N to update every N seconds  

There are currently three partitions ('queues') but the partitions and their resources will vary over time as systems are migrated from SL6 to Centos7:
Queue name Max RAM/CPU Max CPUs/job Default/Max Walltime CPUs Notes
desktop 1000MB 2 24hrs/24hrs 16 Runs on desktops so keep IO light
gpu 1900MB 8 24hrs/48hrs 16 Dedicated old batch nodes, fast network
compute 5300MB 24 24hrs/48hrs 96 Default. Dedicated batch nodes, fast network.
All queues can have jobs submitted from desktops, but advanced operations that require connecting directly to the compute nodes will only work from interactive nodes (most job scripts shouldn't need this).

A simple example jobscript to be submitted with sbatch (for a job that runs with 4 threads per process, and needs 4hrs 30minutes to run on the 'medium' partition)
#!/bin/bash
#SBATCH -N 1
#SBATCH -c 4
#SBATCH -p compute
#SBATCH -J myfirstjob
#SBATCH -t 04:30:00

#run the application:
/path/to/my/binary

Or a job that needs 1GPU of any type and 2CPUs
#!/bin/bash
#SBATCH -N 1
#SBATCH --gres=gpu:1
#SBATCH -p gpu
#SBATCH -c 2
#SBATCH -J gpujob
/path/to/my/binary

Job resources are constrained by Linux cgroups. This effectively means that job processes won't be allowed to exceed their CPU or RAM limits (default or requested) eg if your job requests 1 CPU but runs 4 processes they will only get 25% of a CPU each.

GPU Acceleration

Some Centos7 desktops (and kappa and hepcuda1 interactive nodes) contain a NVidia graphics card with a minimum of 2GB video RAM. They are available for CUDA and OpenCL GPGPU processing tasks. The CUDA development kit is available under /usr/local/cuda/ and is enabled by default.

Configuring Development tools

Centos7 comes with a number of development packages. This is supplemented by local installs located in /user/software. If you require a package that isn't installed let the admins know on helpdesk.

Note that the short commands below (eg gccSetup) only work in login shells eg SSH logins or batch jobs.

GCC

The system compiler provided with Centos7 is version 4.8.5 and is enabled by default.

Newer versions of GCC are available. They can be configured by calling the setup script eg for version 6.2.0
  • gccSetup 6.2.0
  • or source /user/software/gcc/gccSetup.sh 6.2.0
If you supply a version that doesn't exist all available versions for the current OS and architecture will be listed.

By default the compiler will optimise for a generic 64bit intel CPU. You can enable extra CPU optimisations with the -march=ARCH switch. Suggested architecture levels for use on the HEP linux network are
  • core2 (all Linux systems including desktops)
  • westmere (all interactive, batch and grid nodes)
  • haswell (some newer interactive nodes and all 'compute' batch nodes)
  • native (only for use on the node being compiled on)
The optimisations enable more math-based instructions which could speed up CPU-bound simulations.

Python

The system version of Python provided with Centos7 is 2.7.5. Most standard Python add-ons eg numpy, are installed.

Newer versions of Python are available, including Python 3.x. They can be configured by calling the setup script eg for version 3.6.3
  • pythonSetup 3.6.3
  • or source /user/software/python/pythonSetup.sh 3.6.3
If you supply a version that doesn't exist all available versions for the current OS and architecture will be listed.

Virtualenv

Extra Python modules can be installed in virtual environment in your home directory using the virtualenv command.

Configure the version of Python you wish to use. Then create the directory to contain your new environment eg
  • virtualenv myenv
Activate your environment with
  • source myenv/bin/activate
You can then use standard Python package managers eg pip to install extra modules eg
  • pip install module_name
The virtual environment will copy some base Python packages as well your additional modules. This will take up space on whichever device you have placed your environment directory on. Be aware this could impact on quotas.

Adding additional modules can be done simply without a virtual environment, although it requires more work and configuration. Some modules may not work properly with this method.

Create a directory you want to use for this eg ~/python. Configure whichever version of Python you want eg

  • pythonSetup.sh 2.7.12
Add your personal python directory to the PYTHONPATH environment variable eg

  • export PYTHONPATH=~/python:$PYTHONPATH
When installing the module specify this directory as the home for the installation with eg

  • python setup.py install --home=~/python

Anaconda

We provide a central install of Anaconda. This includes a large number of extra packages, usually more recent versions than supplied with Centos7 itself. This can be configured with
  • source /user/software/anaconda/anaconda-2019.03-python3.7-x86_64/bin/activate
for python 3.6 or
  • source /user/software/anaconda/anaconda-5.2.0-python2.7-x86_64/bin/activate
for python 2.7.

If you need any packages which aren't included by default please contact Helpdesk.

ROOT

ROOT is supplied in the local software area and enabled by default.

Many versions of ROOT are available. They can be configured by calling the setup script eg for version 6.06.08
  • rootSetup 6.06.08
  • or source /user/software/root/rootSetup.sh 6.06.08
If you supply a version that doesn't exist all available versions for the current OS and architecture will be listed.

CMake

A newer version of CMake is supplied in the local software area.

Many versions of CMake are available. They can be configured by calling the setup script eg for version 3.7.1
  • cmakeSetup 3.7.1
  • or source /user/software/cmake/cmakeSetup.sh 3.7.1
If you supply a version that doesn't exist all available versions for the current OS and architecture will be listed.

Geant4

Geant4 is available in /user/software/geant4 and can be configured with the built in scripts eg
  • source /user/software/geant4/geant4.10.03-x86_64-cc7-48/bin/geant4.sh
  • source /user/software/geant4/geant4.10.03-x86_64-cc7-48/share/Geant4-10.3.0/geant4make/geant4make.sh

LCG Views

CERN provides extensive collections of HEP-related software and development tools (eg gcc, python, root, lhapdf, geant4) via CVMFS. These can be configured using
  • source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh <LCG_directory> <arch-os-compiler>
eg
  • source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_96b x86_64-centos7-gcc8-opt
or for the same view supporting Python 3
  • source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_96bpython3 x86_64-centos7-gcc8-opt
These software installs should also be available on any system where CVMFS is available (eg lxplus, grid jobs etc).

Others

Newer versions of Boost are available in
  • /user/software/boost/
CADENCE can be configured with
  • source /eda/cadence-config.sh
Sentaurus TCAD can be configured with
  • source /eda/tcad-config.sh

Printing

Centos7 systems should come with one print queue enabled: StaffQueueSouth. This prints to the University Ricoh printer system. You will need to input your MWS username and password for the print jobs to be accepted. Most GNOME tools will ask for this information, but many KDE or other non-standard tools may not, in which case the print jobs will not be accepted.

Further instructions are available on the CSD website http://www.liv.ac.uk/csd/printing/staff-printing/ .

Audio

The audio system should be working. There are two possible microphone and headphone outputs; the rear ports or the front panel. You will need to select which microphone/capture device you are using with eg the Gnome Sound Preferences application (this should also work with most USB headsets).

Gnome Sound Preferences Input

Gnome Sound Preferences Output

Using Vidyo

The Vidyo client should be installed and loaded on login so you shouldn't need to do anything to enable it. If it has crashed it can be reloaded from the System menus, under Internet.

Follow the instructions for accessing the Vidyo conference as per any other operating system.

To change the audio/video devices load up the Configuration window and choose the Devices tab.

Vidyo devices tab

Known Issues

  • There are no screensavers, the desktop can be locked but only with a blank screen.
  • Vidyo clients can be unreliable, but this is hardly unique to Centos7.
Topic attachments
I Attachment Action Size Date Who Comment
soundpreferences.pngpng soundpreferences.png manage 29 K 18 Nov 2016 - 09:10 JohnBland  
soundpreferences2.pngpng soundpreferences2.png manage 37 K 18 Nov 2016 - 09:10 JohnBland  
vidyodevices.pngpng vidyodevices.png manage 40 K 18 Nov 2016 - 09:10 JohnBland  
Topic revision: r32 - 03 Aug 2021, JohnBland
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback