You are here: Foswiki>Computing Web>WebHome (11 Nov 2024, MarkWong)Edit Attach

HEP Computing Documentation

FAQs about Remote Working

During the ongoing pandemic situation we're keeping a page updated with advice about WorkingFromHome and accessing HEP and University services from off-campus.

Registering for a HEP Linux account

To access HEP computing resources you will need to register for a HEP Linux account. This must be submitted by your supervisor or line manager. See AccountRegistration for more details on registration and starting to use Linux.

For external collaborators, an MWS account needs to first be created. See https://www.liverpool.ac.uk/media/livacuk/computingservices/help/mwsregform.pdf

Obtaining Support for Experiments and Projects

Experiments and projects within the Particle Physics group who require computing support including hardware, software, networking and procurement should communicate their projects requirements to the local computer support team as as soon as possible. General guidance can be gound in the ObtainingSupport topic.

Using HEP Linux Software

If you have access to a HEP Linux desktop you can use this directly. There are also high performance interactive nodes provided for intensive work and remote use. They all have high throughput network connections to local and off-site storage systems.

Node Name	OS	CPUs	RAM	GPU
phi.ph.liv.ac.uk	Centos7	40	128GB	2xRTX A4000 16GB
hepcuda1.ph.liv.ac.uk	Centos7	8	16GB	1xGTX980Ti 6GB
gamma.ph.liv.ac.uk	Centos7	64	384GB	NA
kappa.ph.liv.ac.uk	Centos7	16	72GB	NA
livdat1.ph.liv.ac.uk (LIVDAT only)	Centos7	64	128GB	NA

If you have large amounts of simulation or analysis to perform please use the batch systems. See BatchProcessing for more information

For advice on using the 64bit Centos7 software environments and known issues see the Centos7Desktops topic.

For advice on using the deprecated 64bit SL6 software environments and known issues see the SL6Desktops topic.

Remote Shell Access

You can connect to our systems from off-campus or personal laptops and desktops. This is done through an SSH client, which provides a secure connection to a command line. Linux and OS X systems should have an SSH client built in. Windows users should download an SSH client, eg putty. MobaXterm also provides graphical support and can be installed on MWS or downloaded on personal systems. Usage on-campus is simple

ssh username@hostname
eg ssh joebloggs@phi

For more information, particularly for off-site access and Duo 2 factor authentication see the HepSSHGuide.

Remote Desktops with VNC

A desktop can be used remotely with VNC. There is more information about how to do this in the HepVNCGuide.

Email

HEP email can be accessed via IMAPv4, webmail or from the command line with Alpine. See EmailGuide for more information.

Filestores

Particle Physics produces a large quantity of data which, in turn, require large quantities of storage. There are a number of file stores in use and the nature of the data determines where it should be stored. For more information see the FilestoresGuide.

CERN EOS Storage

Bulk Data and user files stored on CERN EOS storage can be accessed directly from Liverpool HEP systems, without having to copy files first. For more information see the EosAccessGuide.

Grid Storage on DPM

Complementing the dedicated local filestores is the large capacity grid storage area (currently accessed using DPM). This is a shared resource for all grid users, with different levels of space reservation and experiment specific quotas. To learn more about storing and accessing files on DPM see the GridStorageGuide.

WISP Cloud Storage

The WISP Integrated Storage Portal is available for all HEP Linux account users and provides a web-based storage interface and synchronisation service. For more information on accessing and using it see the WispStorageGuide.

3rd Party Cloud Storage (Onedrive, Google Drive etc)

Most 3rd party cloud storage systems such as MS Onedrive can be accessed through a web browser to browse, upload and download files. This should be possible using Firefox or Chromium on HEP Linux systems as normal.

Direct access to external storage systems from the command line is possible using the rclone utility. This isn't officially supported but is provided for convenience. There is a brief guide for usage on the HEP network in the RcloneGuide.

Version Control (Git, SVN)

The Linux desktops should have the usual range of version control utilities installed (git, svn, cvs). We run a local Git service at https://gitlab.ph.liv.ac.uk, just use your usual HEP login to access it. Basic instructions should be available when you log in. This is intended for local development, particularly projects that are private. Large scale public projects with other institutes may be better served by github.

Continuous Integration and Deployment is supported. Code can be built/tested in the standard HEP Centos 7 environment by specifying the hep-centos7 tag. Code can also be built on any platform supported by freely available docker containers eg Ubuntu, through the default Kubernetes runner.

Containers

The officially supported container technology on managed HEP systems is Singularity. This can be used to run both Singularity and Docker images, while also integrating seamlessly with local user accounts and storage systems. Docker is not directly supported due to a number of security concerns about running it on shared-user systems such as HEP Linux.

Pre-built containers can be used from files or Docker and Singularity registries. New containers can be created either on your own systems or on HEP Linux if you have the necessary permissions. For more information about using Singularity and accessing our local container library see the HEPContainerGuide.

Mattermost Team-based Messaging

We provide a local Mattermost service for team-based messaging and collaboration. This is available directly via the web at https://mattermost.ph.liv.ac.uk or via dedicated clients on all main platforms. Mattermost clients should be preinstalled on SL6 and Centos7 desktops.

For more information on creating accounts and using Mattermost see our LiverpoolMatterMost topic.

Backups

Some user data is backed up on a daily basis in case of accidental deletion or corruption. For more information about which data is backed up and how to retrieve it see the BackupsGuide.

Printers

The HEP print system provides monochrome and colour printers for all users. See the PrintersGuide for more information.

Account Security and Passwords

HEP systems have two main authentication systems, one for Linux and one for Samba (for printing and filestore access from Windows). Your Linux password will be assigned when the account is created. You can change it at any time using passwd from any HEP linux system. This will change the password for your shell access and email access.

For Windows access see SambaGuide for more information.

As well as your HEP password you may have to have access to various experiment sites or systems with other accounts or logins. Keeping track of multiple passwords, or securely sharing passwords within a group can be tricky. We recommend using a password safe such as Keepass. We have clients installed by default on SL6 and Centos7 systems.

We have some general guidance about protecting your account and securing your password in the PasswordSecurity topic.

Networking

Adding a machine to the HEP network with MACAddressRegistration.

Guest wireless access can be provided for meetings and conferences in Room 337. Email helpdesk@hepREMOVETHIS.ph.liv.ac.uk with your requirements (including number of accounts and duration in days).

Batch and Interactive Processing

A number of queues are available for local analysis using dedicated HEP machines. See BatchProcessing for more information.

GPGPU (CUDA) Access and Usage, Machine Learning (Tensorflow/Torch)

CUDA libraries are pre-installed on HEP Linux systems where appropriate. CUDA development software is available in /usr/local/cuda. The version will change over time, if your software needs a particular version please use the full path eg /usr/local/cuda-11.2. Cudnn libraries are provided in /user/software/cuda/.

phi.ph.liv.ac.uk has 2 RTX A4000s and hepcuda1.ph.liv.ac.uk has 1 GTX 980Ti card available for interactive GPU work. High performance RTX 2080Ti cards are available on the main Slurm compute queue for batch processing. See BatchProcessing for more information.

We provide TensorFlow software for Centos7 systems. See HEPTensorFlow for more information. Other Machine Learning frameworks can be supported.

To monitor GPU usage:

$ watch -d -n 0.5 nvidia-smi

Video Conferencing

Most day to day video conferencing needs can be easily met with a webcam and a suitable head set. If you are using desktop conferencing software (eg Vidyo or Skype) and need extra hardware see the admins in room 220.

Zoom should be pre-installed on Centos7 HEP Desktops. Skype is available on Centos7 HEP Desktops.

We have a dedicated video conferencing room with built in hardware for groups or high quality conferencing. See the VideoConferenceRoom guide for more details.

Procurement Advice

HEP admins can assist with HEP computer equipment procurement. Please read the ComputerProcurement guide before submitting any requests to helpdesk.

Calendar

Computing Web Utilities

Topic revision: r52 - 11 Nov 2024, MarkWong

Computing

Categories

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback