Grid

Introduction

Information on the LHC Computing Grid (LCG) can be found here. In particular, the LCG-2 User Guide can be very useful, if a little detailed. The best place to clear up what all that grid jargon means is the Grid Acronym Soup. For a guide to getting started see the "Using the Grid" section of the ATLAS Workbook

Certificate

To find out how to get a grid certificate and an ATLAS VO see:

• GridCertificate

Commands

• UsefulGridCommands

Tools

• UsefulGridTools

Ganga

Ganga is a grid interface for ATLAS and LHC.

• The main ganga web page is here
• For information on using ganga in ATLAS see here
• Simple setup instructions for ganga version 5 are here (the legacy page is here).

pAthena

pAthena is an interface for the OSG, where jobs get sent to grid sites for analysis.

• One command will send a job off, and put your output in a DQ2 dataset. Your athena release area must be set up correctly, but it seems to be very efficient:

pathena <JobOptions>.py --inDS <inputdataset> --outDS user10.<nickname>.<outputname>

• Monitor your jobs by logging in here and going to your page
• Once the job is finished, just do dq2_get -r user10.<nickname>.<outputname>
• See the UsingPathena page for full details.

DQ2

The dq2 tool allows global access to atlas datasets stored on all flavours of the grid.

• For information on the dq2 tool see here
• For information on the dq2 end-user tools see here or here
• There is also a web-based dataset browser

Making datasets and registering them to DQ2

Making your locally stored files into a dataset which can be accessed from the grid is relatively simple. All you need to do is:

• asetup 16.0.2

  export LFC_HOST=lfc0448.gridpp.rl.ac.uk

  export LFC_HOME=/grid/atlas/datafiles/

  source /batchsoft/atlas/grid/dq2/setup.sh

  voms-proxy-init -voms atlas -valid 90:00

  dq2-put -s <SOURCE-DIRECTORY> <DATASET-NAME>

Notes

1). Your dataset name must be unique and follow the naming rules (https://twiki.cern.ch/twiki/bin/view/Atlas/DQ2ClientsHowTo#DatasetNameRules).

2). The individual file names should also be unique on the grid. Since you need an athena release to use the dq2-put command, you'll need to do this from one of the interactive nodes (alpha, theta etc) rather than hepgrid1.

3). I found that sourcing the Liverpool ATLAS grid setup script (/batchsoft/atlas/grid/setup.sh) gave an 'LFC exception: python bindings missing' error. Setting up dq2 only (i.e. without ganga and panda) worked for me, though maybe this was just a temporary glitch...

The Grid at Liverpool

• The grid user interface (UI) at Liverpool is hepgrid1. You can log on to this from any internal machine by doing

   ssh hepgrid1

• The following variables set the LHC file catalog (LFC) to use and the default base directory for the LFC commands.

  export LFC_HOST=lfc0448.gridpp.rl.ac.uk export LFC_HOME=/grid/atlas/datafiles/

• To use dq2 at Liverpool log on to the UI and do

  source /batchsoft/atlas/grid/dq2/setup.sh

• To get your files, do

  voms-proxy-init -voms atlas -valid time in hours and minutes 

  It then then asks for your password. Then do

  dq2_get -r dataset files

• To use the latest version of ganga at Liverpool log on to the UI and do

  source /batchsoft/atlas/grid/ganga/setup.sh

  To use an older version try

  source /hepstore/store1/ganga/setup.sh < version >

  An example .gangarc file for atlas can be found at /hepstore/store1/ganga/ExampleGangaRC

• All the ntuples produced here are written to the catlaog under /grid/atlas/datasets/liverpool. You can view these via

  lfc-ls /grid/atlas/datafiles/liverpool

  or if you have set LFC_HOME as above just

  lfc-ls liverpool

Other

• DatasetList

-- CarlGwilliam - 02 May 2007