You are here: Foswiki>ATLAS Web>GridInformation (22 Feb 2011, KatharineLeney)Edit Attach

Grid

Introduction

Information on the LHC Computing Grid (LCG) can be found here. In particular, the LCG-2 User Guide can be very useful, if a little detailed. The best place to clear up what all that grid jargon means is the Grid Acronym Soup. For a guide to getting started see the "Using the Grid" section of the ATLAS Workbook

Certificate

To find out how to get a grid certificate and an ATLAS VO see:

Commands

Tools

Ganga

Ganga is a grid interface for ATLAS and LHC.

  • The main ganga web page is here
  • For information on using ganga in ATLAS see here
  • Simple setup instructions for ganga version 5 are here (the legacy page is here).

pAthena

pAthena is an interface for the OSG, where jobs get sent to grid sites for analysis.

  • One command will send a job off, and put your output in a DQ2 dataset. Your athena release area must be set up correctly, but it seems to be very efficient:
pathena <JobOptions>.py --inDS <inputdataset> --outDS user10.<nickname>.<outputname>
  • Monitor your jobs by logging in here and going to your page
  • Once the job is finished, just do dq2_get -r user10.<nickname>.<outputname>
  • See the UsingPathena page for full details.

DQ2

The dq2 tool allows global access to atlas datasets stored on all flavours of the grid.

  • For information on the dq2 tool see here
  • For information on the dq2 end-user tools see here or here
  • There is also a web-based dataset browser

Making datasets and registering them to DQ2

Making your locally stored files into a dataset which can be accessed from the grid is relatively simple. All you need to do is:
  • asetup 16.0.2
    export LFC_HOST=lfc0448.gridpp.rl.ac.uk
    export LFC_HOME=/grid/atlas/datafiles/
    source /batchsoft/atlas/grid/dq2/setup.sh
    voms-proxy-init -voms atlas -valid 90:00
    dq2-put -s <SOURCE-DIRECTORY> <DATASET-NAME>
Notes

1). Your dataset name must be unique and follow the naming rules (https://twiki.cern.ch/twiki/bin/view/Atlas/DQ2ClientsHowTo#DatasetNameRules).

2). The individual file names should also be unique on the grid. Since you need an athena release to use the dq2-put command, you'll need to do this from one of the interactive nodes (alpha, theta etc) rather than hepgrid1.

3). I found that sourcing the Liverpool ATLAS grid setup script (/batchsoft/atlas/grid/setup.sh) gave an 'LFC exception: python bindings missing' error. Setting up dq2 only (i.e. without ganga and panda) worked for me, though maybe this was just a temporary glitch...

The Grid at Liverpool

  • The grid user interface (UI) at Liverpool is hepgrid1. You can log on to this from any internal machine by doing
     ssh hepgrid1
  • The following variables set the LHC file catalog (LFC) to use and the default base directory for the LFC commands.
    export LFC_HOST=lfc0448.gridpp.rl.ac.uk export LFC_HOME=/grid/atlas/datafiles/
  • To use dq2 at Liverpool log on to the UI and do
    source /batchsoft/atlas/grid/dq2/setup.sh
  • To get your files, do
    voms-proxy-init -voms atlas -valid time in hours and minutes 
    It then then asks for your password. Then do
    dq2_get -r dataset files
  • To use the latest version of ganga at Liverpool log on to the UI and do
    source /batchsoft/atlas/grid/ganga/setup.sh
    To use an older version try
    source /hepstore/store1/ganga/setup.sh < version >
    An example .gangarc file for atlas can be found at /hepstore/store1/ganga/ExampleGangaRC

  • All the ntuples produced here are written to the catlaog under /grid/atlas/datasets/liverpool. You can view these via
    lfc-ls /grid/atlas/datafiles/liverpool
    or if you have set LFC_HOME as above just
    lfc-ls liverpool

Other

-- CarlGwilliam - 02 May 2007
Topic attachments
I Attachment Action Size Date Who Comment
ttbarFiles.shsh ttbarFiles.sh manage 589 bytes 11 Oct 2007 - 12:01 KatharineLeney Example shell script for copying files from Liverpool to Grid
Topic revision: r19 - 22 Feb 2011, KatharineLeney
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback