Writing and Running Applications on Xeon Phi

Xeon Phi 7120P co-processor cards based on the Knights Corner chipset are available for MIC evaluation and development. The cards are installed in the interactive node phi.ph.liv.ac.uk, accessible from the HEP network.

Accessing the Co-processors

The co-processors can be accessed directly from phi.ph.liv.ac.uk using ssh. Log in with your usual HEP Linux username/password. Your default shell must be bash. Home directories are in /user, please contact the admins about moving your home directory to the default location if it can't be found (only new users are put into the default location).

There are three cards, with the following hostnames and assignments

Name IP Address Allocation
hep-mic 192.168.178.202 General development
lhcb-mic 192.168.178.203 LHCb development
neutrino-mic 192.168.178.204 T2K /Snoplus development
If you need access to more than one card for a project please contact the admins.

The standard filestores are available (/scratch, /hepstore, /batchsoft) along with user home directories.

The cards run a limited Linux distribution, compiled for the MIC architecture. Standard x86_64 binaries won't run (scripts should be fine).

Acceleration modes (local or offload)

The cards can run native MIC code locally eg by ssh-ing to the card and running the application as on a normal node.

Applications can also run on the host phi.ph.liv.ac.uk, and offload certain functions to the co-processors, called off-loading. The code must be compiled with the off-load compiler and with the correct use of extra compiler directives. Any libraries that the code links to on the x86_64 host must also be made available in MIC format on the co-processor.

See the developer documentation for more information.

Cross-compiling for the MIC architecture

Intel Compiler

The IntelĀ® Parallel Studio XE for Linux compiler suite is installed for evaluation and student development. Academic staff working on projects specifically funded for Xeon Phi development will need to acquire a full license for themselves.

The suite includes C (icc), C++ (icpc) and Fortran (ifort) compilers. On phi.ph.liv.ac.uk configure your shell to use these compilers by including their setup with

source /opt/intel/bin/compilervars.sh intel64

To enable cross compiling for the MIC architecture add -mmic to the compiler arguments eg

icc -mmic -o code code.c

Any libraries that the code links against will also need to be cross-compiled with -mmic, standard x86_64 libraries cannot be linked against MIC binaries. Link with the Intel compiler.

If using autotools eg ./configure, the host type can be specified to force cross-compiling with --host=x86_64-k1om-linux eg

CC=icc CFLAGS=-mmic LDFLAGS=-mmic ./configure --host=x86_64-k1om-linux

Again ensuring external libraries are compiled for MIC.

GCC

Currently (16/05/2014) GCC does not natively support compiling for the MIC architecture used on Knights Corner so cannot be used for MIC development. It is probable that support will be added in the future, although this may only be for newer MIC architectures (eg Knights Landing).

The MIC software suite includes a patched GCC 4.7 which is used for producing the Linux distribution that runs on the co-processors. This does not include any options for vectorisation and so does not produce binaries that exploit the full Xeon Phi performance, we do not recommend this be used for performance-critical work, if at all.

Monitoring activity

Standard tools like top, ps, free etc are available on the co-processors directly. Please use these sparingly as they can be a performance drain in some circumstances.

Real time co-processor activity can be viewed with the micsmc utilitity. This produces a window with processor and RAM utilisation, as well as general system states and errors.

This utilitity can also be used to show the processor state on the command line with eg

micsmc -a

Historical activity metrics are generated using Ganglia, adding them to the phi.ph.liv.ac.uk entry. See the phi ganglia page. Metrics are recorded as phi-METRIC-micN.

Further Reading

There is a Liverpool MIC development mailing list which we strongly encourage any interested developers to subscribe to. Contact the admins if you have trouble subscribing.

Some quick start guides for MIC development are attached to this topic.

The Intel website has numerous documents for MIC development, debugging and optimisation.

Administration

After a kernel update the 'mic' module needs to be rebuilt for the new kernel otherwise the system becomes unstable. The module sources are kept in /batchsoft/mpss/mpss-3.2.1/src/. The new modules can be built with
  • rpmbuild --rebuild mpss-modules-3.2.1-1.el6.src.rpm
The new modules should be installed and the system rebooted.

-- JohnBland - 15 May 2014

Topic attachments
I Attachment Action Size Date Who Comment
intel-xeon-phi-coprocessor-quick-start-developers-guide-mpss-3.2.pdfpdf intel-xeon-phi-coprocessor-quick-start-developers-guide-mpss-3.2.pdf manage 785 K 16 May 2014 - 08:05 JohnBland MIC Developer Quick Start Guide
intel-xeon-phi-systemssoftwaredevelopersguide.pdfpdf intel-xeon-phi-systemssoftwaredevelopersguide.pdf manage 3 MB 16 May 2014 - 08:07 JohnBland MIC System Software Developer's Guide
Topic revision: r6 - 11 Dec 2017, JohnBland
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback