Iceberg: University of Sheffield’s older HPC cluster

Introduction

Iceberg is the older of the University of Sheffield’s two computer clusters. The majority of the computational resources are available to all researchers with ShARC/Iceberg accounts but INSIGNEO and certain sub-groups/projects have purchased hardware for Iceberg that only they have access to.

The documentation on getting started with High-Performance Computing at the University of Sheffield covers connecting, usage (inc. submitting jobs and filestores. There is a separate section for the software that is readily available on Iceberg. All users should consult that documentation when getting started.

INSIGNEO members will then want to consult the following for additional information on making use of INSIGNEO-specific resources in Iceberg and for guidance on INSIGNEO-specific workflows.

Warning

Researchers should use ShARC instead of Iceberg unless there is good reason to do so:

• Many nodes in Iceberg are no longer under warranty and may not be repaired/replaced if they fail.

• New research software packages or updates to research software will only be provided on ShARC.

INSIGNEO-related hardware

INSIGNEO has purchased nodes for Iceberg, as have three related groups/projects: IMSB (for the MultiSim project), Polaris and NoTremor.

• Restricted access to the sub-group/project that purchased the nodes (and potentially other related groups if sharing agreements are reached);

• Shorter job queue times for researchers as there is less contention for resources;

• Specific workloads as the nodes may have e.g. more RAM than is typical for the cluster.

Details as of 2017-10-03:

hostname	host_groups	cpu_arch	num_proc	m_socket	m_core	ram_gb	num_gpgpus	gpgpu_arch	gfx_gpu
node097.iceberg.shef.ac.uk	@insigneo-gpu,@allnodes,@insigneo-imsb	intel-e5-2670	16	2	16	256	0		True
node098.iceberg.shef.ac.uk	@insigneo-gpu,@allnodes,@insigneo-imsb	intel-e5-2670	16	2	16	256	0		True
node099.iceberg.shef.ac.uk	@insigneo-gpu,@allnodes,@insigneo-imsb	intel-e5-2670	16	2	16	256	0		True
node100.iceberg.shef.ac.uk	@insigneo,@allnodes,@insigneo-imsb	intel-e5-2670	16	2	16	256	0		False
node101.iceberg.shef.ac.uk	@insigneo,@allnodes,@insigneo-imsb	intel-e5-2670	16	2	16	256	0		False
node102.iceberg.shef.ac.uk	@insigneo,@allnodes,@insigneo-imsb	intel-e5-2670	16	2	16	256	0		False
node103.iceberg.shef.ac.uk	@insigneo,@allnodes	intel-e5-2670	16	2	16	256	0		False
node104.iceberg.shef.ac.uk	@insigneo,@allnodes	intel-e5-2670	16	2	16	256	0		False
node105.iceberg.shef.ac.uk	@insigneo,@allnodes	intel-e5-2670	16	2	16	256	0		False
node106.iceberg.shef.ac.uk	@insigneo,@allnodes	intel-e5-2670	16	2	16	256	0		False
node107.iceberg.shef.ac.uk	@insigneo,@allnodes	intel-e5-2670	16	2	16	256	0		False
node117.iceberg.shef.ac.uk	@insigneo-notremor,@allnodes	intel-e5-2650v2	16	2	16	64	0		False
node219.iceberg.shef.ac.uk	@insigneo-gpu-k80,@allnodes	intel-e5-2660v3	20	2	20	384	1	nvidia-k80m	True
node220.iceberg.shef.ac.uk	@insigneo-gpu-k80,@allnodes	intel-e5-2660v3	20	2	20	384	1	nvidia-k80m	True

Warning

All nodes apart from those containing NVIDIA K80 GPUS are no longer under warranty and may not be repaired/replaced if they fail. The K80 GPU nodes are under warranty until 2019-07-05 but will be moved from Iceberg to ShARC in spring 2020.

Gaining access to these nodes

Users need to be explicitly added to particular user groups in order to be able to run jobs on these nodes (in addition to the ‘public’ nodes).

If a researcher would like access then a relevant PI needs to contact the INSIGNEO tech team.

Using these nodes

To run jobs on these ‘private’ nodes you need to follow the standard instructions for starting interactive sessions and submitting batch jobs but make sure you specify a Project and Queue, the values of which depend on which research group you are in:

Research group	Project	Queue
IMSB (MultiSim)	insigneo-imsb	insigneo-imsb.q
NoTremor	insigneo-notremor	insigneo-notremor.q
INSIGNEO (non-specific)	insigneo-default	insigneo.q

Here’s how you specify a Project and Queue when starting an interactive session (in this case on an IMSB-owned node):

qrshx -P insigneo-imsb -q insigneo-imsb.q -l rmem=256G

And here’s how to specify a Project and Queue in a batch job submission script (in this case for a job that you wish to run on a general INSIGNEO-owned node):

#!/bin/bash
#$ -l h_rt=24:00:00
#$ -l rmem=6G
#$ -P insigneo-default
#$ -q insigneo.q
#$ -pe openmp 16
#$ -m youremail@sheffield.ac.uk

module load apps/java/1.8u71
java -jar MyProgram.jar

To see all jobs that are running in a particular queue or are waiting for a particular queue:

qstat -q queuename.q -u \*

e.g.

qstat -q insigneo-default.q -u \*