SPECTRE Architecture

SPECTRE is a large system built from a number of components. The diagram below shows the basic architecture, and each component is described beneath.

SPECTRE schematic

Login nodes

The login nodes provide the user interface to the cluster. When users log in to spectre.le.ac.uk, they will be connected to one of the six login nodes; internally these are known as spectre01, spectre02, ... spectre06. They are completely identical in use.

It is from the login nodes that users perform any actions with the cluster: compiling code, transferring data in and out of the cluster, and submitting jobs to the scheduler. 

Compute nodes

There are 67 compute nodes available for job execution. Once a job is submitted to the job queue, the scheduler will decide which compute node(s) the job should run on, depending on a combination of the job's requirements and available resources.

A user requesting to run an interactive job will be automatically logged in to an available compute node in order to run the job.

65 of the compute nodes have a pair of four-core AMD Opteron CPUs running at either 2.1Ghz or 2.3Ghz, and 16GB of RAM. There is approximately 35GB of local storage available on each node, available in /local. The remaining two compute nodes (comp101 and comp102) each have four, four-core AMD Opteron CPUs running at 1.9 Ghz and 32GB of RAM.

Head nodes

There are two nodes whose task is to run the essential services for the cluster, such as the job scheduler and management tools. These are inaccessible to users and are common between ALICE and SPECTRE.

Storage

A high performance Panasas storage system provides user home directories whilst scratch storage is delivered from an Oracle X4500 server providing approximately 16TB of temporary working space.

Interconnect

There are two, physically independent, Gigabit Ethernet networks. One of these is used primarily for moving data to an from the storage systems whilst the other is dedicated to communications for multi-node parallel applications (MPI parallel).