Page tree
Skip to end of metadata
Go to start of metadata

Plato is a Linux-based, heterogeneous, high-performance computing (HPC) cluster at the University of Saskatchewan. It is used for research projects and for training, and is restricted to USask users and their collaborators. Plato is managed by ICT’s Advanced Research Computing (ARC) team.

Although it is not a Compute Canada cluster, Plato is configured to be as similar as possible to Compute Canada clusters. For instance, Plato has the same scientific and general software stack, and uses the same scheduler.

Plato cannot provide all the computing power required for USask research projects. Researchers with important computing needs should consider Plato a stepping stone to test their projects locally before moving to Compute Canada clusters.

Contents

Basic information

SSH hostnameplato.usask.ca  (VPN required when off-campus)
Globus accessusask#gateway /plato/<nsid>  (home directory)
System statusARC main page, and cluster-info command
Current usagecluster-info command

Storage

Home filesystem

/home

  • 24T total volume
  • Location of home directories
  • Each home directory has a small (500G) quota
  • No backup
  • For active research data and user programs
  • Inactive research data must be moved to Datastore

Datastore service

/datastore

  • 723T total volume
  • Also accessible from outside Plato
  • Faculty members are entitled to 3T free space and can obtain more
  • Backed-up
  • For long-term storage and inactive research data
  • Read-only access on the compute nodes, read-write on the login nodes

Compute nodes

/local

  • Temporary storage for jobs
  • Total volume varies by compute node type
  • Use $SLURM_TMPDIR to get the temporary directory for your current job, in the form: /local/$USER/$SLURM_JOB_ID
  • Files are deleted when a job ends

Interconnect

Most Plato compute nodes are connected by a 1Gb Ethernet link. Login nodes are connected to compute nodes and to the University network by a 10Gb Ethernet link. Some compute nodes (see below) are connected by 10Gb Ethernet or FDR InfiniBand.

Scheduler

Plato uses the SLURM scheduler. Job duration is limited to 21 days, except GPU (7 days) and large-memory (30 days) nodes. (This limit does not apply to contributed hardware.) Shorter jobs get increased priority, according to the categories below. The default allocation is for a single task on one CPU with 512M of memory.

Maximum duration
(d-hh:mm)

Priority factor
21-00:001
4-00:002
12:004
04:008

Nodes

CountTypePublic

Cores per node

Memory per nodeCPUHighest SIMDGPU/local storageInterconnect
64“Pipit”X16

31000M (30G)

2 x Intel Xeon E5-2640 v2 @ 2.00GHz “Ivy Bridge”AVX-347G1Gb Ethernet
32“Penguin”X40190000M (185G)2 x Intel Xeon Gold 6148 @ 2.40GHz “Skylake”AVX512-781G10Gb Ethernet
1Large-memoryX482048000 (2000G)4 x Intel Xeon E7-4850 v2 @ 2.30GHz “Ivy Bridge”AVX-3.2T10Gb Ethernet
2GPUX1631000M (30G)2 x Intel Xeon E5-2640 v3 @ 2.60GHz “Haswell”AVX22 x NVIDIA K40805G1Gb Ethernet
20GWF-1631000M (30G)2 x Intel Xeon E5-2640 v2 @ 2.00GHz “Ivy Bridge”AVX-347G1Gb Ethernet
2Tse group-32250000M (244G)2 x Intel Xeon E5-2683 v4 @ 2.10GHz “Broadwell”AVX22 x NVIDIA K80768GFDR InfiniBand

Choosing nodes

Plato will choose the appropriate node type for your job according to your resource requirements (cores per node, memory, GPUs). It is therefore not necessary to request a specific node type. Doing so may reduce the number of nodes eligible to run your job, increasing your wait time. You can specify a node type using SLURM options: --constraint=ivybridge (for Pipit nodes) or --constraint=skylake (for Penguin nodes).

Login nodes

Plato has a single login node, platolgn01, accessible by SSH at plato.usask.ca. The login node should be used to prepare jobs and submit them to the scheduler, to compile programs, and to run short calculations that require little memory and processing power. Intensive processes must never be run on the login node; they must be submitted to the scheduler to run on the compute nodes.


Other topics (subpages)

  • No labels