Cluster user update

We will have a cluster update meeting on Friday at 10AM in Watson Room 440D (within the suite of the Digital Humanities group).

Today the Center for Research Computing announced the re-opening of the compute cluster. A number of features we have come to depend on were removed. All of the CRMDA documentation ( will need to be revised. This will take some time. These changes were not well publicized during the six-month-long runup to the cluster administration changeover, we are playing catchup.

They have declined to support NoMachine GUI connections and that the cluster storage is not externally accessible via Windows Server or Network File System protocols. We will have to find ways to navigate around those changes.

The top priority right now is updating the hpc example collection,

Most of that work has been kindly attended to by Wes Mason at KU ITTC.

Here is a copy of the announcement.

KU Community Cluster Users,

Over the course of the last few weeks we have been working to transition the administration of the KU Community Cluster to the Center for Research Computing (CRC). We have completed testing with a subset of users and we are now restoring access for all users who are part of an owner group. If you know someone in your group that did not get this announcement, please email

We have kept the underlying legacy software environment the same to make this transition simpler, but have made some improvements and updates that you will need to be aware of to use the cluster. We will be building upon these initial improvements over the coming months to standardize, implement best practices, update and integrate the software stack, provide transparency of resources utilization, integrate with KU, and help you optimize your use of the cluster.


We have integrated with KU's identity management system so you will use your KU username and password to access the cluster. We have 2 login nodes that you will randomly be assigned to when you login to the address:



'env-selector' was removed and only 'module' is available to load different software packages.

When issuing the command:

> module avail

you will see the new software we have compiled that is optimized for the latest version of the CPUs in the cluster.

To see the software installed before this transition, you must enter:

> module load legacy

and then you can see all legacy software by entering the command:

> module avail

You must place these commands in your job submit scripts as well if you choose to use the legacy software.


'qsub' has been replaced w‚Äčith 'msub'. All your submit scripts will still work with 'msub'. The #PBS directives in your job submit scripts are also compatible with "msub', but we suggest when you create new job submit scripts to use the #MSUB directives.


Your home directory now has a 100GB quota. We have integrated the cluster with KU's identity management system so your home directory also matches the KU home directory path (e.g., /home/a123b456).

All data from /research, /projects, /data, and if you had your own root directory (for example: /compbio), this has all been placed in

/panfs/pfs.local/work/<owner group>/<user>

If your owner group has used all their storage allocation or if your group does not have a storage allocation, some of your data had to be moved to $SCRATCH:

/panfs/pfs.local/scratch/<owner group>/<user>

We organized the data to better keep track of usage for owner groups. Scratch has been set up in the same manner. Some groups were previously allocated more storage than they purchased and you will see your quota for your $HOME, $WORK, and $SCRATCH directories when you log on. If you see any directory at 100%, then you must remove files before writing to it.

To see your quota, group, and queue stats at anytime, run:

> mystats

on the submit nodes.

NO data was deleted. If you see that you are missing something, please contact Please check all paths first, please.


Your default queue will be displayed when you log in. This is the queue you will run in if you do not specify a queue name. If you wish to run across the whole cluster, you must specify:

#MSUB -q sixhour

in your job script or from command line:

> msub -q sixhour

You may only run a maximum of 6 hours on the 'sixhour' queue, but your jobs goes across all nodes.

Most users will only have access to their owner group queue and the 'sixhour' queue. Others will be part of multiple groups and have access to other queues as well.

All of this information will be displayed when you login to the cluster for at least the first few months after coming back online.

We are continuing to write documentation and help pages about the new setup of the cluster. These pages can be found at under the HPC tab and more will be added as time goes on so check back often. We will also have an introduction to the cluster next Wednesday, March 8, at 10:30am during our regular monthly HPC meeting (location TBD).

We understand that change can some times be a little jarring so if you have any questions feel free to contact us at and we will get back to you as soon as we can.

Thank you, Center for Research Computing Team

This entry was posted in Data Analysis. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *