Danger: new smaller memory default!
At the user meeting on April 12, we found out that requesting 1 core will automatically provide only 500MB of memory. This is a BIG change, because in older cluster we received 2GB per core and that was generally sufficient. That is to say, we almost always did not specify memory.
The default interactive session is not likely to be sufficient, so it will be required to specify memory.
As a result, the command to ask for 1 node with 1 processor (core) on that node would be
msub -X -I -l nodes=1:ppn=1,pmem=2048m
This asks for graphics X11 forwarding (-X). The memory can also be specified as "2gb".
If you only want 1 core on 1 node, the simpler notation would be to use the flag "procs".
msub -X -I -l procs=1,pmem=2048m
To ask for several cores on 1 node (test multicore project), run
msub -X -I -l nodes=1:ppn=5,pmem=2048m
** Specify a queue **
Interactive jobs can be run on any queue. By default, they go to the user's nodes.
The default queue is displayed with 'mystats'. If you wish to run on a node that is not in your owner group, like a GPGPU node, you will then need to specify the sixhour queue and the node name. You will only have a maximum of 6 hours on this node. There is no time limit to your default queue.
msub -X -I -l nodes=1:ppn=5,pmem=2048m -q sixhour
One can specify a particular node, "g0001", with a request likee:
msub -X -I -lnodes=g001:ppn=1 -q sixhour
CRC made a page regarding queues and has relocated it at http://crc.ku.edu/using-hpc#Submitting http://crc.ku.edu/queues
We requested a simpler way to launch the usual type of interactive session--one node, one core--as we had in the old cluster. The administrators created a script "qxlogin" which the user can run from the login node.
$ qxlogin qsub: waiting for job 40565091.sched to start qsub: job 40565091.sched ready
We suggest caution with this, since the new memory default limit is 500MB and CRMDA users have regularly reported frustration with unanticipated job failures.
In case you want to write your own login script, you can take an example from the new qxlogin, which I found is installed in /usr/local/bin on the new cluster.
$ cat /usr/local/bin/qxlogin #!/bin/sh ARGS=$@ /opt/moab/bin/msub -X -I -lnodes=1:ppn=1 $ARGS
If you want more interactive nodes, or more ppn, just change the 1's. To test that, suppose you save it as "qxlogin2", then run
$ sh qxlogin2
If you enjoy the result, save that file in your $HOME/bin directory, make it executable, and then it will be more generally available within your sessions. After that, there is no need to run "sh" before "qxlogin2". Try it out, let me know if there is trouble.