Robert Nicholas (ren10@psu.edu), Earth and Environmental Systems Institute, Penn State University

updated 11 April 2016

Systems and Services

SCRiM’s resources with ICS-ACI comprise a computing allocation of 400 “standard-memory” cores and a 400 TB storage allocation, as codified in a 5-year service level agreement (SLA) that ends on 31 January 2020. Services available to SCRiM researchers under this agreement include:

  • ICS-ACI cluster aci-b:
    • replacement for soon-to-be-retired Lion-X systems
    • shell access via SSH: ssh -Y username@aci-b.aci.ics.psu.edu
    • time-averaged usage of up to 400 cores (averaged over a 90-day moving window) and instantaneous (a.k.a. “burst”) usage of up to 1600 cores with a guaranteed respsone time of one hour or less on the “kzk10_collab” queue
    • unlimited use of the “open” queue up to 20 cores (total, not per job) with a maximum wall clock time of 24 hours and a maximum of 100 queued jobs
  • ICS-ACI interactive system aci-i:
    • for interactive sessions only; similar to the old Hammer system
    • shell access via SSH: ssh -Y username@aci-i.aci.ics.psu.edu
    • full graphical desktop via Exceed onDemand with hostname “aci-i.aci.ics.psu.edu”, Xconfig “Desktop_Mode_1400x1050.cfg”, and Xstart “Gnome_Desktop.xs”
  • storage:
    • up to 10 GB in your home directory, /storage/home/<username>
    • up to 128 GB in your work directory, /storage/work/<username>
    • up to 400 TB in our group storage pool, /storage/group/kzk10_collab, shared with all other SCRiM users
    • up to 1 million files in your scratch directory, /gpfs/scratch/<username>; these storage resource is intended for temporary files and is not backed up; files residing here for more than 30 days may be automatically deleted
    • all filesystems are accessible from both aci-b and aci-i
    • move files to and from these storage pools via special file transfer node fsgw01.aci.ics.psu.edu using scp from the command line or a graphical SFTP client (note that transfers to/from woju should use hostname woju-rn.scrim.psu.edu to bypass a currently-unresolved network issue)

These systems are managed separately from other SCRiM computing resources (woju, napa, RStudio Server, and the web-based download server), which are hosted by the Penn State Meteorlogy Computing Facility.

Getting an Account

The above resources are available to all SCRiM researchers, including non-PSU SCRiM participants. Contact SCRiM Managing Director Robert Nicholas (ren10@psu.edu) for details on how to gain access.

Submitting Jobs on the Batch System

  • qsub -A kzk10_collab <yourrunscript>
  • alternately, add #PBS –A kzk10_collab to your run script
  • to use the open queue, replace kzk10_collab with open

Using the Modules System

The modules system controls which softare packages are available in your current environement.

  • to list all available modules: module avail
  • to search available modules: module avail <keyword>
  • to load a specific software package: module load <modulename>
  • to show modules currently loaded in your environment: module list
  • to show paths for executables and libraries associated with a particular module (whether it is loaded or not): module show <modulename>

Monitoring Usage on the Batch System

For quick update on your allocated and available hours for the current 90-day moving window, try:

/usr/bin/python /var/aci/zACI-useralloc.pyc

For most users, this utility is run automatically when you log in to aci-b. For some of the same information in a less-nice format, you can use the glsfund command (try glsfund --man for complete documentation on this command).

The current state of the batch queue may be obtained using the showq command. To show the queue state for just the SCRiM allocation, use:

showq –w acct=kzk10_collab

To show the status of the open queue, use:

showq –w acct=open

To monitor your own overall usage, use:

glsusage –u <username>

Getting Support

You can obtain professional support for ICS-ACI resourcs in the following ways:

Support hours are Mon-Fri 9:00am-6:00pm and Sat-Sun 1:00pm-4:00pm. ICS has not prepared a comprehensive user guide for ACI, but you may also find the ACI Onboarding Materials to be a helpful resource.