scp and
ssh-agentat, batch, and
cron jobsPlease mail me any questions you may have, and when you get access if you find problems or other issues please let me know so I can bring them to OIT's attention. Also I would be happy to have any comments on or suggestions about this document.
The new cluster is named the Grace cluster ("Glue Research and Academic Computing Environment"). There are four machines, two are SPARC running Solaris (these have four 1.6 GHz UltraSPARC IIIi processors and 16G memory), and the other two are AMD64 running Red Hat Enterprise Linux (these have four 2.4 GHz Opteron processors and 16G memory). There is also a fifth machine specifically for Oracle database use. There's a webpage with some information about the systems for instructors and students at http://www.grace.umd.edu/.
If you're teaching a course where you give students some precompiled object files to link with their code, or you provide other files that are architecture-dependent, you might want to tell them to always specifically use one of the machine types. Note that there are obviously some differences between the architectures; although all common UNIX/Linux commands should be identical or very similar, some utilities are only on one type of machine (for instance, valgrind is a Linux-only application). The Solaris machines are much less heavily used, so if it doesn't matter what architecture your students use you may want to have them log into solaris.grace.umd.edu.
Note that insecure access (telnet and ftp)
to these systems is not possible; secure access (ssh,
slogin, scp) must be used instead.
tap
These systems use a program called tap so users
(students) can get access to certain software without having to
permanently modify their dotfiles. Some software is not on the standard
search path but running tap modifies the user's path and environment to
give access to it for the duration of the current login session. For
example, mathematica isn't on the default search path, but it can be run
after typing tap mathematica. "tap" without any
arguments lists the applications available via this mechanism.
Java 1.5.0 is the default on the two Grace Solaris machines.
Running java on the two Grace Linux machines gives the Red Hat
default GNU Java. Oracle Java 1.6.0 is available on both the
Solaris and Linux machines by typing tap -q
java6. (Just using tap -q java accesses the
most current version of Oracle Java that's installed.)
Eclipse 3.1.0 is available by typing tap
eclipse.
Allegro Common Lisp is available on the two Grace Solaris
machines after typing tap allegro70, for
version 7.0. Allegro Common Lisp is only licenced for the two
Solaris hosts, and is not available on the two Grace Linux
machines, so just tell students to log in to
solaris.grace.umd.edu to use it.
.cshrc.mine in your home directory, which gets executed
upon every login (this is assuming the default shell tcsh
is used; use .bashrc.mine for bash).
"Disposable" class accounts for each course, which disappear at the
end of the semester, are used with these systems. Students, faculty,
instructors, and TAs can access the systems using their campus
TerpConnect account (TerpConnect accounts were previously called Glue or
WAM accounts). Anyone affilitated with the University can get a
TerpConnect account online
at www.oit.umd.edu/new/.
Students registered for a course will then get login permission on the
Grace machines when the course instructor requests class access (see the
last section below).
Faculty and instructors will no longer have to deal with student
password problems, login difficulties, etc.; since students will be
using their TerpConnect accounts they would go to the OIT Help Desk for
such things (1400 CSS Building, x51500,
www.helpdesk.umd.edu/). For courses that use these systems students
also won't have to set up their environment (.emacs file, aliases, etc.)
every semester for the separate account for every course as with the old
OIT systems.
The Grace machines share a filesystem with the regular TerpConnect
machines. The differences between the Grace systems and the regular
TerpConnect machines are as follows. The Grace machines are dedicated
for instructional use, so only students taking courses can log into
them, while TerpConnect accounts persist as long as their owners are
associated with the University, so anyone can log into the regular
TerpConnect machines. The Grace machines are faster. The regular
TerpConnect machines are just Solaris, while there are two Solaris Grace
machines but also two Linux ones. Lastly, when class access is created
for the Grace systems, students registered for the course get extra disk
space (see below), and shared course disk space is created where files
may be provided to students.
Note that students may be using their TerpConnect account for more
than one course, if they're taking multiple courses using the Grace
cluster. They won't need separate class accounts for each course as
with the old cluster, they'll just get additional separate disk space
for each course, and access to shared course disk space created for each
course.
Note that a user's University directory ID and password are their
login ID and password for the TerpConnect and Grace systems. Although
some University applications (for example the registration system) allow
uppercase letters in the University ID when logging in, the Grace and
TerpConnect systems use the all-lowercase form of the directory ID, so
if students have trouble logging in one thing to ask is if they're using
uppercase letters in their directory ID.
tcsh is the default shell on the Grace machines. Although bash is
supported, and some other shells are installed, some settings may not
work correctly without tweaking if users change their shell. Various
shell and environment variables) in the system login scripts may not all
get set correctly, for example, if someone changes their shell. If
students have trouble with any tasks (things like resource limits,
submitting projects, using the right version of Java, etc., discussed
below) you may want to ask them if they've changed their shell. If so
they should either change it back, or, if they're proficient enough with
UNIX to have a preference between shells they may be able to figure out
has to be modified in their dotfiles to use another shell.
If a user accidentally deletes or trashes one of their dotfiles, but
can still log in and execute commands, default versions of all the
dotfiles as of when new accounts are created can be found in the
directory
TerpConnect uses the AFS file system. This allows instructors, and
TAs to grant varying levels of access to directories to specific lists
of users, without having to get staff to create groups. (Users can
create and administer their own groups in AFS as needed, although OIT's
web interface can usually handle most situations.) Each directory has
an Access Control List (ACL) which can be manipulated to control access.
Of course, it requires learning the AFS commands if you want to do these
things, if you don't know them already; these are described
at
www.helpdesk.umd.edu/documents/1/1222. A few of the things
described in more detail on that page are:
Accounts
Default dotfiles
/local/lib/skel.
AFS and Kerberos
fs
listacl
fs
setacl
The systems use Kerberos for authentication. When a user logs in they
get a Kerberos ticket and an AFS token (used for file permission and
authentication) by the login process. These both expire after a day;
the command klist lists Kerberos tickets and the time they
expire, while command tokens lists AFS tokens and when they
expire.
When you request space for your class the following directory will be created (suppose this is for CMSC 123, Fall 2008, section 0101):
/afs/glue.umd.edu/class/fall2008/cmsc/123/0101Under this several directories will be created (this chart is based on www.grace.umd.edu/instructors/filesystem.html):
| directory | student permissions | instructor permissions | purpose |
| .db/ | none | none | needed by OIT staff |
| public | read | read/write | place for instructor to post any materials for students to have access to |
| instructor | none | read/write | instructor's personal space |
| backup | read | read | holds a one-day-old copy of all the directories and subdirectories in this directory |
student/<loginID> |
read/write | read | individual students' AFS space for this course |
submit/<loginID> |
insert only | read/write | OIT's submit script writes files here |
There's a chart on the Grace website at www.grace.umd.edu/instructors/timeline.html indicating when after the semester is over student access to these directories will end, and when they're deleted.
As mentioned in the chart above, students are given extra disk space
for every course using the Grace cluster which they're registered for.
Instead of the disk space being in their home directory it's in the AFS
volume for that course. For example, a student whose login ID was
larry, taking CMSC 123, Fall 2008, section 0101 would get a directory
created named
/afs/glue.umd.edu/class/fall2008/cmsc/123/0101/student/larry.
To avoid students having problems with quota in their home directories
you may want to tell them how how to create a symlink from their home
directory to this AFS space, and to cd there to do all their coursework
for your course.
Note that after a semester ends students can just use cp
if they want to save their work, by copying it from this location to
their regular TerpConnect home directory tree (provided they have
sufficient disk quota avaialable; see below). Since TerpConnect
accounts persist as long as someone is associated with the University
they would have access to anything in their home directory tree for the
duration of their time at UMCP.
Quota shouldn't be as much of an issue to deal with as on the old
detective cluster, because students will get extra quota for each
course, and any questions about quota would be between a student and
OIT. The AFS class account space which is created for a course has a
common quota rather than a per-user quota. fs lq
<directory name> will list the quota for
the AFS volume if you want to check on the quota being used by your
class space.
If you are using the CSD project submission server the command-line
submission tool uses a jarfile, which requires Oracle Java, and won't
work with GNU Java. On the Linux Grace machines, where GNU Java is the
default, either tell students to type tap -q java (once per
login) before running the project submission script, or just have them
add it to their .cshrc.mine file in their home diretory,
which gets executed upon every login.
limit vmemoryuse 1500000 to
their ~/.cshrc.mine file, which is more than sufficient (a
smaller limit may also work if you want to experiment) and log in again.
There is a simpl option for project submission for those who aren't
using the CSD project submission server; as mentioned above the AFS
space for a course includes space and directories already set up for
project submission. The submit program which some faculty used on the
old OIT class cluster, written by Dr. Purtilo, won't work on these
systems as it won't run under AFS. OIT has a simple submit shell script
installed as /usr/local/scripts/submit. To use it a
student just runs submit, and it prompts for the necessary
information. It leaves the submitted file (for example for CMSC 123,
Fall 2008, section 0101) in
/afs/glue.umd.edu/class/fall2008/cmsc/123/0101/submit/<loginID>.
A subdirectory is created there for each project (based on the project
number given when submitting.)
This script lacks some features, for example it doesn't identify late submissions, it doesn't keep a log of submissions made, and the only mechanism to cut off submissions as of a certain time is to change the access permissions for the directory (or to copy the submitted files elsewhere) at the desired time; of course you can write a script to choose which submissions you want to grade based on their timestamp (the timestamp is appended to each submitted file's name). Of course it's possible to copy and modify this script to add features and have your students run your version instead, or to set up your own directory for submission elsewhere and write your own project submission program entirely if preferred.
This should be pretty obvious to faculty and TAs, but could be
confusing for some students. There are two separate Grace platforms
with access to the same filesystem: students can access their files when
logged in to one of the regular TerpConnect machines (SPARC 64 bit), to
the Solaris Grace hosts (SPARC 64 bit), or to the Linux Grace hosts (AMD
64 bit). An executable for example compiled on one of the Linux hosts
won't run when logged in to the others, and vice versa. Students may
want to always log in to the same type of Grace host, or at least know
about the uname command to be able to tell which type of
Grace host they're logged in to.
Ira Gold is not working closely with the new Cluster as he did with the old detective Cluster. Rather than emailing issues to him, problems can be reported using the "request" system- from the new machines (or from any TerpConnect machine) run "request" and type in the description of the issue. This directs it to the sysadmins. If you can't log in to a TerpConnect machine the request application is also available on the web at https://www.itsc.umd.edu, or just send mail about the problem to request@glue.umd.edu. To submit a request using the web interface of the ITSC request system click on "Home" at the link above and log in (before first-time use of the web interface use the registration link "New Users: Sign Up Here"). The procedure for requesting class account space is described below.
New TerpConnect accounts are set up to autoforward email to the user's University email account, which should be a Terpmail account for students, or a mail.umd.edu account for faculty and staff. Although it's doubtful anyone would want to do this, users who for some reason don't want their mail forwarded (who want to read mail sent to their TerpConnect account locally on the TerpConnect or Grace machines) can use OIT's request system (described right above) to ask that mail autoforwarding on their TerpConnect account to mail.umd.edu be turned off. If after that you want to forward email elsewhere, the .forward file should be put in an atypical location: if your account ID is "user" it would be placed in in /mail/user/.forward rather than in your home directory. You can point students who are novices to mail forwarding to more detailed procedures on the mail forwarding specific to TerpConnect on the Help Desk page www.helpdesk.umd.edu/os/unix/usage/410; this also explains to the student the format of the .forward file.
The Kerberos tickets and AFS tokens used for authentication will
expire after a day. If you leave an xterm running for longer than that
your session won't have any permissions, but just run renew
and type your password to restore access.
If you have a long-running or background job which will take longer to
run than the duration of your Kerberos tickets and AFS tokens, OIT has
an autorenew command which will extend tickets and tokens
before they expire (man autorenew has a brief explanation).
scp and ssh-agent
Due to Kerberos authentication it's not possible to log in to
TerpConnect and Grace machines without typing the password, even if
you're running
ssh-agent on your local machine. You also can't remotely
scp files to or from TerpConnect machines without typing the
password, for the same reason. However, you can run
ssh-agent and ssh-add on a TerpConnect or Grace
machine and scp files in either direction, without typing the password,
by just running scp on the TerpConnect or Grace host. This
requires typing your password once to log in to the Grace host and once
when running ssh-add on it, and then once every 25 hours
when you would need to run the renew command mentioned
above to renew your Kerberos tickets and AFS tokens.
at, batch, and cron jobs
Due to Kerberos authentication issues, users won't be able to run
at/batch/cron/ jobs on these
systems (you can run them, but since they wouldn't get any
authentication tickets they won't be able to access files). If there's
something which really has to be run as one of these types of jobs the
staff can set it up if you use the request system described above.
Request access for your course using the form at https://learningonline.umd.edu/LTtools/grace/request-grace-course.seam.
You have to have a TerpConnect account yourself in order for the request to be processed, otherwise you won't be able to request or access the class space. If you don't have a TerpConnect account already you can request one at www.oit.umd.edu/new. Once your account exists you can request class access.
When requesting Grace access, first you select the desired semester, and the system determines what courses you're listed as teaching. If you're teaching multiple courses you next choose which one to create Grace access for. If it's a course with multiple sections (all taught by you) you can create access for the sections separately, or for only one section or some sections, or for all sections separately. (If multiple instructors are teaching a course see below before proceeding.) More likely, if you are teaching a course with multiple sections you want to create one common site for all sections, meaning they would all share common AFS groups and common class disk space. Click on "Submit Request" and when the class access is created (which may take a day or so) you will get an email.
When class access has been set up you can then log into the ( same site used to request access in order to administer the class space, by cicking on "Manage site(s)". Click on "Manage" next to the course you want to administer, which takes you to a self-explanatory web interface that allows you to perform activities like requesting an increase in quota, add TAs to the course (who are automatically given access to the course space, just like students) modifying permissions (for example which directories TAs can access), etc.
Note: if multiple instructors are teaching different sections of the same course and want a shared Grace space for all sections there is currently no automatic mechanism to set this up. Before requesting access one of the instructors will need to submit a request explaining to OIT that you want a shared Grace space between different instructors' sections of the same courses, which they will set up manually. (Alternately, one instructor can create Grace access for their sections combined, then send the request asking for the other instructor's sections to be added to the shared space.), before the second instructor has requested
Note that your own login access to the Grace systems will disappear, as will the students', sometime after each semester is over, although you will remain able to log into the regular TerpConnect machines as long as you are associated with the University. At some point after each semester you would lose access to files created in the disk space created for the course (see the Grace access removal timeline at www.grace.umd.edu/instructors/timeline.html), so you would have to copy any files created in that space that you want to save to your TerpConnect home directory tree, or to some other machine, before file access is lost. If you want to have permanent login permission to the Grace machines you can request a research account at www.grace.umd.edu/research/. Note that even with a research account you would still lose access to course files according to the timeline linked to above, however you would continue to be able to log into the Grace machines even after your class access expires.