There are two types of SURAgrid user accounts to establish on your resource:

• SURAgrid accounts for your local user(s) (minimally for successful administration of your site's resources on SURAgrid, and eventually for access to SURAgrid for *your site's users*)
• SURAgrid accounts to enable access by users from other SURAgrid sites (Under current policies, resources added to SURAgrid are considered available to all SURAgrid application users when the application's system requirements can be accommodated.)

5.1 SURAgrid accounts for local users

All SURAgrid accounts are created and maintained via the SURAgrid Central User Authentication and Authorization System tool at: https://www.pki.virginia.edu/suragrid/. This system creates and maintains user entries in the SURAgrid LDAP directory. Figure 2 shows an example user entry.

Figure 2. SURAgrid LDAP Directory User Entry

Any site that is adding resources to SURAgrid needs to designate a SURAgrid User Site Administrator to initially configure, test and then properly maintain the resource as part of SURAgrid. A SURAgrid account should be created for this user and also used to obtain the most accurate results within the configuration and operation sections of this document.

Most likely, a site that is contributing resources to SURAgrid will also want to eventually enable access to SURAgrid resources for their local users. These SURAgrid accounts are also created by the SURAgrid User Site Administrator, which is typically the same person that manages the site's SURAgrid resources, but does not need to be.

To establish your site's SURAgrid User Administrator account, send an email request to sg-implementation@sura.org.

5.2 Establishing Access for Users From Other SURAgrid Sites

Globus assumes that standard Unix user accounts exist on all systems including account password file entries (e.g., Name, UID, GID, shell) and home directories. Globus maps PKI Subject Distinguished Names (DNs) to local Unix IDs. The web page https://www.pki.virginia.edu/sura-bridge/scl/ explains the steps necessary to manually create Unix accounts for all SURAgrid users and map SURAgrid user accounts to Unix accounts.

However, these mappings are maintained in the grid-mapfile on each Globus gatekeeper and the complexity of the DN naming convention makes manual maintenance of the grid-mapfile cumbersome (at best) and problematic (e.g., likeliness of typo's, scalability issues with dozens of sites and large numbers of researchers and students). Therefore, this web page also explains several automation tools provided by SURAgrid that simplify the creation of Unix accounts and the required home directories for all SURAgrid users. In addition, it provides several levels of automation that a site can implement to keep their grid-mapfile mappings of SURAgrid user accounts to Unix accounts current.

The steps involved in SURAgrid user account management and the related automation tools are summarized below.

5.2.1 Create Unix Accounts for SURAgrid Users

You need to add all of the SURAgrid users to your local Unix user database. (These accounts generally have the form sitename-username, such as uva-jaj. The uid numbers for these accounts start at 1,000,000 in order to avoid conflicts with the uids of local user accounts.) The SURAgrid LDAP server contains posixAccount schema data. To view the discussion of how you can automate replacement of Unix password files, see https://www.pki.virginia.edu/sura-bridge/scl/#adduser. In general, this includes changes to /etc/nsswitch.conf:

passwd: files ldap

and /etc/ldap.conf:

host sura-ldap1.itc.Virginia.EDU
base ou=SuraGrid,dc=sura,dc=org

Changes to /etc/nsswitch.conf and /etc/ldap.conf take effect immediately, although processes started before you change nsswitch.conf will not reread it and will therefore not see the change. (Note these changes must be propogated to all nodes if your host is a cluster.)

5.2.2 Create Home Directories for SURAgrid Users

The bulleted section entitled "Create Home Directories for SURAgrid Users" at https://www.pki.virginia.edu/sura-bridge/scl/#homedir discusses how to use the homedir.pl Perl script to create home directories for SURAgrid users. (You must edit this script before running it; the comments in the script include instructions.) This script, which must be run as superuser, does not create the account itself, but rather directories with consistent uid numbers across SURAgrid. You must create the associated account.

You will need to issue certificates to each of your site's users created by homedir.pl and create a .globus subdirectory under the directory created by homedir.pl (since all globus commands expect the directory created by homedir.pl to have the user's certificate.) For example, if using SURAgrid SimpleCA:

> cd /home/jdoe
> mkdir .globus
> cd .globus
> /usr/local/bin/usercert-request.sh
     Your full name:  John Q. Doe
     Enter your email address:  jdoe@somewhere.net
          ...
     Enter PEM pass phrase:
          ...

This will generate your user's private and public keys in usercert.csr and userkey.pem. As the administrator, you should sign the key:

> /usr/local/sura-grid-simpleCA/usercert-sign.sh usercert.csr

Users from other sites will use the certificate credentials issued by their home site. For example, "/home/username/.globus" is the original directory set up with user certificate and "/home/sitename-username" (where sitename is the 3-5 character acronym for the user's institution) is the directory the script homedir.pl creates. You will need to copy the .globus directory where the user certificate is stored to the "home/sitename-username" directory.

> mkdir /home/SURA-jdoe/.globus
> cp /home/jdoe/.globus/* /home/SURA-jdoe/.globus/

5.2.3 Map SURAgrid Users to Unix Accounts

You have the option of either using the SURAgrid LDAP callout to control local account mapping and authorization or simply setting up a grid-mapfile to do so.

• LDAP callout option

  SURAgrid provides a Globus module called "ldap_authz_callout" (commonly referred to as simply the "LDAP Callout") that augments your grid-mapfile by looking up a SURAgrid user by subject DN on the SURAgrid LDAP server to obtain the corresponding Unix loginid. The bulleted section entitled "Map SURA Grid Users to Unix Accounts" at https://www.pki.virginia.edu/sura-bridge/scl/#gridmapfile explains how to download the callout source and provides related installation instructions.

• Manual grid-mapfile maintenance option

  To get started, you can simply put an entry for your own personal certificate into a local grid-mapfile. This file is located at /etc/grid-security/grid-mapfile. So that you can perform testing, create the /etc/grid-security/grid-mapfile file. In this file, create a line containing your certificate subject DN (run grid-cert-info -subject) in quotes and your username. For example:

  "/C=US/O=SURADC/OU=SuraGrid/CN=John Doe/UID=jdoe" jdoe

  This will map the indicated user identified by the name /C=US/O=SURADC/OU=SuraGrid/CN=John Doe/UID=jdoe to the local Unix user jdoe.

  For further information, see the grid-mapfile section of the SURAgrid PKI Bridge Certification Authority and User Management System pages. Note that this topic is undergoing some evolution within SURAgrid, and we hope in particular to provide a mechanism for a fully-accredited approach in cooperation with the International Grid Trust Federation soon.

5.2.4 Synchronizing Passwords for SURAgrid Users

The SURAgrid stack recommendation for GSI-SSH on all SURAgrid resources and the SURAgrid LDAP Directory Server schema allow SURAgrid user to access all SURAgrid resources with just their SURAgrid account. As mentioned above to configure your Linux system to support SURAgrid account/password access to your resource, configure your LDAP to augment your local /etc/passwd file by editing one line in /etc/nsswitch.conf file:

passwd: files ldap

Grid users authenticate to Globus using certificates, so they do not need Unix passwords. Currently the LDAP server has no Unix passwords for SURAgrid users, so there is no need to augment /etc/shadow (which stores Unix passwords) with LDAP. (See https://www.pki.virginia.edu/sura-bridge/scl/#adduser.)

Note that there are several situations in which SURAgrid users must use a local account to access a site's resources:

• to access resources at SURAgrid sites that are cross-certified (allowing them to take advantage of the BCA for certificate exchange) but have local policies that require that local accounts be used for resource access, and
• to access resources at SURAgrid sites that are not cross-certified.

5.2.5 Update and Maintain Mappings of SURAgrid Users to Unix Accounts

Whether a site uses the LDAP Callout or the grid-mapfile option, the mappings stored in the grid-mapfile on the SURAgrid directory server need to be kept synchronized with the mappings on all SURAgrid resources. While use of the LDAP Callout should mean resource sites do not need to create or manage the grid-mapfile for true Globus applications, unfortunately at present GSISSH (GSI-OpenSSH) only uses the grid-mapfile. Therefore, in order to allow SURAgrid users to run GSISSH on SURAgrid resources, both those sites that use the LDAP Callout and those that choose the grid-mapfile need to install and run the grid-mapfile.pl script periodically with a cron job. (Note that GSISSH is currently a recommended component of the SURAgrid stack and is likely to become required in future stack updates.)

Before you add your resource to the SURAgrid Portal, you need to configure and verify the functionality of its basic grid services using the information below.

6.1 Security Services and Recommendations, Port Settings

6.1.1 Firewall Settings

If your resource is behind a firewall, make sure that the Globus ports (globus-gatekeeper, gsissh, gsiftp, and gram) can be accessed through the firewall. Only a specific range of ports needs to be opened.

A separate document that more fully addresses this important topic is available at Firewall Instructions for SURAgrid Resources. Globus also provides reference information at http://www.globus.org/toolkit/security/firewalls/.

6.1.2 Globus Account

We recommend running the Globus web services container as an ordinary user for increased security. The typical userid is 'globus' and you should create this account on your system if it does not already exist.

6.1.3 GSISSH

The GSI version of SSH is like regular SSH (similarly, GSISCP is like the regular SCP service). However, there is one difference — the GSI-version of these services can use the grid security infrastructure to provide authentication. By enabling GSISSH and providing SURAgrid with the port number you're using, you ensure that a SURAgrid user is authenticated with their grid credentials and will not be prompted for a password when accessing your resource. They are automatically mapped to their local, assigned grid userid. (See "Step 5: User Accounts" sections "Synchronizing Passwords for SURAgrid Users" and "Update and Maintain Mappings of SURAgrid Users to Unix Accounts" for more detail.)

It is recommended that you start GSISSH on a separate port. SURAgrid default port selections are 22 and 2222. However, you may choose to open another port for GSISSH if your site has local reasons that make ports 22 or 2222 unworkable for you.

The easiest way to do this, after using the SURAgrid automated stack installation, is to edit the $VDT_LOCATION/post-install/sshd file to add any options you may need (for example, specifying the port number using the "-p nnnnn" option.) Alternately, to make GSISSH listen on a different port, add the following line to the SXXsshd command:

(copied $GLOBUS_LOCATION/sbin/SXXsshd as /etc/rc.d/init.d/SXXsshd )
Start the server with $/etc/rc.d/init.d/SXXsshd start
#
# SSHD arguments can be added here within the following
# set of double quotes.
#
SSHD_ARGS="-p XXXX" (where XXXX=port #)

Next, you must enable your gsissh server by issuing (once only) the command:

> vdt-register-service --name gsisshd --type init --enable --protocol tcp --init-script \ 
          $VDT_LOCATION/post-install/sshd

After doing this step, the gsisshd service can be started or stopped using the commands:

> vdt-control --on gsisshd
or
> vdt-control --off gsisshd

You should then test your gsissh server or proceed directly to using it by simply changing to your user account, setting up either the SURAgrid client or server package, getting a grid proxy and then connecting to your server resource using the command "gsissh (your server) -p nnnnn" where nnnnn is the port you selected during the configuration step above.

6.1.4 Setting other special local conditions

To set the a port range to match your local preferred values, or to set any other special local conditions for your installation, you can put commands into the files $VDT_LOCATION/vdt/etc/vdt-local-setup.sh AND .csh (formatted appropriately for each shell.) For example, for globus tcp:

In $VDT_LOCATION/vdt/etc/vdt-local-setup.sh 
export GLOBUS_TCP_PORT_RANGE="40000,50000"
export GLOBUS_TCP_SOURCE_RANGE="40000,50000"

AND in $VDT_LOCATION/cat vdt/etc/vdt-local-setup.csh
setenv GLOBUS_TCP_PORT_RANGE "40000,50000"
setenv GLOBUS_TCP_SOURCE_RANGE "40000,50000"

Add any other commands as needed to handle variations that may apply to your local cluser environment.

Once such local conditions have been set, a "vdt-control --off" followed by "vdt-control --on" should start your SURAgrid services with the local settings applied. Use "vdt-control --list" to see what you have installed and whether it is enabled to respond to vdt-control --on and --off control commands.

6.1.5 Web Services GRAM

The Web Services GRAM (WS-GRAM) component allows remote users to execute applications on your system.

In order to configure WS-GRAM, sudo must be available. The Globus web services container executes as the globus user for improved security. But, when a user submits an application to execute, the WS-GRAM service needs to become that user to submit and manage the application. This is accomplished using sudo.

An easy way to configure sudo is to give the globus user permission to run the commands it needs to run, as any other user than root. This is accomplished by adding the following to your /etc/sudoers file (replace the "/usr/local" with your installation directory). Note that you must give the EXACT path to the respective components in the additions below; soft-links will not work, so give the exact path. Use visudo to make these changes:

Runas_Alias GLOBUSUSERS = ALL, !root
globus ALL=(GLOBUSUSERS) \
      NOPASSWD: /usr/local/suragrid/globus/libexec/globus-gridmap-and-execute \
      -g /etc/grid-security/grid-mapfile \
      /usr/local/suragrid/globus/libexec/globus-job-manager-script.pl *
globus ALL=(GLOBUSUSERS) \
      NOPASSWD: /usr/local/suragrid/globus/libexec/globus-gridmap-and-execute \
      -g /etc/grid-security/grid-mapfile \
      /usr/local/suragrid/globus/libexec/globus-gram-local-proxy-tool *

By default, WS-GRAM can start applications only on the host it is running on. However, when WS-GRAM is running on a cluster, it needs to submit the application to the batch scheduler. This support can currently be provided for the Condor, LSF, and PBS scheduling systems. Please contact the suragrid-support@sura.org email list if you require support for a different scheduler. To set up WS-GRAM to interact with your scheduler, execute one of the following commands. (NOTE: Double-check that the cache is the same one your installation used in the above step, and adjust the appropriate command below before issuing it if not!):

> pacman -get http://vdt.cs.wisc.edu/vdt_181_cache:Globus-WS-Condor-Setup
   or
> pacman -get http://vdt.cs.wisc.edu/vdt_181_cache:Globus-WS-LSF-Setup
   or
> pacman -get http://vdt.cs.wisc.edu/vdt_181_cache:Globus-WS-PBS-Setup

Please refer to the WS-GRAM Admin Guide for more information on configuring WS-GRAM services http://www.globus.org/toolkit/docs/4.0/admin/docbook/ch11.html.

6.1.6 Pre-WS GRAM

We recommend using WS-GRAM when joining SURAgrid. But you may find it necessary to run Pre-WS GRAM. If this is the case, the Pre-WS gatekeeper requires that a port (default TCP port 2119) be open for job submission and so that the SURAgrid Portal can get information about your resource. You can test if the gatekeeper is running with the globusrun command. For example:

> globusrun -a -r mileva.hpc.odu.edu
GRAM Authentication test successful

The job manager uses a port in a specified range to query your resource. For this purpose, a port range can be specified using the Globus environment GLOBUS_TCP_PORT_RANGE (see See "Setting other local special conditions" above or "Section 4" of http://www.globus.org/toolkit/security/firewalls/). If your site's local security policies prevent this, the next best thing might be to open the port to a selected set of machines (i.e., all SURAgrid resources.)

An additional set of ports needs to be opened so that Globus can use them for things such as the GRAM job manager and gridftp callbacks.

6.1.7 MyProxy services

The SURAgrid Portal uses MyProxy services to manage SURAgrid user credentials and enable them to be authenticated on SURAgrid resources. The SURAgrid portal has a current list of SURAgrid MyProxy servers (https://gridportal.sura.org/gridsphere/gridsphere?cid=documentation&Java...). Refer to the "Using the Portal" page of the Documentation tab in the SURAgrid Portal for information on how to upload and manage MyProxy credentials, submit and manage jobs, and transfer files using the SURAgrid Portal.

In general, using ODU as the MyProxy server:

> grid-proxy-init
> myproxy-init -s mileva.hpc.odu.edu -l mysite-myaccount

Note that the machine that you run the myproxy-init from should first be cross-certified with the SURAgrid BCA and should have all the necessary files copied into the /etc/grid-security/certificates directory (see "Cross-certify your site with the SURAgrid BCA" in Step 4). If your site does not have SURAgrid bridge cross-certification, send the signing_policy and .0 files for the CA that issued your host certificate to the MyProxy server's resource administrator and ask them to install the files on their MyProxy server (see "Establishing trust when cross-certification is not an option" in Step 4).

More information about MyProxy can be found at http://grid.ncsa.uiuc.edu/myproxy and http://www.globus.org/alliance/publications/papers/myproxy.pdf.

6.2 Data Services

6.2.1 GridFTP

The Globus GridFTP provides high-performance file movement between SURAgrid systems. The GridFTP server runs out of xinetd and no configuration is required after using the SURAgrid automated stack installation.

You can test your GridFTP installation using globus-url-copy or uberftp. To verify that gridftp is setup correctly, test your resource's ability to move files on SURAgrid by using a utility such as globus-url-copy against SURAgrid's Bandera resource (or any operational SURAgrid host listed on the portal). For example:

> globus-url-copy file://bandera.tacc.utexas.edu/tmp/file1 gsiftp://bandera.tacc.utexas.edu/tmp/file2

Be sure that the user running this has their .globus directory properly configured (see "Create Home Directories for SURAgrid Users" in Step 5.)

6.2.2 Reliable File Transfer (RFT)

Configuration of this service is optional. RFT is included in the SURAgrid automated stack installation.

6.2.3 Replica Location Services (RLS)

Configuration of this service is optional.

6.3 Starting Services

You can now start your Globus services, including the container that contains the WS-GRAM service, by executing:

> vdt-control --on

Note that if you had pre-existing services of the same name left over from previous installations, you can add the parameter "--force" to the vdt-control command the first time you issue it. The previous services will be renamed and retained but not started by the VDT once this is done. You may want to go through one cycle of "vdt-control --off --force" followed by "vdt-control --on" to get to a known state your first cycle through. After this, "vdt-control --off" or "--on" should be enough to turn off or on your grid services.

6.4 Job Management

As mentioned above, it is strongly recommend that you set up a local resource manager and a scheduler, especially if your resource is a compute cluster or a symmetric multiprocessor. The section below includes a discussion of how to configure your resource to accept job requests from SURAgrid users. While there are many ways of submitting jobs, from basic to advanced, we discuss some fairly common means of job submission here.

6.4.1 Web Services

Before proceeding with the instructions below, ensure RFT has been configured for job staging.

WS-GRAM job submission

You can test if WS GRAM was correctly setup with the following commands:

>  globusrun-ws -submit -c /bin/true
Submitting job...Done.
Job ID: uuid:3304e3f2-55f2-11da-8b8f-00d0b7b7c0bc
Termination time: 11/16/2005 16:09 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.
>  echo $?
0
>  globusrun-ws -submit -c /bin/false
Submitting job...Done.
Job ID: uuid:456b7c9a-55f2-11da-9b0d-00d0b7b7c0bc
Termination time: 11/16/2005 16:09 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.
>  echo $?
1

Please refer to section 4 of the Globus document "GT 4.0 WS GRAM: User's Guide", http://www.globus.org/toolkit/docs/4.0/execution/wsgram/user-index.html#..., for more details about various usage scenarios and to section 4.6 specifically for additional information about configuring your local resource manager with WS GRAM. See section 3.5.2. "Local resource manager configuration" and section 3.9. "WS GRAM auto-registration with default WS MDS Index Service" in the Globus document "GT 4.0 WS GRAM : System Administrator's Guide", http://www.globus.org/toolkit/docs/4.0/execution/wsgram/admin-index.html... Information about registering with mds-servicegroup-add and the Index Service is also available from Globus.org and can be found at http://www.globus.org/toolkit/docs/4.0/info/aggregator/re01.html#mds-ser...

Note that you will need to edit the file:

$GLOBUS_LOCATION/etc/globus_wsrf_mds_aggregator/example-aggregator-registration.xml

to represent your local resource (in particular, edit the hostname and resource manager name.)

Here are some quick links to the commands:

• globusrun-ws - Official job submission client for WS GRAM
• managed-job-globusrun - (DEPRECATED) Java-based job submission client for GRAM
• globus-job-run-ws - Interactive job submission script for WS GRAM (requires globus_wsrf_gram_client_tools update package from http://www.globus.org/toolkit/downloads/development/)
• globus-job-submit-ws - Batch job submission script for WS GRAM (requires globus_wsrf_gram_client_tools update package from http://www.globus.org/toolkit/downloads/development/)
• globus-job-get-output-ws - Job output fetch script for WS GRAM (requires globus_wsrf_gram_client_tools update package from http://www.globus.org/toolkit/downloads/development/)
• globus-job-clean-ws - Destroy and clean up a batch job for WS GRAM (requires globus_wsrf_gram_client_tools update package from http://www.globus.org/toolkit/downloads/development/)

6.4.2 Pre-WS Services

Pre-WS Job submission

You can test job submissions via the job manager by submitting a job request to your resource using the globus-job-submit command. For example:

> globus-job-submit  -np 1 /bin/date

and:

> globusrun -o -r mileva.hpc.odu.edu/jobmanager-pbs '&(executable="/bin/echo") \
          (arguments="Grid Status Test")'
Grid Status Test

In order to determine which Job manager is in use, you can use grid-info-search command. See Appendix A for example results from a SURAgrid resource.

6.5 Information Services

Access to information about resources on the grid is necessary for monitoring of and access to those resources. SURAgrid uses several common information services:

• Globus Monitoring and Discovery System (MDS)
• Gridport Grid Portals Information Repository (GPIR)
• Integration of MDS and/or GPIR with Resource Manager (PBS, SGE, etc.)

MDS provides information to resource providers such as host data, memory size, processor type, current load, and so forth. It also provides information to end users about queues, the number of processors available, current job counts, and memory usage. It also provides file transfer information. MDS is included in the SURAgrid automated stack installation.

The GPIR service provides the pathway between the MDS data services and the GridPort portal. See Step 8 for more information about using GPIR and the Portal link for a look at the SURAgrid portal.

Pre-WS MDS

Although you can query a different MDS server than the host of the GRAM and GridFTP servers, it is typically the MDS running on the same node as the services that is queried. An example query is:

> grid-info-search -anonymous -L -h myresource.myinstitution.org –nowrap 

There are two methods that can be used to configure pre-WS MDS. One is to provide anonymous (and perhaps less secure) queries, while the second is to provide LDAP-based (and perhaps more secure) queries. When using the LDAP option, the -pre-ws-mds configure switch must be used during Globus installation in order to generate some LDAP certificates. However, several things should be noted:

• There is a security warning from Globus Alliance noting that pre-WS MDS uses port 2135 as the default port. Some SURAgrid sites have therefore restricted this service (port) to certain SURAgrid users' machines (e.g., SCOOP/RENCI for the ADCIRC application, http://www.sura.org/programs/docs/sura_grid_adcirc_sep_05.pdf); this method allows the application to be supported but should not compromise your system's security.
• There is a known issue in Rocks 4.1 Grid Roll that causes grid-info-search hangs. This problem is limited to using the pre-built binaries contained in the Grid Roll 4.0.1 (it is unclear whether the problem extended to 4.2 as well). One workaround is of course to not use the Grid Roll and to instead build Globus from source or use the globus.org binary download. However since the Grid Roll makes for a quick initial code load, another option is to still use the Grid Roll and to then go back and set each service separately. For more current/peer advice, you can send an email to the SURAgrid support listserv (suragrid-support@sura.org ).

Additional Information

To integrate pre-WS MDS with PBS, see the reference material at http://calclab.math.tamu.edu/grid/ADCIRC.xhtml)

Part of the SURAgrid stack specification is the configuration of at least the required environment variables. These are set automatically during the one-button stack installation.

If you are performing a manual installation or who would like more information about the environment variables, see the SURAgrid Environment Variables document (Version 1, http://www.sura.org/programs/SURAgrid_EnvVar.htm) which lists both the required and recommended variables to be set on all SURAgrid resources.

SURAgrid's resources can be monitored and accessed through the SURAgrid Portal (http://gridportal.sura.org). Please follow the instructions below to add your resource to the SURAgrid Portal and to allow SURAgrid to run test scripts periodically to verify the operability of your resource's required grid services. If you need assistance while working through the steps below, send an email to PortalAdmin@sura.org.

8.1 SURAgrid Resource Data Collection Form

First, fill out and submit the "SURAgrid Resource Data Collection Form" form at http://sura.org/programs/RDCform.html. The details of your resource are published in various places in the SURAgrid portal and are also used for SURAgrid technical and administrative support purposes.

(Please remember to resubmit the form when you make changes to your resources.)

8.2 Install GPIR Resource Monitor Code

Install the remote GPIR provider code for use with SURAgrid's GPIR resource monitor. This code gathers data about your resource (e.g., job, load) that is returned to and published in the SURAgrid Portal.

• PBS, LoadLeveler, LSF, and Condor:
  • Download: http://gridport.net/services/gpir/gpir-download.html
  • Install and configure instructions: http://gridport.net/services/gpir/providers.html
• SGE:
  • http://projects.uabgrid.uab.edu/gpir-sge#DownloadInstall

Note 1: It is not necessary to install the full GPIR service contained in the downloaded package. Only a subset (the GPIR provider) needs to be installed.

Note 2: For step six in the GridPort configuration instructions, use the following values:

• hostname=yourHostname
• gpir.contact=cuero.tacc.utexas.edu:12080/gpir/webservices
• admin.email=yourAdminEmailAddress

8.3 Enable Automated Resource Verification via SURAgrid Test Scripts

SURAgrid's GPIR scripts run periodically during each hour under the SGmonitor account. This script tests the following on your SURAgrid resource:

• Gatekeeper authentication
• GSIFTP Server availability
• GSISSH Server operation
• Grid Programs in PATH
• Simple Test of jobmanager
• GSIFTP Job Test of jobmanager

The test script runs using a certificate issued by TACC and under a user account of SGmonitor, which must be established on each resource. To enable the test script to run:

• Refer to Step 4 for instructions on use of certificates between SURAgrid sites. If you are ready to verify your resource but cross-certification is still in planning or progress for your site, you may enable access through exchange of your CA's .0 and signing_policy files directly with TACC by sending them to PortalAdmin@sura.org.
• Refer to Step 5 for instructions on establishing SURAgrid user accounts in general. Specifically, ensure that your grid-mapfile allows SGmonitor access by including the following authorized account information:
  

  "/C=US/O=UTAustin/OU=TACC/CN=SuraGrid Monitor/UID=sgmonitor" SuraGrid Monitor

  For more information see Step 5. "Map SURAgrid Users to Unix Accounts".

8.4 Verify the Accessibility of Your Resource Across SURAgrid

In order to verify that your resource has been properly added to the SURAgrid portal, that it's fully accessible for testing, and that SURAgrid users can run jobs on it:

• Find your machine in the Resource Monitor subtab of the SURAgrid portal's Welcome tab (https://gridportal.sura.org/gridsphere/gridsphere?cid=resource-monitor&J...)
• Follow the "Click Here" link under the Test Status column. This will bring you to a page which shows the test results for all SURAgrid resources.
• If your resource shows a "Pass" indicator under all tests, send an email to PortalAdmin@sura.org so that SURA can formally thank you and announce the availability of your resource to SURAgrid users and other resource administrators.
• If your resource does *not* show a "PASS" indicator under any of the tests, see the test documentation further down the page. Contact PortalAdmin@sura.org for assistance in troubleshooting test failures.
• In addition, refer to the "Using the Portal" page of the Documentation tab in the SURAgrid Portal for instructions on using the portal (e.g., portal user authorization and login; working with files and jobs.)

Support of your SURAgrid resource

It is understood in SURAgrid that contributed resources are supported at varying levels, in keeping with what each resource owner is able to provide. Your site’s support levels and methods for each of your SURAgrid resources should be documented as fully as possible and are published in the SURAgrid portal. As described in Step 8 above, all SURAgrid resources are verified regularly through a script used by the SGmonitor user and the results of these tests are also published for SURAgrid users in the the SURAgrid Portal.

Support for resource owners

Support as well as troubleshooting assistance for installation, configuration and integration of SURAgrid resources is available from the SURAgrid implementation team ( sg-implementation@sura.org ) and via peer support through the SURAgrid support listserv: suragrid-support@sura.org . The list includes threaded and searchable archives for access by subscribers (http://www.sura.org/archives/suragrid-support) and can support subject line queries. Your questions and comments are always welcome and will be used by the SURAgrid team to modify this document as needed as well as in the analysis of SURAgrid’s infrastructure and policies.

Appendix 1. Web Services versus Pre-Web Services

While early grids were built with something called "pre-web services", today the recommended approach is through "web services." The advantages of web services (WS) over pre-web services (pre-WS) include platform and language independence, interoperability, ease of use and reuse, standards-based, and more. See the Web Services Primer for more information.

As the following diagram shows, SURAgrid can contain both pre-WS and WS resources. While we recommend WS, we show usage examples of each in this appendix.

(Click on the diagram for a larger version.)

Web Services Example

An example that displays a Web Services ServiceGroup Overview can be found at http://mileva.hpc.odu.edu:8080/webmds/webmds?info=indexinfo&xsl=serviceg.... As shown:

If we click on the "detail" for GRAM, we get:

(To see the full listing, go to http://mileva.hpc.odu.edu:8080/webmds/webmds?info=indexinfo&xsl=sgedetai....)

Pre-Web Services Example

[17:38] mileva:~]grid-info-search -h mileva.hpc.odu.edu -x
version: 2

#
# filter: (objectclass=*)
# requesting: ALL
#

# mileva.hpc.odu.edu, local, grid
dn: Mds-Host-hn=mileva.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsComputer
objectClass: MdsComputerTotal
objectClass: MdsFsTotal
objectClass: MdsHost
objectClass: MdsMemoryRamTotal
objectClass: MdsMemoryVmTotal
objectClass: MdsNet
objectClass: MdsNetTotal
objectClass: MdsOs
Mds-Computer-isa: x86_64
Mds-Computer-platform: x86_64
Mds-Computer-Total-nodeCount: 1
Mds-Fs-freeMB: 1005
Mds-Fs-freeMB: 18979
Mds-Fs-freeMB: 4644
Mds-Fs-freeMB: 485
Mds-Fs-freeMB: 6925
Mds-Fs-freeMB: 8991
Mds-Fs-freeMB: 9281
Mds-Fs-sizeMB: 1005
Mds-Fs-sizeMB: 14762
Mds-Fs-sizeMB: 24861
Mds-Fs-sizeMB: 487
Mds-Fs-sizeMB: 9844
Mds-Fs-Total-count: 11
Mds-Fs-Total-freeMB: 126226
Mds-Fs-Total-sizeMB: 170091
Mds-Host-hn: mileva.hpc.odu.edu
Mds-keepto: XXXX
Mds-Memory-Ram-freeMB: 384
Mds-Memory-Ram-sizeMB: 2010
Mds-Memory-Ram-Total-freeMB: 384
Mds-Memory-Ram-Total-sizeMB: 2010
Mds-Memory-Vm-freeMB: 4093
Mds-Memory-Vm-sizeMB: 6000
Mds-Memory-Vm-Total-freeMB: 4093
Mds-Memory-Vm-Total-sizeMB: 6000
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-name: eth0
Mds-Net-name: eth1
Mds-Net-name: lo
Mds-Net-name: vmnet1
Mds-Net-name: vmnet8
Mds-Net-netaddr: XXX.XXX.XXX.XXX/X
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-Total-count: 5
Mds-Os-name: Linux
Mds-Os-release: 2.6.9-22.ELsmp
Mds-Os-version: 1 SMP Sat Oct 8 21:32:36 BST 2005
Mds-validfrom: 20070109223843Z
Mds-validto: 20070109223843Z

# memory, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-Group-name=memory, Mds-Host-hn=mileva.hpc.odu.edu,Mds-Vo-name=l
 ocal,o=grid
objectClass: MdsMemoryRamTotal
objectClass: MdsMemoryVmTotal
objectClass: MdsDeviceGroup
Mds-Device-Group-name: memory
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070112173204Z
Mds-Memory-Ram-Total-sizeMB: 2010
Mds-Memory-Ram-Total-freeMB: 384
Mds-Memory-Vm-Total-sizeMB: 6000
Mds-Memory-Vm-Total-freeMB: 4093
Mds-Memory-Ram-sizeMB: 2010
Mds-Memory-Ram-freeMB: 384
Mds-Memory-Vm-sizeMB: 6000
Mds-Memory-Vm-freeMB: 4093

# physical memory, memory, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=physical memory, Mds-Device-Group-name=memory, Mds-Host-hn
 =mileva.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: Mds
objectClass: MdsDevice
objectClass: MdsMemoryRam
Mds-Device-name: physical memory
Mds-Memory-Ram-sizeMB: 2010
Mds-Memory-Ram-freeMB: 384
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070112173204Z

# virtual memory, memory, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=virtual memory, Mds-Device-Group-name=memory, Mds-Host-hn=
 mileva.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: Mds
objectClass: MdsDevice
objectClass: MdsMemoryVm
Mds-Device-name: virtual memory
Mds-Memory-Vm-sizeMB: 6000
Mds-Memory-Vm-freeMB: 4093
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070112173204Z

# filesystems, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-Group-name=filesystems, Mds-Host-hn=mileva.hpc.odu.edu,Mds-Vo-n
 ame=local,o=grid
objectClass: MdsFsTotal
objectClass: MdsDeviceGroup
Mds-Device-Group-name: filesystems
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070110043844Z
Mds-Fs-freeMB: 1005
Mds-Fs-freeMB: 18979
Mds-Fs-freeMB: 4644
Mds-Fs-freeMB: 485
Mds-Fs-freeMB: 6925
Mds-Fs-freeMB: 8991
Mds-Fs-freeMB: 9281
Mds-Fs-sizeMB: 1005
Mds-Fs-sizeMB: 14762
Mds-Fs-sizeMB: 24861
Mds-Fs-sizeMB: 487
Mds-Fs-sizeMB: 9844
Mds-Fs-Total-count: 11
Mds-Fs-Total-freeMB: 126226
Mds-Fs-Total-sizeMB: 170091

# /export, filesystems, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=/export, Mds-Device-Group-name=filesystems, Mds-Host-hn=mi
 leva.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsFs
Mds-Device-name: /export
Mds-Fs-sizeMB: 24861
Mds-Fs-freeMB: 18979
Mds-Fs-mount: /export
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070110043844Z

# /opt, filesystems, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=/opt, Mds-Device-Group-name=filesystems, Mds-Host-hn=milev
 a.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsFs
Mds-Device-name: /opt
Mds-Fs-sizeMB: 14762
Mds-Fs-freeMB: 6925
Mds-Fs-mount: /opt
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070110043844Z

# /scratch, filesystems, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=/scratch, Mds-Device-Group-name=filesystems, Mds-Host-hn=m
 ileva.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsFs
Mds-Device-name: /scratch
Mds-Fs-sizeMB: 9844
Mds-Fs-freeMB: 9281
Mds-Fs-mount: /scratch
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070110043844Z

# /tmp, filesystems, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=/tmp, Mds-Device-Group-name=filesystems, Mds-Host-hn=milev
 a.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsFs
Mds-Device-name: /tmp
Mds-Fs-sizeMB: 9844
Mds-Fs-freeMB: 8991
Mds-Fs-mount: /tmp
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070110043844Z

# /home/thorpe, filesystems, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=/home/thorpe, Mds-Device-Group-name=filesystems, Mds-Host-
 hn=mileva.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsFs
Mds-Device-name: /home/thorpe
Mds-Fs-sizeMB: 24861
Mds-Fs-freeMB: 18979
Mds-Fs-mount: /home/thorpe
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070110043844Z

# /home/uab-afgane, filesystems, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=/home/uab-afgane, Mds-Device-Group-name=filesystems, Mds-H
 ost-hn=mileva.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsFs
Mds-Device-name: /home/uab-afgane
Mds-Fs-sizeMB: 24861
Mds-Fs-freeMB: 18979
Mds-Fs-mount: /home/uab-afgane
Mds-validfrom: 20070109223844Z
Mds-validto: 20070109233844Z
Mds-keepto: 20070110043844Z

# networks, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-Group-name=networks, Mds-Host-hn=mileva.hpc.odu.edu,Mds-Vo-name
 =local,o=grid
objectClass: MdsNetTotal
objectClass: MdsNet
objectClass: MdsDeviceGroup
Mds-Device-Group-name: networks
Mds-validfrom: 20070109223844Z
Mds-validto: 20070110043844Z
Mds-keepto: 20070110043844Z
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-Net-name: eth0
Mds-Net-name: eth1
Mds-Net-name: lo
Mds-Net-name: vmnet1
Mds-Net-name: vmnet8
Mds-Net-netaddr: XXX.XXX.XXX.XXX/8
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-Total-count: 5

# eth0, networks, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=eth0, Mds-Device-Group-name=networks, Mds-Host-hn=mileva.h
 pc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsNet
Mds-Device-name: eth0
Mds-Net-name: eth0
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-validfrom: 20070109223844Z
Mds-validto: 20070110043844Z
Mds-keepto: 20070110043844Z

# eth1, networks, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=eth1, Mds-Device-Group-name=networks, Mds-Host-hn=mileva.h
 pc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsNet
Mds-Device-name: eth1
Mds-Net-name: eth1
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-addr: XXX.XXX.XXX.XXX
Mds-validfrom: 20070109223844Z
Mds-validto: 20070110043844Z
Mds-keepto: 20070110043844Z

# lo, networks, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=lo, Mds-Device-Group-name=networks, Mds-Host-hn=mileva.hpc
 .odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsNet
Mds-Device-name: lo
Mds-Net-name: lo
Mds-Net-netaddr: 127.0.0.0/8
Mds-Net-addr: 127.0.0.1
Mds-validfrom: 20070109223844Z
Mds-validto: 20070110043844Z
Mds-keepto: 20070110043844Z

# vmnet1, networks, mileva.hpc.odu.edu, local, grid
dn: Mds-Device-name=vmnet1, Mds-Device-Group-name=networks, Mds-Host-hn=mileva
 .hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: MdsDevice
objectClass: MdsNet
Mds-Device-name: vmnet1
Mds-Net-name: vmnet1
Mds-Net-netaddr: XXX.XXX.XXX.XXX/24
Mds-Net-addr: 192.168.92.1
Mds-validfrom: 20070109223844Z
Mds-validto: 20070110043844Z
Mds-keepto: 20070110043844Z

# operating system, mileva.hpc.odu.edu, local, grid
dn: Mds-Software-deployment=operating system, Mds-Host-hn=mileva.hpc.odu.edu,M
 ds-Vo-name=local,o=grid
objectClass: MdsSoftware
objectClass: MdsOs
Mds-Software-deployment: operating system
Mds-Os-name: Linux
Mds-Os-release: 2.6.9-22.ELsmp
Mds-Os-version: 1 SMP Sat Oct 8 21:32:36 BST 2005

# MDS, mileva.hpc.odu.edu, local, grid
dn: Mds-Software-deployment=MDS, Mds-Host-hn=mileva.hpc.odu.edu,Mds-Vo-name=lo
 cal,o=grid
objectClass: MdsSoftware
objectClass: MdsService
objectClass: MdsServiceLdap
Mds-Software-deployment: MDS
Mds-Service-type: ldap
Mds-Service-hn: mileva.hpc.odu.edu
Mds-Service-port: 2135
Mds-Service-Ldap-timeout: 30
Mds-Service-admin-contact: unspecified
Mds-Service-Executable-PID: 6318
Mds-Service-Path: /opt/globus-4.0.2
Mds-Service-admin-comment: This is an MDS 2.4 deployment.
Mds-validfrom: 20070109223844Z
Mds-validto: 20070110043844Z
Mds-keepto: 20070112173204Z

# jobmanager-pbs, mileva.hpc.odu.edu, local, grid
dn: Mds-Software-deployment=jobmanager-pbs, Mds-Host-hn=mileva.hpc.odu.edu,Mds
 -Vo-name=local,o=grid
objectClass: Mds
objectClass: MdsSoftware
objectClass: MdsService
objectClass: MdsServiceGram
objectClass: MdsComputer
objectClass: MdsOs
Mds-Software-deployment: jobmanager-pbs
Mds-Service-type: x-gram
Mds-Service-hn: mileva.hpc.odu.edu
Mds-Service-port: 2119
Mds-Service-url: x-gram://mileva.hpc.odu.edu:2119/jobmanager-pbs:/O=ODU/OU=ODU
 grid/OU=ODUgrid-sly.hpc.odu.edu/CN=host/mileva.hpc.odu.edu
Mds-Service-protocol: 0.1
Mds-Computer-isa: x86_64
Mds-Os-release: 2.6.9-22.ELsmp
Mds-Os-name: Linux
Mds-Computer-manufacturer: unknown
Mds-Service-Gram-schedulertype: pbs
Mds-validfrom: 200701092238.44Z
Mds-validto: 200701092239.14Z
Mds-keepto: 200701092239.14Z

# batch, jobmanager-pbs, mileva.hpc.odu.edu, local, grid
dn: Mds-Job-Queue-name=batch, Mds-Software-deployment=jobmanager-pbs, Mds-Host
 -hn=mileva.hpc.odu.edu,Mds-Vo-name=local,o=grid
objectClass: Mds
objectClass: MdsSoftware
objectClass: MdsJobQueue
objectClass: MdsComputerTotal
objectClass: MdsComputerTotalFree
objectClass: MdsGramJobQueue
Mds-Job-Queue-name: batch
Mds-Computer-Total-nodeCount: 4
Mds-Computer-Total-Free-nodeCount: 4
Mds-Memory-Ram-Total-sizeMB: 0
Mds-Memory-Ram-sizeMB: 0
Mds-Gram-Job-Queue-maxtime: 0
Mds-Gram-Job-Queue-maxcputime: 0
Mds-Gram-Job-Queue-maxcount: 4
Mds-Gram-Job-Queue-maxrunningjobs: 0
Mds-Gram-Job-Queue-maxjobsinqueue: 0
Mds-Gram-Job-Queue-whenactive: 0
Mds-Gram-Job-Queue-status: enabled
Mds-Gram-Job-Queue-dispatchtype: batch
Mds-Gram-Job-Queue-priority: NULL
Mds-Gram-Job-Queue-jobwait: NULL
Mds-Gram-Job-Queue-schedulerSpecific: NULL
Mds-validfrom: 200701092238.44Z
Mds-validto: 200701092239.14Z
Mds-keepto: 200701092239.14Z

# local, Grid
dn: Mds-Vo-name=local,o=Grid
objectClass: GlobusStub

# search result
search: 2
result: 0 Success

# numResponses: 28
# numEntries: 27
[17:38] mileva:~]