TAMUQ Research Computing Policies

From TAMUQ Research Computing User Documentation Wiki
Jump to navigation Jump to search
Acronyms: 
  HPC:	  High Performance Computing
  IVF:	  Immersive Visualization Facility
  MoU:	  Memorandum of Understanding
  QF:	  Qatar Foundation
  RC:	  Research Computing
  TAMU:  Texas A&M University 
  TAMUQ: Texas A&M University at Qatar


1. HPC Policies

a. Account Eligibility

Anyone employed by TAMUQ or another QF entity is eligible for an account, provided a legitimate academic or research need exists. HPC Cluster accounts shall be granted based on the following criteria (in order of priority, from highest to lowest):

  1. Applicant is TAMUQ faculty, or research staff supervised by TAMUQ faculty, or a research collaborator of TAMUQ faculty.
  2. Applicant belongs to a Qatar Foundation (or other Qatar-based) entity with which TAMUQ has an agreement for the use of HPC resources.
  3. Applicant belongs to an entity within Qatar, but one with which TAMUQ does not have an agreement for the use of HPC resources.

All HPC Cluster account requests must be submitted through our online account application form.

b. US Export Control Clearance

As per University rules and procedures, access to HPC resources is subject to Visual Compliance screening for purposes of compliance with U.S. export control requirements. The account application process itself includes procedures for obtaining export control clearance for the user, and requires completion of a checklist that is reviewed by TAMU’s Export Control Office, or designee, as part of this process.

c. Account Validity

All accounts are made to expire on an annual basis and must be renewed by submitting new account applications. Existing users are also required to submit new applications when their affiliation changes, even if the account is otherwise valid. An affiliation change is defined as either a change of the user’s institution, or a change of the user’s Principal Investigator (PI), or both.

Accounts that are not renewed before their expiry deadline are de-activated the day after the deadline. In such cases account data may be archived by RC staff until the account is renewed. If renewal does not take place within 6 months of account de-activation, and if alternate arrangements are not made (see section 3.g), the user’s data may be permanently deleted. Details about account expiry and the renewal process may be found in the user guides at the RC website.

d. Acceptable Use Statement

  1. Users should adhere to the Texas A&M Computing Policies, paying particular attention to the Rules for Responsible Computing and the Computer Security Policy, while using the supercomputers.
  2. All users must adhere to and comply with long and short term operational practices when logged on to a system. These practices are stipulated in the user guides for each system and, occasionally, in the system "message of the day".
  3. RC accounts are to be used only for the purpose for which they are authorized and are not to be used for non-RC related activities. Unauthorized use of a RC account/system may constitute sufficient grounds for account termination and/or legal sanctions.
  4. Users shall not share their account(s) with anyone. This includes sharing the password to the account, providing access via an ssh key entry or other means of sharing. It is a violation of state law to share accounts and passwords.
  5. A user must not engage in any activity that violates the export control laws of the United States and the cyber crime laws of the state of Qatar.
  6. Users are responsible for protecting any information used and/or stored on/in their accounts. Consult the user guides for guidelines on protecting your account and information using the standard system protection mechanisms.
  7. Users are required to report any weaknesses in computer security, and any incidents of possible misuse or violation of this agreement to the proper authorities by contacting RC (email: rc.admin@qatar.tamu.edu).
  8. Users shall not attempt to access any data or programs contained on systems for which they do not have authorization or explicit consent of the owner of the data/program or the university security authorities.
  9. Users must adhere to the provisions of our data storage policy. Please see the relevant item in the Data Storage Policies section.
  10. Users must have proper authorization to use copyrighted files. Unauthorized use of such files renders a user liable to a variety of sanctions. These include account suspension, account termination, and/or appropriate legal action.
  11. Users shall not make copies of system configuration files (e.g. /etc/passwd) for unauthorized personal use or to provide to other people/users for unauthorized uses.
  12. Users shall not intentionally engage in activities to: harass other users; degrade the performance of systems; deprive an authorized user access to a resource; obtain extra resources beyond those allocated; circumvent computer security measures or gain access to a system for which proper authorization has not been granted; misuse batch queues or other resources in ways not authorized or intended.
  13. Electronic communication facilities (such as E-mail) are for authorized use only. Fraudulent, harassing, obscene or sexually explicit messages and/or materials shall not be sent from, or stored on the RC systems.
  14. Users shall not download, install, or run security programs or utilities which reveal weaknesses in the security of a system. For example, RC users shall not run password cracking programs.

2. Data Storage Policies

a. Who can store data?

Only RC staff and RC users can store data on RC systems, whether those systems are meant for computation, or are dedicated exclusively to data storage or archiving.

b. Who owns the data?

The data stored under a user’s account is owned by the user’s Principal Investigator. The user’s PI may in fact legitimately request RC to suspend an account at any time. The PI is also authorized to make decisions about the disposal of the data in accounts under his/her charge.

c. What type of data may be stored?

  1. Data that serves as input to executables that are run on RC systems
  2. Data that is output from executables that are run on RC systems
  3. Executable files that are run on RC systems
  4. Other directly relevant files: source code, object files, user libraries, include files, make files, etc

d. What type of files should be regularly deleted?

Users should regularly delete non-essential files in their directories. Examples include simulation log files that would be irrelevant to research needs once job results have been correctly generated, or core files produced by application crashes.

e. When are home & project directories backed up?

Full backups of user home directories are generated on Fridays, once every 4 weeks. Furthermore, differential backups on those same directories are generated every other night. User files that are not stored in users' home directories are not backed up.

Backup schedules for project directories are determined on a case by case basis, and depend primarily on the volume of data concerned. Typically, a full backup at least once every 8 weeks, and differential backups at least once a week will be attempted. However, in cases where project directories are unusually large (e.g. more than 4TB) RC may not commit to any backups at all. Groups members for project directories should agree with RC on the backup schedule to be applied to their project directories.

f. How much data may be stored?

For the Lustre file system, default per-user disk quotas are defined in the user guides, but these quotas apply to all usage under /ddn, and not just to /ddn/home (in other words, usage under /ddn/scratch will count towards a user’s quota). If the need for a larger user limit can be demonstrated, RC will consider extensions to the default quotas on a case by case basis.

RC staff may employ alternative or complementary methods (to the traditional lustre quotas) to detect, warn about, and ultimately cap disk usage to enforce theses limits.

g. How long will data be retained on the system?

All disk-resident user data on all compute servers will be deleted six months after account de-activation unless a user has made prior arrangements with RC staff. An account is de-activated permanently if it is not renewed annually within 6 months of expiry. Within this 6 month period, if an account is not renewed, the account-holder’s PI can still direct RC to:

  1. delete the archived data
  2. copy the data to an external, non-RC storage device
  3. move the data to another user’s HPC account (without violating disk quota limits)
  4. extend the tenure of the data, subject to storage availability
  5. any combination of the above

h. How is /ddn/scratch to be used?

The /ddn/scratch directory is made available to meet the current needs of active jobs. This space is not meant in any way to provide long-term storage. Users are expected to continually delete or move elsewhere files that are not being used by current jobs. RC staff reserves the right to delete user files from /ddn/scratch at any time after the corresponding jobs have completed. Jobs intending to employ this scratch space should follow RC guidelines to name the temporary directories created therein. These guidelines are available in the user guides.

3. HPC Software Policies

This section of the policy pertains specifically to packages meant for HPC systems. Guidance on purchasing research software for desktops and workstations may be found in the “Guidance for Research Software Procurement” document found on the RC website.

a. Acquisition

RC has many commercial and freeware packages installed on its HPC systems. Our approach to acquisition of additional software depends upon multiple factors: its compatibility with our HPC systems, its impact on system security, performance and usability, financial & budgeting considerations, licensing restrictions, and user interest. In all cases in which a party requires the system-wide installation of licensed software currently unavailable on an HPC system, RC must be consulted before the software purchase is made in order to confirm our ability and willingness to install the package. Failure to do so may lead the requesting end users to a disappointing outcome.

  1. Single user interest
    2. If the package is of interest to a single TAMUQ user or research group, its license may be purchased by that user or group, and his/her department or sponsor. RC staff can then install or help install the package on an RC HPC Cluster and make it accessible exclusively by the software license holder(s).
  2. Group interest
    3. If a package is of interest to a group of several TAMUQ users, the best approach at first is for one user to act as a primary sponsor and arrange to split the procurement/licensing costs among the group. In this case, RC may assist in the procurement and billing of other group members as well as doing the installation.
  3. Broad interest
    4. RC will consider acquiring and supporting software packages that have broad interest among TAMUQ users. Full facility support will depend on the cost of the package and our ability to comply with any restrictive licensing conditions.
  4. Interest from TAMUQ partner institutes
    5. Licensed software required by partner institutes and their users must be purchased and procured by those institutes, and RC will then install the software and make it available exclusively to users belonging to that institute. As a general rule, software licensed exclusively for TAMUQ users will not be accessible to non-TAMUQ users. However, institutional agreements may enable the sharing of software licenses in some cases.

b. Licensing & Access Restrictions

Based on the terms of a given software license, RC has the ability to the restrict access to any software package to (a) an individual user, (b) an arbitrary but defined group of users, (c) all users affiliated with a specific institution, or (d) all users on the system. RC shall honor the terms of all software licenses and control access accordingly.

c. Compilation & Installation

Many HPC software packages adhere to open source licensing and are thus generally available for all users. However, these packages must usually be compiled before installation, sometimes requiring specialized knowledge and skills, and a great deal of time. Advanced users have the option to perform compilations themselves, whereas novice users may have to rely on assistance from system administrators for this purpose.

  1. Within this context, users are allowed to compile and install software within their home directories. Users should not, however, loosen access restrictions on their home directories in order to grant other colleagues access to such software. Instead, shared project folders should be requested from RC as the sharing mechanism in such cases.
  2. Alternatively, RC can compile such complex software on behalf of users and install it in locations accessible system-wide. Where such compilations are time consuming, RC will only commit to repeat compilations of minor revisions of a given package on a best-effort basis.

4. Restricted Software & Data Agreement

a. The User agrees to only employ RC systems for fundamental research purposes in compliance with all U.S. export control regulations, including but not limited to the Export Administration Regulations (EAR) and the International Traffic in Arms Regulations (ITAR). In this regard, the User is strictly prohibited from uploading or otherwise storing any export controlled software or data onto RC systems by means of a TAMUQ User account.

b. In the event that the User wishes to upload or store any software program or data that is export controlled onto RC systems, the User shall notify the Director of RC at least 30 days in advance of such intention upon which the Director will advise on the permissibility of such activity. Failure to adhere to this condition shall be considered a breach of this Agreement, resulting in the termination of the User’s RC account.

5. IVF Policies

a. The Immersive Visualization Facility at TAMUQ is intended primarily for research and teaching purposes.

b. At least one RC staff member must be present in the facility while the IVF is in operation.

c. Users must reserve the facility with at least one week advance notice. RC staff must be supplied with relevant visualization data (if any) at least one week in advance of a scheduled session.

d. To run a new application on the IVF systems, the end-user must supply the data three weeks in advance. The provided data format must be supported by the visualization system.

e. Unless authorized by the director of RC, an IVF session requires at least 3 attendees. Also, due to capacity limitations, no more than 18 attendees can be accommodated simultaneously.

f. Demo requests from departments not directly engaged in research or teaching activities should emanate from the directors of those departments.

g. Demo sessions for which attendees are delayed by more than 15 minutes are subject to cancellation by RC. If delays are expected, they must be communicated to RC in advance of the scheduled start time, and otherwise as early as possible.

Retrieved from "https://rcdocs.mywikis.wiki/w139/index.php?title=Policies&oldid=3200"