Submitting Condor Jobs to Globus Toolkit 4

This technical note explains how to submit Condor jobs to a Globus Toolkit 4 frontend node. It expands on the official documentation provided by the Condor Project. Our examples demonstrate job submissions to the OFFIS Cluster, which is a part of the German Grid and thus only accessible by its participants. However, the general procedure, a deeper understanding of the behind-the-scenes mechanisms and our troubleshooting advice may be helpful to other Condor administrators and users who wish to take advantage of the Grid resources available to them.

This documentation is provided by the WISENT project, sponsored by the German Federal Ministry of Education of Research. Please contact Jan.Ploski (at) offis.de if you have corrections, questions or remarks regarding this document.

Background and Motivation

Condor provides an environment for the management of batch jobs typical in scientific applications. In our project WISENT, Condor is in productive use at our partner DLR PA. Condor's features make it particularly suitable for managing a pool of desktop resources and for so-called "cycle scavenging". That is, jobs utilize the computing resources as they become available. Also, a job can be transferred to another machine if the availabilty of its current host changes.

Our project WISENT is also associated with the German Grid and maintains a high-performance computing cluster at OFFIS. This cluster can be accessed by all D-Grid participants, including DLR PA. Like most D-Grid computing resources, it is accessible from the Internet using interfaces exposed by Globus Toolkit 4 - the GRAM protocol for job submission and GridFTP for job-related data transfers.

In order to make the cluster at OFFIS available to the Condor users at DLR PA, we rely on Condor's capability to submit jobs to a Globus Toolkit 4 host. Thus, the users at DLR PA do not have to learn a new set of tools for submitting and monitoring jobs. They can use the familiar Condor tools also applicable in their own local environment. Additionally, Condor provides some convenience features which are absent in the Globus Toolkit:

  • the possibility for each user to manage their jobs in a queue
  • automatic renewal of credentials (proxy certificate) for long-running jobs
  • email notification upon job completion

While Globus Toolkit is mainly a Grid middleware for developers wishing to build Grid applications, Condor provides command-line tools suitable for end users.

Condor-GT4 Integration

Follow the link(s) according to your role: