EGEE Home | Technical Pages Home | Search | EDMS Documents | People | Calendar | Agenda maker | Glossary

JRA1: Workload Management

Tools Testing Integration Information
Services
Workload
Management
Data
Management
Security Management

JRA1 Home | Workload Management Home | Mandate | People | Meetings | Presentations | Savannah Portal| Useful links | Actions list | EDMS



WMProxy Client Quick start guide


WMProxy Client Quick Start:
===========================

The WMProxy client commands options are basically the same as the one provided by the WMS-UI 
python commands (glite-job-*). In the following we provide a brief description (including 
examples) for each WMProxy client command mostly highlighting the differencies and novelties 
with respect to the WMS-UI commands.

The endpoint URL of the WMProxy service to contact is read from the configuration by all commands not 
taking a jobId as input. The value in the configuration can be overridden using the --endpoint option.
The commands taking a jobId as input get the WMProxy endpoint URL from the LB server.



*** glite-wms-job-delegate-proxy ***

  This commands allows delegating a user proxy to the WMProxy service. The user can either specify
  the delegationId to be associated with the delegated proxy by using the --delegationid option:

  [trinity] /home/fpacini > glite-wms-job-delegate-proxy -d myfirstdlegationID

  or make the command automaticallt generate an ID by using the --autm-delegation option:

  [trinity] /home/fpacini > glite-wms-job-delegate-proxy -a

  The delegationId is needed for performing job submission later. In particular glite-wms-job-submit and
  glite-wms-job-list-match are the only commands needing proxy delegation.


*** glite-wms-job-submit ***  

  This commands allows submitting a job, DAG or collection to the WMProxy. The request to be submitted
  has to be described by a JDL file as specified by document https://edms.cern.ch/document/590869/1.
  Submission can be performed either by using a previously delegated proxy (advised option), e.g.:

  [trinity] /home/fpacini > glite-wms-job-delegate-proxy -d myfirstdlegationID

  or delegating a new proxy at each new submission, e.g.:

  [trinity] /home/fpacini > glite-wms-job-submit --autm-delegation -o myids JDL/mydag.jdl

  To submit complex requests such as DAGs and Collections there is no change in the command syntax; it 
  suffices passing a JDL file describing such requests to the job as shown above.

  For job collections there is a further way to perform submission. The --collection option allows indeed to
  specify the path to a directory containing all the single JDL files of the jobs you want the collection is 
  made of. The glite-wms-job-submit will generate the corresponding collection JDL and submit it. E.g.:

  [trinity] /home/fpacini > glite-wms-job-submit -d myfirstdlegationID -o myids --collection /home/fpacini/KOLL/

  where /home/fpacini/KOLL/ is the directory containing all the JDL files.

  It is important to note that files that are shared from the sandboxes of different jobs will be transferred 
  only once to the WMS node.

  In all cases, upon success, the command returns an ID to the user that can be used as the handle to follow 
  up the progress of the request itself.


*** glite-wms-job-list-match ***

  This command provides the list of CEs matching the requirements of the job specified in the JDL. If the 
  --rank option is specified it also prints the rank corresponding to each listed CE. E.g.:

  [trinity] /home/fpacini > glite-wms-job-list-match -d myfirstdlegationID --rank JDL/tt.jdl 

    Connecting to the service https://tigerman.cnaf.infn.it:7443/glite_wms_wmproxy_server

    ==========================================================================
                         COMPUTING ELEMENT IDs LIST 
    The following CE(s) matching your job requirements have been found:

           *CEId*                                    *Rank*
    - atlfarm006.mi.infn.it:2119/blah-pbs-infinite      0
    - atlfarm006.mi.infn.it:2119/blah-pbs-long          0
    - atlfarm006.mi.infn.it:2119/blah-pbs-short         0
    - lxb2077.cern.ch:2119/blah-pbs-short               0
    - lxde01.pd.infn.it:2119/blah-lsf-grid01            0
    - lxde01.pd.infn.it:2119/blah-lsf-grid03            0
    - lxde01.pd.infn.it:2119/blah-lsf-grid04            0
    - lxb2077.cern.ch:2119/blah-pbs-infinite            -624999
    - lxb2077.cern.ch:2119/blah-pbs-long                -624999
    ==========================================================================

  This command does not support complex requests such as DAGs and collections.



*** glite-wms-job-cancel ***

  This command allows cancelling a previously submitted request. It takes as input a request ID, e.g.:

  [trinity] /home/fpacini > glite-wms-job-cancel  https://tigerman.cnaf.infn.it:9000/K4Tsz4jkqiAqZ1fcWU-pBQ
  
  or a list of requests IDs either on the command line or from an input file, e.g.:

  [trinity] /home/fpacini > glite-wms-job-cancel -i myids

  It is important to note that nodes of a DAG cannot be cancelled singularly as this could break dependencies.



*** glite-wms-job-output ***

  This command performs the retrieval of the output sandbox files of a previously submitted request. Of course 
  file retrieval is only allowed when the job has finished its execution (either successing or failing).
  For complex requests the command retrieves recursively all the output sandboxes of the request sub-jobs.

  The user can specify the location on the local file system where the fiules have to be stored using the --dir 
  option, e.g.:

  [trinity] /home/fpacini > glite-wms-job-output --dir /home/fpacini/myout https://tigerman.cnaf.infn.it:9000/6WBIXoTt1kk1SSHJc5hmcw

  If only a job ID is specified the retrieved files are stored in the specified dir; if more than one job is 
  given as input to the command, e.g.:

  [trinity] /home/fpacini > glite-wms-job-output --dir /home/fpacini/myout -i myids

  then a subdir (named <user name>_<jobId unique string> )is created for each job in /home/fpacini/myout for containing
  the files for the given job. The same applies if the specified handle is the ID of a complex request.

  The --list-only option allows listing the file of the output sandbox of a job without retrieving them.



 Commands Sequence:
 ==================

 We report here an example of the sequence of steps that have to be performed to submit a job and to monitor 
 its progress.

 - Create a valid VOMS proxy as follows:

  [trinity] /home/fpacini > voms-proxy-init -voms EGEE
  Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=DATAMAT DSAGRD/CN=Fabrizio Pacini
  Enter GRID pass phrase:
  Creating temporary proxy .................................... Done
  Contacting  kuiken.nikhef.nl:15001 [/O=dutchgrid/O=hosts/OU=nikhef.nl/CN=kuiken.nikhef.nl] "EGEE" Done
  Creating proxy ........................................ Done
  Your proxy is valid until Tue Jul 12 07:42:35 2005  


- Delegate your proxy to the WMProxy service:

  [trinity] /home/fpacini > glite-wms-job-delegate-proxy -d delID1234

  Delegating Credential to the service https://tigerman.cnaf.infn.it:7443/glite_wms_wmproxy_server

  ================== glite-wms-job-delegate-proxy Success ==================

  Your proxy has been successfully delegated to the WMProxy:
  https://tigerman.cnaf.infn.it:7443/glite_wms_wmproxy_server

  with the delegation identifier: delID1234

  ========================================================================== 


- Submit a job to the WMProxy associating the previously delegated proxy to it:

  [trinity] /home/fpacini > glite-wms-job-submit -d delID1234 -o ids JDL/tt.jdl 

  Connecting to the service https://tigerman.cnaf.infn.it:7443/glite_wms_wmproxy_server

  ====================== glite-wms-job-submit Success ======================

  The job has been successfully submitted to the WMProxy
  Your job identifier is:

  https://tigerman.cnaf.infn.it:9000/NEs1GSI8uYDE33SaobBIyA

  The job identifier has been saved in the following file:
  /home/fpacini/ids

  ==========================================================================
   

- Submit a collection to the WMProxy associating the same delegated proxy to it:

  [trinity] /home/fpacini > glite-wms-job-submit -d delID1234 -o ids --collection /home/fpacini/JDLS

  Connecting to the service https://tigerman.cnaf.infn.it:7443/glite_wms_wmproxy_server

  ====================== glite-wms-job-submit Success ======================

  The job has been successfully submitted to the WMProxy
  Your job identifier is:

  https://tigerman.cnaf.infn.it:9000/stshjZvp-yPdn7jGVX64Ag

  The job identifier has been saved in the following file:
  /home/fpacini/ids

  ==========================================================================


- Monitor the status of the job (using the legacy glite-job-status command):

  [trinity] /home/fpacini > glite-job-status -i ids 

  ------------------------------------------------------------------
  1 : https://tigerman.cnaf.infn.it:9000/NEs1GSI8uYDE33SaobBIyA
  2 : https://tigerman.cnaf.infn.it:9000/stshjZvp-yPdn7jGVX64Ag
  a : all
  q : quit
  ------------------------------------------------------------------

  Choose one or more jobId(s) in the list - [1-2]all:1


  *************************************************************
  BOOKKEEPING INFORMATION:

  Status info for the Job : https://tigerman.cnaf.infn.it:9000/NEs1GSI8uYDE33SaobBIyA
  Current Status:     Ready 
  Status Reason:      unavailable
  Destination:        lxde01.pd.infn.it:2119/blah-lsf-grid03
  Submitted:          Mon Jul 11 19:48:38 2005 CEST
  *************************************************************


- Retrieve the output for the collection:

  [trinity] /home/fpacini > glite-wms-job-output --dir /home/fpacini/coll_output -i ids 
  ------------------------------------------------------------------
  1 : https://tigerman.cnaf.infn.it:9000/NEs1GSI8uYDE33SaobBIyA
  2 : https://tigerman.cnaf.infn.it:9000/stshjZvp-yPdn7jGVX64Ag
  a : all
  q : quit
  ------------------------------------------------------------------

  Choose one or more jobId(s) in the list - [1-2]all (use , as separator or - for a range): 2

  ================================================================================

                        JOB GET OUTPUT OUTCOME
  Output files for the job:
  https://tigerman.cnaf.infn.it:9000/stshjZvp-yPdn7jGVX64Ag
  are stored in the directory: /home/fpacini/coll_output

  ================================================================================

  [trinity] /home/fpacini > ll /home/fpacini/coll_output
  total 12
  drwxr-xr-x    2 fpacini  grid         4096 Jul 11 20:03 fpacini_FxBa61Ws4g4Z0wa3k28ckA
  drwxr-xr-x    2 fpacini  grid         4096 Jul 11 20:02 fpacini_vWDkzyzNOU6GG_ENPk-QEQ
  drwxr-xr-x    2 fpacini  grid         4096 Jul 11 20:03 fpacini_ZKpwNUXXyrbbUjDZd5B0dQ


- Check the status of the Collection and its nodes:

  [trinity] /home/fpacini > glite-job-status -v 0 -i ids

  ------------------------------------------------------------------
  1 : https://tigerman.cnaf.infn.it:9000/NEs1GSI8uYDE33SaobBIyA
  2 : https://tigerman.cnaf.infn.it:9000/stshjZvp-yPdn7jGVX64Ag
  a : all
  q : quit
  ------------------------------------------------------------------

  Choose one or more jobId(s) in the list - [1-4]all:2

  *************************************************************
  BOOKKEEPING INFORMATION:

  Status info for the Job : https://tigerman.cnaf.infn.it:9000/stshjZvp-yPdn7jGVX64Ag
  Current Status:     Done (Success)

  - Nodes information for: 
      Status info for the Job : https://tigerman.cnaf.infn.it:9000/vWDkzyzNOU6GG_ENPk-QEQ
      Current Status:     Done (Success)
      Status info for the Job : https://tigerman.cnaf.infn.it:9000/FxBa61Ws4g4Z0wa3k28ckA
      Current Status:     Done (Success)
      Status info for the Job : https://tigerman.cnaf.infn.it:9000/ZKpwNUXXyrbbUjDZd5B0dQ
      Current Status:     Done (Success)
  *************************************************************


       Disclaimer Contact   Last Modified: Tuesday, 12-Jul-2005 18:11:36 CEST