View source for LSF

Platform Load Sharing Facility (or simply LSF) is a job scheduler and workload manager developed by IBM. It is similar to [[Slurm]].

== Usage ==
=== Tasks ===
{| class="wikitable"
! Command
! Description
|-
| {{code|bjobs -u all -a}}
| Shows all jobs of all users
|-
| {{code|bjobs 101 102}}
| Shows jobs with job_ID 101 and 102
|-
| {{code|bsub}}
| Submits a batch job
|-
| {{code|bhosts}}
| Shows all host status
|-
| {{code|badmin}}
| LSF administration shell
|}

=== Admin ===
Use the {{code|lsadmin}} and {{code|badmin}} commands to do most admin related things.

==== Startup ====
Use {{code|lsfstartup}}.

A Load Information Manager ('''LIM''') daemon needs to be running on each server host. This daemon collects host load and configuration information and forwards it to the master LIM service on the master host.

A Remote Execution Server ('''RES''') daemon needs to be running on each server host in order for them to accept remote execution requests.

To start LIM and RES on all hosts, run:
{{highlight|lang=terminal|code=
# lsadmin limstartup all
# lsadmin resstartup all
}}

{{code|sbatchd}} needs to be running on all hosts as well and can be started by running
{{highlight|lang=terminal|code=
# badmin hstartup all
}}

==== Restart ====
Use {{code|lsfrestart}}.

If some hosts are in a 'closed' state, you may need to restart LIM and RES on the host.

Fix 'closed' status by running on crick:
{{highlight|lang=terminal|code=
## restart the 'lsf' service and also lim and res.
# lsadmin resrestart all
# lsadmin limrestart all
}}

==== Shutdown ====
Use {{code|lsfshutdown}} to prevent users from submitting jobs.

To fully shut down, you must turn off {{code|sbatchd}}, LIM, and RES.
{{highlight|lang=terminal|code=
# badmin  hshutdown all
# lsadmin resshutdown all
# lsadmin limshutdown all
}}


== Tasks ==
=== Extend Job Run Time ===
If a job was created with too little wall time (eg. {{code|bsub -W}}), you will see the following output when listing jobs using {{code|bjobs -WL, -WF, or -WP}}:

{{highlight|lang=terminal|code=
# bjobs -u all -WL
JOBID   USER    STAT  QUEUE      FROM_HOST   EXEC_HOST   JOB_NAME   SUBMIT_TIME   TIME_LEFT
916742  user001 RUN   interactiv compute001  node001     bash       Jan  5 22:58     -       
933794  user001 RUN   normal     compute001  63*node001  *_de_novo. Jan 13 21:28  59:47 X    
934759  user001 RUN   normal     compute001  56*node001  *test_pasa Jan 17 16:21     -       
936079  user001 RUN   normal     compute001  56*node001  *e_guided. Jan 21 07:03  237:22 E   
}}

The {{code|TIME_LEFT}} column shows the time left in hours and minutes. The state is one of:
* {{code|E}}: The job has an estimated run time that has not been exceeded.
* {{code|L}}: The job has a hard run time limit specified but either has no estimated run time or the estimated run time is more than the hard run time limit.
* {{code|X}}: The job has exceeded its estimated run time and the time displayed is the time remaining until the job reaches its hard run time limit.
* {{code|-}}: A dash indicates that the job has no estimated run time and no run limit, or that it has exceeded its run time but does not have a hard limit and therefore runs until completion.

A job's run time limit can be adjusted using {{code|bmod -W HH:MM Job_ID}} or removed using {{code|bmod -Wn Job_ID}} .

== See Also ==
* [https://www.ibm.com/support/knowledgecenter/en/SSETD4_9.1.2/lsf_command_ref/bjobs.1.html IBM Documentation: bjobs reference]
* [https://www.ibm.com/support/knowledgecenter/en/SSETD4_9.1.2/lsf_command_ref/lsadmin.8.html IBM Documentation: lsadmin reference]
* https://bmi.cchmc.org/resources/software/lsf-examples