<PREV> <INDEX> <NEXT>

Schedulers

Evolution:

• Sun Grid Engine (SGE)
• OpenPBS/Torque
• SLURM
• Mesos/Zookeeper/Marathon

SLURM

Advantages:

• popular (de facto standard)
• integrates with MPI through srun

Disadvantages:

• really basic resource management unless doing advanced accounting
• accounting introduces high maintenance overhead
• not good with containers
  

Structure:  

• slurmctld: Handles submitted jobs, queues, and scheduling (one process per cluster)
  
• slurmdbd: Handles accounting and resource management (one per cluster)
  
• slurmd: Handles local jobs and monitoring (one on every node)
• munge: Handles authentication of the nodes in the cluster (one on every node)
  

Job submission:
sbatch (options) -> slurmd -> slurmctld
             (partition -> accounting -> resource allocation) -> code runs -> acks sent in reverse

Configuration files: /etc/slurm-llnl/

Options inside job file (sbatch script): --> man sbatch
    #SBATCH -flag value

• -n number_of_tasks(processes)
• -N min_nodes-max_nodes
  
• -m distribution_method
  
• -J job_name
• -o stdout->file
• -p partition
  
• --priority=priority
• -d dependencies

Utilities to interact with SLURM:

sbatch	submit jobs
sinfo	info about cluster load
squeue	info about partitions/queues
scontrol	manage policies and configuration
srun	run jobs (MPI aware)
scancel	stop and remove a job (queued or running)
salloc	allocate resources for a job (before srun)
sacctmgr	view/configure accounting

• Kubernetes
• Docker -- singularity
• Snaps
  

• Zookeeper -> defines a cluster of computers and facilitates communication
  
• Mesos -> distributed kernel for job/container scheduling, scalability
  
• Marathon -> orchestrates containers (load balancing, OS resources), web UI, fault tolerance
  

• Hadoop Common: utilities that support the other Hadoop module
• Hadoop Distributed File System (HDFS): distributed filesystem
  
• YARN: framework for job scheduling and cluster resource management
• MapReduce: YARN-based system for parallel processing of large data sets <key, value>