What is a chunk in layman's term

I can’t wrap my mind about the concept of a chunk in PBS. Quoting from the users guide :

A chunk is a set of resources that are allocated as a unit to a job. All parts of a chunk come from the same host/node. A chunk -level resource or a host-level resource is a resource available at host level.

So in practical terms what does it mean when I select=2 chunks and send a job with qsub ?

thanks !

If the user submits a job without requesting any resources, then the default resources that is requested for that job can be called a chunk. However, if the user is given a chance to submit jobs on one compute node, he can submit jobs , For example user has 1 compute node with 16 cores and 16GB RAM

  • User can submit 16 jobs requesting 1 core and 1 GB RAM, then you will have 16 jobs running on the system, any other jobs would be queued ( chunk = -l select=1:ncpus=1:mem=1gb)
  • User can submit 4 jobs each requesting 4 cores and 4GB RAM , then you will have 4 jobs running on the system, any other jobs would be queued ( chunk = -l select=1:ncpus=4:mem=4gb)
  • User can submit 2 jobs each requesting 8 cores and 8GB RAM and so on ( chunk = -l select=1:ncpus=8:mem=8gb)
  • If user requests 1 core and 16gb RAM, then you can run one job only, rest of the jobs would be queued ( chunk = -l select=1:ncpus=16:mem=16gb)

Chunk gives an opportunity to the end user of how they want to run the job, also you can use split chunks qsub -l select=1:ncpus=3:mem=10gb+2:ncpus=4:mem=3gb+3:ncpus=1:mem=15gb

You are requesting 2 chunks each with 1 cpus ( -l select=2:ncpus=1 )

hope this helps, also please check -l place=(free or scatter or pack or exc ) , it gives you more control on chunk placement.

1 Like