LAMMPS Process Mapping in OpenMPI - (3) --map-by ppr:<N>:<resource>

ppr : process per resource

resources : numa, socket, core..

 

OpenMPI에서 --report-bindings은 stderr로 보내진다. 따라서, stderr을 파일로 보내려면, 이렇게

2> bindings.log

 

리눅스 쉘에서 2는 stderr을 의미한다.

 

ppr 테스트, rank 8개씩 NUMA 노드로 분배

mpirun -np 32 --map-by ppr:8:numa --report-bindings lmp

[dell7875:528836] Rank 0 bound to package[0][core:0-7,32-39,64-71]
[dell7875:528836] Rank 2 bound to package[0][core:0-7,32-39,64-71]
[dell7875:528836] Rank 3 bound to package[0][core:0-7,32-39,64-71]
[dell7875:528836] Rank 1 bound to package[0][core:0-7,32-39,64-71]
[dell7875:528836] Rank 4 bound to package[0][core:0-7,32-39,64-71]
[dell7875:528836] Rank 6 bound to package[0][core:0-7,32-39,64-71]
[dell7875:528836] Rank 7 bound to package[0][core:0-7,32-39,64-71]
[dell7875:528836] Rank 5 bound to package[0][core:0-7,32-39,64-71]
[dell7875:528836] Rank 10 bound to package[0][core:8-15,40-47,72-79]
[dell7875:528836] Rank 9 bound to package[0][core:8-15,40-47,72-79]
[dell7875:528836] Rank 13 bound to package[0][core:8-15,40-47,72-79]
[dell7875:528836] Rank 14 bound to package[0][core:8-15,40-47,72-79]
[dell7875:528836] Rank 8 bound to package[0][core:8-15,40-47,72-79]
[dell7875:528836] Rank 11 bound to package[0][core:8-15,40-47,72-79]
[dell7875:528836] Rank 17 bound to package[0][core:16-23,48-55,80-87]
[dell7875:528836] Rank 18 bound to package[0][core:16-23,48-55,80-87]
[dell7875:528836] Rank 12 bound to package[0][core:8-15,40-47,72-79]
[dell7875:528836] Rank 15 bound to package[0][core:8-15,40-47,72-79]
[dell7875:528836] Rank 16 bound to package[0][core:16-23,48-55,80-87]
[dell7875:528836] Rank 21 bound to package[0][core:16-23,48-55,80-87]
[dell7875:528836] Rank 22 bound to package[0][core:16-23,48-55,80-87]
[dell7875:528836] Rank 19 bound to package[0][core:16-23,48-55,80-87]
[dell7875:528836] Rank 20 bound to package[0][core:16-23,48-55,80-87]
[dell7875:528836] Rank 23 bound to package[0][core:16-23,48-55,80-87]
[dell7875:528836] Rank 25 bound to package[0][core:24-31,56-63,88-95]
[dell7875:528836] Rank 24 bound to package[0][core:24-31,56-63,88-95]
[dell7875:528836] Rank 26 bound to package[0][core:24-31,56-63,88-95]
[dell7875:528836] Rank 28 bound to package[0][core:24-31,56-63,88-95]
[dell7875:528836] Rank 27 bound to package[0][core:24-31,56-63,88-95]
[dell7875:528836] Rank 29 bound to package[0][core:24-31,56-63,88-95]
[dell7875:528836] Rank 30 bound to package[0][core:24-31,56-63,88-95]
[dell7875:528836] Rank 31 bound to package[0][core:24-31,56-63,88-95]

 

짝이 정확히 안맞으면 에러

$ mpirun -np 32 --map-by ppr:4:numa --report-bindings lmp
$ cat bindings.log
Exception caught: rsmi_init.
--------------------------------------------------------------------------
Your job has requested more processes than the ppr for
this topology can support:

  App: lmp
  Number of procs:  32
  Procs mapped:  16
  Total number of procs:  16
  PPR: 4:numa

Please revise the conflict and try again.
--------------------------------------------------------------------------

 

mpirun -np 32 --map-by ppr:16:numa --report-bindings lmp 

[dell7875:529009] Rank 1 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 0 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 5 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 2 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 3 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 4 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 7 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 6 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 8 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 9 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 10 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 11 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 12 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 14 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 13 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 15 bound to package[0][core:0-7,32-39,64-71]
[dell7875:529009] Rank 16 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 17 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 18 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 19 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 20 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 21 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 22 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 23 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 24 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 25 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 26 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 27 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 28 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 29 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 30 bound to package[0][core:8-15,40-47,72-79]
[dell7875:529009] Rank 31 bound to package[0][core:8-15,40-47,72-79]