LAMMPS Process Mapping in OpenMPI - (2) --map-by numa:PE

PE=n의 효과

 

PE=n을 적용하지 않으면, NUMA 노드 범위 내에서 프로세스는 떠 있는다.

[dell7875:527316] Rank 1 bound to package[0][core:0-7,32-39,64-71]
[dell7875:527316] Rank 0 bound to package[0][core:0-7,32-39,64-71]
[dell7875:527316] Rank 2 bound to package[0][core:0-7,32-39,64-71]
[dell7875:527316] Rank 4 bound to package[0][core:0-7,32-39,64-71]
[dell7875:527316] Rank 3 bound to package[0][core:0-7,32-39,64-71]
[dell7875:527316] Rank 6 bound to package[0][core:0-7,32-39,64-71]
[dell7875:527316] Rank 5 bound to package[0][core:0-7,32-39,64-71]
[dell7875:527316] Rank 7 bound to package[0][core:0-7,32-39,64-71]

[dell7875:527316] Rank 8 bound to package[0][core:8-15,40-47,72-79]
[dell7875:527316] Rank 10 bound to package[0][core:8-15,40-47,72-79]
[dell7875:527316] Rank 11 bound to package[0][core:8-15,40-47,72-79]
[dell7875:527316] Rank 9 bound to package[0][core:8-15,40-47,72-79]
[dell7875:527316] Rank 12 bound to package[0][core:8-15,40-47,72-79]
[dell7875:527316] Rank 13 bound to package[0][core:8-15,40-47,72-79]
[dell7875:527316] Rank 14 bound to package[0][core:8-15,40-47,72-79]
[dell7875:527316] Rank 15 bound to package[0][core:8-15,40-47,72-79]

[dell7875:527316] Rank 16 bound to package[0][core:16-23,48-55,80-87]
[dell7875:527316] Rank 17 bound to package[0][core:16-23,48-55,80-87]
[dell7875:527316] Rank 18 bound to package[0][core:16-23,48-55,80-87]
[dell7875:527316] Rank 19 bound to package[0][core:16-23,48-55,80-87]
[dell7875:527316] Rank 20 bound to package[0][core:16-23,48-55,80-87]
[dell7875:527316] Rank 21 bound to package[0][core:16-23,48-55,80-87]
[dell7875:527316] Rank 22 bound to package[0][core:16-23,48-55,80-87]
[dell7875:527316] Rank 23 bound to package[0][core:16-23,48-55,80-87]

[dell7875:527316] Rank 24 bound to package[0][core:24-31,56-63,88-95]
[dell7875:527316] Rank 25 bound to package[0][core:24-31,56-63,88-95]
[dell7875:527316] Rank 26 bound to package[0][core:24-31,56-63,88-95]
[dell7875:527316] Rank 27 bound to package[0][core:24-31,56-63,88-95]
[dell7875:527316] Rank 28 bound to package[0][core:24-31,56-63,88-95]
[dell7875:527316] Rank 29 bound to package[0][core:24-31,56-63,88-95]
[dell7875:527316] Rank 30 bound to package[0][core:24-31,56-63,88-95]
[dell7875:527316] Rank 31 bound to package[0][core:24-31,56-63,88-95]

 

PE를 지정하면, 프로세스 당 코어가 고정됨

mpirun -np 32 --map-by numa:PE=1 --report-bindings lmp

[dell7875:528509] Rank 2 bound to package[0][core:2]
[dell7875:528509] Rank 0 bound to package[0][core:0]
[dell7875:528509] Rank 3 bound to package[0][core:3]
[dell7875:528509] Rank 1 bound to package[0][core:1]
[dell7875:528509] Rank 6 bound to package[0][core:6]
[dell7875:528509] Rank 4 bound to package[0][core:4]
[dell7875:528509] Rank 7 bound to package[0][core:7]
[dell7875:528509] Rank 5 bound to package[0][core:5]
[dell7875:528509] Rank 8 bound to package[0][core:8]
[dell7875:528509] Rank 9 bound to package[0][core:9]
[dell7875:528509] Rank 10 bound to package[0][core:10]
[dell7875:528509] Rank 11 bound to package[0][core:11]
[dell7875:528509] Rank 12 bound to package[0][core:12]
[dell7875:528509] Rank 13 bound to package[0][core:13]
[dell7875:528509] Rank 14 bound to package[0][core:14]
[dell7875:528509] Rank 15 bound to package[0][core:15]
[dell7875:528509] Rank 16 bound to package[0][core:16]
[dell7875:528509] Rank 17 bound to package[0][core:17]
[dell7875:528509] Rank 18 bound to package[0][core:18]
[dell7875:528509] Rank 19 bound to package[0][core:19]
[dell7875:528509] Rank 20 bound to package[0][core:20]
[dell7875:528509] Rank 21 bound to package[0][core:21]
[dell7875:528509] Rank 22 bound to package[0][core:22]
[dell7875:528509] Rank 23 bound to package[0][core:23]
[dell7875:528509] Rank 24 bound to package[0][core:24]
[dell7875:528509] Rank 25 bound to package[0][core:25]
[dell7875:528509] Rank 26 bound to package[0][core:26]
[dell7875:528509] Rank 27 bound to package[0][core:27]
[dell7875:528509] Rank 28 bound to package[0][core:28]
[dell7875:528509] Rank 29 bound to package[0][core:29]
[dell7875:528509] Rank 30 bound to package[0][core:30]
[dell7875:528509] Rank 31 bound to package[0][core:31]

 

예를 들어 PE=2

[dell7875:528584] Rank 0 bound to package[0][core:0-1]
[dell7875:528584] Rank 2 bound to package[0][core:4-5]
[dell7875:528584] Rank 1 bound to package[0][core:2-3]
[dell7875:528584] Rank 3 bound to package[0][core:6-7]
[dell7875:528584] Rank 6 bound to package[0][core:36-37]
[dell7875:528584] Rank 5 bound to package[0][core:34-35]
[dell7875:528584] Rank 4 bound to package[0][core:32-33]
[dell7875:528584] Rank 7 bound to package[0][core:38-39]
[dell7875:528584] Rank 9 bound to package[0][core:10-11]
[dell7875:528584] Rank 8 bound to package[0][core:8-9]
[dell7875:528584] Rank 10 bound to package[0][core:12-13]
[dell7875:528584] Rank 11 bound to package[0][core:14-15]
[dell7875:528584] Rank 12 bound to package[0][core:40-41]
[dell7875:528584] Rank 13 bound to package[0][core:42-43]
[dell7875:528584] Rank 14 bound to package[0][core:44-45]
[dell7875:528584] Rank 15 bound to package[0][core:46-47]
[dell7875:528584] Rank 16 bound to package[0][core:16-17]
[dell7875:528584] Rank 17 bound to package[0][core:18-19]
[dell7875:528584] Rank 18 bound to package[0][core:20-21]
[dell7875:528584] Rank 19 bound to package[0][core:22-23]
[dell7875:528584] Rank 20 bound to package[0][core:48-49]
[dell7875:528584] Rank 21 bound to package[0][core:50-51]
[dell7875:528584] Rank 22 bound to package[0][core:52-53]
[dell7875:528584] Rank 23 bound to package[0][core:54-55]
[dell7875:528584] Rank 24 bound to package[0][core:24-25]
[dell7875:528584] Rank 25 bound to package[0][core:26-27]
[dell7875:528584] Rank 26 bound to package[0][core:28-29]
[dell7875:528584] Rank 27 bound to package[0][core:30-31]
[dell7875:528584] Rank 28 bound to package[0][core:56-57]
[dell7875:528584] Rank 29 bound to package[0][core:58-59]
[dell7875:528584] Rank 30 bound to package[0][core:60-61]
[dell7875:528584] Rank 31 bound to package[0][core:62-63]

 

7995WX Pro의 경우 map by numa는 L3 Cache와 동일한 효과를 가진다.