MPI의 이해

YADE에서는 Distribued memeory를 이용한 병렬 가속화를 지원한다. 명령어는 mpirun이다.

mpirun을 실행하면 신기한 현상이 하나 생기는데, body가 증가하는 것이다. 오잉?

 

예를 들어보자.

 

Box와 Sphere 생성모델

 

초기생성모델이다. 박스를 먼저, Sphere를 다음에, 그리고 Topcap을 생성하였다.

그렇다면 O.bodies로 body object가 어떻게 생성되었는지 확인해 보자.

 

In [1]: len(O.bodies)
Out[1]: 613

body는 총 613개가 생성되었다.

In [2]: O.bodies[0].dict()
Out[2]: 
{'id': 0,
 'groupMask': 1,
 'flags': 1,
 'subdomain': 0,
 'material': <FrictMat instance at 0x55b30223a610>,
 'state': <State instance at 0x55b300dfb800>,
 'shape': <Facet instance at 0x55b301cccc80>,
 'bound': None,
 'clumpId': -1,
 'chain': -1,
 'iterBorn': 0,
 'timeBorn': 0.0}

In [3]: O.bodies[9].dict()
Out[3]: 
{'id': 9,
 'groupMask': 1,
 'flags': 1,
 'subdomain': 0,
 'material': <FrictMat instance at 0x55b30223a610>,
 'state': <State instance at 0x55b301da6a90>,
 'shape': <Facet instance at 0x55b301da6870>,
 'bound': None,
 'clumpId': -1,
 'chain': -1,
 'iterBorn': 0,
 'timeBorn': 0.0}

In [4]: O.bodies[10].dict()
Out[4]: 
{'id': 10,
 'groupMask': 1,
 'flags': 1,
 'subdomain': 0,
 'material': <FrictMat instance at 0x55b30223a610>,
 'state': <State instance at 0x55b301da7570>,
 'shape': <Sphere instance at 0x55b302239f80>,
 'bound': None,
 'clumpId': -1,
 'chain': -1,
 'iterBorn': 0,
 'timeBorn': 0.0}

body id 0부터 9까지 10개의 facet이 생성된 것을 알 수 있다. (뚜껑이 열린 box 이므로) body id 10번 부터는 sphere이다.

마지막 body를 확인해 보면,

In [5]: O.bodies[-1].dict()
Out[5]: 
{'id': 612,
 'groupMask': 1,
 'flags': 1,
 'subdomain': 0,
 'material': <FrictMat instance at 0x55b30223a610>,
 'state': <State instance at 0x55b302370440>,
 'shape': <Facet instance at 0x55b3023b50f0>,
 'bound': None,
 'clumpId': -1,
 'chain': -1,
 'iterBorn': 0,
 'timeBorn': 0.0}

In [6]: O.bodies[613].dict()
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
/usr/bin/yade in <module>
----> 1 O.bodies[613].dict()

IndexError: Body id out of range.

In [7]: O.bodies[612].dict()
Out[7]: 
{'id': 612,
 'groupMask': 1,
 'flags': 1,
 'subdomain': 0,
 'material': <FrictMat instance at 0x55b30223a610>,
 'state': <State instance at 0x55b302370440>,
 'shape': <Facet instance at 0x55b3023b50f0>,
 'bound': None,
 'clumpId': -1,
 'chain': -1,
 'iterBorn': 0,
 'timeBorn': 0.0}

In [8]: O.bodies[611].dict()
Out[8]: 
{'id': 611,
 'groupMask': 1,
 'flags': 1,
 'subdomain': 0,
 'material': <FrictMat instance at 0x55b30223a610>,
 'state': <State instance at 0x55b30236f260>,
 'shape': <Facet instance at 0x55b3023b4ec0>,
 'bound': None,
 'clumpId': -1,
 'chain': -1,
 'iterBorn': 0,
 'timeBorn': 0.0}

In [9]: O.bodies[610].dict()
Out[9]: 
{'id': 610,
 'groupMask': 1,
 'flags': 1,
 'subdomain': 0,
 'material': <FrictMat instance at 0x55b30223a610>,
 'state': <State instance at 0x55b30233dc60>,
 'shape': <Sphere instance at 0x55b30233db70>,
 'bound': None,
 'clumpId': -1,
 'chain': -1,
 'iterBorn': 0,
 'timeBorn': 0.0}

마지막 두개가 facet, 뚜껑이다.

 

그럼 이어서 mpirun을 실행 후 어떻게 변화되는지 확인해 보자. 아래는 mpirun을 위한 명령어 셋이다.

mp.ERASE_REMOTE_MASTER = False 
mp.DOMAIN_DECOMPOSITION= True
mp.mpirun(nSteps=10000, np = 5, withMerge=True)

실행결과 Domain이 나뉘었다.

 

body의 개수를 확인해 보자. 그런데 오잉?

Number of bodies =  597
Master: will spawn  4  workers running: /usr/bin/yade ['230412.py'] 
<WARNING> SpherePack:336 long int yade::SpherePack::makeCloud(yade::Vector3r, yade::Vector3r, yade::Real, yade::Real, int, bool, yade::Real, const std::vector<double>&, const std::vector<double>&, bool, int, yade::Matrix3r): Exceeded 1000 tries to insert non-overlapping sphere to packing. Only 577 spheres were added, although you requested 5000.
<WARNING> SpherePack:336 long int yade::SpherePack::makeCloud(yade::Vector3r, yade::Vector3r, yade::Real, yade::Real, int, bool, yade::Real, const std::vector<double>&, const std::vector<double>&, bool, int, yade::Matrix3r): Exceeded 1000 tries to insert non-overlapping sphere to packing. Only 582 spheres were added, although you requested 5000.
<WARNING> SpherePack:336 long int yade::SpherePack::makeCloud(yade::Vector3r, yade::Vector3r, yade::Real, yade::Real, int, bool, yade::Real, const std::vector<double>&, const std::vector<double>&, bool, int, yade::Matrix3r): Exceeded 1000 tries to insert non-overlapping sphere to packing. Only 603 spheres were added, although you requested 5000.
<WARNING> SpherePack:336 long int yade::SpherePack::makeCloud(yade::Vector3r, yade::Vector3r, yade::Real, yade::Real, int, bool, yade::Real, const std::vector<double>&, const std::vector<double>&, bool, int, yade::Matrix3r): Exceeded 1000 tries to insert non-overlapping sphere to packing. Only 612 spheres were added, although you requested 5000.

 ak
Number of bodies =  589

 ak
Number of bodies =  615

 ak

 ak
Number of bodies =  594
Number of bodies =  624

body의 개수가 제각각이다? number of thread 만큼 subdomain instancec가 추가되어 일어나는 일이다.