Skip to content

Commit a1531a7

Browse files
steven112163steven112163
committed
Add readme for hw4
1 parent b8bc484 commit a1531a7

File tree

1 file changed

+169
-0
lines changed

1 file changed

+169
-0
lines changed

‎HW4/README.md‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# Parallel Programming Assignment 4
2+
3+
4+
5+
###### tags: `平行程式作業`
6+
[TOC]
7+
8+
9+
10+
## Part 1
11+
### Q1
12+
1. **How do you control the number of MPI processes on each node?**
13+
1. Put `slot=#` after hostnames in the hostfile
14+
2. Run the program like `mpirun -H aa -np 1 program : -H bb,cc -np 2 program`. It will make host aa run the program with 1 process, and host bb and cc run the program with 2 processes respectively.
15+
3. Use `--map-by` to map each process to each node, slot, etc.
16+
2. **Which functions do you use for retrieving the rank of an MPI process and the total number of processes?**
17+
`int MPI_Comm_rank ( MPI_Comm comm, int *rank )` is used for retrieving the rank of an MPI process, and `int MPI_Comm_size ( MPI_Comm comm, int *size )` is used to retrieve the total number of processes.
18+
19+
### Q2
20+
1. **Why MPI_Send and MPI_Recv are called “blocking” communication?**
21+
They are called "blocking" is because they both wait until the data transfer finished. If the data size is small, MPI_Send blocks until transferring the data into the system buffer of receiving node is complete. If the data size is large, it has to block until the data transfer to the user's buffer is complete. MPI_Recv blocks until the data is copied into the user's buffer.
22+
2. **Measure the performance (execution time) of the code for 2, 4, 8, 12, 16 MPI processes and plot it.**
23+
The measurement is conducted 10 times.
24+
|# of MPI Processes|Performance (s)|
25+
|:-:|:-:|
26+
|2|6.742710|
27+
|4|3.544033|
28+
|8|1.730747|
29+
|12|1.172859|
30+
|16|0.923084|
31+
32+
![](https://i.imgur.com/K7z9OAF.png)
33+
34+
### Q3
35+
1. **Measure the performance (execution time) of the code for 2, 4, 8, 16 MPI processes and plot it.**
36+
The measurement is conducted 10 times.
37+
|# of MPI Processes|Performance (s)|
38+
|:-:|:-:|
39+
|2|6.777657|
40+
|4|3.422625|
41+
|8|1.803603|
42+
|16|0.941322|
43+
44+
![](https://i.imgur.com/Z8FbbFd.png)
45+
2. **How does the performance of binary tree reduction compare to the performance of linear reduction?**
46+
Based on the measurements, their performances are close.
47+
3. **Increasing the number of processes, which approach (linear/tree) is going to perform better? Why? Think about the number of messages and their costs.**
48+
The measurement is conducted 10 times.
49+
|# of MPI Processes|Performance of Linear Reduction (s)|Performance of Binary Tree Reduction (s)|
50+
|:-:|:-:|:-:|
51+
|2|6.742710|6.777657|
52+
|4|3.544033|3.422625|
53+
|8|1.730747|1.803603|
54+
|16|0.923084|0.941322|
55+
|32|0.524869|0.480484|
56+
57+
![](https://i.imgur.com/fvaxYoR.png)
58+
According to the result, their performances are still close. However, there is a slight difference during the summation process. In linear reduction, all processes send their results to process 0, which needs $process - 1$ data transfer times. In contrast, binary tree reduction needs $2 \times (process - 1)$ data transfer times. Whereas we can not see that the linear reduction performs significantly better than the binary tree reduction in the above graph. It might be because the number of processes is not large enough or the transferred data is not large enough.
59+
60+
### Q4
61+
1. **Measure the performance (execution time) of the code for 2, 4, 8, 12, 16 MPI processes and plot it.**
62+
The measurement is conducted 10 times.
63+
|# of MPI Processes|Performance (s)|
64+
|:-:|:-:|
65+
|2|6.799243|
66+
|4|3.768059|
67+
|8|1.723564|
68+
|12|1.169002|
69+
|16|1.002430|
70+
71+
![](https://i.imgur.com/Aqv7KXR.png)
72+
2. **What are the MPI functions for non-blocking communication?**
73+
- **Send**
74+
- **MPI_Isend**: standard-mode nonblocking send
75+
- **MPI_Issend**: nonblocking synchronous send
76+
- **MPI_Irsend**: ready-mode nonblocking send
77+
- **MPI_Ibsend**: nonblocking buffered send
78+
- **Receive**
79+
- **MPI_Irecv**: standard-mode nonblocking receive
80+
- **MPI_Imrecv**: non-blocking receive for a matched message
81+
3. **How the performance of non-blocking communication compares to the performance of blocking communication?**
82+
|# of MPI Processes|Blocking Performance (s)|Non-Blocking Performance (s)|
83+
|:-:|:-:|:-:|
84+
|2|6.742710|6.799243|
85+
|4|3.544033|3.768059|
86+
|8|1.730747|1.723564|
87+
|12|1.172859|1.169002|
88+
|16|0.923084|1.002430|
89+
90+
![](https://i.imgur.com/s9uGWtA.png)
91+
According to the graph above, their performances are close. It might be because there is no extra computation between receive and wait on the master. Thus, we can not see a considerable performance improvement.
92+
93+
### Q5
94+
1. **Measure the performance (execution time) of the code for 2, 4, 8, 12, 16 MPI processes and plot it.**
95+
The measurement is conducted 10 times.
96+
|# of MPI Processes|Performance (s)|
97+
|:-:|:-:|
98+
|2|6.667590|
99+
|4|3.554108|
100+
|8|1.784533|
101+
|12|1.370821|
102+
|16|1.002686|
103+
104+
![](https://i.imgur.com/y1U1ZqL.png)
105+
106+
### Q6
107+
1. **Measure the performance (execution time) of the code for 2, 4, 8, 12, 16 MPI processes and plot it.**
108+
The measurement is conducted 10 times.
109+
|# of MPI Processes|Performance (s)|
110+
|:-:|:-:|
111+
|2|6.761534|
112+
|4|3.475385|
113+
|8|1.778904|
114+
|12|1.164414|
115+
|16|0.971082|
116+
117+
![](https://i.imgur.com/mRQJ7Ww.png)
118+
119+
### Q7
120+
1. **Measure the performance (execution time) of the code for 2, 4, 8, 12, 16 MPI processes and plot it.**
121+
The measurement is conducted 10 times.
122+
|# of MPI Processes|Performance (s)|
123+
|:-:|:-:|
124+
|2|6.674785|
125+
|4|3.530522|
126+
|8|1.745863|
127+
|12|1.197773|
128+
|16|0.925097|
129+
130+
![](https://i.imgur.com/HxXTq48.png)
131+
2. **Which approach gives the best performance among the 1.2.1-1.2.6 cases? What is the reason for that?**
132+
One-sided communication is the best approach because the number of messages in its communication is the same as linear reduction. However, its sending and receiving is non-blocking. Therefore, workers can directly write their results into the buffer of the master.
133+
134+
### Q8
135+
1. **Plot ping-pong time in function of the message size for cases 1 and 2, respectively.**
136+
- **Case 1**
137+
![](https://i.imgur.com/mpTgWsk.png)
138+
- **Case 2**
139+
![](https://i.imgur.com/QPA2RX9.png)
140+
2. **Calculate the bandwidth and latency for cases 1 and 2, respectively.**
141+
- **Case 1**
142+
- $T(n) = 1.52104146 \times 10^{-10} n + 3.18012197 \times 10^{-5}$
143+
- $\Rightarrow \ bandwidth = \frac{1}{1.52104146 \times 10^{-10}} = 6.57444275056 \times 10^{9} = 6.574 \ GB/s$
144+
- $\Rightarrow \ latency = 3.18012197 \times 10^{-5} = 0.0318 \ ms$
145+
- **Case 2**
146+
- $T(n) = 8.59101630 \times 10^{-9} n + 6.15770777 \times 10^{-4}$
147+
- $\Rightarrow \ bandwidth = \frac{1}{8.59101630 \times 10^{-9}} = 0.116400663796 \times 10^{9} = 0.116 \ GB/s$
148+
- $\Rightarrow \ latency = 6.15770777 \times 10^{-4} = 0.6158 \ ms$
149+
150+
151+
152+
## Part 2
153+
### Q9
154+
1. **Describe what approach(es) were used in your MPI matrix multiplication for each data set.**
155+
![](https://i.imgur.com/1shonWT.png)
156+
The master process reads variables n, m, and l. Then, it uses *non-blocking send* to send them to the worker processes. Worker processes use *standard receive* to get the variables. Once these variables are received, a process can construct matrices a and b.
157+
![](https://i.imgur.com/brt3nrG.png)
158+
Every process needs to calculate the corresponding region of the matrix result to each worker process. Each region contains multiple rows.
159+
![](https://i.imgur.com/gdN7eBX.png)
160+
![](https://i.imgur.com/eSaV9KC.png)
161+
After calculating regions, the master process reads matrices a and b. Then, it uses *non-blocking send* to send the corresponding region of matrix a to each worker process. Next, worker processes use *standard receive* to retrieve their region and put it into the corresponding region of matrix a. While matrix b is transferred intact.
162+
![](https://i.imgur.com/WSdL1Zz.png)
163+
After transferring matrices a and b, the master process creates the matrix result and a window for retriving the products.
164+
![](https://i.imgur.com/vPJcWc2.png)
165+
Each worker process multiplies matrix a with matrix b and puts the product into matrix result. The multiplication is split into blocks to be cache-friendly. Also, the loop of middle_idx is not the innermost loop for the same reason.
166+
![](https://i.imgur.com/5PJsvG8.png)
167+
After multiplication, each worker process sends their corresponding region of matrix result back to the master process through the created window.
168+
![](https://i.imgur.com/A7GFzIz.png)
169+
Finally, the master process prints the matrix result after all worker processes have finished multiplication.

0 commit comments

Comments
 (0)