Parallel and Distributed Computing
Introduced| English | Meaning |
|---|---|
| sequential computing/siːˈkwenʃl kəmˈpjuːtɪŋ/ | sequential computing |
| Parallel computing/ˈpærəlel kəmˈpjuːtɪŋ/ | Dividing suitable work into parts that run at the same time on multiple processors. |
| processors/ˈprəʊsesəz/ | The component that executes instructions and performs the operations of a computer or similar system. |
| Distributed computing/ˈdɪstrɪbjuːtɪd kəmˈpjuːtɪŋ/ | Sharing computation across multiple connected computers that communicate and coordinate their work. |
| speedup/ˈspiːdʌp/ | speedup |
One step at a time
- Traditionally a computer runs steps one at a time.
- This is sequential computing 顺序计算: each step must finish before the next begins.
- It is simple, but it can be slow for big jobs.
- To go faster, we can do several steps at once.
In sequential computing:
Difficulty 1/5 · 100 coins + a small bonus on first completion today
One step at a time, in order.
Parallel and distributed
- Parallel computing 并行计算 uses several processors 处理器 in one machine to run parts of a task at the same time.
- Distributed computing 分布式计算 spreads a task across many separate computers, often in different places, connected by a network.
- Parallel = many processors, one machine. Distributed = many machines.
- Both aim to finish a large task faster.
Parallel computing: many processors work at the same time
Sequential, parallel, or distributed?
Sequential runs one step at a time; parallel uses several processors in one machine at once; distributed spreads a task across many separate computers.
Match each term to its meaning.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Parallel is one machine; distributed is many machines.
Speedup and its limit
- The benefit of parallel computing is measured by speedup 加速比:
- $\text{speedup} = \dfrac{\text{time using one processor}}{\text{time using several processors}}$.
- But any part that must run sequentially cannot be sped up by adding processors.
- So the sequential portion sets a floor — a hard cap on the total speedup.
A task takes 100 s on one processor and 40 s on several. What is the speedup?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
speedup = 100 / 40 = 2.5×.
If 20 s of a 100 s task must run sequentially, what is the maximum possible speedup?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Even with infinite processors, 20 s remains: 100 / 20 = 5×.
Why can adding more processors never remove the speedup cap?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
The sequential portion sets a floor on the total time.
Distributed computing is only useful for small jobs that fit on one machine.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
False. Distributed computing suits huge jobs too big for one machine, such as indexing the web, by splitting the work across many computers.
When distributed shines
- Distributed solutions shine on huge jobs — enormous data sets, or a search engine indexing the whole web.
- The work is too big for one machine and splits naturally across thousands.
Speedup with a cap. A 100-second task has 80 s parallel and 20 s sequential. On 4 processors the parallel part is $80 / 4 = 20$ s, plus 20 s sequential = 40 s. Speedup $= 100 / 40 = 2.5\times$. Even with a million processors, the 20 s sequential remains, so speedup can never beat $100 / 20 = 5$.
Sequential computing runs one step at a time. Parallel computing uses several processors in one machine at once; distributed computing spreads work across many machines. Parallel gain is measured as speedup (one-processor time ÷ many-processor time), but the sequential part caps it (here, 5×).