Message Passing Leads to Better Scalability in Parallel Systems - Revision history

Kevlin at 13:26, 17 August 2009

2009-08-17T13:26:19Z

←Older revision		Revision as of 13:26, 17 August 2009
Line 7:		Line 7:
	So can we eschew shared memory? Definitely.		So can we eschew shared memory? Definitely.

-	Instead of using threads and shared memory as our programming model, we can use processes and message passing. ''Process'' here just means a protected independent state with executing code, not necessarily an operating system process. Languages such as Erlang (and occam before it) have shown that processes are a very successful mechanism for programming concurrent and parallel systems. Such systems do not have all the synchronization stresses that shared memory, multi-threaded systems have. Moreover there is a formal model &~~ndash~~; Communicating Sequential Processes (CSP) &~~ndash~~; that can be applied as part of the engineering of such systems.	+	Instead of using threads and shared memory as our programming model, we can use processes and message passing. ''Process'' here just means a protected independent state with executing code, not necessarily an operating system process. Languages such as Erlang (and occam before it) have shown that processes are a very successful mechanism for programming concurrent and parallel systems. Such systems do not have all the synchronization stresses that shared memory, multi-threaded systems have. Moreover there is a formal model — Communicating Sequential Processes (CSP) — that can be applied as part of the engineering of such systems.

	We can go further and introduce dataflow systems as a way of computing. In a dataflow system there is no explicitly programmed control flow. Instead a directed graph of operators, connected by data paths, is set up and then data fed into the system. Evaluation is controlled by the readiness of data within the system. Definitely no synchronization problems.		We can go further and introduce dataflow systems as a way of computing. In a dataflow system there is no explicitly programmed control flow. Instead a directed graph of operators, connected by data paths, is set up and then data fed into the system. Evaluation is controlled by the readiness of data within the system. Definitely no synchronization problems.

Kevlin at 13:17, 17 August 2009

2009-08-17T13:17:52Z

←Older revision		Revision as of 13:17, 17 August 2009
Line 12:		Line 12:

	Having said all this, languages such as C, C++, Java, Python, and Groovy are the principal language of systems development and all of these are presented to programmers as languages for developing shared memory, multi-threaded systems. So what can be done? The answer is to use — or, if they don't exist, create — libraries and frameworks that provide process models and message passing, avoiding all use of shared mutable memory.		Having said all this, languages such as C, C++, Java, Python, and Groovy are the principal language of systems development and all of these are presented to programmers as languages for developing shared memory, multi-threaded systems. So what can be done? The answer is to use — or, if they don't exist, create — libraries and frameworks that provide process models and message passing, avoiding all use of shared mutable memory.
-
-	In the C and C++ (also Fortran) worlds, MPI (Message Passing Interface) is a popular tool. The problem it has is that it doesn't provide support for non-primitive data types, programmers have to work with primitive data types or provide a lot of extra infrastructure.
-
-	Various people have tried to apply MPI in Java but it has failed — and this is probably a good thing. Instead there is a growing and increasingly sophisticated infrastructure based on using special data structures to replace any access to shared memory: abstraction is used correctly to raise the level at which programmers work to develop concurrent and parallel systems. These data structures are tools for sending and receiving messages.
-
-	Python is going a different route due to implementation issues with the standard system. In Python parallel systems make use of multiple instances of the Python system passing messages.

	All in all, not programming with shared memory, but instead using message passing, is likely to be the most successful way of implementing systems that harness the parallelism that is now endemic in computer hardware. Bizarrely perhaps, although processes predate threads as a unit of concurrency, the future seems to be in using threads to implement processes.		All in all, not programming with shared memory, but instead using message passing, is likely to be the most successful way of implementing systems that harness the parallelism that is now endemic in computer hardware. Bizarrely perhaps, although processes predate threads as a unit of concurrency, the future seems to be in using threads to implement processes.

Kevlin at 13:16, 17 August 2009

2009-08-17T13:16:23Z

←Older revision		Revision as of 13:16, 17 August 2009
Line 1:		Line 1:
-	Programmers are taught from the very outset of their study of computing that concurrency, and especially ~~the~~ special subset of concurrency &~~ndash; parallelism &ndash~~; is hard, that only the very best can ever hope to get it right, and even they get it wrong. There is invariably great focus on threads, semaphores, monitors, and how hard it is to get concurrent access to variables to be thread-safe. True, there are many difficult problems, and they can be very hard to solve. But what is the core of the problem? Shared memory. Almost all the problems of concurrency that people go on and on about relate to the use of shared, mutable memory: race conditions, deadlock, livelock, etc. The answer seems obvious, either forgo concurrency, or eschew shared memory!	+	Programmers are taught from the very outset of their study of computing that concurrency — and especially parallelism, a special subset of concurrency — is hard, that only the very best can ever hope to get it right, and even they get it wrong. There is invariably great focus on threads, semaphores, monitors, and how hard it is to get concurrent access to variables to be thread-safe.

-	~~Forgoing concurrency~~ is ~~almost certainly not an option~~. ~~Computers have more and more cores on an almost quarterly basis, so harnessing parallelism (a special case~~ of concurrency~~) becomes more~~ and ~~more important. We can no longer rely~~ on ~~ever increasing processor clock speeds~~ to ~~improve application performance. Only by applying parallelism will~~ the ~~performance~~ of ~~applications improve. Obviously~~, ~~not improving performance is an option~~, ~~but it is unlikely to be acceptable to users~~.	+	True, there are many difficult problems, and they can be very hard to solve. But what is the root of the problem? Shared memory. Almost all the problems of concurrency that people go on and on about relate to the use of shared mutable memory: race conditions, deadlock, livelock, etc. The answer seems obvious: Either forgo concurrency or eschew shared memory!

-	So can ~~we eschew shared memory? Definitely~~.	+	Forgoing concurrency is almost certainly not an option. Computers have more and more cores on an almost quarterly basis, so harnessing true parallelism becomes more and more important. We can no longer rely on ever increasing processor clock speeds to improve application performance. Only by exploiting parallelism will the performance of applications improve. Obviously, not improving performance is an option, but it is unlikely to be acceptable to users.

-	~~Instead of using threads and shared memory as our programming model,~~ we can use processes and message passing. Process here just means a protected independent state with executing code, not necessarily an operating system process. Languages such as Erlang (and occam before it) have shown that processes are a very successful mechanism for programming concurrent and parallel systems. Such systems do not have all the synchronization stresses that shared memory~~, multi-threaded systems have. Moreover there is a formal model – Communicating Sequential Processes (CSP) – that can be applied as part of the engineering of such systems~~.	+	So can we eschew shared memory? Definitely.

-	~~We can go further~~ and ~~introduce dataflow systems~~ as ~~a way of computing~~. In a ~~dataflow~~ system ~~there is no explicitly programmed control flow~~. ~~Instead~~ a ~~directed graph of operators, connected by data paths, is set up~~ and ~~then data fed into~~ the ~~system~~. ~~Evaluation~~ is ~~controlled by the readiness~~ of ~~data within~~ the ~~system. Definitely no synchronization problems~~.	+	Instead of using threads and shared memory as our programming model, we can use processes and message passing. ''Process'' here just means a protected independent state with executing code, not necessarily an operating system process. Languages such as Erlang (and occam before it) have shown that processes are a very successful mechanism for programming concurrent and parallel systems. Such systems do not have all the synchronization stresses that shared memory, multi-threaded systems have. Moreover there is a formal model – Communicating Sequential Processes (CSP) – that can be applied as part of the engineering of such systems.

-	~~Having said all this, languages such~~ as C, ~~C++, Java, Python~~, and ~~Groovy are~~ the ~~principal language of systems development and all of these are presented to programmers as languages for developing shared memory, multi-threaded systems~~. ~~So what can be done?~~ ~~The answer~~ is ~~to use – or, if they don't exist, create – libraries and frameworks that provide processes and message passing and avoid all use~~ of ~~shared mutable memory~~.	+	We can go further and introduce dataflow systems as a way of computing. In a dataflow system there is no explicitly programmed control flow. Instead a directed graph of operators, connected by data paths, is set up and then data fed into the system. Evaluation is controlled by the readiness of data within the system. Definitely no synchronization problems.

-	~~In the~~ C ~~and~~ C++ ~~(also Fortran) worlds~~, ~~MPI (Message Passing Interface) is a popular tool~~. The ~~problem it~~	+	Having said all this, languages such as C, C++, Java, Python, and Groovy are the principal language of systems development and all of these are presented to programmers as languages for developing shared memory, multi-threaded systems. So what can be done? The answer is to use — or, if they don't exist, create — libraries and frameworks that provide process models and message passing, avoiding all use of shared mutable memory.
-	~~has~~ is ~~that it doesn~~'t ~~provide support for non-primitive data types~~, ~~programmers have to work with primitive~~	+
-	~~data types or~~ provide ~~a lot~~ of ~~extra infrastructure~~.	+

-	Various people have tried to apply MPI in Java but it has failed &~~ndash~~; and this is probably a good thing. Instead there is a growing and increasingly sophisticated infrastructure based on using special data structures to replace any access to shared memory: abstraction is used correctly to raise the level at which programmers work to develop concurrent and parallel systems. These data structures are tools for sending and receiving messages.	+	In the C and C++ (also Fortran) worlds, MPI (Message Passing Interface) is a popular tool. The problem it has is that it doesn't provide support for non-primitive data types, programmers have to work with primitive data types or provide a lot of extra infrastructure.
		+
		+	Various people have tried to apply MPI in Java but it has failed — and this is probably a good thing. Instead there is a growing and increasingly sophisticated infrastructure based on using special data structures to replace any access to shared memory: abstraction is used correctly to raise the level at which programmers work to develop concurrent and parallel systems. These data structures are tools for sending and receiving messages.

	Python is going a different route due to implementation issues with the standard system. In Python parallel systems make use of multiple instances of the Python system passing messages.		Python is going a different route due to implementation issues with the standard system. In Python parallel systems make use of multiple instances of the Python system passing messages.

-	All in all, not programming with shared memory, but instead using message passing, is likely to be the most successful way of implementing systems that harness the parallelism that is now endemic in computer hardware. Bizarrely perhaps, although processes predate threads as a unit of concurrency, the future seems to be in using threads to implement processes.	+	All in all, not programming with shared memory, but instead using message passing, is likely to be the most successful way of implementing systems that harness the parallelism that is now endemic in computer hardware. Bizarrely perhaps, although processes predate threads as a unit of concurrency, the future seems to be in using threads to implement processes.

	By [[Russel Winder]]		By [[Russel Winder]]

Russel at 10:51, 17 August 2009

2009-08-17T10:51:13Z

←Older revision		Revision as of 10:51, 17 August 2009
Line 1:		Line 1:
-	Programmers are taught from the very outset of their study of computing that concurrency and parallelism is hard, that only the very best can ever hope to get it right, and even they get it wrong. There is invariably great focus on threads, semaphores, monitors, and how hard it is to get concurrent access to variables to be thread-safe. True, there are many difficult problems, and they can be very hard to solve. But what is the core of the problem? Shared memory. Almost all the problems of concurrency ~~and parallelism~~ that people go on and on about relate to the use of shared, mutable memory. The answer seems obvious, either forgo concurrency ~~and parallelism~~, or eschew shared memory!	+	Programmers are taught from the very outset of their study of computing that concurrency, and especially the special subset of concurrency – parallelism – is hard, that only the very best can ever hope to get it right, and even they get it wrong. There is invariably great focus on threads, semaphores, monitors, and how hard it is to get concurrent access to variables to be thread-safe. True, there are many difficult problems, and they can be very hard to solve. But what is the core of the problem? Shared memory. Almost all the problems of concurrency that people go on and on about relate to the use of shared, mutable memory: race conditions, deadlock, livelock, etc. The answer seems obvious, either forgo concurrency, or eschew shared memory!

-	Forgoing concurrency ~~and parallelism~~ is almost certainly not an option. Computers have more and more cores on an almost quarterly basis, so harnessing parallelism becomes more and more important. We can no longer rely on ever increasing processor clock speeds to improve application performance. Only by applying parallelism will the performance of applications improve. Obviously, not improving performance is an option, but it is unlikely to be acceptable to users.	+	Forgoing concurrency is almost certainly not an option. Computers have more and more cores on an almost quarterly basis, so harnessing parallelism (a special case of concurrency) becomes more and more important. We can no longer rely on ever increasing processor clock speeds to improve application performance. Only by applying parallelism will the performance of applications improve. Obviously, not improving performance is an option, but it is unlikely to be acceptable to users.

	So can we eschew shared memory? Definitely.		So can we eschew shared memory? Definitely.

-	Instead of using threads and shared memory as our programming model, we can use processes and message passing. Languages such as Erlang (and occam before it) have shown that ~~concurrency and parallelism based on~~ processes is a very successful ~~way of~~ programming concurrent and parallel systems. Such systems do not have all the synchronization stresses that shared memory, multi-threaded systems have. Moreover there is a ~~mathematics~~ &~~mdash~~; Communicating Sequential Processes (CSP) &~~mdash~~; that can be applied as part of the engineering of ~~concurrent and parallel~~ systems.	+	Instead of using threads and shared memory as our programming model, we can use processes and message passing. Process here just means a protected independent state with executing code, not necessarily an operating system process. Languages such as Erlang (and occam before it) have shown that processes are a very successful mechanism for programming concurrent and parallel systems. Such systems do not have all the synchronization stresses that shared memory, multi-threaded systems have. Moreover there is a formal model – Communicating Sequential Processes (CSP) – that can be applied as part of the engineering of such systems.

-	We can go further and introduce dataflow systems as a way of computing. In a dataflow system there is no explicitly programmed control flow. Instead a directed graph of operators connected by data paths is set up and then data fed into the system. Evaluation is controlled by the readiness of data within the system. Definitely no synchronization problems.	+	We can go further and introduce dataflow systems as a way of computing. In a dataflow system there is no explicitly programmed control flow. Instead a directed graph of operators, connected by data paths, is set up and then data fed into the system. Evaluation is controlled by the readiness of data within the system. Definitely no synchronization problems.

-	Having said all this, languages such as C, C++, Java, Python, and Groovy are the principal language of systems development and	+	Having said all this, languages such as C, C++, Java, Python, and Groovy are the principal language of systems development and all of these are presented to programmers as languages for developing shared memory, multi-threaded systems. So what can be done? The answer is to use – or, if they don't exist, create – libraries and frameworks that provide processes and message passing and avoid all use of shared mutable memory.
-	all of these are presented to programmers as languages for developing shared memory, multi-threaded systems. So what can be done? The answer is to use &~~mdash~~; or, if they don't exist, create &~~mdash~~; libraries and frameworks that provide processes and message passing and avoid all use of shared mutable memory.	+

	In the C and C++ (also Fortran) worlds, MPI (Message Passing Interface) is a popular tool. The problem it		In the C and C++ (also Fortran) worlds, MPI (Message Passing Interface) is a popular tool. The problem it
Line 16:		Line 15:
	data types or provide a lot of extra infrastructure.		data types or provide a lot of extra infrastructure.

-	Various people have tried to apply MPI in Java but it has failed &~~mdash~~; and this is probably a good thing. Instead there is a growing and increasingly sophisticated infrastructure based on using special data structures to replace any access to shared memory: abstraction is used correctly to raise the level at which programmers work to develop concurrent and parallel systems. These data structures are tools for sending and receiving messages.	+	Various people have tried to apply MPI in Java but it has failed – and this is probably a good thing. Instead there is a growing and increasingly sophisticated infrastructure based on using special data structures to replace any access to shared memory: abstraction is used correctly to raise the level at which programmers work to develop concurrent and parallel systems. These data structures are tools for sending and receiving messages.

	Python is going a different route due to implementation issues with the standard system. In Python parallel systems make use of multiple instances of the Python system passing messages.		Python is going a different route due to implementation issues with the standard system. In Python parallel systems make use of multiple instances of the Python system passing messages.

Kevlin at 12:33, 24 June 2009

2009-06-24T12:33:30Z

←Older revision		Revision as of 12:33, 24 June 2009
Line 1:		Line 1:
	Programmers are taught from the very outset of their study of computing that concurrency and parallelism is hard, that only the very best can ever hope to get it right, and even they get it wrong. There is invariably great focus on threads, semaphores, monitors, and how hard it is to get concurrent access to variables to be thread-safe. True, there are many difficult problems, and they can be very hard to solve. But what is the core of the problem? Shared memory. Almost all the problems of concurrency and parallelism that people go on and on about relate to the use of shared, mutable memory. The answer seems obvious, either forgo concurrency and parallelism, or eschew shared memory!		Programmers are taught from the very outset of their study of computing that concurrency and parallelism is hard, that only the very best can ever hope to get it right, and even they get it wrong. There is invariably great focus on threads, semaphores, monitors, and how hard it is to get concurrent access to variables to be thread-safe. True, there are many difficult problems, and they can be very hard to solve. But what is the core of the problem? Shared memory. Almost all the problems of concurrency and parallelism that people go on and on about relate to the use of shared, mutable memory. The answer seems obvious, either forgo concurrency and parallelism, or eschew shared memory!

-	Forgoing concurrency and parallelism is almost certainly not an option. Computers have more and more cores	+	Forgoing concurrency and parallelism is almost certainly not an option. Computers have more and more cores on an almost quarterly basis, so harnessing parallelism becomes more and more important. We can no longer rely on ever increasing processor clock speeds to improve application performance. Only by applying parallelism will the performance of applications improve. Obviously, not improving performance is an option, but it is unlikely to be acceptable to users.
-	on an almost quarterly basis, so harnessing parallelism becomes more and more important. We can no longer	+
-	rely on ever increasing processor clock speeds to improve application performance. Only by applying	+
-	parallelism will the performance of applications improve. Obviously, not improving performance is an	+
-	option, but it is unlikely to be acceptable to users.	+

	So can we eschew shared memory? Definitely.		So can we eschew shared memory? Definitely.

-	Instead of using threads and shared memory as our programming model, we can use processes and message	+	Instead of using threads and shared memory as our programming model, we can use processes and message passing. Languages such as Erlang (and occam before it) have shown that concurrency and parallelism based on processes is a very successful way of programming concurrent and parallel systems. Such systems do not have all the synchronization stresses that shared memory, multi-threaded systems have. Moreover there is a mathematics — Communicating Sequential Processes (CSP) — that can be applied as part of the engineering of concurrent and parallel systems.
-	passing. Languages such as Erlang (and occam before it) have shown that concurrency and parallelism based	+
-	on processes is a very successful way of programming concurrent and parallel systems. Such systems do not	+
-	have all the synchronization stresses that shared memory, multi-threaded systems have. Moreover there is a	+
-	mathematics -- Communicating Sequential Processes (CSP) -- that can be applied as part of the engineering of	+
-	concurrent and parallel systems.	+

-	We can go further and introduce dataflow systems as a way of computing. In dataflow, there is no explicitly	+	We can go further and introduce dataflow systems as a way of computing. In a dataflow system there is no explicitly programmed control flow. Instead a directed graph of operators connected by data paths is set up and then data fed into the system. Evaluation is controlled by the readiness of data within the system. Definitely no synchronization problems.
-	programmed control flow. Instead a directed graph of operators connected by data paths is set up and then	+
-	data fed into the system. Evaluation is controlled by the readiness of data within the system. Definitely	+
-	no synchronization problems.	+

-	Having said all this C, C++, Java, Python and Groovy are the principal language of systems development and	+	Having said all this, languages such as C, C++, Java, Python, and Groovy are the principal language of systems development and
-	all of these are presented to programmers as languages for developing shared memory, multi-threaded systems.	+	all of these are presented to programmers as languages for developing shared memory, multi-threaded systems. So what can be done? The answer is to use — or, if they don't exist, create — libraries and frameworks that provide processes and message passing and avoid all use of shared mutable memory.
-	So what can be done? The answer is to use (or if they don't exist, create) libraries and frameworks that	+
-	provide processes and message passing and avoid all use of shared,mutable memory.	+

	In the C and C++ (also Fortran) worlds, MPI (Message Passing Interface) is a popular tool. The problem it		In the C and C++ (also Fortran) worlds, MPI (Message Passing Interface) is a popular tool. The problem it
Line 30:		Line 16:
	data types or provide a lot of extra infrastructure.		data types or provide a lot of extra infrastructure.

-	Various people have tried to apply MPI in Java but it has failed -- and this is probably a good thing.	+	Various people have tried to apply MPI in Java but it has failed — and this is probably a good thing. Instead there is a growing and increasingly sophisticated infrastructure based on using special data structures to replace any access to shared memory: abstraction is used correctly to raise the level at which programmers work to develop concurrent and parallel systems. These data structures are tools for sending and receiving messages.
-	Instead there is a growing and increasingly sophisticated infrastructure based on using special data	+
-	structures to replace any access to shared memory: abstraction is used correctly to raise the level at which	+
-	programmers work to develop concurrent and parallel systems. These data structures are tools for sending	+
-	and receiving messages.	+

-	Python is going a different route due to implementation issues with the standard system. In Python parallel	+	Python is going a different route due to implementation issues with the standard system. In Python parallel systems make use of multiple instances of the Python system passing messages.
-	systems make use of multiple instances of the Python system passing messages.	+

-	All in all, not programming with shared memory, but instead using message passing, is likely to be the most	+	All in all, not programming with shared memory, but instead using message passing, is likely to be the most successful way of implementing systems that harness the parallelism that is now endemic in computer hardware. Bizarrely perhaps, although processes predate threads as a unit of concurrency, the future seems to be in using threads to implement processes.
-	successful way of implementing systems that harness the parallelism that is now endemic in computer	+
-	hardware. Bizarrely perhaps, although processes predate threads as a unit of concurrency, the future seems	+	By [[Russel Winder]]
-	to be in using threads to implement processes.	+
		+	This work is licensed under a [http://creativecommons.org/licenses/by/3.0/us/ Creative Commons Attribution 3]
		+
		+
		+
		+	Back to [[97 Things Every Programmer Should Know]] home page

Russel: New page: Programmers are taught from the very outset of their study of computing that concurrency and parallelism is hard, that only the very best can ever hope to get it right, and even they get i...

2009-02-07T19:04:21Z

New page: Programmers are taught from the very outset of their study of computing that concurrency and parallelism is hard, that only the very best can ever hope to get it right, and even they get i...

New page

Programmers are taught from the very outset of their study of computing that concurrency and parallelism is hard, that only the very best can ever hope to get it right, and even they get it wrong. There is invariably great focus on threads, semaphores, monitors, and how hard it is to get concurrent access to variables to be thread-safe. True, there are many difficult problems, and they can be very hard to solve. But what is the core of the problem? Shared memory. Almost all the problems of concurrency and parallelism that people go on and on about relate to the use of shared, mutable memory. The answer seems obvious, either forgo concurrency and parallelism, or eschew shared memory!

Forgoing concurrency and parallelism is almost certainly not an option. Computers have more and more cores
on an almost quarterly basis, so harnessing parallelism becomes more and more important. We can no longer
rely on ever increasing processor clock speeds to improve application performance. Only by applying
parallelism will the performance of applications improve. Obviously, not improving performance is an
option, but it is unlikely to be acceptable to users.

So can we eschew shared memory? Definitely.

Instead of using threads and shared memory as our programming model, we can use processes and message
passing. Languages such as Erlang (and occam before it) have shown that concurrency and parallelism based
on processes is a very successful way of programming concurrent and parallel systems. Such systems do not
have all the synchronization stresses that shared memory, multi-threaded systems have. Moreover there is a
mathematics -- Communicating Sequential Processes (CSP) -- that can be applied as part of the engineering of
concurrent and parallel systems.

We can go further and introduce dataflow systems as a way of computing. In dataflow, there is no explicitly
programmed control flow. Instead a directed graph of operators connected by data paths is set up and then
data fed into the system. Evaluation is controlled by the readiness of data within the system. Definitely
no synchronization problems.

Having said all this C, C++, Java, Python and Groovy are the principal language of systems development and
all of these are presented to programmers as languages for developing shared memory, multi-threaded systems.
So what can be done? The answer is to use (or if they don't exist, create) libraries and frameworks that
provide processes and message passing and avoid all use of shared,mutable memory.

In the C and C++ (also Fortran) worlds, MPI (Message Passing Interface) is a popular tool. The problem it
has is that it doesn't provide support for non-primitive data types, programmers have to work with primitive
data types or provide a lot of extra infrastructure.

Various people have tried to apply MPI in Java but it has failed -- and this is probably a good thing.
Instead there is a growing and increasingly sophisticated infrastructure based on using special data
structures to replace any access to shared memory: abstraction is used correctly to raise the level at which
programmers work to develop concurrent and parallel systems. These data structures are tools for sending
and receiving messages.

Python is going a different route due to implementation issues with the standard system. In Python parallel
systems make use of multiple instances of the Python system passing messages.

All in all, not programming with shared memory, but instead using message passing, is likely to be the most
successful way of implementing systems that harness the parallelism that is now endemic in computer
hardware. Bizarrely perhaps, although processes predate threads as a unit of concurrency, the future seems
to be in using threads to implement processes.