Parallel Computing - Revision history

Angeliu: /* What to expect at runtime */

2023-01-07T13:53:13Z

What to expect at runtime

← Older revision		Revision as of 13:53, 7 January 2023
Line 132:		Line 132:
	< inp__3.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__3.rsf		< inp__3.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__3.rsf

	/usr/bin/ssh node1.utexas.edu "cd /home/test ; < ~~spike__4~~.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__4.rsf "		/usr/bin/ssh node1.utexas.edu "cd /home/test ; < inp__4.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__4.rsf "

	< inp__2.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__2.rsf		< inp__2.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__2.rsf

Morgon: /* MPI (external) */

2017-07-05T07:48:01Z

MPI (external)

← Older revision		Revision as of 07:48, 5 July 2017
Line 39:		Line 39:
	===MPI (external)===		===MPI (external)===
	To parallelize a data-parallel task using MPI but without including MPI calls in your source code, try <tt>sfmpi</tt>, as follows:		To parallelize a data-parallel task using MPI but without including MPI calls in your source code, try <tt>sfmpi</tt>, as follows:
	<bash>		<syntaxhighlight lang="bash">
	mpirun -np 8 sfmpi sfradon np=100 p0=0 dp=0.01 input=inp.rsf output=out.rsf split=2		mpirun -np 8 sfmpi sfradon np=100 p0=0 dp=0.01 input=inp.rsf output=out.rsf split=2
	</~~bash~~>		</syntaxhighlight>
	where the argument after <tt>-np</tt> specifies the number of processors involved. sfmpi will use this number to split the input along the slowest axis (presumed to be data-parallel) and to run it through parallel threads. Notice that the keywords <tt>input</tt>, <tt>output</tt>, and <tt>split</tt> are specific to <tt>sfmpi</tt>. They are used to specify the standard input and output streams of your program and the input axis to split.		where the argument after <tt>-np</tt> specifies the number of processors involved. sfmpi will use this number to split the input along the slowest axis (presumed to be data-parallel) and to run it through parallel threads. Notice that the keywords <tt>input</tt>, <tt>output</tt>, and <tt>split</tt> are specific to <tt>sfmpi</tt>. They are used to specify the standard input and output streams of your program and the input axis to split.

Sripanichy: /* Parallel Flow() using split and reduce */

2014-08-01T20:31:57Z

Parallel Flow() using split and reduce

← Older revision		Revision as of 20:31, 1 August 2014
Line 89:		Line 89:
	Flow('radon','spike','radon adj=y p0=-4 np=200 dp=0.04',split=[3,1000],reduce='cat')		Flow('radon','spike','radon adj=y p0=-4 np=200 dp=0.04',split=[3,1000],reduce='cat')
	</syntaxhighlight>		</syntaxhighlight>
	Concatenation on the same axis as specified by <tt>split=</tt> is the default reduction method. Possible other valid options are <tt>reduce='add'</tt>, <tt>reduce='cat axis=1'</tt>, etc. ~~An example~~ can be found in [http://www.ahay.org/RSF/book/rsf/school/data.html $RSFSRC/book/rsf/school/data/SConstruct].		Concatenation on the same axis as specified by <tt>split=</tt> is the default reduction method. Possible other valid options are <tt>reduce='add'</tt>, <tt>reduce='cat axis=1'</tt>, etc. Examples can be found in [http://www.ahay.org/RSF/book/rsf/school/data.html $RSFSRC/book/rsf/school/data/SConstruct] and $RSFSRC/book/trip/pscons/SConstruct.

	If flows that are run by <tt>pscons</tt> contain both serial and parallel targets, care must be exercised in order to not create bottlenecks, in which tasks are distributed to multiple nodes, but the nodes sit idle while waiting for other nodes to finish computing dependencies. Tasks that are not explicitly parallelized will be sped up by <tt>pscons</tt> if they are independent from each other. For example, compiling Madagascar itself with <tt>pscons</tt> instead of scons results in a visible speedup on a multithreaded machine.		If flows that are run by <tt>pscons</tt> contain both serial and parallel targets, care must be exercised in order to not create bottlenecks, in which tasks are distributed to multiple nodes, but the nodes sit idle while waiting for other nodes to finish computing dependencies. Tasks that are not explicitly parallelized will be sped up by <tt>pscons</tt> if they are independent from each other. For example, compiling Madagascar itself with <tt>pscons</tt> instead of scons results in a visible speedup on a multithreaded machine.

Sripanichy: /* Parallel Flow() using split and reduce */

2014-08-01T19:39:15Z

Parallel Flow() using split and reduce

← Older revision		Revision as of 19:39, 1 August 2014
Line 89:		Line 89:
	Flow('radon','spike','radon adj=y p0=-4 np=200 dp=0.04',split=[3,1000],reduce='cat')		Flow('radon','spike','radon adj=y p0=-4 np=200 dp=0.04',split=[3,1000],reduce='cat')
	</syntaxhighlight>		</syntaxhighlight>
	Concatenation on the same axis as specified by <tt>split=</tt> is the default reduction method. Possible other valid options are <tt>reduce='add'</tt>, <tt>reduce='cat axis=1'</tt>, etc.		Concatenation on the same axis as specified by <tt>split=</tt> is the default reduction method. Possible other valid options are <tt>reduce='add'</tt>, <tt>reduce='cat axis=1'</tt>, etc. An example can be found in [http://www.ahay.org/RSF/book/rsf/school/data.html $RSFSRC/book/rsf/school/data/SConstruct].

	If flows that are run by <tt>pscons</tt> contain both serial and parallel targets, care must be exercised in order to not create bottlenecks, in which tasks are distributed to multiple nodes, but the nodes sit idle while waiting for other nodes to finish computing dependencies. Tasks that are not explicitly parallelized will be sped up by <tt>pscons</tt> if they are independent from each other. For example, compiling Madagascar itself with <tt>pscons</tt> instead of scons results in a visible speedup on a multithreaded machine.		If flows that are run by <tt>pscons</tt> contain both serial and parallel targets, care must be exercised in order to not create bottlenecks, in which tasks are distributed to multiple nodes, but the nodes sit idle while waiting for other nodes to finish computing dependencies. Tasks that are not explicitly parallelized will be sped up by <tt>pscons</tt> if they are independent from each other. For example, compiling Madagascar itself with <tt>pscons</tt> instead of scons results in a visible speedup on a multithreaded machine.

Sripanichy: /* MPI (internal) */

2014-08-01T18:58:43Z

MPI (internal)

← Older revision		Revision as of 18:58, 1 August 2014
Line 35:		Line 35:
	[http://www.mcs.anl.gov/research/projects/mpi/ MPI] (Message-Passing Interface) is the dominant standard framework for parallel processing on different computer architectures including '''distributed-memory''' systems. Several MPI implementations (such as [http://www.open-mpi.org/ Open MPI] and [http://www.mcs.anl.gov/research/projects/mpich2/ MPICH2]) are available.		[http://www.mcs.anl.gov/research/projects/mpi/ MPI] (Message-Passing Interface) is the dominant standard framework for parallel processing on different computer architectures including '''distributed-memory''' systems. Several MPI implementations (such as [http://www.open-mpi.org/ Open MPI] and [http://www.mcs.anl.gov/research/projects/mpich2/ MPICH2]) are available.

	An example of compiling a program with <tt>mpicc</tt> and running it under <tt>mpirun</tt> can be found in [http://www.ahay.org/RSF/book/rsf/bash/mpi.html $RSFSRC/book/rsf/bash/mpi/SConstruct]		An example of compiling a program with <tt>mpicc</tt> and running it under <tt>mpirun</tt> can be found in [http://www.ahay.org/RSF/book/rsf/bash/mpi.html $RSFSRC/book/rsf/bash/mpi/SConstruct]. Note that Madagascar has a requirement that all internally-executing MPI programs must contain string 'mpi' in the program name as it is needed for SCons to switch to a mpi compiler such as mpicc.

	===MPI (external)===		===MPI (external)===

Sripanichy: /* Setting the environment variables */

2014-08-01T18:42:20Z

Setting the environment variables

← Older revision		Revision as of 18:42, 1 August 2014
Line 59:		Line 59:
	Simply running pscons with no special environment variable set is equivalent to running <tt>scons -j nproc</tt>, where <tt>nproc</tt> is the auto-detected number of threads on your system. To fully use the potential of <tt>pscons</tt> for running on a distributed-memory computer, you need to set the environment variables <tt>RSF_CLUSTER</tt> and <tt>RSF_THREADS</tt>, and to use <tt>split</tt> and <tt>reduce</tt> arguments in your SConstruct Flow statements where appropriate.		Simply running pscons with no special environment variable set is equivalent to running <tt>scons -j nproc</tt>, where <tt>nproc</tt> is the auto-detected number of threads on your system. To fully use the potential of <tt>pscons</tt> for running on a distributed-memory computer, you need to set the environment variables <tt>RSF_CLUSTER</tt> and <tt>RSF_THREADS</tt>, and to use <tt>split</tt> and <tt>reduce</tt> arguments in your SConstruct Flow statements where appropriate.

	===Setting the environment variables===		===Setting the environment variables and how to run===

	The <tt>RSF_CLUSTER</tt> variable holds, for each node, the name or IP address of that node (in a format that can be used by ssh), followed by the number of threads on the node. For example, creating 26 threads and sending them on 4 nodes, using respectively 6 CPUs on the first node, 4 CPUs on the second, and 8 CPUs on each of the last two nodes:		The <tt>RSF_CLUSTER</tt> variable holds, for each node, the name or IP address of that node (in a format that can be used by ssh), followed by the number of threads on the node. For example, creating 26 threads and sending them on 4 nodes, using respectively 6 CPUs on the first node, 4 CPUs on the second, and 8 CPUs on each of the last two nodes:

Sripanichy: /* The split and reduce options in Flow() */

2014-08-01T18:40:32Z

The split and reduce options in Flow()

← Older revision		Revision as of 18:40, 1 August 2014
Line 84:		Line 84:
	</syntaxhighlight>.		</syntaxhighlight>.

	===~~The~~ split and reduce ~~options in Flow()~~===		===Parallel Flow() using split and reduce===
	The split option specifies the number of the axis to be split and the size of that axis. For an axis 3 of length 1000 on the standard in file, and collection by concatenation:		The split option specifies the number of the axis to be split and the size of that axis. For an axis 3 of length 1000 on the standard in file, and collection by concatenation:
	<syntaxhighlight lang="python">		<syntaxhighlight lang="python">

Sripanichy: /* Setting the environment variables */

2014-08-01T18:37:58Z

Setting the environment variables

← Older revision		Revision as of 18:37, 1 August 2014
Line 79:		Line 79:
	</syntaxhighlight>		</syntaxhighlight>

	To execute using this method, one can then use the command <tt>pscons</tt> or if the environment variables ~~aren't specified~~,		To execute using this method, one can then use the command <tt>pscons</tt> or avoid specifying the environment variables altogether by using,
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	scons -j 26 CLUSTER='140.168.1.236 6 140.168.1.235 4 140.168.1.234 8 140.168.1.233 8'		scons -j 26 CLUSTER='140.168.1.236 6 140.168.1.235 4 140.168.1.234 8 140.168.1.233 8'

Sripanichy: /* pscons */

2014-08-01T18:36:21Z

pscons

← Older revision		Revision as of 18:36, 1 August 2014
Line 78:		Line 78:
	export TMPDATAPATH=/tmp/		export TMPDATAPATH=/tmp/
	</syntaxhighlight>		</syntaxhighlight>

			To execute using this method, one can then use the command <tt>pscons</tt> or if the environment variables aren't specified,
			<syntaxhighlight lang="bash">
			scons -j 26 CLUSTER='140.168.1.236 6 140.168.1.235 4 140.168.1.234 8 140.168.1.233 8'
			</syntaxhighlight>.

	===The split and reduce options in Flow()===		===The split and reduce options in Flow()===

Sripanichy at 18:24, 1 August 2014

2014-08-01T18:24:12Z

← Older revision		Revision as of 18:24, 1 August 2014
Line 2:		Line 2:
	Many of the data processing operations are '''data-parallel''': different traces, shot gathers, frequency slices, etc. can be processed independently. Madagascar provides several mechanisms for handling this type of embarrassingly parallel applications on computers with multiple processors.		Many of the data processing operations are '''data-parallel''': different traces, shot gathers, frequency slices, etc. can be processed independently. Madagascar provides several mechanisms for handling this type of embarrassingly parallel applications on computers with multiple processors.

	==OpenMP (internal)==		==OpenMP and MPI==

			===OpenMP (internal)===
	[https://secure.wikimedia.org/wikipedia/en/wiki/OpenMP OpenMP] is a standard framework for parallel applications on '''shared-memory''' systems. It is supported by the latest versions of [http://gcc.gnu.org/ GCC] and by some other compilers.		[https://secure.wikimedia.org/wikipedia/en/wiki/OpenMP OpenMP] is a standard framework for parallel applications on '''shared-memory''' systems. It is supported by the latest versions of [http://gcc.gnu.org/ GCC] and by some other compilers.

Line 15:		Line 17:
	On the last check (2014-02-09), 139 standalone programs (approximately 11% of Madagascar programs) were using OMP. Running a similar command in the directory <tt>$RSFSRC/api/c</tt> will yield a few library functions parallelized with OMP.		On the last check (2014-02-09), 139 standalone programs (approximately 11% of Madagascar programs) were using OMP. Running a similar command in the directory <tt>$RSFSRC/api/c</tt> will yield a few library functions parallelized with OMP.

	==OpenMP (external)==		===OpenMP (external)===

	To run on a multi-core shared-memory machine a data-parallel process that does not contain OpenMP calls, use <tt>sfomp</tt>. Thus, a call like		To run on a multi-core shared-memory machine a data-parallel process that does not contain OpenMP calls, use <tt>sfomp</tt>. Thus, a call like
Line 30:		Line 32:
	</syntaxhighlight>		</syntaxhighlight>

	==MPI (internal)==		===MPI (internal)===
	[http://www.mcs.anl.gov/research/projects/mpi/ MPI] (Message-Passing Interface) is the dominant standard framework for parallel processing on different computer architectures including '''distributed-memory''' systems. Several MPI implementations (such as [http://www.open-mpi.org/ Open MPI] and [http://www.mcs.anl.gov/research/projects/mpich2/ MPICH2]) are available.		[http://www.mcs.anl.gov/research/projects/mpi/ MPI] (Message-Passing Interface) is the dominant standard framework for parallel processing on different computer architectures including '''distributed-memory''' systems. Several MPI implementations (such as [http://www.open-mpi.org/ Open MPI] and [http://www.mcs.anl.gov/research/projects/mpich2/ MPICH2]) are available.

	An example of compiling a program with <tt>mpicc</tt> and running it under <tt>mpirun</tt> can be found in [http://www.ahay.org/RSF/book/rsf/bash/mpi.html $RSFSRC/book/rsf/bash/mpi/SConstruct]		An example of compiling a program with <tt>mpicc</tt> and running it under <tt>mpirun</tt> can be found in [http://www.ahay.org/RSF/book/rsf/bash/mpi.html $RSFSRC/book/rsf/bash/mpi/SConstruct]

	==MPI (external)==		===MPI (external)===
	To parallelize a data-parallel task using MPI but without including MPI calls in your source code, try <tt>sfmpi</tt>, as follows:		To parallelize a data-parallel task using MPI but without including MPI calls in your source code, try <tt>sfmpi</tt>, as follows:
	<bash>		<bash>
Line 44:		Line 46:
	Some older MPI implementations do not support system calls implemented in <tt>sfmpi</tt> and therefore may not support this feature.		Some older MPI implementations do not support system calls implemented in <tt>sfmpi</tt> and therefore may not support this feature.

	==MPI + OpenMP (both external)==		===MPI + OpenMP (both external)===

	It is possible to combine the advantages of shared-memory and distributed-memory architectures by using OpenMP and MPI together.		It is possible to combine the advantages of shared-memory and distributed-memory architectures by using OpenMP and MPI together.