Madagascar - User contributions [en]

Shanghai 2017

2017-07-17T07:29:07Z

Morgon: /* Additional Installation Requirements */

[[Image:Shanghai2017.jpg|center|frame|800px]]

<center><big>'''2017 Madagascar School in Shanghai'''</big></center>

 

== Introduction ==

Madagascar provides a complete environment for organizing one's research, from new software development to running computational experiments to publishing the experimental results in papers and reports, archiving them for future usage, and sharing them with colleagues and sponsors. Madagascar school has been organized many times around the word. It provides a platform for Madagascar users to learn and communicate with the developers. This Madagascar school will introduce the advanced usages for the users with fundamental knowledge of Madagascar. Besides the lecture section, this school will also features a discussion section, in which the participants are welcome to share the research experiences using Madagascar. This Madagascar school is convened by Prof. Sergey Fomel from the University of Texas at Austin and hosted by Prof. Jiubing Cheng from Tongji University.

== Dates ==

July 10-11, 2017

== Agenda ==

{| align="center" border="1" cellpadding="5" cellspacing="0"
! colspan="3" style="background:#ffdead;" | Before the school: Sunday, July 9, 2017
|-
| 14:00-18:00
! colspan="2" | Bring your laptop and get individual help with Madagascar
|-
|
! colspan="2" | Peng Zou, Chenlong Wang, Gang Fang
|-
|}

 

{| align="center" border="1" cellpadding="5" cellspacing="0"
! colspan="3" style="background:#ffdead;" | Day 1: Monday, July 10, 2017
|-
| 8:30-8:40
! colspan="2" | Welcome (Jiubing Cheng)
|-
| 8:40-9:30
! colspan="2" | Madagascar principles (Sergey Fomel)
|-
| 9:30-9:40
! colspan="2" style="background:#efefef;" | break
|-
| 9:40-10:30
! colspan="2" | Madagascar development: writing programs in C, C++ (Gang Fang)
|-
| 10:30-11:30
! colspan="2" | Plotting Guidance and SConstruct Flow (Xufei Gong)
|-
| 11:30-14:00
! colspan="2" style="background:#efefef;" | Lunch
|-
| 14:00-15:30
! colspan="2" | Parallel programming (MPI, OpenMP, Pscons) in Madagascar (Chenlong Wang)
|-
|
| colspan="2" |
[http://www.ahay.org/wikilocal/docs/chenlongw.pdf Slides] (484K)
|-
| 15:30-15:50
! colspan="2" style="background:#efefef;" | break
|-
| 15:50-17:00
! colspan="2" |Writing a reproducible paper using LaTeX and Madagascar (Jiubing Cheng)
|}

 

{| align="center" border="1" cellpadding="5" cellspacing="0"
! colspan="3" style="background:#ffdead;" | Day 2: Tuesday, July 11, 2017
|-
| 8:30-10:00
! colspan="2" | Seismic data processing with Madagascar (Yang Liu)
|-
| 10:00-10:30
! colspan="2" style="background:#efefef;" | break
|-
| 10:30-11:30
! colspan="2" |Python interface and contributing to Madagascar (Sergey Fomel)
|-
| 11:30-14:00
! colspan="2" style="background:#efefef;" | Lunch
|-
| 14:00-17:00
! colspan="2" | Open discussion（Sharing your experiences in Madagascar）
|}

== Location ==

Room 205B, Rui’an Building, Tongji University, Shanghai, China

同济大学，瑞安楼 205B室，上海市杨浦区四平路1239号

== Registration ==

The Madagascar School participation is free but requires an application. Please send your name and affiliation to [mailto:1533006@tongji.edu.cn 1533006@tongji.edu.cn]. '''The deadline is July 5, 2017.'''
== Pre-workshop Assignment ==

You must [[download]] and [[Installation|install]] the Madagascar package in the days before the school. If issues come up, there will an opportunity to get installation help on Sunday, July 9. Remember to bring your laptop (Linux, Mac) to the session!
=== Additional Installation Requirements ===
#Latex
#MPICH2/OPENMPI

=== QQ group ===

Madagascar advanced school QQ group: 468749704

== Organizations ==

The "Madagascar School of Reproducible Computational Geophysics Shanghai 2017" is organized by
* '''Jiubing Cheng''' from School of Ocean and Earth Science, Tongji University (Shanghai, China)
* '''Sergey Fomel''' from Jackson School of Geosciences, The University of Texas at Austin (Austin, USA)
* '''Yang Liu''' from College of Geo-exploration Science and Technology, Jilin University (Changchun, China)
* '''Gang Fang''' from Qingdao Institute of Marine Geology, China Geological Survey (Qingdao, China)
and hosted by '''Jiubing Cheng'''

== Instructors ==

* '''Jiubing Cheng''' is a Professor of School of Ocean and Earth Science, Tongji University. He received a Ph.D. in Geophysics from Tongji University in 2003. http://mgg.tongji.edu.cn/space/cjb/
* '''Sergey Fomel'''is a Professor at the Jackson School of Geosciences, the University of Texas at Austin. He received a Ph.D. in Geophysics from Stanford University in 2001. Sergey started work on Madagascar (at that time named RSF for Regularly Sampled Format) in 2003. http://www.jsg.utexas.edu/researcher/sergey_fomel/
* '''Yang Liu''' is a Professor of Geophysics at College of Geo-exploration science and technology at Jilin University, China. He received a Ph.D. in Geophysics from Jilin University in 2006 and was a Postdoctoral fellow at Bureau of Economic Geology, The University of Texas at Austin from 2007 to 2010. His research focuses mainly on seismic data processing. http://gest.jlu.edu.cn/?mod=info&act=view&id=2296
* '''Gang Fang''' is an assistant research fellow at Qingdao Institute of Marine Geology, China Geological Survey. He received a Ph.D in Geophysics from China University of Petroleum (East China) in 2014 and was a visiting Ph.D student at Bureau of Economic Geology, The University of Texas at Austin from 2012 to 2013 sponsored by China State Scholarship Fund. His research interests are seismic modeling and seismic imaging. https://www.linkedin.com/profile/view?id=92186291

Parallel Computing

2017-07-05T07:48:01Z

Morgon: /* MPI (external) */

[[Image:Cluster.jpg|right|frame|[http://www.freedigitalphotos.net/images/view_photog.php?photogid=1152 Image: jscreationzs / FreeDigitalPhotos.net]]]
Many of the data processing operations are '''data-parallel''': different traces, shot gathers, frequency slices, etc. can be processed independently. Madagascar provides several mechanisms for handling this type of embarrassingly parallel applications on computers with multiple processors.

==OpenMP and MPI==

===OpenMP (internal)===
[https://secure.wikimedia.org/wikipedia/en/wiki/OpenMP OpenMP] is a standard framework for parallel applications on '''shared-memory''' systems. It is supported by the latest versions of [http://gcc.gnu.org/ GCC] and by some other compilers.

To use OpenMP in your program, you do not need to add anything to your SConstruct. Just make sure the OMP libraries are installed on your system before you configure Madagascar, (or -- reinstall them and rerun the configuration command). Of course, you need to use the appropriate pragmas in your code. To find Madagascar programs that use OpenMP and that you can take as a model, run the following command:

<syntaxhighlight lang="bash">
grep "pragma omp" $RSFSRC/*/*/M*.c |\
awk -F ':' '{ print $1 }' |\
uniq |\
awk -F '/' '{ print $NF }'
</syntaxhighlight>
On the last check (2014-02-09), 139 standalone programs (approximately 11% of Madagascar programs) were using OMP. Running a similar command in the directory <tt>$RSFSRC/api/c</tt> will yield a few library functions parallelized with OMP.

===OpenMP (external)===

To run on a multi-core shared-memory machine a data-parallel process that does not contain OpenMP calls, use <tt>sfomp</tt>. Thus, a call like
<syntaxhighlight lang="bash">
sfradon np=100 p0=0 dp=0.01 < inp.rsf > out.rsf
</syntaxhighlight>
becomes
<syntaxhighlight lang="bash">
sfomp sfradon np=100 p0=0 dp=0.01 < inp.rsf > out.rsf
</syntaxhighlight>
<tt>sfomp</tt> splits the input along the slowest axis (presumed to be data-parallel) and runs it through parallel threads. The number of threads is set by the <tt>OMP_NUM_THREADS</tt> environmental variable or (by default) by the number of available CPUs. For example,
<syntaxhighlight lang="bash">
export OMP_NUM_THREADS=number of threads
</syntaxhighlight>

===MPI (internal)===
[http://www.mcs.anl.gov/research/projects/mpi/ MPI] (Message-Passing Interface) is the dominant standard framework for parallel processing on different computer architectures including '''distributed-memory''' systems. Several MPI implementations (such as [http://www.open-mpi.org/ Open MPI] and [http://www.mcs.anl.gov/research/projects/mpich2/ MPICH2]) are available.

An example of compiling a program with <tt>mpicc</tt> and running it under <tt>mpirun</tt> can be found in [http://www.ahay.org/RSF/book/rsf/bash/mpi.html $RSFSRC/book/rsf/bash/mpi/SConstruct]. Note that Madagascar has a requirement that all internally-executing MPI programs must contain string 'mpi' in the program name as it is needed for SCons to switch to a mpi compiler such as mpicc.

===MPI (external)===
To parallelize a data-parallel task using MPI but without including MPI calls in your source code, try <tt>sfmpi</tt>, as follows:
<syntaxhighlight lang="bash">
mpirun -np 8 sfmpi sfradon np=100 p0=0 dp=0.01 input=inp.rsf output=out.rsf split=2
</syntaxhighlight>
where the argument after <tt>-np</tt> specifies the number of processors involved. sfmpi will use this number to split the input along the slowest axis (presumed to be data-parallel) and to run it through parallel threads. Notice that the keywords <tt>input</tt>, <tt>output</tt>, and <tt>split</tt> are specific to <tt>sfmpi</tt>. They are used to specify the standard input and output streams of your program and the input axis to split.

Some older MPI implementations do not support system calls implemented in <tt>sfmpi</tt> and therefore may not support this feature.

===MPI + OpenMP (both external)===

It is possible to combine the advantages of shared-memory and distributed-memory architectures by using OpenMP and MPI together.
<syntaxhighlight lang="bash">
mpirun -np 32 sfmpi sfomp sfradon np=100 p0=0 dp=0.01 input=inp.rsf output=out.rsf
</syntaxhighlight>
will distribute the job on 32 nodes and split it again on each node using shared-memory threads.

==pscons==
To get SCons to cut your inputs into slices, run in parallel on one multi-cpu workstation or on multiple cluster nodes and then collect, use the <tt>pscons</tt> wrapper to <tt>scons</tt>. Unlike the OpenMP or MPI utilities, this has fault tolerance -- in case of a node failing, restarting the job will allow it to complete.

Simply running pscons with no special environment variable set is equivalent to running <tt>scons -j nproc</tt>, where <tt>nproc</tt> is the auto-detected number of threads on your system. To fully use the potential of <tt>pscons</tt> for running on a distributed-memory computer, you need to set the environment variables <tt>RSF_CLUSTER</tt> and <tt>RSF_THREADS</tt>, and to use <tt>split</tt> and <tt>reduce</tt> arguments in your SConstruct Flow statements where appropriate.

===Setting the environment variables and how to run===

The <tt>RSF_CLUSTER</tt> variable holds, for each node, the name or IP address of that node (in a format that can be used by ssh), followed by the number of threads on the node. For example, creating 26 threads and sending them on 4 nodes, using respectively 6 CPUs on the first node, 4 CPUs on the second, and 8 CPUs on each of the last two nodes:
<syntaxhighlight lang="bash">
export RSF_CLUSTER='140.168.1.236 6 140.168.1.235 4 140.168.1.234 8 140.168.1.233 8'
</syntaxhighlight>

The <tt>RSF_THREADS</tt> variable holds the sum of the numbers of threads on all nodes, i.e.:
<syntaxhighlight lang="bash">
export RSF_THREADS=26
</syntaxhighlight>
If <tt>RSF_CLUSTER</tt> is not defined, <tt>RSF_THREADS</tt> can be used to override the auto-detected number of threads used on the local host. This can be useful in the case of processes using a large amount of memory.

In Beowulf-type clusters in which communication of the processor with the local disk is much faster than with the shared network storage, it is important to set in the shell resource file the temporary file location to a local disk, and the <tt>DATAPATH</tt> variable to a network-visible location for global collection of results, i.e.:

<syntaxhighlight lang="bash">
export DATAPATH=/disk1/data/myname/
export TMPDATAPATH=/tmp/
</syntaxhighlight>

To execute using this method, one can then use the command <tt>pscons</tt> or avoid specifying the environment variables altogether by using,
<syntaxhighlight lang="bash">
scons -j 26 CLUSTER='140.168.1.236 6 140.168.1.235 4 140.168.1.234 8 140.168.1.233 8'
</syntaxhighlight>.

===Parallel Flow() using split and reduce===
The split option specifies the number of the axis to be split and the size of that axis. For an axis 3 of length 1000 on the standard in file, and collection by concatenation:
<syntaxhighlight lang="python">
Flow('radon','spike','radon adj=y p0=-4 np=200 dp=0.04',split=[3,1000],reduce='cat')
</syntaxhighlight>
Concatenation on the same axis as specified by <tt>split=</tt> is the default reduction method. Possible other valid options are <tt>reduce='add'</tt>, <tt>reduce='cat axis=1'</tt>, etc. Examples can be found in [http://www.ahay.org/RSF/book/rsf/school/data.html $RSFSRC/book/rsf/school/data/SConstruct] and $RSFSRC/book/trip/pscons/SConstruct.

If flows that are run by <tt>pscons</tt> contain both serial and parallel targets, care must be exercised in order to not create bottlenecks, in which tasks are distributed to multiple nodes, but the nodes sit idle while waiting for other nodes to finish computing dependencies. Tasks that are not explicitly parallelized will be sped up by <tt>pscons</tt> if they are independent from each other. For example, compiling Madagascar itself with <tt>pscons</tt> instead of scons results in a visible speedup on a multithreaded machine.

=== Computing on the local node only by using the option local=1 ===

By default, with '''pscons''', SCons attempts to run all the commands of the <tt>SConstruct</tt> file in parallel.
The option '''local=1''' forces SCons to compute locally on the head node of the cluster. It can be useful for preventing serial
parts of your python script to be distributed across multiple nodes.
<syntaxhighlight lang="python">
Flow('spike',None,'spike n1=100 n2=300 n3=1000',local=1)
</syntaxhighlight>

===What to expect at runtime===
SCons will create intermediate input and output slices in the current directory. For example, for
<syntaxhighlight lang="bash">
Flow('out','inp','radon np=100 p0=0 dp=0.01',split=[3,256])
</syntaxhighlight>
and
<syntaxhighlight lang="bash">
RSF_THREADS=8
RSF_CLUSTER='localhost 4 node1.utexas.edu 4'
</syntaxhighlight>
the SCons output will look like:
<syntaxhighlight lang="bash">
< inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=0 squeeze=n > inp__0.rsf

< inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=42 squeeze=n > inp__1.rsf

/usr/bin/ssh node1.utexas.edu "cd /home/test ; /bin/env < inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=84 squeeze=n > inp__2.rsf "

< inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=126 squeeze=n > inp__3.rsf

< inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=168 squeeze=n > inp__4.rsf

/usr/bin/ssh node1.utexas.edu "cd /home/test ; /bin/env < inp.rsf /RSFROOT/bin/sfwindow f3=210 squeeze=n > inp__5.rsf "

< inp__0.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__0.rsf

/usr/bin/ssh node1.utexas.edu "cd /home/test ; /bin/env < inp__1.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__1.rsf "

< inp__3.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__3.rsf

/usr/bin/ssh node1.utexas.edu "cd /home/test ; < spike__4.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__4.rsf "

< inp__2.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__2.rsf

< inp__5.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__5.rsf

< out__0.rsf /RSFROOT/bin/sfcat axis=3 out__1.rsf out__2.rsf out__3.rsf out__4.rsf out__5.rsf > out.rsf
</syntaxhighlight>

Note that operations were sent for execution in parallel, but the display is necessarily serial.

Runtime job monitoring can be achieved with '''sftop'''. To kill a distributed job, use '''sfkill'''.