{"id":160,"date":"2007-12-27T11:00:41","date_gmt":"2007-12-27T11:00:41","guid":{"rendered":"http:\/\/ahay.org\/blog\/?p=160"},"modified":"2015-08-04T23:51:42","modified_gmt":"2015-08-04T23:51:42","slug":"parallel-processing","status":"publish","type":"post","link":"https:\/\/ahay.org\/blog\/2007\/12\/27\/parallel-processing\/","title":{"rendered":"Parallel processing"},"content":{"rendered":"

Many of the data processing operations are data-parallel<\/strong>: different traces, shot gathers, frequency slices, etc. can be processed independently. Madagascar<\/tt> provides several mechanisms for handling this type of embarrassingly parallel<\/em> applications on computers with multiple processors.<\/p>\n

    \n
  1. OpenMP<\/strong><\/li>\n

    OpenMP<\/a> is a standard framework for parallel applications on shared-memory<\/strong> systems. It is supported by the latest versions of GCC <\/a>and by some other compilers.
    \nTo run a data-parallel processing task like<\/p>\n

    <\/i>
    <\/i><\/a><\/div><\/div>
    sfradon np=100 p0=0 dp=0.01 < inp.rsf > out.rsf<\/pre><\/div>\n
    on a shared-memory computer with multiple processors (such as a multi-core PC), try sfomp<\/strong>, as follows:.<\/p>\n
    <\/i> 
    <\/i><\/a><\/div><\/div>
    sfomp sfradon np=100 p0=0 dp=0.01 < inp.rsf > out.rsf<\/pre><\/div>\n
    sfomp<\/strong> splits the input along the slowest axis (presumed to be data-parallel) and runs it through parallel threads. The number of threads is set by the OMP_NUM_THREADS<\/strong> environmental variable or (by default) by the number of available CPUs.<\/p>\n
  2. MPI<\/strong><\/li>\n
    MPI<\/a> (Message-Passing Interface) is a standard framework for parallel processing on different computer architectures including distributed-memory<\/strong> systems. Several MPI implementations (such as MPICH<\/a>) are available.
    \nTo parallelize a task using MPI, try sfmpi<\/strong>, as follows:<\/p>\n
    <\/i> 
    <\/i><\/a><\/div><\/div>
    mpirun -np 8 sfmpi sfradon np=100 p0=0 dp=0.01 input=inp.rsf output=out.rsf<\/pre><\/div>\n
    where the argument after -np<\/b> specifies the number of processors involved. sfmpi<\/strong> will use this number to split the input along the slowest axis (presumed to be data-parallel) and to run it through parallel threads.
    \nNote:<\/em> Some MPI implementations do not support system calls implemented in sfmpi<\/tt> and therefore will not support this option.<\/p>\n
  3. MPI + OpenMP<\/strong><\/li>\n
    It is possible to combine the advantages of shared-memory and distributed-memory architectures by using OpenMP and MPI together.<\/p>\n
    <\/i> 
    <\/i><\/a><\/div><\/div>
    mpirun -np 32 sfmpi sfomp sfradon np=100 p0=0 dp=0.01 input=inp.rsf output=out.rsf<\/pre><\/div>\n
    will distribute the job on 32 nodes and split it again on each node using shared-memory threads.<\/p>\n
  4. SCons<\/strong><\/li>\n
    If you process data using SCons<\/a>, another option is available. Change<\/p>\n
    <\/i> 
    <\/i><\/a><\/div><\/div>
    Flow('out','inp','radon np=100 p0=0 dp=0.01')<\/pre><\/div>\n
    in your SConstruct<\/tt> file to<\/p>\n
    <\/i> 
    <\/i><\/a><\/div><\/div>
    Flow('out','inp','radon np=100 p0=0 dp=0.01',split=[3,256])<\/pre><\/div>\n
    where the optional split=<\/strong> parameter contains the axis that needs to be split and the size of this axis. Then run something like<\/p>\n
    <\/i> 
    <\/i><\/a><\/div><\/div>
    scons -j 8 CLUSTER='localhost 4 node1.utexas.edu 2' out.rsf<\/pre><\/div>\n
    The -j<\/strong> options instructs SCons to run in parallel creating 8 threads, while the CLUSTER=<\/strong> option supplies it with the list of nodes to use and the number of processes to involve for each node. The output may look like<\/p>\n
    <\/i> 
    <\/i><\/a><\/div><\/div>
    \n< inp.rsf \/RSFROOT\/bin\/sfwindow n3=42 f3=0 squeeze=n > inp__0.rsf\n< inp.rsf \/RSFROOT\/bin\/sfwindow n3=42 f3=42 squeeze=n > inp__1.rsf\n\/usr\/bin\/ssh node1.utexas.edu \"cd \/home\/test ; \/bin\/env < inp.rsf \/RSFROOT\/bin\/sfwindow n3=42 f3=84 squeeze=n > inp__2.rsf \"\n< inp.rsf \/RSFROOT\/bin\/sfwindow n3=42 f3=126 squeeze=n > inp__3.rsf\n< inp.rsf \/RSFROOT\/bin\/sfwindow n3=42 f3=168 squeeze=n > inp__4.rsf\n\/usr\/bin\/ssh node1.utexas.edu \"cd \/home\/test ; \/bin\/env < inp.rsf \/RSFROOT\/bin\/sfwindow f3=210 squeeze=n > inp__5.rsf \"\n< inp__0.rsf \/RSFROOT\/bin\/sfradon p0=0 np=100 dp=0.01 > out__0.rsf\n\/usr\/bin\/ssh node1.utexas.edu \"cd \/home\/test ; \/bin\/env < inp__1.rsf \/RSFROOT\/bin\/sfradon p0=0 np=100 dp=0.01 > out__1.rsf \"\n< inp__3.rsf \/RSFROOT\/bin\/sfradon p0=0 np=100 dp=0.01 > out__3.rsf\n\/usr\/bin\/ssh node1.utexas.edu \"cd \/home\/test ; < spike__4.rsf \/RSFROOT\/bin\/sfradon p0=0 np=100 dp=0.01 > out__4.rsf \"\n< inp__2.rsf \/RSFROOT\/bin\/sfradon p0=0 np=100 dp=0.01 > out__2.rsf\n< inp__5.rsf \/RSFROOT\/bin\/sfradon p0=0 np=100 dp=0.01 > out__5.rsf\n< out__0.rsf \/RSFROOT\/bin\/sfcat axis=3 out__1.rsf out__2.rsf out__3.rsf out__4.rsf out__5.rsf > out.rsf\n<\/pre><\/div>\n
    Splitting the input with sfwindow<\/strong> and putting the output back together with sfcat<\/strong> are immediately apparent. The advantage of the SCons-based approach (in addition to documentation and reproducible experiments) is fault tollerance<\/em>: If one of the nodes dies during the process, one should be able to restart the computation without recreating parts that are already computed.<\/ol>\n
    All these options will continue to evolve and improve with further testing. Please report your experiences and suggestions.<\/p>\n","protected":false},"excerpt":{"rendered":"
    Many of the data processing operations are data-parallel: different traces, shot gathers, frequency slices, etc. can be processed independently. Madagascar provides several mechanisms for handling this type of embarrassingly parallel applications on computers with multiple processors. OpenMP OpenMP is a standard framework for parallel applications on shared-memory systems. It is supported by the latest versions […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","activitypub_content_warning":"","activitypub_content_visibility":"local","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"","footnotes":""},"categories":[7],"tags":[],"class_list":["post-160","post","type-post","status-publish","format-standard","hentry","category-examples"],"_links":{"self":[{"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/posts\/160","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/comments?post=160"}],"version-history":[{"count":1,"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/posts\/160\/revisions"}],"predecessor-version":[{"id":755,"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/posts\/160\/revisions\/755"}],"wp:attachment":[{"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/media?parent=160"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/categories?post=160"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ahay.org\/blog\/wp-json\/wp\/v2\/tags?post=160"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}