Madagascar School in Zürich

October 31, 2016 Celebration No comments

Filippo Broggini reports:

The 2016 Madagascar School on Reproducible Computational Geophysics took place in Zürich, Switzerland, on June 6-7, 2016, and was hosted by the Exploration and Environmental Geophysics (EEG) group at ETH Zürich.

The school attracted more than 15 participants from 5 countries and 10 different universities. The program included lectures given by 5 different instructors and hands-on exercises on different topics in the use of the Madagascar software framework. The school materials are available on the website.

The genesis of Madagascar

November 12, 2015 Celebration No comments

The November 2015 issue of The Leading Edge contains an article about Madagascar: The Genesis of Madagascar by John Holden.

In July 2014, an unusual meeting took place at Rice University in Houston, Texas. Two dozen participants from numerous organizations gathered in a conference room for a workshop. Instead of the usual presentations and long talks one associates with scientific workshops, the participants immediately broke into small teams and gathered around laptop computers to write software code. Intense code-hacking sessions were interrupted only by necessary group discussions. This was the second “working workshop” of the Madagascar open-source software project.

One of the article’s features are user testimonials:

William W. Symes, Noah Harding Professor in Computational and Applied Mathematics and professor of earth science at Rice University, Houston, says, “I use Madagascar for everything! All of my computational research projects and those of my students take advantage of both the utility side of Madagascar (that is, its many useful commands for data manipulation and processing) and the reproducible research side. I have been convinced since learning the concept from J. F. Claerbout many years ago that reproducible research, in the sense epitomized by Madagascar, is not just an intellectually satisfactory mode (arguably the only such) for computational science research but a tremendous labor saver. Over the years, I have built a number of my own reproducible research (RR) frameworks but junked them all in favor of Madagascar several years ago. The utility suite is an incredible achievement, but the truly exceptional feature of Madagascar, in my opinion, is the suite of reproducible research tools. These are based on a full-featured language (Python) making them easily extensible, and they integrate TeX shell commands and (increasingly) HPC tools — this integration makes Madagascar the most powerful realization of the reproducible-research concept of which I am aware. It doesn’t solve all of the problems of RR — notably, the inability of software to keep itself maintained without human intervention — but it represents a quantum leap beyond other RR frameworks.”

Yang Liu, professor at the College of Geo-exploration Science and Technology, Jilin University, China, says, “I am using Madagascar software to implement new research ideas and provide reproducible examples for techniques in oil-gas exploration. For me the most attractive feature of the Madagascar open-source software is its reproducibility of computational modules, data-processing scripts, and research papers. Anyone could generate all examples and papers just by using several simple commands. The software package is also well maintained, and there is a group of developers who are continuously contributing their codes, which keeps the Madagascar software up to date. Many natural phenomena, including geologic events and geophysical data, are fundamentally nonstationary. So figuring out how to recover nonstationary signals from a noisy environment is a persistent problem in my field. We developed a 3D t-x-y adaptive prediction filter (APF) for random-noise attenuation in seismic exploration. The method is also able to deal with random noise in other fields, e.g., imaging processing. The core of the proposed method is based on existing source codes in the Madagascar environment. Therefore, we can implement the new theory in a short time. The computational modules and the corresponding paper are also reproducible in the updated Madagascar software package.”

Jeffrey Shragge, Woodside Professor of Computational Geoscience at the School of Earth and Environment/School of Physics, University of Western Australia, says, “My students and I use Madagascar for an integrated R & D environment on all aspects of my research. We develop and apply codes within the $RSFSRC/user/area and port these over to our R & D and local public (termed IVEC) HPC clusters. We are integrating Madagascar into our teaching through developing reproducible seismic labs. The package is unique because it integrates all of my R & D activities — code development, testing and verification, development of examples, easy extension to interface with HPC cluster systems, writing of manuscripts, etc. Most important, all of these activities can be undertaken in a reproducible environment. We have used codes written in the Madagascar package to address what type of microseismic signals we would expect from a large CO2 project being developed south of Perth. This required performing large-scale 3D elastic modeling on grids of the size 20003 using MPI+OpenMP codes developed using the Madagascar API. We have also used various Madagascar 2D/3D seismic-imaging codes for imaging geologic structures at this site to help prepare for pilot CO2 injection studies. We also use Madagascar for various near-surface geophysics investigations, including archaeological and forensic projects throughout Western Australia.”

SEG Working Workshop 2015 – Land 3D Processing

September 23, 2015 Celebration No comments

A working workshop to create processing results for Land 3D seismic data held on August 19-22, 2015. Working Workshops as opposed to “talking workshops” are meetings where the participants collaborate in small groups to develop new software code or to conduct computational experiments addressing a particular problem.

This workshop invited participants to create processing results on the small, open data, 3D land project (Teapot Dome 3D). Participants were encouraged to use their own proprietary or open software. Our goal was a final presentation describing the programs, parameters, and data (input and output) in sufficient detail so the results can be reproduced after the workshop. Participants were encouraged to download the Teapot Dome 3D data before the workshop.

The photograph at the top of this post shows some of the 25 participants from 16 organizations who participated in the event. They were from diverse backgrounds including students, post docs, academic staff, faculty, early career professionals, and senior professionals.

A GitHub repository was established to share processing scripts. Final presentations were made in a Lightning talk (5 minute presentation) session on the final day. Presentations were saved and are publicly available.

The agenda for the event was: Wednesday – Optional informal Tutorials Install software and Teapot Dome data Thursday – Working started Introductions, team formation, team work Lunch and learn – SeaView Team work, team check in Friday – continue work sessions Team work, check in, workshop dinner Saturday – wrap up and presentations Create presentation, lunch, lightning talks

An overview of some of the presentations follows.

Bjorn Olofsson was invited to on Thursday to make a lunch and learn presentation on SEAVIEW the viewer from the OpenSeaSeis package. Seaview supports multiple seismic formats including SeaSeis, SEGY, SU, and Madagascar. Bjorn described issues related to visualization including how to handle aliasing. See Bjorn’s presentation and the full distribution of his demo.

Huang, Liau, Yu described the workshop effort to edit bad traces on the Teapot Dome survey. They used the Python API in IPython notebooks to access the data. They used the Mayavi Python library for 3D visualization. They observed three types of noise and developed a Python script to build a mask to edit one type of noise traces. The presentation is available.

Merzlikin and Cvetkovic presented results using diffraction imaging and oriented filtering on the teapot dome project. They estimated a 3D dip field and used it for 3D diffraction and structure oriented smoothing. The presentation is available.

Worthy-Blackwell and Staal worked on Velocity interpolation and NMO. They interpolated the commercial velocity field using scipy.interpolate.Rbf and applied it to a small subset of the data using sfbinint3 and sfnmo.

Yangkang Chen applied noise suppression to the post stack volume and evaluated results on output and difference volumes.

Bill Symes applied a basic processing sequence with Seismic Unix. He approximated the processing sequence in Excell’s processing report from the original processing. His sequence was tpow, decon, agc, nmo, and stack. No surface consistant decon or scaling is available in SU, so he used single trace processing. He showed incremental improvement at each stage on midpoint gathers and stacks. He also checked for residual moveout on CMP gathers and made updates to the velocity field. This was the most complete processing sequence produced at the workshop. His presentation is available.

In summary, the workshop was attended by a good mixture of academics and professionals with a broad range of experience. Turnout for the optional tutorial day was much larger then expected, about 20. Most attendees used Madagascar, but the most complete processing sequence was applied using Seismic Unix. Madagascar and Seismic Unix were both developed as a platform for university research and many basic processes are missing. Applying a full processing sequence to 3D land data with these systems is a challenging task. Unfortunately nobody used a commercial processing system to help guide improvements to these academic systems. SeaView shows promise as a viewer to help explore your data. Participants enjoyed the opportunity to learn, contribute and network.

Let’s do it again next year using the Stratton 3D Land seismic survey. Contact me at seismic.working.workshop@gmail.com to help organize the event.


August 30, 2015 Celebration 1 comment

The first Madagascar school for advanced users took place in Qingdao, China, on August  8-9, 2015, and was hosted by the China University of Petroleum (East China).

The local organizers were Professor Qizhen Du (China University of Petroleum) and Dr. Gang Fang (Qingdao Institute of Marine Geology).

The school attracted more than 100 participants from 15 different organizations, including 10 Chinese universities. The program included not only lectures on different advanced topics in the use of Madagascar but also presentations from different users about their personal experiences with using Madagascar for geophysical research. The school materials are available on the website.

The school timing and location cleverly coincided with the famous Qingdao International Beer Festival, the biggest one of its kind in Asia.

Qingdao Beer Festival

Program of the month: sfmutter

July 10, 2015 Celebration No comments

sfmutter performs a mute operation, common in prestack seismic data processing.

The following example from bei/wvs/vscan shows a seismic CMP gather before and after a mute.

sfmutter multiples the input by a weight, which follows the equation

$$W(t,x) = \left\{\begin{array}{rl} 0 & \quad \mbox{for} \quad t – t_0 < p_0\,(x-x_0) \\ 1 & \quad \mbox{for} \quad t – t_0 > \Delta t + p_1\,(x-x_0) \end{array}\right.$$

The weight is smoothly interpolated in between the two regions. Different parameters appearing in this equation can be specified as inputs for sfmutter: t0= and x0= correspond to $t_0$ and $x_0$; tp= corresponds to $\Delta t$; slope0= and slopep= correspond to $p_0$ and $p_1$, respectively. The slope can be specified by velocity v0= such that $p_0=1/v_0$. By default, $p_1=p_0$.

The parameter half= indicates whether the horizontal axis corresponds to half-offset instead of the full offset (true by default). In case of irregular offsets, they can be provides in a file specified by offset=. The parameter abs= controls whether to use the absolute value of the offset in the calculation of the mute (true by default).

To make a hyperbolic mute, instead of the default linear mute, use hyper=y. In this case, the equation changes to

$$W(t,x) = \left\{\begin{array}{rl} 0 & \quad \mbox{for} \quad (t-t_0)^2 < p_0^2\,(x-x_0)^2 \\ 1 & \quad \mbox{for} \quad (t-t_0)^2 > (\Delta t)^2 + p_1^2\,(x-x_0)^2 \end{array}\right.$$

The following example from xjtu/mcaseislet/sep2 applies a hyperbolic mute for an SRME prediction of surface-related multiple reflections:

For inner mute, specified by inner=y, the signs in the equation are reversed. The following example from milano/taupvel/cmp applies an inner mute to a Radon-transformed CMP gather:

10 previous programs of the month:

Program of the month: sfintbin

June 10, 2015 Celebration No comments

sfintbin uses trace headers to arrange input traces in a 3-D cube.

The input to this program typically comes from reading a SEGY file with sfsegyread.

The following example from tccs/phase/boon3 shows a typical output:

sfintbin takes as an input a 2-D trace file and its corresponding trace header file (specified by head=). The header file is assumed to contain integer values. The output 3-D cube is generated according to trace header keys, which can be specified by name (xk=, yk=) or by number (xkey=, ykey=). In the example above, the parameters are xk=cdp and yk=fldr. If you are unsure which keys to use, try sfheaderattr. The range of values can be controlled with xmin=, xmax=, ymin=, ymax=.

sfintbin performs a very simple operation: it goes through the input file and assigns traces to the output according to their header keys. If there are several traces with the same x and y keys, the last one survives. If there are bins in the output with no input, they are filled with zeros. You can output the mask of empty traces using mask=. You can output the mapping of traces using map=. The latter is particularly useful for the inverse operation (mapping from 3-D back to 2-D), which can be performed with inv=y.

For an absent trace header key, which corresponds to a trace number in a gather, you can set xkey= to a negative number. For this to work, the input needs to be sorted in the corresponding gathers, which can be accomplished with sfheadersort.

To produce a 4-D output (for example, a set of gathers in 3-D), try sfintbin3.

10 previous programs of the month:


May 27, 2015 Celebration No comments

Madagascar passes a symbolic mark of one million lines of code. Black Duck Open Hub reports that

In a Nutshell, Madagascar… has had 12,444 commits made by 85 contributors representing 1,098,223 lines of code

Summer events

May 5, 2015 Celebration No comments

Two exciting events are being planned for this summer. The Madagascar School for Advanced Users is being planned for August 8-9, 2015. The school will take place in Qingdao, China, and will be hosted by the China University of Petroleum. Unlike previous Madagascar schools, the intended audience are not beginners but current users of Madagascar who want to learn more about advanced topics (parallel computing; graphical user interfaces; interfaces to C++, Python, and Matlab, etc.) More information is available at http://www.ahay.org/wiki/Qingdao2015. The Working Workshop on 3D Seismic Data Processing is being planned for August 19-22, 2015. The workshop will take place in Houston and will be hosted by Rice University. Unlike previous working workshops, the participations is not limited to Madagascar users. Users of other software packages with interest in seismic field data processing are encouraged to participate. The workshop is being organized by Karl Schleicher, with support of SEG. More information is available at http://www.ahay.org/wiki/SEG3DSeismicProcessingWorkingWorkshopHouston2015-Land3D

Program of the month: sfbox

May 1, 2015 Celebration No comments

sfbox draws a boxed baloon-style label, which is useful for annotating graphics.

The following example from sep/avo/imp annotates several reflectors on a seismic section:

sfbox has a large number of parameters to control the box appearance. The most important are: label= for the text on the label, x0= and y0 for the position of the pointer tip (in Vplot coordinates), xt= and yt= for the relative shift of the label with respect to the pointer tip.

The following example from tccs/timelapse/duri uses xt=0 and yt=0 to hide the pointer under the label.

Selecting proper parameters for the location of the label can be tricky. The task is made interactive by the vpannotate script. vpannotate takes a Vplot file as an input and adds an annotation to it. It can do it in the batch mode (when used with batch=y) or in an interactive mode (when used with batch=n). In the later case, the selected parameters are saved in a text file, which can be used as a parameter file for sfbox (or, alternatively, for vpannotate batch=y).

10 previous programs of the month

madagascar-1.7 released

April 13, 2015 Celebration No comments

The 1.7 stable release features 21 new reproducible papers and multiple other enhancements including improved tools for parallel computing developed at the Second Working Workshop.

According to the SourceForge statistics, the previous 1.5 stable distribution has been downloaded more than 4,000 times. The top country (with 24% of all downloads) was USA, followed by China, Colombia, Germany, and Brazil. According to Openhub.net (last updated in January 2015), the year of 2014 was a period of a high development activity, with 33 contributors making 1,876 commits to the repository (up 16% from the previous year). Openhub.net says that Madagascar “has a well established, mature codebase maintained by a very large development team with stable year-over-year commits” and estimated 239 man-years of effort (an estimated development cost of $13 million).