Editing
Reproducible computational experiments using SCons
(section)
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
===Tools for reproducible research=== The reproducible research system developed at Stanford is based on "make" (Stallman et al., 2004<ref>Stallman, R. M., R. McGrath, and P. D. Smith, 2004, GNU make: A program for directing recompilation: GNU Press.</ref>), a Unix software construction utility. Initially, SEP used "cake," a dialect of "make" (Nichols and Cole, 1989<ref>Nichols, D. and S. Cole, 1989, Device independent software installation with CAKE, ''in'' SEP-61, 341--344. Stanford Exploration Project.</ref>;Claerbout and Nichols, 1990<ref>Claerbout, J. F. and D. Nichols, 1990, Why active documents need cake, ''in'' SEP-67, 145--148. Stanford Exploration Project.</ref>;Claerbout, 1992b<ref>-------- 1992b, How to use Cake with interactive documents, ''in'' SEP-73, 451--460. Stanford Exploration Project.</ref>;Claerbout and Karrenbach, 1993<ref>Claerbout, J. F. and M. Karrenbach, 1993, How to use cake with interactive documents, ''in'' SEP-77, 427--444. Stanford Exploration Project.</ref>). The system was converted to "GNU make," a more standard dialect, by Schwab and Schroeder (1995<ref>Schwab, M. and J. Schroeder, 1995, Reproducible research documents using GNUmake, ''in'' SEP-89, 217--226. Stanford Exploration Project.</ref>). The "make" program keeps track of dependencies between different components of the system and the software construction targets, which, in the case of a reproducible research system, turn into figures and manuscripts. The author specifies the targets and commands for their construction in "makefiles," which serve as databases for defining source and target dependencies. A dependency-based system leads to rapid development because when one of the sources changes, only parts that depend on this source get recomputed. Buckheit and Donoho (1995<ref>Buckheit, J. and D. L. Donoho, 1995, Wavelab and reproducible research, ''in'' Wavelets and Statistics, volume '''103''', 55--81. Springer-Verlag.</ref>) based their system on MATLAB, a popular integrated development environment produced by MathWorks (Sigmon and Davis, 2001<ref>Sigmon, K. and T. A. Davis, 2001, MATLAB primer, sixth edition: Chapman & Hall.</ref>). While MATLAB is an adequate tool for prototyping numerical algorithms, it may not be sufficient for large-scale computations typical for many applications in computational geophysics. "Make" is a handy utility employed by thousands of software development projects. Unfortunately, it is not well designed from the perspective of user experience. "Make" employs an obscure and limited special language (a mixture of Unix shell and special-purpose commands), which often appears confusing to inexperienced users. According to Peter van der Linden, a software expert from Sun Microsystems (van der Linden, 1994<ref>van der Linden, P., 1994, Expert C programming: Prentice Hall.</ref>), <blockquote> "Sendmail" and "make" are two well-known programs that are pretty widely regarded as originally being debugged into existence. That's why their command languages are so poorly thought out and difficult to learn. It's not just you -- everyone finds them troublesome. </blockquote> The inconvenience of the "make" command language is also in its limited capabilities. The reproducible research system developed by Schwab et al. (2000<ref>Schwab, M., M. Karrenbach, and J. Claerbout, 2000, Making scientific computations reproducible: Computing in Science & Engineering, '''2''', 61--67.</ref>) includes not only custom "make" rules but also an obscure and hardly portable agglomeration of shell and Perl scripts that extend "make" (Fomel et al., 1997<ref>Fomel, S., M. Schwab, and J. Schroeder, 1997, Empowering SEP's documents, ''in'' SEP-94, 339--361. Stanford Exploration Project.</ref>). Several alternative systems for dependency-checking software construction have been developed in recent years. One of the most promising new tools is SCons, enthusiastically endorsed by Dubois (2003<ref>Dubois, P. F., 2003, Why Johnny can't build: Computing in Science & Engineering, '''5''', 83--88.</ref>). The SCons initial design won the Software Carpentry competition sponsored by Los Alamos National Laboratory in 2000 in the category of "a dependency management tool to replace make." Some of the main advantages of SCons are: *SCons configuration files are Python scripts. Python is a modern programming language praised for its readability, elegance, simplicity, and power (Rossum, 2000a<ref>Rossum, G. V., 2000a, Python reference manual: Iuniverse Inc.</ref>;Rossum, 2000b<ref>-------- 2000b, Python tutorial: Iuniverse Inc.</ref>). Scales and Ecke (2002<ref>Scales, J. A. and H. Ecke, 2002, What programming languages should we teach our undergraduates?: The Leading Edge, '''21''', 260--267.</ref>) recommend Python as the first programming language for geophysics students. *SCons offers reliable, automatic, and extensible dependency analysis and creates a global view of all dependencies—no more "make depend," "make clean," or multiple build passes of touching and reordering targets to get all the dependencies. *SCons has built-in support for many programming languages and systems, including C, C++, Fortran, Java, and LaTeX. *While "make" relies on timestamps to detect file changes (creating numerous problems on platforms with different system clocks), SCons uses a more reliable detection mechanism, employing MD5 signatures by default. It can detect changes not only in files but also in commands used to build them. *SCons provides integrated support for parallel builds. *SCons provides configuration support analogous to the "autoconf" utility for testing the environment on different platforms. *SCons is designed from the ground up as a cross-platform tool. It works equally well on POSIX systems (Linux, Mac OS X, Solaris, etc.) and Windows. *The stability of SCons is assured by an incremental development methodology utilizing comprehensive regression tests. *SCons is publicly released under a liberal open-source license<ref>As of this writing, SCons is in a beta version of 0.96, approaching the 1.0 official release. See http://www.scons.org/.</ref>. In this paper, we propose to adopt SCons as a new platform for reproducible research in scientific computing.
Summary:
Please note that all contributions to Madagascar are considered to be released under the GNU Free Documentation License 1.3 or later (see
My wiki:Copyrights
for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource.
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
English
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
View history
More
Search
Getting Madagascar
download
Installation
GitHub repository
SEGTeX
Introduction
Package overview
Tutorial
Hands-on tour
Reproducible documents
Hall of Fame
User Documentation
List of programs
Common programs
Popular programs
The RSF file format
Reproducibility with SCons
Developer documentation
Adding programs
Contributing programs
API demo: clipping data
API demo: explicit finite differences
Community
Conferences
User mailing list
Developer mailing list
GitHub organization
LinkedIn group
Development blog
Twitter
Slack
Tools
What links here
Related changes
Special pages
Page information