Editing Reproducibility (section)

=Reproducible science in practice=

* Some [http://sepwww.stanford.edu/doku.php?id=sep:research:reports Stanford Exploration Project reports] between 1990 and 2007
* [http://software.seg.org/ software.seg.org]: Even if the algorithms are available, the input data is also needed for a completely reproducible numerical experiment. Besides publishing code for papers in Geophysics, this SEG site also makes some datasets publicly available
* [http://www.agu.org/pubs/esupp_about.html AGU Electronic Data Supplements]: Since 1994, software and data were accepted, peer reviewed and made available through FTP. More information on the [http://www.agu.org/pubs/agu_elec.html AGU Electronic Services and Publications] page
* [http://itk.org/ The Insight Toolkit], which attempts to provide reference implementations of published algorithms in Medical Image Processing, and [http://www.insight-journal.org/ the Insight Journal], which allows authors to publish open articles which are automatically verified with CMake
* [http://lcavwww.epfl.ch/reproducible_research/ LCAV] &ndash; Audiovisual Communications Laboratory, Ecole Polytechnique Federale de Lausanne
* [http://www-stat.stanford.edu/~wavelab/ WaveLab]: Implementation of the concept of reproducibile research in Matlab from the Dept. of Statistics, Stanford University
* [http://lcavwww.epfl.ch/reproducible_research/ Reproducible Neurophysiological Data Analysis]: a setup from U. Paris 5 for reproducible research in neurophysiology with Sweave. Sweave itself is advertised as a [http://www.stat.umn.edu/~charlie/Sweave/ package for reproducible research]
* [http://drexel-coas-elearning.blogspot.com/2006/09/open-notebook-science.html Open Notebook Science]
* [http://www.reproducibleresearch.net/ reproducibleresearch.net]

The links above point to explicit attempts to bundle together whole scientific experiments, as described in scientific papers. It is worth noting that:
* It was not uncommon for scientific articles and books written during the late 1970's and early 1980's to contain printouts of the computer programs used. This alone helped clarify the algorithm and remove any doubts about the implementation. If input parameters were mentioned in the text and data was created synthetically, then the article can be considered reproducible for all practical purposes. The brevity of the computer programs of those times helped make them publishable in print and made them easy to understand, so in practice those results may be even more reproducible than a live working paper based on thousands of lines of poorly written code. Volumes of [http://www.segdl.org/geophysics/ Geophysics] with sections dedicated to computer programs are 45/1980(nr 3,7,11), 44/1979 (nr 12) and others [Note to self: complete list. The earliest example probably gains the title of earliest reproducibile experiment]
* Outside the scientific world there are many instances of using software to automatically re-create a document with figures whenever the underlying data changes. Internet searches on "automatic reporting" or "automatic report generation" yield useful information on such utilities used in finance, meteorology, computer system administration, medicine, government administration, etc.
* Some proprietary software packages have offered reproducibility capabilities:
** Matlab notebooks
** The whole Mathcad
** Mathematica may have "live document" capabilities