Adding new programs to Madagascar

Whether you want to share your work with the world or keep it for yourself, madagascar is fully extensible. The following instructions explain the simplest way to add your own programs to the package.

How to add your program

1. Create a directory for yourself or your group under $RSFSRC/user, where $RSFSRC is the top source directory. Put your programs there.
2. The following conventions are adopted:
   • Files containing main programs start with M, i.e. Mmyprog.c or Mmyprog.py. Each main program file should start with a short one-line comment describing its purpose.
   • Files containing subroutines start with small letters. A header/interface file myprog.h is automatically generated from myprog.c. By using header files, dependencies between different functions will be figured out automatically. Include a comment of the form /*^*/ after any block of text that you want to be included in the header file. Such blocks must not contain any all-whitespace lines. Include a comment of the form /*< My function description >*/ after a function definition if you want its interface included in the header file. It is a good habit to comment interfaces to all your functions.
3. Create a SConstruct file in your directory by following examples from other user directories. Inside the triple quotation marks after progs=, replace the names of their programs with the names of your programs with spaces between the names.
4. Note that running scons inside your user directory compiles programs with debugging flags to make them suitable for debugging with common debuggers such as gdb, dbx, or TotalView. Running scons install inside $RSFSRC will compile your programs with optimization flags and install them in $RSFROOT.

How to share your program with others

Follow the guide on Contributing new programs to Madagascar.

How to document your program

There are three levels of documentation. From the most simple to the most complex, they are: in-code documentation, the Wiki, and reproducible papers.

In-code documentation

Of course, do comment your code -- focusing on "why" (the code itself describes the "how"). Keep comments sparse so that as much code as possible can be kept in mind at the same time by the human reader. Use the other levels for more information.

If you follow a certain syntax for some of your comments, then the scripts in the Madagascar framework will be able to automatically produce a self-doc page for your program. This page will be displayed in the terminal (using basic text-mode formatting) when the program is invoked without any arguments, and a HTML version will be automatically generated in $RSFDOC (default: $RSFROOT/doc). The self-doc page has the following sections: Name, Description, Synopsis, Comments, Parameters, Used in, Source and Version.

Self-doc for C programs

To automatically populate the Description and Comments sections, use the following syntax. Start the file with:

/* Short description line
Comments here blablabla lorem ipsum dolores sit amet...

You can use several paragraphs for comments, no problem.*/

/* Copyright notice */

Notice that: (1) the description line is the first line in the file; (2) Comments are closed and then open again on a new line for Copyright (so that Copyright does not show up in the documentation).

To make input parameters and their default values (if they exist) automatically appear in the Synopsis and Parameters sections, make sure to read the parameters in the code like in the following example:

if (!sf_getbool("su",&su)) su=false;
/* y if input is SU, n if input is SEGY */

Notice that there are no complex logical operations inside the if, that there is no space between the equal sign and the right and left hand, and that a short comment follows immediately. If by necessity the parameter is read in a complex fashion, use angular brackets inside the short line following the read block, i.e.:

... sf_getfloat ... complex stuff ...
/*< parameter_name parameter meaning >*/

Self-doc for Python programs

To automatically populate the Description page, and Comments sections, use the following syntax. Start the file with:

#! /usr/bin/env python
'''My one-line description of this program
If I have comments, put them in the lines below.

I can use several paragraphs for that'''

# Copyright notice...

Then, make sure to

import rsfprog

To make input parameters and their default values (if they exist) automatically appear in the Synopsis and Parameters sections, make sure to follow each argument read with a comment, and call selfdoc if no argument was given. Here is an example from user/ivlad/Mpclip.py:

    par = rsf.Par(argv)

    inp = par.string('inp') # input file
    out = par.string('out') # output file
    if None in (inp, out):
        rsfprog.selfdoc()   # self-doc
        return error

    verb = par.bool('verb', False) # if y, print system commands, outputs

Self-doc for Fortran 90 programs

To automatically populate the Description section (Comments is not available), use the following syntax. Start the file with:

! One-line short description, leave a blank line after it

! Copyright notice...

To make input parameters and their default values (if they exist) automatically appear in the Synopsis and Parameters sections, make sure to read the parameters in the code like in the following example:

call from_par( "n1", nt, 512 ) ! Param description on same line or first line after

Wiki documentation

The self-doc is good as a reminder of what the parameters mean and what their default values are, but more is usually needed for someone who never used the tool. This is what the Guide to programs is for. Whenever you change or create a program, make sure to add a section for it in there too, if possible with an implementation section. Documenting programs that already exist is also a great idea.

You may have noticed that each program section on that page starts with a table that has the same content as the self-doc. That is created automatically the following way. Say you want to create this table for sfattr. At a command line, type:

sfdoc -m $HOME attr

This will create the file $HOME/sfattr.wiki. Its content can be copied and pasted into the Madagascar wiki, as they already are in the MediaWiki format. Another directory can be used instead of $HOME.

If your program/script reads an environment variable, make sure you check the list of environment variables used by Madagascar and add your new variable or your peculiar usage of an existing variable.

After creating the appropriate section in the Guide to programs, add your program name without the sf prefix to the docprogs list at the beginning of RSFSRC/framework/rsf/doc.py . This will ensure that a link to the wiki section is created in the auto-generated HTML program page in $RSFDOC.

Add your program to the Task-centric program list so that it can be found by those users who do not already know they need it.

Reproducible documents

Madagascar programs are tested and illustrated by means of reproducible documents. See the Guide to SCons interface for reproducible computations for a brief explanation of what a reproducible paper is and how to produce one.

Style guide

• Use components (macros, types, functions) from the Madagascar library to maximize component reuse and to make sure you do not reinvent the wheel. Be familiar with the Library Reference.
• User interface convention: if your program provides a verbosity option, use verb=n [y/n]
• Do not keep commented code in your program. This does not refer to writing comments about code, but to commenting out actual code. You can always use the version control system to retrieve the older version
• Initialize all pointers (including variables of type sf_file) with NULL
• If you have one or more calls to sf_input in your program, call sf_close before exit to avoid file leaks (temporary files left in $DATAPATH after an operation involving pipes)

How to test your program

The testing mechanism is semi-automatic. Testing can be done for any program that is used in a reproducible document in the RSF/book directory. To test whether the change you made in a program called, say, sfmyprog introduced any new bugs, make sure you have the latest set of stored figures from the figures repository and do the following:

1. cd RSF/book
2. Run

   scons sfmyprog.test

   If the error messages are not helpful you can omit them from the console, but keep them in a file like this (sh):

   scons sfmyprog.test 2> sfmyprog.test.err

   or like this (csh):

   (scons sfmyprog.test > /dev/stdin) >& sfmyprog.test.err

3. If testing fails, find the directory and figure where failure happened from the "Comparing ... and ..." statement just before the beginning of the error messages. The directory name will appear before that in a line " Testing in ...". cd to that directory. Let's say the figure that failed is called myfig.vpl. Run

   scons myfig.flip

   That will open a graphical display window that will quickly alternate between the stored figure and your figure.
   • If the figures look identical, then it means the differences are due to rounding errors and other insignificant differences between platforms (work is done on improving the comparison mechanism). Load your figure into your copy of the figures repository with

     scons myfig.lock

   • If you see differences between the two figures, then it is time to debug your program.

Tips and tricks

Backups

You may want to back up only your developer directory, not the entire Madagascar source tree. If you do not want to or do not have the permissions to change the backup scripts, you will need to make your RSF user directory a link to somewhere else. To get that to work, you need to modify your username/SConstruct. First thing in the file, define

RSF_src_root = '/path/to/madagascar/source/root/'

Then, replace ../.. throughout the SConstruct with

os.path.join(RSF_src_root,'whatever_followed_after_dotdots')

External libraries

To link your programs to an external library, add to the env.Prepend statement in username/Sconstruct the paths to its include dir, lib dir and the name of the library you are linking to.

Emacs customization

If you use Emacs, the following might be helpful.

• Install the python mode (if you don't have it installed already) and put

<lisp> (setq auto-mode-alist

     (cons '("SConstruct" . python-mode) auto-mode-alist))

</lisp>

in your .emacs file to enable it on SConstruct files.

• Put

<lisp> (set-default 'compile-command "scons") </lisp>

in your .emacs file to have scons as the default compile command.

• Use

<lisp> (add-hook 'c-mode-common-hook

  '(lambda () (c-set-style "linux")
       (c-set-offset 'case-label 4)
       (setq c-basic-offset 4)))

</lisp>

to make sure that the identation in C files follows the previously adopted convention.

Vim customization

To enable syntax highlighting on SConstruct files in vim, add the following to your ~/.vimrc file:

<bash> au BufRead,BufNewFile SConstruct setfiletype python

syntax enable </bash>