Editing RSF Comprehensive Description (section)

==RSF components==
The information to be encoded in a RSF dataset, described in the previous section, consists of two distinct parts: the metainformation and the data sequence. Depending on the context, the two parts can be in a single file, in a stream following each other, or in two separate files. So, depending the context, RSF can be a file format, a protocol or a metaformat &ndash; hence the preference for the weaselly term "dataset". This section defines the context-independent content of each of the two parts, and the sections that follow shows how they come together in various contexts.

===ASCII header===
The previous section mentioned that each program will record metainformation "to the extent that is technically practical". The current interpretation of this statement is that in the case of a linear workflow (i.e. <tt>prog1 | prog2 | prog3...</tt>) all of them will record the metainformation. However, in the case of a merged workflow (i.e. <tt>prog1 > file1.rsf; prog2 > file2.rsf; prog3 file1 file2 >file3.rsf </tt>) only the metainformation entries from one file (usually the one going into stdin) is kept. 

Using the nomenclature described in the previous section, a metainformation entry made by a program is formatted as follows:

<pre>
prog dir: user@host datetime

        in="pointer"
        data_format="format_type"
        esize=esize
        n1=nelem_axis1
        o1=orig_axis1
        d1=sampl_axis1
        unit1="unit_axis1"
        label1="label_axis1"

</pre>

Datasets of dimensions higher than 1 will have values for parameters describing the other axes such as n2, n3, etc.

The header must contain only [http://en.wikipedia.org/wiki/ASCII#ASCII_printable_characters ASCII printable characters].

All entries must be of the form: <tt>key=val</tt>, with no spaces adjacent to the equal sign. Lines that 

The mandatory fields are those in the first line, <tt>in</tt>, <tt>data_format</tt>, <tt>esize</tt> and <tt>n1</tt>. These are the minimum necessary for a sequence of data values to be read. The first line of the recording (<tt>prog dir: user@host datetime</tt>) is written by all Madagascar programs, but is currently not used in any way (i.e. it is not mandatory). Indentation of the fields after the first line and the presence of the blank lines are not mandatory either.

The order of the entries after the first line is not mandatory. 

The ''pointer'' can be either a path to a separate file holding the data and only the data, or the string <tt>stdin</tt> (when the RSF dataset is contained in a single file or transmitted over a stream).

The valid values of ''format'' are <tt>native</tt>, <tt>xdr</tt> and <tt>ascii</tt>, and the valid values of type can be <tt>short</tt>, <tt>int</tt>, <tt>float</tt>, <tt>double</tt>, <tt>complex</tt>, <tt>uchar</tt>, <tt>byte</tt>. These correspond to the similarly-named C language data types. The <tt>data_format</tt> with the best support and highest amount of optimization throughout Madagascar is ''native_float''.

Given the encodings currently implemented in madagascar, <tt>esize</tt> can be 1, 2, 4 or 8.

Axis length values (n1, n2, ... n9) must be greater than zero. It is allowed to not specify axis length values of higher dimensions than the intrinsic dimensionality of the data. For example, for a 1000x20x500 data cube, it is allowed to not specify any values for n4, n5, ... n9. It is also allowed to set the higher-dimension axis length values to 1. Gaps in the axis length sequence are not allowed (example: if n4 is given a value greater than one, then n1, n2 and n3 must be specified, even if their values are set to one.

Programs write to the ASCII header only those non-compulsory fields whose values have been modified. For example, a program performing summation of a 3-dimensional array over axis 3 will write to the header only <tt>n3=1</tt>, but not the values for <tt>n1</tt> and <tt>n2</tt>, which have not been changed. Madagascar programs read the ASCII headers from the beginning to the end, and overwrite existing fields if new values appear.

===Data sequence===
Like arrays stored in computer memory, the sequence of numerical values is ordered in progressive order along consecutive dimensions. For example, the array 
<pre>
a11 a12 a13
a21 a22 a23
a31 a32 a33
</pre>
will be written as:
<pre>
a11 a21 a31 a12 a22 a32 a13 a23 a33
</pre>
A array with three dimensions will be ordered as follows:
<pre>
a111 a211 a311 a121 a221 a321 a131 a231 a331 a112 a212 a312 a122 a222 a322 a132 a232 a332
</pre>
In this example dimension 1 is defined as the column direction, dimension 2 as the line direction, and dimension 3 as the "out of page" direction. This example follows the linear algebra / Fortran convention. It must be noticed that of only the reality of ordering of numerical values in the file is of substance; the imaginary representation with "vertical" and "horizontal" directions is not (the C language actually uses an inverse order for the dimensions). Thus, dimension 1 in a RSF dataset is in general defined as the dimension along which the first two elements in the data file are organized, then dimension 2 as the dimension adjacent to it, and so on.