Contributing new programs to Madagascar: Difference between revisions

From Madagascar
Jump to navigation Jump to search
Nick (talk | contribs)
m Removed image. It was illustrating idea of maintenance of old stuff, not of contribution/sharing
Sfomel (talk | contribs)
No edit summary
 
(20 intermediate revisions by 2 users not shown)
Line 2: Line 2:
# Follow the guide on [[Adding new programs to Madagascar]].
# Follow the guide on [[Adding new programs to Madagascar]].
# If your employer holds the copyright for your software, obtain a permission to distribute under a [http://www.gnu.org/copyleft/gpl.html GPL license] and  place a copyright and GPL license notice in each file with code.  
# If your employer holds the copyright for your software, obtain a permission to distribute under a [http://www.gnu.org/copyleft/gpl.html GPL license] and  place a copyright and GPL license notice in each file with code.  
# [http://sourceforge.net/account/registration/ Register at SourceForge].
# [https://github.com/ Sign up at GitHub].
# Find out who the project administrators are (Look for "Project Admins" on [http://sourceforge.net/projects/rsf/ this page]) and pass your SourceForge user name to one of them.
# Upload your directory to the [https://github.com/ahay/src Github repository] by making a pull request.
# Upload your directory to the repository using <tt>[http://svnbook.red-bean.com/en/1.0/re01.html svn add]</tt> and <tt>[http://svnbook.red-bean.com/en/1.1/re06.html svn commit]</tt>.
# Announce your changes in the release notes for the upcoming stable version ([https://github.com/ahay/src/blob/master/NEWS.txt $RSFSRC/NEWS.txt])
# Make sure to add reproducible examples of using your program under the <tt>MADAGASCAR/book</tt> tree. Create new directories if necessary and commit them to the repository. As a rule, programs without examples are not included in the release version.
# Make sure to add reproducible examples of using your program under the <tt>$RSFSRC/book</tt> tree. Create new directories if necessary and commit them to the repository. As a rule, programs without examples are not included in the release version.


Some additional useful information is included below.
Some additional useful information is included below.


==Style suggestions==
==Repository procedure suggestions==
Please try to:
Please try to:
* Do [http://en.wikipedia.org/wiki/Atomic_commit atomic commits].
* Do [http://en.wikipedia.org/wiki/Atomic_commit atomic commits].
* Use <tt>svn commit -m "your message here"</tt> to let others know what changed.
* If you use Subversion, run  <pre>svn commit -m "your message here"</pre> to let others know what changed.
** Alternatively, set the <tt>EDITOR</tt> or <tt>SVN_EDITOR</tt> environmental variables to your favorite editor and edit your message there.
** Alternatively, set the <tt>EDITOR</tt> or <tt>SVN_EDITOR</tt> environmental variables to your favorite editor and edit your message there.
* If you plan to do large-scale substantive changes, create a Subversion [http://en.wikipedia.org/wiki/Branching_(software) branch] by running
 
<pre>
For coding style suggestions, please see [[Adding_new_programs_to_Madagascar#Style_guide | the "Adding new programs" section]].
svn copy https://rsf.svn.sourceforge.net/svnroot/rsf/trunk https://rsf.svn.sourceforge.net/svnroot/rsf/yourbranchname
svn co https://rsf.svn.sourceforge.net/svnroot/rsf/yourbranchname
</pre>


==Backward compatibility features==
==Backward compatibility features==
Line 31: Line 28:
* In <tt>user/ivlad/ivlad.py</tt>: Everything in the <tt>have_subprocess=False</tt> branches
* In <tt>user/ivlad/ivlad.py</tt>: Everything in the <tt>have_subprocess=False</tt> branches
* The entire <tt>api/python/rsfbak.py</tt> (also needed on systems which do not have recent versions of numpy ''and'' SWIG)
* The entire <tt>api/python/rsfbak.py</tt> (also needed on systems which do not have recent versions of numpy ''and'' SWIG)
===Python 2.4 and older===
* try (try...except)... finally construct instead of try... except... finally in <tt>framework/rsfdoc.py</tt>
==Mitigating missing dependencies==
The main idea is to attempt to provide [http://en.wikipedia.org/wiki/Graceful_degradation graceful degradation] when facing a missing dependency. Quoting the Wikipedia page on fault-tolerant systems, this "enables a system to continue operating properly in the event of the failure of (or one or more faults within) some of its components. If its operating quality decreases at all, the decrease is proportional to the severity of the failure, as compared to a naively-designed system in which even a small failure can cause total breakdown."
==Reasons for the existence of certain features==
===Why pyc files are generated during the install step===
Python bytecode files, with the pyc extensions, are created during the <tt>scons install</tt> in order to:
* Register them in SCons' dependency tree, so that they are removed by <tt>scons -c install</tt>
* Avoid a situation in which a user tries to execute a Madagascar Python program, but creating the pyc file during a module import fails because of lack of write access to <tt>$RSFROOT/lib</tt>.


==cfortran.h==
==cfortran.h==
This source code file provides a machine-independent interface between C procedures, Fortran procedures and global data (more details [http://www-zeus.desy.de/~burow/cfortran/ on the website of its initial author]). It is a dependency of the F77 and F90 APIs, as well as of vplot. It is included in m8r in two places. Version 4.3 (2002) is still distributed through the website of the original author. Another version (fork?) is [http://root.cern.ch/viewvc/trunk/montecarlo/eg/inc/cfortran.h?view=log maintained at CERN] and distributed in the include directory of the interface to event generators in the Monte Carlo libraries in the [http://cernlib.web.cern.ch/cernlib/ CERN Program Library] (CERNlib). Debian distributes the Free Software part of this package as <tt>cernlib</tt>. Under Fedora, <tt>cfortran.h</tt> is included in <tt>cernlib-g77-devel</tt> and <tt>cernlib-devel</tt>, which happily overwrite each other's files and create copies of <tt>cfortran.h</tt> in two different locations deeply nested under <tt>/usr/include</tt>. CERNlib's Debian maintainer states that [http://people.debian.org/~kmccarty/cernlib/index.html "CERNlib is an ancient mass of mostly Fortran code"] and that [http://people.debian.org/~kmccarty/physics-software-rant.html "Most components of CERNLIB are completely broken on modern 64-bit architectures"], so m8r developers should be aware that CERNlib will probably be superseded at some point in the future by [http://en.wikipedia.org/wiki/ROOT ROOT], with upstream <tt>cfortran.h</tt> maintenance possibly continuing this way.
This source code file provides a machine-independent interface between C procedures, Fortran procedures and global data (more details [http://www-zeus.desy.de/~burow/cfortran/ on the website of its initial author]). It is a dependency of the F77 and F90 APIs, as well as of vplot. It is included in m8r in <tt>api/f90</tt> with a link in <tt>api/f77</tt>. Version 4.3 (2002) is still distributed through the website of the original author. Another version (fork?) is [http://root.cern.ch/viewvc/trunk/montecarlo/eg/inc/cfortran.h?view=log maintained at CERN] and distributed in the include directory of the interface to event generators in the Monte Carlo libraries in the [http://cernlib.web.cern.ch/cernlib/ CERN Program Library] (CERNlib). Debian distributes the Free Software part of this package as <tt>cernlib</tt>. Under Fedora, <tt>cfortran.h</tt> is included in <tt>cernlib-g77-devel</tt> and <tt>cernlib-devel</tt>, which happily overwrite each other's files and create copies of <tt>cfortran.h</tt> in two different locations deeply nested under <tt>/usr/include</tt>. CERNlib's Debian maintainer states that [http://people.debian.org/~kmccarty/cernlib/index.html "CERNlib is an ancient mass of mostly Fortran code"] and that [http://people.debian.org/~kmccarty/physics-software-rant.html "Most components of CERNLIB are completely broken on modern 64-bit architectures"], so m8r developers should be aware that CERNlib will probably be superseded at some point in the future by [http://en.wikipedia.org/wiki/ROOT ROOT], with upstream <tt>cfortran.h</tt> maintenance possibly continuing this way.


==Static code checks==
==Static code checks==
Madagascar contains the beginning of an implementation of static code checks for security vulnerabilities and coding mistakes with [http://www.splint.org/ Splint]. This is visible in the code through the presence of comments like <tt>/*@out@*/</tt> and <tt>/*@null@*/</tt>, which instruct Splint about the intent of different variables and function return values. Future plans include using also dynamic checking by [http://valgrind.org/ Valgrind] or IBM's [http://www-306.ibm.com/software/awdtools/purify/ Rational Purify].
Madagascar contains the beginning of an implementation of static code checks for security vulnerabilities and coding mistakes with [http://www.splint.org/ Splint]. This is visible in the code through the presence of comments like <tt>/*@out@*/</tt> and <tt>/*@null@*/</tt>, which instruct Splint about the intent of different variables and function return values. Future plans include using also dynamic checking by [http://valgrind.org/ Valgrind] or IBM's [http://www-306.ibm.com/software/awdtools/purify/ Rational Purify]. Madagascar code metrics are [http://www.ohloh.net/p/m8r/analyses/latest assessed periodically by Ohloh].
 
==Maintaining the maintenance tools==
Please document scripts used to maintain the Madagascar codebase, and save them in <tt>$RSFSRC/admin</tt>. Cleanup procedures may need to be repeated in the future, as authors are human and can make the same mistakes again.


==Information flow map==
==Information flow map==
Line 42: Line 52:
[[Image:M8rmap.png]]
[[Image:M8rmap.png]]
<references/>
<references/>
==Dual licensing Madagascar code==
When in a contractual work relationship, you may find it handy to reuse some of your own existing public code for which you own the copyright. This may also apply if you are doing contract work for a client on behalf of your employer, who owns the copyright.
If your code does not have any GPL-ed dependencies, and the copyright owner agrees to releasing the code under a different license than the GPL, then you are free to go ahead and do it. However, make sure that in every file you state:
* who is the copyright owner,
* what are the terms of the proprietary license that it is being released under,
* that the code has already been licensed to the public under the GNU GPL prior to this as part of the Madagascar package
It is important to specify all these items, lest years later, someone in your client's company finds the code, realizes that similar code is part of Madagascar, and wrongly believes that the copyright belonged to his company, and that you infringed it by releasing the code under the GPL.
Make sure that proper copyright and licensing statements are in place in case of dual licensing, even if you are on very good terms with your client, and are certain that they will not falsely accuse you of wrongdoing. Companies get acquired, employees and even owners leave companies, policies and business strategies change, and old contracts, licenses and legal documents get discarded or lost in a mass of old junk. Each source code file should contain all the relevant copyright and licensing information.

Latest revision as of 15:40, 30 August 2016

If you want to share your code with other Madagascar users:

  1. Follow the guide on Adding new programs to Madagascar.
  2. If your employer holds the copyright for your software, obtain a permission to distribute under a GPL license and place a copyright and GPL license notice in each file with code.
  3. Sign up at GitHub.
  4. Upload your directory to the Github repository by making a pull request.
  5. Announce your changes in the release notes for the upcoming stable version ($RSFSRC/NEWS.txt)
  6. Make sure to add reproducible examples of using your program under the $RSFSRC/book tree. Create new directories if necessary and commit them to the repository. As a rule, programs without examples are not included in the release version.

Some additional useful information is included below.

Repository procedure suggestions[edit]

Please try to:

  • Do atomic commits.
  • If you use Subversion, run
    svn commit -m "your message here"
    to let others know what changed.
    • Alternatively, set the EDITOR or SVN_EDITOR environmental variables to your favorite editor and edit your message there.

For coding style suggestions, please see the "Adding new programs" section.

Backward compatibility features[edit]

If you introduce or notice a feature that is used solely for backwards compatibility with an old version of a dependency, please document it here, so that the feature can be eliminated when the Madagascar community stops supporting that version of that dependency[1].

Python 2.2 and older[edit]

  • In framework/rsfdoc.py: Everything in the have_datetime_module=False branches

Python 2.3 and older[edit]

  • In configure.py: Everything in the have_subprocess=False branches
  • In configure.py: See code after comment "For Py 2.4 and up this can be done more elegantly..."
  • In user/ivlad/ivlad.py: Everything in the have_subprocess=False branches
  • The entire api/python/rsfbak.py (also needed on systems which do not have recent versions of numpy and SWIG)

Python 2.4 and older[edit]

  • try (try...except)... finally construct instead of try... except... finally in framework/rsfdoc.py

Mitigating missing dependencies[edit]

The main idea is to attempt to provide graceful degradation when facing a missing dependency. Quoting the Wikipedia page on fault-tolerant systems, this "enables a system to continue operating properly in the event of the failure of (or one or more faults within) some of its components. If its operating quality decreases at all, the decrease is proportional to the severity of the failure, as compared to a naively-designed system in which even a small failure can cause total breakdown."

Reasons for the existence of certain features[edit]

Why pyc files are generated during the install step[edit]

Python bytecode files, with the pyc extensions, are created during the scons install in order to:

  • Register them in SCons' dependency tree, so that they are removed by scons -c install
  • Avoid a situation in which a user tries to execute a Madagascar Python program, but creating the pyc file during a module import fails because of lack of write access to $RSFROOT/lib.

cfortran.h[edit]

This source code file provides a machine-independent interface between C procedures, Fortran procedures and global data (more details on the website of its initial author). It is a dependency of the F77 and F90 APIs, as well as of vplot. It is included in m8r in api/f90 with a link in api/f77. Version 4.3 (2002) is still distributed through the website of the original author. Another version (fork?) is maintained at CERN and distributed in the include directory of the interface to event generators in the Monte Carlo libraries in the CERN Program Library (CERNlib). Debian distributes the Free Software part of this package as cernlib. Under Fedora, cfortran.h is included in cernlib-g77-devel and cernlib-devel, which happily overwrite each other's files and create copies of cfortran.h in two different locations deeply nested under /usr/include. CERNlib's Debian maintainer states that "CERNlib is an ancient mass of mostly Fortran code" and that "Most components of CERNLIB are completely broken on modern 64-bit architectures", so m8r developers should be aware that CERNlib will probably be superseded at some point in the future by ROOT, with upstream cfortran.h maintenance possibly continuing this way.

Static code checks[edit]

Madagascar contains the beginning of an implementation of static code checks for security vulnerabilities and coding mistakes with Splint. This is visible in the code through the presence of comments like /*@out@*/ and /*@null@*/, which instruct Splint about the intent of different variables and function return values. Future plans include using also dynamic checking by Valgrind or IBM's Rational Purify. Madagascar code metrics are assessed periodically by Ohloh.

Maintaining the maintenance tools[edit]

Please document scripts used to maintain the Madagascar codebase, and save them in $RSFSRC/admin. Cleanup procedures may need to be repeated in the future, as authors are human and can make the same mistakes again.

Information flow map[edit]

The vector graphics original of this map in Dia format is available upon request through the rsf-devel mailing list.

  1. This could be more elegantly done by using a special string to flag a backwards compatibility comment in the code, then have the documentation builder build this list automatically. This would make it more likely for this page to be kept in synch with the codebase.

Dual licensing Madagascar code[edit]

When in a contractual work relationship, you may find it handy to reuse some of your own existing public code for which you own the copyright. This may also apply if you are doing contract work for a client on behalf of your employer, who owns the copyright.

If your code does not have any GPL-ed dependencies, and the copyright owner agrees to releasing the code under a different license than the GPL, then you are free to go ahead and do it. However, make sure that in every file you state:

  • who is the copyright owner,
  • what are the terms of the proprietary license that it is being released under,
  • that the code has already been licensed to the public under the GNU GPL prior to this as part of the Madagascar package

It is important to specify all these items, lest years later, someone in your client's company finds the code, realizes that similar code is part of Madagascar, and wrongly believes that the copyright belonged to his company, and that you infringed it by releasing the code under the GPL.

Make sure that proper copyright and licensing statements are in place in case of dual licensing, even if you are on very good terms with your client, and are certain that they will not falsely accuse you of wrongdoing. Companies get acquired, employees and even owners leave companies, policies and business strategies change, and old contracts, licenses and legal documents get discarded or lost in a mass of old junk. Each source code file should contain all the relevant copyright and licensing information.