Automatic Testing - Revision history

Sfomel: /* 4. Summarize tests */

2014-04-15T14:59:35Z

4. Summarize tests

← Older revision		Revision as of 14:59, 15 April 2014
Line 37:		Line 37:

	===4. Summarize tests===		===4. Summarize tests===
	<bash>sftestlist outfile=testlist.txt book</~~bash~~>		<syntaxhighlight lang="bash">sftestlist outfile=testlist.txt book</syntaxhighlight>

	Step 4 runs a different script called '''sftestlist'''. This script just collects data from all the <tt>.rsftest</tt> files and makes a summary.		Step 4 runs a different script called '''sftestlist'''. This script just collects data from all the <tt>.rsftest</tt> files and makes a summary.

Sfomel: /* 3. Compare the results */

2014-04-15T14:59:07Z

3. Compare the results

← Older revision		Revision as of 14:59, 15 April 2014
Line 31:		Line 31:

	===3. Compare the results===		===3. Compare the results===
	<bash>rm book///*/.rsftest		<syntaxhighlight lang="bash">rm book///*/.rsftest
	sfbooklist size=1024 list=filter skipfile=admin/skiplist.txt command="sffiglist rsftest=y list=none > list.log" book</~~bash~~>		sfbooklist size=1024 list=filter skipfile=admin/skiplist.txt command="sffiglist rsftest=y list=none > list.log" book</syntaxhighlight>

	Step 3 is like Step 2 except for a different command string. This command string runs the script '''sffiglist''' in each directory to compare the <tt>*.vpl</tt> files in the <tt>Fig</tt> directory with those in the lock directory. A summary of the comparison is put into an <tt>.rsftest</tt> file in each directory.		Step 3 is like Step 2 except for a different command string. This command string runs the script '''sffiglist''' in each directory to compare the <tt>*.vpl</tt> files in the <tt>Fig</tt> directory with those in the lock directory. A summary of the comparison is put into an <tt>.rsftest</tt> file in each directory.

Sfomel: /* 2. Run the examples */

2014-04-15T14:58:43Z

2. Run the examples

← Older revision		Revision as of 14:58, 15 April 2014
Line 24:		Line 24:

	===2. Run the examples===		===2. Run the examples===
	<bash>sfbooklist size=1024 list=filter skipfile=admin/skiplist.txt timer=file command="scons >scons.log 2>&1" book</~~bash~~>		<syntaxhighlight lang="bash">sfbooklist size=1024 list=filter skipfile=admin/skiplist.txt timer=file command="scons >scons.log 2>&1" book</syntaxhighlight>

	Step 2 is the same as Step 1 except I also specify a command string to run in each directory. The command string runs <tt>scons</tt> to update the examples and capture the console output in log files. There is also an option '''timer=file''' which appends a single line execution-time summary to a .rsftimer file in each directory. The other options for timer are <tt>none</tt> and <tt>log</tt> which sends the summary to the console instead of a file.		Step 2 is the same as Step 1 except I also specify a command string to run in each directory. The command string runs <tt>scons</tt> to update the examples and capture the console output in log files. There is also an option '''timer=file''' which appends a single line execution-time summary to a .rsftimer file in each directory. The other options for timer are <tt>none</tt> and <tt>log</tt> which sends the summary to the console instead of a file.

Sfomel: /* 1. Get an inventory */

2014-04-15T14:58:17Z

1. Get an inventory

← Older revision		Revision as of 14:58, 15 April 2014
Line 14:		Line 14:

	===1. Get an inventory===		===1. Get an inventory===
	<bash>sfbooklist size=1024 public=y list=filter skipfile=admin/skiplist.txt book</~~bash~~>		<syntaxhighlight lang="bash">sfbooklist size=1024 public=y list=filter skipfile=admin/skiplist.txt book</syntaxhighlight>

	This step is not really necessary, but I do it to get a quick list of all the examples and a list of the subset that will be tested. I usually capture the output in a file and do a diff with the previous run. That way I will notice when new examples have been added to book since the last time I ran the tests.		This step is not really necessary, but I do it to get a quick list of all the examples and a list of the subset that will be tested. I usually capture the output in a file and do a diff with the previous run. That way I will notice when new examples have been added to book since the last time I ran the tests.

Sfomel: /* 1. Get an inventory */

2014-04-15T14:57:33Z

1. Get an inventory

← Older revision		Revision as of 14:57, 15 April 2014
Line 18:		Line 18:
	This step is not really necessary, but I do it to get a quick list of all the examples and a list of the subset that will be tested. I usually capture the output in a file and do a diff with the previous run. That way I will notice when new examples have been added to book since the last time I ran the tests.		This step is not really necessary, but I do it to get a quick list of all the examples and a list of the subset that will be tested. I usually capture the output in a file and do a diff with the previous run. That way I will notice when new examples have been added to book since the last time I ran the tests.

	The filters <tt>size= public= skipfile=</tt> limit the collection of examples to be executed. The '''size=''' filter skips any example that makes more than 1 GB of data. '''public=y''' means it will run examples that download public data from the server (the default is <tt>public=n</tt>). The default options for '''local=''' and '''private=''' are both <tt>n</tt>, so examples that use that kind of external data will be skipped. '''skipfile=''' specifies a list of examples by name that will be skipped. An example skipfile is [http://~~rsf.svn.~~sourceforge.net/~~viewvc~~/rsf/trunk/admin/skiplist.txt~~?revision=6224&view=markup~~ RSFSRC/admin/skiplist.txt], it specifies examples I can't run because of stuff I haven't installed and others that take a long time to run on my laptop. There is also a default filter to skip any example that does not yet have an .rsfproj file.		The filters <tt>size= public= skipfile=</tt> limit the collection of examples to be executed. The '''size=''' filter skips any example that makes more than 1 GB of data. '''public=y''' means it will run examples that download public data from the server (the default is <tt>public=n</tt>). The default options for '''local=''' and '''private=''' are both <tt>n</tt>, so examples that use that kind of external data will be skipped. '''skipfile=''' specifies a list of examples by name that will be skipped. An example skipfile is [http://sourceforge.net/p/rsf/code/HEAD/tree/trunk/admin/skiplist.txt $RSFSRC/admin/skiplist.txt], it specifies examples I can't run because of stuff I haven't installed and others that take a long time to run on my laptop. There is also a default filter to skip any example that does not yet have an .rsfproj file.

	Running this step will tell you which examples pass these set of filters. The following options will run all examples:		Running this step will tell you which examples pass these set of filters. The following options will run all examples:
	<bash>sfbooklist rsfproj=both public=y private=y local=y book</~~bash~~>		<syntaxhighlight lang="bash">sfbooklist rsfproj=both public=y private=y local=y book</syntaxhighlight>

	===2. Run the examples===		===2. Run the examples===

Sfomel: /* Quick Test */

2014-04-15T14:55:31Z

Quick Test

← Older revision		Revision as of 14:55, 15 April 2014
Line 3:		Line 3:
	To run a "quick" test run this command from <tt>$RSFSRC</tt>:		To run a "quick" test run this command from <tt>$RSFSRC</tt>:

	<bash>admin/quick_test.csh</~~bash~~>		<syntaxhighlight lang="bash">admin/quick_test.csh</syntaxhighlight>

	This script currently (June 19, 2010) runs a set of 147 examples in <tt>$RSFSRC/book</tt> that make 4GB of total data. They run in about 2 minutes total on a 2.4 GHz Intel Core 2 Duo MacBook when nothing needs to be updated, and 15 minutes when everything is rebuilt from scratch. Your mileage will vary.		This script currently (June 19, 2010) runs a set of 147 examples in <tt>$RSFSRC/book</tt> that make 4GB of total data. They run in about 2 minutes total on a 2.4 GHz Intel Core 2 Duo MacBook when nothing needs to be updated, and 15 minutes when everything is rebuilt from scratch. Your mileage will vary.

	~~Seven~~ of the 147 examples do not pass all the tests on my system~~, perhaps because of bugs not yet found :-( These examples are:~~		Some of the 147 examples do not pass all the tests on my system. Your mileage will probably vary on this as well. Indeed, I will be curious to see how the tests work on your system (hint, hint :-) Post your testing report on [https://lists.sourceforge.net/lists/listinfo/rsf-devel rsf-devel].

	~~# book/gee/mda/levint~~
	~~# book/jsg/lpf/lpf~~
	~~# book/sep/bspl/plane3~~
	~~# book/sep/pwd/alias~~
	~~# book/sep/pwd/hole~~
	~~# book/sep/pwd/signoi~~
	~~# book/sep/stack/miginv~~

	Your mileage will probably vary on this as well. Indeed, I will be curious to see how the tests work on your system (hint, hint :-) Post your testing report on [https://lists.sourceforge.net/lists/listinfo/rsf-devel rsf-devel].

	==Explanation of different steps==		==Explanation of different steps==

Sfomel: /* Explanation of different steps */ testlist

2010-06-24T21:43:37Z

Explanation of different steps: testlist

← Older revision		Revision as of 21:43, 24 June 2010
Line 21:		Line 21:
	==Explanation of different steps==		==Explanation of different steps==

	The following commands use [http://www.ahay.org/RSF/sfbooklist.html sfbooklist] ~~and~~ [http://www.ahay.org/RSF/sffiglist.html sffiglist]. Run them from <tt>$RSFSRC</tt>.		The following commands use [http://www.ahay.org/RSF/sfbooklist.html sfbooklist], [http://www.ahay.org/RSF/sffiglist.html sffiglist], and [http://www.ahay.org/RSF/sftestlist.html sftestlist]. Run them from <tt>$RSFSRC</tt>.

	===1. Get an inventory===		===1. Get an inventory===

Sfomel: /* 2. Run the examples */

2010-06-24T21:42:43Z

2. Run the examples

← Older revision		Revision as of 21:42, 24 June 2010
Line 34:		Line 34:

	===2. Run the examples===		===2. Run the examples===
	<bash>sfbooklist size=1024 list=filter skipfile=admin/skiplist.txt timer=file command="scons 2>&1 ~~>scons.log~~" book</bash>		<bash>sfbooklist size=1024 list=filter skipfile=admin/skiplist.txt timer=file command="scons >scons.log 2>&1" book</bash>

	Step 2 is the same as Step 1 except I also specify a command string to run in each directory. The command string runs <tt>scons</tt> to update the examples and capture the console output in log files. There is also an option '''timer=file''' which appends a single line execution-time summary to a .rsftimer file in each directory. The other options for timer are <tt>none</tt> and <tt>log</tt> which sends the summary to the console instead of a file.		Step 2 is the same as Step 1 except I also specify a command string to run in each directory. The command string runs <tt>scons</tt> to update the examples and capture the console output in log files. There is also an option '''timer=file''' which appends a single line execution-time summary to a .rsftimer file in each directory. The other options for timer are <tt>none</tt> and <tt>log</tt> which sends the summary to the console instead of a file.

Sfomel: /* 1. Get an inventory */

2010-06-22T15:21:33Z

1. Get an inventory

← Older revision		Revision as of 15:21, 22 June 2010
Line 30:		Line 30:
	The filters <tt>size= public= skipfile=</tt> limit the collection of examples to be executed. The '''size=''' filter skips any example that makes more than 1 GB of data. '''public=y''' means it will run examples that download public data from the server (the default is <tt>public=n</tt>). The default options for '''local=''' and '''private=''' are both <tt>n</tt>, so examples that use that kind of external data will be skipped. '''skipfile=''' specifies a list of examples by name that will be skipped. An example skipfile is [http://rsf.svn.sourceforge.net/viewvc/rsf/trunk/admin/skiplist.txt?revision=6224&view=markup RSFSRC/admin/skiplist.txt], it specifies examples I can't run because of stuff I haven't installed and others that take a long time to run on my laptop. There is also a default filter to skip any example that does not yet have an .rsfproj file.		The filters <tt>size= public= skipfile=</tt> limit the collection of examples to be executed. The '''size=''' filter skips any example that makes more than 1 GB of data. '''public=y''' means it will run examples that download public data from the server (the default is <tt>public=n</tt>). The default options for '''local=''' and '''private=''' are both <tt>n</tt>, so examples that use that kind of external data will be skipped. '''skipfile=''' specifies a list of examples by name that will be skipped. An example skipfile is [http://rsf.svn.sourceforge.net/viewvc/rsf/trunk/admin/skiplist.txt?revision=6224&view=markup RSFSRC/admin/skiplist.txt], it specifies examples I can't run because of stuff I haven't installed and others that take a long time to run on my laptop. There is also a default filter to skip any example that does not yet have an .rsfproj file.

	Running this step will tell you which examples pass these set of filters.		Running this step will tell you which examples pass these set of filters. The following options will run all examples:
			<bash>sfbooklist rsfproj=both public=y private=y local=y book</bash>

	===2. Run the examples===		===2. Run the examples===

Sfomel: /* Quick Test */

2010-06-22T15:20:23Z

Quick Test

← Older revision		Revision as of 15:20, 22 June 2010
Line 17:		Line 17:
	# book/sep/stack/miginv		# book/sep/stack/miginv

	Your mileage will probably vary on this as well. Indeed, I will be curious to see how the tests work on your system (hint, hint :-) Post your testing report on rsf-devel.		Your mileage will probably vary on this as well. Indeed, I will be curious to see how the tests work on your system (hint, hint :-) Post your testing report on [https://lists.sourceforge.net/lists/listinfo/rsf-devel rsf-devel].

	==Explanation of different steps==		==Explanation of different steps==