Skip to content

HPC GAP test failures

Jump to bottom Edit New page
Alexander Konovalov edited this page Mar 20, 2015 · 25 revisions

State of the GAP test suite in HPC-GAP

Below is an overview on the state of the test suite in HPC-GAP, with a list of known problems. In a few cases, names are listed indicating people who are looking into these issues (or at least were, during GAP Days Spring 2015).

The GAP test suite consists of several tests:

  • tst/testinstall.g
  • tst/testall.g
  • tests of the manual examples, packages loading, etc...
Test Main thread Background thread Main thread first, then background
testinstall.g 0 0 12
testall.g 0 119 (some because of missing packages) ?
Tutorial examples 9 (mostly harmless - randomness etc.)
Reference manuals examples 57 (mostly harmless - randomness etc.)

Of course in addition to all this, we need to write more codes to increase the overall code coverage.

To start with, we are looking at tst/testinstall.g first.

Main thread, run testinstall.g once

Running the GAP test suite once in the main thread by now works reasonably well.

Main thread, run testinstall.g twice

Running the GAP test suite twice in the main thread produces some harmless failures (objects are printed differently because more information about them is known), but this is also the case in regular GAP. These should be fixed.

Single non-main thread, run testinstall.g once

Running the GAP test suite once in the background thread by now works reasonably well (with no packages installed, except GAPDoc)

Multiple threads, run tests in at least two.

The next stage (apart from writing many more tests ...) now is to run tests in multiple threads, in all kinds of variations: Run a single test (or the whole test suite) in the main thread; then afterwards (or in parallel) in another thread. Or vice versa, etc.

Running individual tests with Test, you may compare input and output up to whitespaces to ignore trivial diffs caused by different formatting. To do this, use optional 2nd argument rec(compareFunction:="uptowhitespace"), for example:

Test("tst/ctblmono.tst",rec(compareFunction:="uptowhitespace"));

(in particular , this is the setting used by Read("tst/testinstallg")).

Running tests in multiple threads uncovers many new problems, which need to be fixed. Note that some of them are just due to re-running tests, i.e. they also occur in GAP when running a test twice. To fix these, the tests should be adjusted.

Here is a list of files which experience problems, created by first running Read("tst/testinstall.g"); in the main thread, then afterwards running it again in another thread:

  • comprvec.tst
  • ctblmoli.tst
  • ctblmono.tst
  • ctblsolv.tst Claus:works
  • grpmat.tst Claus: works after Max's collector fixes
  • grppc.tst
  • mapping.tst Claus: works after Max' collector fixes
  • morpheus.tst
  • relation.tst Claus: works
  • semigrp.tst
  • vspchom.tst

Who is working on what?

In order to avoid duplicate effort, please state here if you are working on a specific tst file:

Add a custom footer
Clone this wiki locally