Testing your docrepo¶
The module ferenda.testutil
contains an assortment of
classes and functions that can be useful when testing code written
against the Ferenda API.
Extra assert methods¶
The FerendaTestCase
is intended to be
used by your unittest.TestCase
based testcases. Your
testcase inherits from both TestCase
and FerendaTestCase
, and
thus gains new assert methods:
Method | Description |
---|---|
assertEqualGraphs() |
Compares two
Graph objects |
assertEqualXML() |
Compares two XML documents (in string or
lxml.etree form) |
assertEqualDirs() |
Compares the files and contents of those files in two directories |
assertAlmostEqualDatetime() |
Compares two datetime objects to a specified precision |
Creating parametric tests¶
A parametric test case is a single unit of test code that, during test
execution, is run several times with different arguments
(parameters). The function parametrize()
creates a single new testcase, based upon a template method, and binds
the specified parameters to the template method. Each testcase is
uniquely named based on the given parameters. Since each invocation
creates a new test case method, specific parameters can be tested in
isolation, and the normal unittest test runner reports exactly which
parameters the test succeeds or fails with.
Often, the parameters to the test is best stored in files. The
function file_parametrize()
creates one
testcase, based upon a template method, for each file found in a
specified directory.
RepoTester¶
Functional tests are written to test a specific functionality of a software system as a whole. This means that functional tests excercize a larger portion of the code and is focused on what the behaviour (output) of the code should be, given a particular input. A typical repository has at least three large units of code that benefits from a functional-level testing: Code that performs downloading of documents, code that extracts metadata from downloaded documents, and code that generates structured XHTML documents from the downloaded documents.
The RepoTester
contains generic,
parametric test for all three of these. In order to use them, you
create test data in some directory of your choice, create a subclass
of RepoTester
specifying the location of your test data and the
docrepo class you want to test: and finally call
parametrize_repotester()
in your top-level
test code to set up one test for each test data file that you’ve
created.
from ferenda.testutil import RepoTester, parametrize_repotester
from ferenda.sources.tech import RFC
class RFCTester(RepoTester):
repoclass = RFC
docroot = "myrepo/tests/files"
parametrize_repotester(RFCTester)
Download tests¶
See download_test()
.
For each download test, you need to create a JSON file under the
source
directory of your docroot, eg:
myrepo/tests/files/source/basic.json
that should look something
like this:
{
"http://www.ietf.org/download/rfc-index.txt": {
"file":"index.txt",
"content-type":"text/plain"
},
"http://tools.ietf.org/rfc/rfc6953.txt": {
"file":"rfc6953.txt",
"content-type": "text/plain",
"expect": "downloaded/6953.txt"
}
}
Each key of the JSON object should be a URL, and the value should be
another JSON object, that should have the key file
that specifies
the relative location of a file that corresponds to that URL.
When each download test runs, calls to requests.get et al are intercepted and the given file is returned instead. This allows you to run the download tests without hitting the remote server.
Each JSON object might also have the key expect
, which indicates
that the URL represents a document to be stored. The value specifieds
the location where the download method should store the corresponding
file, if that particular URL should be stored underneath the
downloaded
directory. In the above example, the index file is no
If you want to test your download code under any specific condition,
you can specify a special @settings
key. Each key and sub-key
underneath this will be set directly on the repo object being
tested. For example, this sets the next_sfsnr
key of the
config
object on the repo to
2014:913
.
{
"@settings": {
"config": {"next_sfsnr": "2014:913"}
}
}
Recording download tests¶
If the environment variable FERENDA_SET_TESTFILE
is set, the
download code runs like normal (calls to requests.get et al are not
intercepted) and instead each accessed URL is stored in the JSON
file. URL accessses that results in downloaded files results in
expect
entries in the JSON file. This allows you to record the
behaviour of existing download code to examine it or just to make sure
it doesn’t change inadvertantly.
Distill and parse tests¶
See distill_test()
and
parse_test()
.
To create a distill or parse test, you first need to create whatever
files that your parse methods will need in the download
directory of
your docroot.
Both distill_test()
and
parse_test()
will run your parse
method, and then compare it to expected results. For distill tests,
the expected result should be placed under
distilled/[basefile].ttl
. For parse tests, the expected result
should be placed under parsed/[basefile].xhtml
.
Recording distill/parse tests¶
If the environment variable FERENDA_SET_TESTFILE
is set, the
parse code runs like normal and the result of the parse is stored in
eg. distilled/[basefile].ttl
or parsed/[basefile].xhtml
. This
is a quick way of recording existing behaviour as a baseline for your
tests.
Py23DocChecker¶
Py23DocChecker
is a small helper to
enable you to write doctest-style tests that run unmodified under
python 2 and 3. The main problem with cross-version compatible
doctests is with functions that return (unicode) strings. These are
formatted u'like this'
in Python 2, and 'like this'
in
Python 3. Writing doctests for functions that return unicode strings
requires you to choose one of these syntaxes, and the result will fail
on the other platform. By strictly running doctests from within the
unittest
framework through the load_tests
mechanism, and
loading your doctests in this way, the tests will work even under
Python 2:
from ferenda.testutil import Py23DocChecker
def load_tests(loader,tests,ignore):
tests.addTests(doctest.DocTestSuite(mymodule, checker=Py23DocChecker()))
return tests
testparser¶
testparser()
is a simple helper that tests
FSMParser
based parsers.