[doc] multiple doc upgrades:
authorVladimir Popescu <vladimir.popescu@logilab.fr>
Fri, 17 Jan 2014 18:32:07 +0100
changeset 1467 d332fd51803d
parent 1466 f3ff429b88ac
child 1468 3644b206edb5
[doc] multiple doc upgrades: - Add Apycot architecture specification. Remove deprecated file. - Add Apycot quick start tutorial which replaces the French step by step - integrate old doc and remove deprecated files - Update checkers doc
wdoc/ChangeLog_en
wdoc/ChangeLog_fr
wdoc/apycot_architecture_en.rst
wdoc/apycot_automation_en.rst
wdoc/apycot_checker_en.rst
wdoc/apycot_en.rst
wdoc/apycot_quick_start_en.rst
wdoc/apycot_step_by_step_fr.rst
wdoc/apycot_test_configuration_en.rst
wdoc/toc.xml
--- a/wdoc/ChangeLog_en	Wed Feb 26 18:15:33 2014 +0100
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,15 +0,0 @@
-.. -*- coding: utf-8 -*-
-.. _`execution reports`: ?vid=apycot.te.summary
-
-2010-05-18  --  1.9
-
-    * new views for `execution reports`_ , based on filterable table of
-      test execution
-
-    * test executions filtering abilities using new facets on branch,
-      status, start time...
-
-    * ease access to test environment archive
-
-    * some enhancements in configuration views
-
--- a/wdoc/ChangeLog_fr	Wed Feb 26 18:15:33 2014 +0100
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,14 +0,0 @@
-.. -*- coding: utf-8 -*-
-.. _`rapports d'éxécution`: ?vid=apycot.te.summary
-
-2010-05-18  --  1.9
-
-    * nouvelles vues de `rapports d'éxécution`_ sous forme de table
-      filtrable des éxécutions
-
-    * possibilité de filtrage des éxécutions via de nouvelles facettes
-      sur la branche, l'état, l'heure de lancement...
-
-    * accès aux archives des environnements de test facilité
-
-    * petites amélioration dans les vues de configurations
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wdoc/apycot_architecture_en.rst	Fri Jan 17 18:32:07 2014 +0100
@@ -0,0 +1,121 @@
+=====================
+ Apycot architecture
+=====================
+
+Apycot is designed to be an extensible test automation tool usable for continuous
+integration and continuous testing.
+
+It can fetch source code from version-controlled repositories (like SVN or Hg),
+run tests, then store the results and generate various reports from the data
+collected. Once the tests are configured, users can be notified in realtime
+when the status of a test changes or get a periodic report about the health
+of the projects their work on.
+
+.. image:: doc/images/apycot_processes.png
+
+Cubicweb instances contain environment and test configurations, as well as test
+execution information that may be used to build useful reports.
+
+Once configured, you can explicitly queue a task (eg run tests for a
+configuration) through a test configuration page. To get actual CI you'll have to
+`automate this`_.
+
+When a task is queued through `apycotclient` or the web user interface, an
+apycot bot will get tasked with it:
+
+* the bot will retrieve the configuration from the instance hosting it
+
+* the bot will execute the task (setup environmenent, run tests) and 
+  store the output in the instance from which it got the configuration.
+
+---------------
+Testing Process
+---------------
+
+Apycot builds upon the narval cube and bot. The process of running a test in
+apycot is based on the way narval runs tests and looks like:
+
+.. aafig::
+
+     Narval bot server                    Web app server
+
+   +-----+  +----------+                +----------------+
+   | bot |  | run-plan |                | apycot web app |
+   +--+--+  +----+-----+                +--------+-------+
+      |          |                               |
+      |          |                               |
+      X poll (HTTP GET)                          X
+      X----------------------------------------->X
+      X<-----------------------------------------X
+      X list of pending plans (eids)             X
+      X          |                               X
+      X spawn (narval run-plan plan-uri)         X
+      X--------->X                               X
+      X          X get plan description (GET)    X
+      X          X------------------------------>X
+      X          X<------------------------------X
+      X          X   plan description (recipe)   X
+      X          |                               X
+      X          X fire setup transition (POST)  X
+      X          X------------------------------>X
+      X          X<------------------------------X
+      X          |                               X
+      X          X execfile(recipe)              X
+      X      ____|_______________________________X____
+      X      | for each checker |                X   | 
+      X      |_________________/                 X   |
+      X      |   |                               X   |
+      X      |   X create                        X   |
+      X      |   X a CheckResult entity          X   |
+      X      |   X (POST)                        X   |
+      X      |   X------------------------------>X   |
+      X      |   X<------------------------------X   |
+      X      |   X      CheckResult cwuri        X   |
+      X      |   X                               X   |
+      X      |   X  refresh log_file (POST)      X   |
+      X      |   X------------------------------>X   |
+      X      |   X                               X   |
+      X      |   X run(checker)                  X   |
+      X      |   X                               X   |
+      X      |   X  refresh log_file (POST)      X   |
+      X      |   X------------------------------>X   |
+      X      |   X                               X   |
+      X      |   X  set status of                X   |
+      X      |   X  CheckResults (POST)          X   |
+      X      |   X------------------------------>X   |
+      X      |___X_______________________________X___|
+      X          X                               X 
+      X          X  refresh log_file (POST)      X
+      X          X------------------------------>X
+      X          X                               X 
+      X          X  set status of                X
+      X          X  TestExecution (POST)         X
+      X          X------------------------------>X
+      X          X<------------------------------X
+      X          X                               X
+      X  stdout  X                               X
+      X<---------X                               X
+      X<---------X                               X
+      X  stderr  X                               X
+      X          |                               X
+      X          |    POST execution_log (POST)  X
+      X----------------------------------------->X
+      X                                          X
+      X          |      poll (HTTP GET)          X
+      X----------------------------------------->X
+      X<-----------------------------------------X
+      X          |                               X
+      X          |                               X
+
+
+
+
+The bot is responsible for checking if there are pending tasks,
+spawning a sub-processes for each task, and monitoring this latter
+sub-process execution (resources limits, crash, etc.).
+
+The `narval-plan` sub-process retrieves the job's description and changes its
+workflow state, prepares the job environment (checkout, dependencies...) and,
+for each checker in the job, runs the checker and transmits all the results to
+a specifically-created CheckResult entity on the webapp. Finally, the
+narval-plan log is uploaded to the web application.
--- a/wdoc/apycot_automation_en.rst	Wed Feb 26 18:15:33 2014 +0100
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,24 +0,0 @@
-Automation
-----------
-
-By integrating to a local repository (vcsfile)
-``````````````````````````````````````````````
-
-This is the recommended way to get deep CI/VCS integration :
-
-* import vcs repository data for your projects using the `vcsfile cube`_
-
-* link a repository to the relevant test configuration through the
-  `local_repository` relation
-
-* set auto-test-mode to 'quick' or 'full' in the instance's all-in-one.conf file.
-
-
-By using apycotclient in cron jobs
-``````````````````````````````````
-
-XXX WRITE ME
-
-See apycotclient --help until then.
-
-.. winclude:: apycot_links
--- a/wdoc/apycot_checker_en.rst	Wed Feb 26 18:15:33 2014 +0100
+++ b/wdoc/apycot_checker_en.rst	Fri Jan 17 18:32:07 2014 +0100
@@ -1,10 +1,11 @@
 Checkers
 --------
 
-A checker provides testing functionnalities. Its return status can be a
-"success" if the test passed, a "failure" if the test failed or an "error"
-if the test could not be run to completion (maybe the environment could not
-be set up or the test program was badly written).
+A checker provides testing functionalities. It is run by the narvalbot and
+multiple checkers can be called in a recipe. Its return status can be
+"success" if the test passed, "skipped" if the test is skipped, a "failure" if
+the test failed or an "error" if the test could not be run to completion (maybe
+the environment could not be set up or the test program was badly written).
 
 Apycot comes with several checkers described below. If you do not find the
 checker you are looking for, write your own by deriving an existing one and
@@ -13,36 +14,35 @@
 Some checkers depend on third-party programs (usually a Python package or
 an external command) and may not be available on your system.
 
-Once the bot is running, you can see checker and preprocessors.
+Once the narval-apycot package is installed on the same machine as the bot, the
+checkers can be accessed by the Narval bot.
 
-General Checkers options
-````````````````````````
-
-All checkers support the following options :
+Running generic test files
+++++++++++++++++++++++++++
 
-+-------------------+-------+--------------------------------------------------+
-|   name            |  req. |   description                                    |
-+===================+=======+==================================================+
-| max_status        |  no   | A maximum result this checker cannot exceed      |
-+-------------------+-------+--------------------------------------------------+
-| status_cap_reason |  no   | The reason why this checker is capped            |
-+-------------------+-------+--------------------------------------------------+
+script_runner
+~~~~~~~~~~~~~
+
+TODO
+
+Debian packaging
+++++++++++++++++
 
-File checkers
-``````````````
+lintian
+~~~~~~~
 
-A file checker acts on files according to their extension. The check succeeds
-if each and every file passes the test independantly.
+TODO
 
-All the file checkers support the following options :
+Javascript
+++++++++++
 
-+----------+-------+----------------------------------------------------------+
-|   name   |  req. |   description                                            |
-+==========+=======+==========================================================+
-| ignore   |  no   | comma separated list some files or directories to ignore |
-+----------+-------+----------------------------------------------------------+
+jslint
+~~~~~~
 
-XXX Generate me
+TODO
+
+For Python code
++++++++++++++++
 
 python_syntax
 ~~~~~~~~~~~~~
@@ -51,87 +51,13 @@
   Checks the syntax of Python files using the compile function coming with
   Python.
 
-html_tidy
-~~~~~~~~~
-:depends on: mxtidy_
-:extensions: .html, .htm
-:description:
-  Checks the syntax of HTML files using the mx Tidy module.
-
-pt_syntax
-~~~~~~~~~
-:depends on: Zope_
-:extensions: .pt, .zpt
-:description:
-  Checks the syntax of Page Template files using Zope's PageTemplates package.
-
-xml_well_formed
-~~~~~~~~~~~~~~~
-:depends on: lxml_
-:extensions: .xml
-:description:
-  Checks the well-formness of XML files using the lxml parser.
-
-xml_valid
-~~~~~~~~~
-:depends on: lxml_
-:extensions: .xml
-:description:
-  Checks the validity of XML files using the lxml validating parser.
-
-rest_syntax
-~~~~~~~~~~~
-:depends on: docutils_
-:extensions: .rst, .txt
-:description:
-  Checks the syntax of ReST files using the docutils package.
+pytest
+~~~~~~
 
-debian_lint
-~~~~~~~~~~~
-:depends on: lintian
-:extensions: .deb, .dsc
-:description:
-  Checks debian packages by parsing the output of lintian.
-
-debian_piuparts
-~~~~~~~~~~~~~~~
-:depends on: piuparts (and sudo configuration)
-:extensions: .deb
-:description:
-  Checks debian packages by installing and uninstalling them with piuparts
-
-
-
-Package checkers
-````````````````
-
-A package checker acts more globally, to locate some data or to make interact
-results on each files to finally make the check succeed or failed.
+TODO
 
-XXX Generate me
-
-python_lint
-~~~~~~~~~~~
-:depends on: pylint_
-:description:
-  Use Pylint to check a score for python package. The check fails if the score is
-  inferior to a given treshold.
-:options:
-  +---------------------+--------+-------------------------------------------+
-  |        name         |  req.  |   description                             |
-  +=====================+========+===========================================+
-  | treshold            |   yes  | the minimal note to obtain for the        |
-  |                     |        | package from PyLint                       |
-  +---------------------+--------+-------------------------------------------+
-  | show_categories     |   no   | comma separated list of letter used to    |
-  |                     |        | filter the message displayed default to   |
-  |                     |        | Error and Fatal                           |
-  +---------------------+--------+-------------------------------------------+
-  | pylintrc            |   no   | The path to a pylint configuration file   |
-  +---------------------+--------+-------------------------------------------+
-
-python_unittest
-~~~~~~~~~~~~~~~
+pyunit
+~~~~~~
 :depends on: pyunit
 :description:
   Execute tests found in the "test" or "tests" directory of the package. The check
@@ -150,63 +76,51 @@
   |                             |      | of tests directory. default to       |
   |                             |      | "test, tests"                        |
   +-----------------------------+------+--------------------------------------+
-  | ignored_python_versions     |  no  | Comma separated version of python to |
-  |                             |      | ignore when running the test.        |
-  +-----------------------------+------+--------------------------------------+
-  | tested_python_versions      |  no  | Comma separated version of python to |
-  |                             |      | test when running the test.          |
-  +-----------------------------+------+--------------------------------------+
-  | use_pkginfo_python_versions |  no  | 0, or 1 (default to 1) run the tests |
-  |                             |      | with the python's versions defined   |
-  |                             |      | in the pkginfo module.               |
-  +-----------------------------+------+--------------------------------------+
-
-    * First, versions defined in the pkginfo module are imported (if enable).
-    * Then versions defined into tested_python_versions are added.
-    * finally version in ignored_python_versions are removed.
 
 pycoverage
-~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~
+
 :depends on: devtools_
 :description:
   When devtools is available, test will be launched in a coverage mode. This test
   will gather coverage information, and will succeed if the test coverage is
   superior to a given treshold. *This checker must be executed after the
-  python_unittest checker.*
+  pyunit checker.*
 :options:
-  +----------+-------+---------------------------------------------------------+
-  |   name   |  req. |   description                                           |
-  +==========+=======+=========================================================+
-  | treshold |  yes  | the minimal note to obtain for the test coverage        |
-  +----------+-------+---------------------------------------------------------+
-
-pkg_doc
-~~~~~~~
-:depends on: `rest_syntax`_, `xml_well_formed`_, `html_tidy`_
-:description:
-  Check some standard package documentation :
-
-  * presence of some required files (README, INSTALL, ChangeLog)
-  * plain text files in the "doc" directory are ReST files
-  * xml files in the "doc" directory are well formed
-  * html files in the "doc" directory are correct
-  
-  The 3 last tests will be done according to the presence of the respective
-  checkers (which depends on external packages).
-:options:
-  +----------+-------+----------------------------------------------------------+
-  |   name   |  req. |   description                                            |
-  +==========+=======+==========================================================+
-  | ignore   |  no   | comma separated list of files or directories to ignore   |
-  +----------+-------+----------------------------------------------------------+
-
-lgp_check
-~~~~~~~~~~
-:depends on: devtools_
-:description:
-  Check a package is conform to the `standard source tree` as described in the
-  devtools package for a Python package. It'll also check the content of some 
-  of the specified files, like __pkginfo__.py, MANIFEST.in...
+  +--------------------+-------+--------------------------------------------------+
+  |   name             |  req. |   description                                    |
+  +====================+=======+==================================================+
+  | coverage_threshold |  yes  | the minimal note to obtain for the test coverage |
+  +--------------------+-------+--------------------------------------------------+
 
 
-.. winclude:: apycot_links
+pylint
+~~~~~~
+
+:depends on: pylint_
+:description:
+  Use Pylint to check a score for python package. The check fails if the score is
+  inferior to a given treshold.
+:options:
+  +---------------------+--------+-------------------------------------------+
+  |        name         |  req.  |   description                             |
+  +=====================+========+===========================================+
+  | pylint_threshold    |   no   | the minimal note to obtain for the        |
+  |                     |        | package from PyLint                       |
+  +---------------------+--------+-------------------------------------------+
+  | show_categories     |   no   | comma separated list of letter used to    |
+  |                     |        | filter the message displayed default to   |
+  |                     |        | Error and Fatal                           |
+  +---------------------+--------+-------------------------------------------+
+  | pylintrc            |   no   | The path to a pylint configuration file   |
+  +---------------------+--------+-------------------------------------------+
+
+pychecker
+~~~~~~~~~
+
+TODO
+
+py.test
+~~~~~~~
+
+TODO
--- a/wdoc/apycot_en.rst	Wed Feb 26 18:15:33 2014 +0100
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,28 +0,0 @@
-
-Apycot is designed to be an extensible test automation tool usable for continuous
-integration and continuous testing.
-
-It can fetch source code from version-controlled repositories (like SVN or Hg),
-run tests, then store the results and generate various reports from the data
-collected. Once the tests are configured, users can be notified in realtime
-when the status of a test changes or get a periodic report about the health
-of the projects their work on.
-
-.. image:: doc/images/apycot_processes.png
-
-Cubicweb instances contain environment and test configurations, as well as test
-execution information that may be used to build useful reports.
-
-Once configured, you can explicitly queue a task (eg run tests for a
-configuration) through a test configuration page. To get actual CI you'll have to
-`automate this`_.
-
-When a task is queued through `apycotclient` or the web user interface, an
-apycot bot will get tasked with it:
-
-* the bot will retrieve the configuration from the instance hosting it
-
-* the bot will execute the task (setup environmenent, run tests) and 
-  store the output in the instance from which it got the configuration.
-
-.. winclude:: apycot_links
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wdoc/apycot_quick_start_en.rst	Fri Jan 17 18:32:07 2014 +0100
@@ -0,0 +1,271 @@
+====================
+ Apycot Quick Start 
+====================
+
+--------
+Overview
+--------
+
+Apycot is designed to be an extensible test automation tool usable for
+continuous integration and continuous testing. It is a CubicWeb-based
+application extending the Narval framework. It adds the notion of preprocessors
+and checkers to Narval. The preprocessors are used mainly to checkout
+version-controlled repositories and setup a test environment while the checkers
+correspond to specific checks (pycoverage, pytest, pylint...). 
+
+Other major differences with respect to Narval include the use of TestExecution
+and CheckResults instead of Plans to store the results and the organisation of
+test environments through ProjectEnvironments and TestConfigs.
+
+The Apycot framework, like Narval is split between:
+
+- a Web application used for storing, presenting and accessing the test results;
+- a bot regularly polling the web application using http, running the tests
+  when necessary and transmitting back the results to the Web application.
+
+Both parts can be installed on the same machine or on different machines able
+to communicate via HTTP. Please read the Narval documentation before installing
+Apycot.
+
+-----------
+Definitions
+-----------
+
+* A **repository** is a version-controlled sources (VCS) database, like SVN or
+  Hg.
+
+* A **preprocessor** allows a specific construction / installation step
+  required to build a test environment.  Example preprocessors: 'setup_install'
+  to install a package by using `python setup.py install`, 'make' call the
+  `make` command...
+
+* A **check** is a single functional test which may be applied to test a
+  package. A **checker** is the object applying this functional test.  Example
+  checks: 'pyunit' to start python unittest, 'pylint' to start pylint...
+
+---------
+Narvalbot
+---------
+
+Installation
+============
+
+In addition to what is described in the Narval documentation, you need to install
+the additional ``narval-apycot`` package (it contains the additional checkers and 
+preprocessors used by apycot).
+
+.. sourcecode:: bash
+  
+  apt-get install narval-bot narval-apycot
+
+Configuration
+=============
+
+Follow the Narval bot configuration documentation to configure the Narval bot.
+
+---------------
+Cubicweb-apycot
+---------------
+
+Installation
+============
+
+To install a complete Apycot environment, you need to install the following packages:
+
+.. sourcecode:: bash
+  
+  apt-get install cubicweb-apycot cubicweb-ctl 
+
+Creating an Apycot instance
+===========================
+
+.. sourcecode:: bash
+
+  cubicweb-ctl create apycot myapycot
+
+As explained in the Narval documentation, you need to fill-in the ``base-url``
+before creating the database as the bot uses this address to connect to the web
+application. This file is usually available in
+``[~]/etc/cubicweb.d/<instance-name>``.
+
+.. sourcecode:: bash
+
+  cubicweb-ctl db-create myapycot
+
+You can then launch the CubicWeb instance:
+
+.. sourcecode:: bash
+
+  cubicweb-ctl start -D myapycot
+
+Make sure the bot manages to connect to the instance by looking at the logs in `/var/log/narval/narval.log`. If you see something like :: 
+  
+  2014-01-15 20:20:40 - (narval.bot) INFO: get pending plan from narval
+  2014-01-15 20:20:40 - (requests.packages.urllib3.connectionpool) INFO: Starting new HTTP connection (1): crater2.logilab.fr
+
+it means everything is working as expected. If the Narval bot cannot connect, you will see this instead::
+
+  Traceback (most recent call last):
+    File "/usr/lib/pymodules/python2.7/narvalbot/server.py", line 107, in _loop
+      for plandata in self.config.cnxh(instance_id).pending_plans():
+    File "/usr/lib/pymodules/python2.7/narvalbot/__init__.py", line 222, in pending_plans
+      '?vid=narval.pending-plans'))
+    File "/usr/lib/pymodules/python2.7/narvalbot/__init__.py", line 182, in http_get
+      return self.http_post(url, method='get')
+    File "/usr/lib/pymodules/python2.7/narvalbot/__init__.py", line 192, in http_post
+      self.connect()
+    File "/usr/lib/pymodules/python2.7/narvalbot/__init__.py", line 212, in connect
+      '?__login=%s&__password=%s' % (user, password)))
+    File "/usr/lib/pymodules/python2.7/narvalbot/__init__.py", line 247, in open
+      return self.opener.open(*args, **kwargs)
+    File "/usr/lib/python2.7/urllib2.py", line 401, in open
+      response = self._open(req, data)
+    File "/usr/lib/python2.7/urllib2.py", line 419, in _open
+      '_open', req)
+    File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
+      result = func(*args)
+    File "/usr/lib/python2.7/urllib2.py", line 1211, in http_open
+      return self.do_open(httplib.HTTPConnection, req)
+    File "/usr/lib/python2.7/urllib2.py", line 1181, in do_open
+      raise URLError(err)
+  URLError: <urlopen error [Errno 111] Connection refused>
+
+Setting up a test environment
+=============================
+
+We first describe the entity types added by Apycot and then provide a complete
+setup procedure. 
+
+Creating a project environment (ProjectEnvironment)
+---------------------------------------------------
+
+To setup a ProjectEnvironment, you need to specify:
+
+- the ProjectEnvironment name
+- the configuration variables. These variables are available in the
+  checkers and preprocessors in the options object. For example ::
+
+  install=python_setup
+  pycoverage=True
+  extra_argument=['-m', 'Corp']
+  test_dirs=['tests']
+  test_prefixes=['test_','unittest_']
+  pylint_threshold=70
+  pylintrc='~/.pylintrc'
+  coverage_threshold=0.7
+  keep_test_dir=True
+  archive=False
+  required=False
+  verbose=True
+
+- the environment variables. These variables will be available in the
+  shell for the preprocessors and checkers. For example::
+
+  NO_SETUPTOOLS=1
+  DISPLAY=:1.0
+
+Usually, when checkers or preprocessors fail, or crash, some of these variables
+are incorrect or missing.
+
+Creating a test configuration (TestConfig) 
+------------------------------------------
+
+Usually, a recipe requires at least a TestConfig. This configuration
+contains all the necessary information to run the tests.
+
+To setup a TestConfig, you need to specify:
+
+- the TestConfig name
+- the launching mode ("manual" by default)
+- the way dependent project environments are tested when the TestConfig
+  is modified; can be "yes" (thus you lauch the tests for these project
+  environments), "no" (thus, you dont lauch the tests), and "inherited" (the
+  value is set elsewhere).
+- the configuration variables and environment variables. These are similar to
+  the configuration variables found in ProjectEnvironment instances.
+- the recipe associated to the TestConfig. This recipe will be launched
+  by the narval instance when this configuration will be used.
+- the "refinement of" specifies the parent TestConfig. The current TestConfig
+  will inherit all the attributes and variables from the parent TestConfig. 
+
+To run tests you must link the TestConfig to a ProjectEnvironment and specify a
+recipe. Make sure the options necessary for each checker and preprocessor are
+availalbe in either the ProjectEnvironment or the TestConfig.
+
+Creating a repository
+---------------------
+
+The Repository is a proxy to a VCS repository holding the project to be tested,
+e.g. Mercurial.
+
+To setup a Repository, you need to specify:
+
+- the type of the VCS handling the Repository;
+- the path to the Repository. 
+
+Make sure the Repository can be accessed by both the Apycot Web application and
+the Narval bot instance.
+
+In the "files" tab you can find all the files in the current revision, whereas
+in the "revisions" tabs you have a table with all revisions known to the
+Repository. 
+
+Full setup procedure
+--------------------
+
+Make sure the cubicweb instance running Apycot and the Narval bot instance are
+both installed, configured (as shown above and in the Narval documentation) and
+able to communicate (see their respective install procedures). 
+
+1. Setup a Repository for the project
+2. Create a TestConfig for each recipe you want to be able to run.
+   - make sure the correct recipe is selected for each TestConfig.
+   - make sure to fill-in the necessary options (``install=python_setup`` is
+     usually mandatory for Python recipes).
+3. Setup a ProjectEnvironment for the project
+   - link the ProjectEnvironment to the previously created Repository
+   - link the ProjectEnvironment to all the necessary TestConfig instances.
+4. From the ProjectEnvironment view in cubicweb, manually launch the tests and
+   watch the results appear on you display (if you do not see the option to
+   launch tests, make sure you are logged in and have the necessary
+   authorizations).
+
+Troubleshooting
+---------------
+
+The Narval bot successfully retrieves the plan and runs the checks, but the
+reports are never transmitted to the Web application:
+
+- The `base-url` is used by the user to connect to the Web application, but
+  also by the Narval bot to transmit its results back to the application.
+  Hence, the Narval bot must be able to connect to the Web application using the
+  base-url provided in the `all-in-one.conf` file. A similar problem arises
+  when the cwuri of the TestExecutions, TestConfigs and Checkresultsare wrong
+  or outdated: the narvalbot uses these cwuri to connect to the instance. You
+  should check these values and make sure there are up to date. 
+
+.. sourcecode:: python 
+
+  >>> for ent in rql('Any X WHERE X cwuri LIKE "None%"').entities(): 
+          ent.cw_set(cwuri=ent.cwuri.replace('None',
+                                        'http://crater2.logilab.fr:10003/'))
+
+The Narval bot never gets the plan and the interval between the bot's polls is
+too long:
+
+- the interval can be modified in `/etc/narval/narval-cw-sources.ini` to add
+  the ``poll-delay`` argument::
+
+  [narval]
+  url=http://instance-name.fr:10003/
+  user=narval
+  password=narval0
+  poll-delay=5 
+
+The checkers crash but the project tests are fine, the project is not installed
+correctly:
+
+- make sure no configuration and environment variables are missing or wrongly
+  set. 
+- check the paths used by Narval bot are writable and free space is available
+  on the disk.
--- a/wdoc/apycot_step_by_step_fr.rst	Wed Feb 26 18:15:33 2014 +0100
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,208 +0,0 @@
-===================================
-Mise en place d'une instance Apycot
-===================================
-
-:Auteurs:
-     Alexandre Richardson,
-     Charles Hébert,
-     Logilab SA
-
-:Version: 1.0 - 2010/01/27
-
-.. contents:: Sommaire
-.. section-numbering::
-
-----------
-Pré-requis
-----------
-
-- Ensemble des dépendances
-
-* Cubicweb et cube apycot (et ses dépendances)
-* Serveur pyro
-* Apycotbot actif
-* extension bfile de mercurial
-
----------
-ApycotBot
----------
-
-Configuration
-=============
-
-Apycotbot a besoin de deux fichiers de configuration:
-
- - `apycotbot.ini` : configuration des paramètres du bot, notamment de pyro
- - `apycotbot-cw-sources.ini`: configuration d'un utilisateur permettant au
-   bot d'utiliser l'instance cubicweb-apycot.
-
-Ces fichiers se trouvent dans le répertoire `/etc/`. Lors d'une installation
-locale (sans droit), ces fichiers se trouvent dans le répertoire `~/etc/`.
-Il faut une installation locale de apycotbot et des outils logilab-common.
-Ne pas oublier de mettre à jour le `PYTHONPATH` faute de quoi ces fichiers
-ne seront pas chargés.
-
-Structure du fichier `apycotbot-cw-sources.ini`
-
-.. sourcecode:: ini
-
-  [nom_instance_cubicweb]
-
-  user=bot_user
-  password=bot_password
-
-Par défaut, lors de la création d'un instance cubicweb-apycot, un utilisateur
-`apycot` est créé.
-
-Structure du fichier `apycotbot.ini`
-------------------------------------
-
-.. sourcecode:: ini
-
-  [PROCESS-CONTROL]
-  threads=1
-  max-cpu-time=5 min
-  max-time=10 min
-  max-memory=500MB
-  max-reprieve=60s
-  [MAIN]
-  cw-inst-id=:cubicweb.myapycot
-  test-dir=/tmp/myapycot/
-  [PYRO-NAME-SERVER]
-  pyro-ns-host=cepheus.logilab.fr:9090
-  [PYRO-SERVER]
-  log-file=/tmp/myapycotbot.log
-  log-threshold=INFO
-  #host=
-  pid-file=/tmp/myapycot.pid
-  pyro-id=:cubicweb.myapycot
-
-Quelques explications :
-
-- `pyro-ns-host`, le serveur pyro,
-- `cw-inst-id`, l'identifiant pyro de l'instance cubicweb-apycot,
-- `pyro-id`, l'identidiant pyro du bot.
-
-Les identifiants sont de la forme `:groupe.instance`.
-
-Quelques commandes Pyro
------------------------
-
-Pyro pour Python Remote Object permet de partager des objets python entre le
-bot et l'instance cubicweb-apycot. Le bot et l'instance cubicweb doivent être
-enregistrés dans le Pyro Name Server.
-
-On obtient la liste des clients avec la commande suivante :
-
-.. class:: commande
-
-  pyro-nsc listall
-
-Il est possible que la suppression d'un client dans le pyro name server ne se
-déroule pas correctement. Pour pouvoir relancer le bot, il faut supprimer ce
-référencement avec la commande :
-
-.. class:: commande
-
-  pyro-nsc remove
-
-
-Configuration
-=============
-
-La mise en place du bot en mode debug :
-
-.. class:: commande
-
-  apycotbot -D
-
----------------
-Cubicweb-apycot
----------------
-
-Créer une instance apycot
-=========================
-
-.. class:: commande
-
-  cubicweb-ctl create apycot myapycot
-
-(mise à jour si besoin du fichier source : db-host=mydbhost)
-
-.. class:: commande
-
-  cubicweb-ctl db-create myapycot
-
-Configurer la communication de l'instance avec le bot
-=====================================================
-
-Modifier le fichier `all-in-one.conf`, section `[APYCOT]`
-(`~/etc/cubicweb.d/myapycot` ou `/etc/cubicweb.d/myapycot`) pour être en
-accord avec la configuration du bot:
-
-.. sourcecode:: ini
-
-  [APYCOT]
-
-  bot-pyro-id=:cubicweb.myapycot
-  bot-pyro-ns=myhost.mydomain.com:9090
-
-- lancer l'instance cubicweb
-
-.. class:: commande
-
-  cubicweb-ctl start -D myapycot
-
-- vérifier que le bot est correctement configuré :
-  http://[moninstance]/view?vid=botstatus
-
-Préparer les tests !
-====================
-
-Création d'un environnement de projet
--------------------------------------
-
-- Spécifierle nom du projet,
-- Choisir le type, nom et url du dépôt, (uniquement à titre informatif car
-  ce n'est pas ce champ qui est utilisé pour connaître les branches du
-  dépôt mais l'instance de l'entité dépôt),
-- si le bot est correctement configuré, des indications sont précisées sous
-  les boîtes `préprocesseurs`, `environnement`, `configuration`.
-
-Créer une entité dépôt
------------------------
-
-Cette entité contient les informations du dépôt de l'outil de gestion de
-configuration.
-
-Les champs devant être saisies sont :
-- le type (mercurial ou subversion)
-- le chemin d'accès au dépôt
-
-L'onglet révision permet de visualiser les différentes révisions.
-Attention : la mise à jour de cet onglet peut-être longue pour nos
-6000 changesets.
-
-Créer une configuration de test
--------------------------------
-
-1. Mise en place d'un environnement de projet
-
-- créer un entrepôt mercurial pointant vers le dépôt de code,
-- lier ce dépot à un nouvel environnement de projet,
-- définir une variable d'environnement (environnement) :
-
-`HGRCPATH=${TESTDIR}/[moninstance]/hgrc`
-
-2. Créer un groupe de configuration de test
-
-- définir un outil de vérification (vérifications) : `pytest`,
-- définir les préprocesseurs (préprocesseurs) : `install=python_setup`,
-- définir une variable d'environnement : `HGRCPATH=${TESTDIR}/[moninstance]/hgrc`,
-
-3. Créer une configuration de test et la faire dépendre du groupe défini ci-dessus,
-
-- définir une option pour `pytest` (configuration) : `pytest_argument=-m Corp`
-- l'option `-m` lance une catégorie de tests
-
-voir la documentation de pytest pour les détails.
--- a/wdoc/apycot_test_configuration_en.rst	Wed Feb 26 18:15:33 2014 +0100
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,98 +0,0 @@
-Test configuration
-------------------
-
-Vocabulary
-``````````
-* A **repository** is a usually version controlled sources (VCS) database, like
-  SVN or Hg.
-
-* A **package** is a generic term for some files related together as a project in
-  a source repository.
-
-* A **preprocessor** allow a specific construction / installation step required
-  to build a test environment.
-  Example preprocessors: 'setup_install' to install a package by using
-  `python setup.py install`, 'make' call the `make` command...
-
-* A **check** is a single functional test which may be applied to test a
-  package. A **checker** is the object applying this functional test.
-  Example checks: 'pyunit' to start python unittest, 'pylint' to start pylint...
-
-- A **test environment** describes how to build the test environment in which
-  checks will be executed: fetch source from vcs repository, fetch and install
-  sources for dependencies...
-
-All this information is gathered from TestConfig entities.
-
-TestConfig and TestConfigGroup attributes
-`````````````````````````````````````````
-
-:VCS access configuration:
-
-:Dependencies:
-
-:Test execution configuration:
-
-_
-
-Repositories
-------------
-
-Apycot supports the following repository types. If you don't find the one you're
-looking for, you may still write your own repository wrapper, and contribute it
-;). The best way to do so for the moment is to look at the existing ones...
-
-cvs
-```
-This repository type is used to fetch sources from a CVS repository.
-
-svn
-```
-This repository type is used to fetch sources from a Subversion repository.
-
-
-hg
-``
-This repository type is used to fetch sources from a Mercurial repository.
-
-
-fs
-``
-This repository type is used to fetch sources from a file system repository (non
-versioned). This may be useful to test projects in your working directory
-(that's actually whats the --fs option of *runtest* is using instead of the
-repository defined in the configuration file).
-
-.. null
-````
-.. Used for internal and test purpose.
-
-.. mock
-..````
-.. Used for internal and test purpose.
-
-Configuration tips
-------------------
-
-* Once the bot is running, you can see `all available variables`_.
-
-* You can give options to preprocessors and checkers by preceding the option
-  name by the preprocessor or checker name. For instance, you can give the
-  treshold option to the python_lint checker by adding ::
-
-    python_lint_treshold = 4
-
-
-* link to repository entity and grant access from the cubicweb instance to the
-  apycot bot, so test launch is controlled by cubicweb and you don't have to
-  maintain cron jobs to do so (though this solution may gives you finer control
-  about which checks should be started at which time)
-
-* vcsfile require local repository access, so those'll have to be available on
-  the cubiciweb repository host
-
-* but it's fine to give repository URL to TestConfiguration
-
-* quick checks should be included in full checks (XXX to be fixed)
-
-.. winclude:: apycot_links
--- a/wdoc/toc.xml	Wed Feb 26 18:15:33 2014 +0100
+++ b/wdoc/toc.xml	Fri Jan 17 18:32:07 2014 +0100
@@ -1,22 +1,12 @@
 <toc>
-    <section resource="apycot" insertafter="standard_usage">
-      <title xml:lang="en">Continuous integration environment with Apycot</title>
-      <title xml:lang="fr">Intégration continue avec Apycot</title>
-        <section resource="apycot_step_by_step">
-          <title xml:lang="en">Apycot Setup step by step</title>
-          <title xml:lang="fr">Mise en place d'Apycot pas à pas</title>
-        </section>
-        <section resource="apycot_test_configuration">
-          <title xml:lang="en">Test Configuration</title>
-          <title xml:lang="fr">Configuration des tests </title>
-        </section>
-        <section resource="apycot_checker">
-          <title xml:lang="en">Available Checkers</title>
-          <title xml:lang="fr">Checkers Disponibles</title>
-        </section>
-        <section resource="apycot_automation">
-          <title xml:lang="en">Testing Automation</title>
-          <title xml:lang="fr">Automatisation des tests </title>
-        </section>
-    </section>
+   <section resource="apycot_quick_start">
+     <title xml:lang="en">Apycot Quick Start</title>
+   </section>
+   <section resource="apycot_checker">
+     <title xml:lang="en">Available Checkers</title>
+     <title xml:lang="fr">Checkers Disponibles</title>
+   </section>
+   <section resource="apycot_architecture">
+     <title xml:lang="en">Architecture</title>
+   </section>
 </toc>