author Christophe de Vienne <>
Wed, 22 Oct 2014 20:24:55 +0200
changeset 19 d06afae95fe4
permissions -rw-r--r--
CWEP 0004 initial draft

 CWEP-0004 Cubes as packages

:champion: Christophe de Vienne
:reviewers: COPIL
:last update: 2014/10/22
:status: draft


This document specifies a proposed standard for cubes layout, installation and

The central idea is to use standard python packages as containers, and rely on
entry-points to discover the cubes.


CubicWeb currently handle the cubes in its own way, without clear advantage
over user standard python packages.

This situation leads to several problematic situations, with no easy solution:

-   If modules are imported from one cube to another, multiple values in
    CUBES_PATH is not possible, because the root "cubes" directory is expected
    to be in the PYTHONPATH.

    This particular issue makes it difficult, if not impossible, to install
    cubes system-wide (or in a virtualenv) and work on cubes in a work

-   The (unperfect but popular) "develop" mode of setuptools and pip does not
    work. It can be worked around but it is a pain (See

-   The schema of a cube can only be in a "" file, not a "schema"

-   The code handling the cubes is complex.

-   The dependencies between cubes are only used to enforce dependencies at
    runtime, but not to install them (except if using the debian packages).

-   The implicit scan and registering of objects depending on the module
    they belong to is a source of confusion, and a limition of possible

The design choices for the current way cubes are handled made sens when it was

Since then however, the python packaging went throught a lot of rationalization
and standardization, and now provides all CubicWeb needs to define a simpler
way to create cubes.

The reading of the `Python Packaging User Guide`_ is recommended to get an
updated summary of the python packaging ecosystem



What is a cube ?

A cube has a name, that should be unique in the cubes ecosystem.

A cube has a version, that can be used to define requirements.

A cube has a schema version, that is used to detect if a the database needs an
upgrade. In the very large majority of cases, this version is the same as the
cube version.

A cube provides:

Things loaded at cube loading time:

-   Properties (uiprops, site_cubicweb)

Things loaded at registration time (almost the same as loading time):

-   Registrable objects (mostly AppObjects)
-   Static resources (indirectly)

Things loaded when schema definition is needed, mainly to initialize a
migration context:

-   Schema

Things loaded exceptionnally, ie when a migration is needed:

-   Migration scripts

Cube Interface

The Cube interface is what it is expected to provide so that CubicWeb is able
to find it and load what is inside.

Currently, the cube interface is a list of file and directory with precise
names, allowing little, if any, customization.

The proposed new interface consists of a few functions/objects:

    The specific configuration options of the cube. Strictly equivalent to
    site_cubicweb.options variable.

def register(vreg):
    Use the vreg methods to:
    - scan module(s) for buildobjs (ie 'schema')
    - scan module(s) for app objects.
    - declare a directory containing static resources
    - declare a directory containing the migration scripts.
    - declare / alter uiprops

def includeme(config)
    Called in the context of a pyramid based cubicweb instance.
    See `Pyramid Advanced Configuration`_ for more information on the expected
    behavior of this function.
    This particular function will always be called _after_ init_cube.

Importing these function should have minimal side-effect, as importing them
does not mean activating them.

Compatible Cubes

The new interface being compatible with the old one, we can have cube sources
that can produce both an old-fashion cube (installed in the cubes path), and
new-fashion cubes (installed as packages).

These cubes will need a special option on the " install" to be
installed the old way.

Cubes lookup

Any python package declaring any entry-point in the "cubicweb" group is
considered a cube.

`pkg_resources` provides a `convenience API`_ to lookup all entry-points
in a group.

A sample code for listing available cubes could be:

.. code-block:: python

    def list_cubes():
        dists = set()
        for entrypoint in pkg_resource.iter_entry_points('cubicweb'):
        return [ for dist in dists]

.. _convenience API:

Migration Path

To avoid forcing all cubes to be converted at once, we need to set up a smooth

Phase 1

Proposed target: Cubicweb 3.20

At this stage, the new cubes interfaces and apis are not frozen, hence people
using them must be ready to fix their cubes between two cubicweb releases.

Phase 2

Proposed target: Cubicweb 3.21

The new cubes api is frozen.

The old fashion cubes are still loaded, but with deprecation warnings.

During this phase, cube maintainers should make their cubes compatible.

Phase 3

Proposed target: Cubicweb 4.0

Old-fashion cubes are not loaded anymore.


.. _Python Packaging User Guide:
.. _Pyramid Advanced Configuration: