README.rst
author Philippe Pepiot <philippe.pepiot@logilab.fr>
Thu, 05 Mar 2020 11:53:20 +0100
changeset 28 f3b733c665ac
parent 26 8f8090626f4a
permissions -rw-r--r--
Fix checkrebuild We checked non cubicweb images, so upgrades of cubicweb where not handled correctly.

CubicWeb docker images
======================

This project helps to build CubicWeb_ based applications as Docker_ images.
Builds include various versions of cubicweb and python and are using Debian_
base images.

.. _CubicWeb: https://www.cubicweb.org/
.. _Docker: https://www.docker.com/
.. _Debian: https://www.debian.org/

Tags
----

Images with cubicweb pre-installed:

* ``latest``, ``3.27``, ``py37-buster-3.27``
* ``dev``, ``py37-buster-dev`` (built using latest mercurial changeset)
* ``3.26``, ``py37-buster-3.26``
* ``3.25``, ``py27-buster-3.25``
* ``py35-stretch-3.27``
* ``py35-stretch-3.26``
* ``py27-buster-3.26``
* ``py27-stretch-3.26``
* ``py27-buster-3.25``
* ``py27-stretch-3.25``

Image without cubicweb installed:

* ``py37``, ``py37-buster``
* ``py35``, ``py35-stretch``
* ``py27``, ``py27-buster``
* ``py27-stretch``

Image for building debian packages:

* ``buildpackage``, ``buster-buildpackage``
* ``stretch-buildpackage``


What's included ?
-----------------

The image include required packages to build a cubicweb application image
suitable for production. It's designed to be used as a parent image for your
application Dockerfile.

The image contains:

* python with psycopg2
* /usr/bin/python and /usr/bin/pip are symlink to the selected python version (2.7, 3.5 or 3.7)
* gettex, graphviz
* uwsgi with a config file in /etc/uwsgi/uwsgi.ini
* a pyramid configuration templated from environment variables in /pyramid.ini
* a entrypoint to configure pyramid.ini, static data files in
  /etc/cubicweb.d/instance/data and automatic upgrade of database.

The entrypoint can also be used to run various command like database creation
with ``db-create`` or to run arbitrary commands.

The image has some expectations:

* The top level cube source code is expected to be found in /src directory. In
  case you install your cube elsewhere or from a pypi release, you will have to
  set the ``CUBE`` environment variable.

* The default instance name is "instance" and its configuration directory is in
  /etc/cubicweb.d/instance, this can be changed with the ``CW_INSTANCE``.
  Although you shouldn't need to modify this.


How to build an image for a cubicweb application ?
--------------------------------------------------

There are multiple ways to use theses images corresponding to different levels
of integration.

Here are some hints to make the best choice:

* Images with cubicweb pre-installed build faster
* Images without cubicweb pre-installed allow to use your own version of cubicweb
* ``onbuild`` images are useful when the Dockerfile in versioned in the source
  tree, except when your cube require a build toolchain to install. However
  they are DEPRECATED.

Example
~~~~~~~

For example, given you're in the source tree of `cubicweb-blog`_::

   FROM logilab/cubicweb:3.27
   USER root
   COPY . /src
   RUN pip install -e /src
   USER cubicweb
   RUN docker-cubicweb-helper create-instance


In case of out-of-source tree or not installing from /src directory, you will
also have to set the ``CUBE`` environment variable::

   FROM logilab/cubicweb:3.27
   USER root
   RUN pip install cubicweb-blog
   USER cubicweb
   ENV CUBE=blog
   RUN docker-cubicweb-helper create-instance

onbuild images
~~~~~~~~~~~~~~

.. warning:: A lot of magic happen with onbuild images. They are DEPRECATED.

All images have a ``onbuild`` version by adding the suffix ``-onbuild``.
The single tag ``onbuild`` is an alias for py37-buster-3.27-onbuild.

These images use the `ONBUILD intruction`_ to copy current code to the build
context and install your cube in develop mode and create an instance of your cube.

.. _ONBUILD intruction: https://docs.docker.com/engine/reference/builder/#onbuild

For example, given you're in the source tree of `cubicweb-blog`_, a Dockerfile would be as simple as::

   FROM logilab/cubicweb:onbuild

You can even build an image without actually writing any Dockerfile::

   echo "FROM logilab/cubicweb:onbuild" | docker build -f - -t cubicweb-blog .

In case you don't want a cubicweb version pre-installed and let your own dependencies control what version to install::

   FROM logilab/cubicweb:py37-onbuild

.. _cubicweb-blog: https://hg.logilab.org/master/cubes/blog


How to run resulting images ?
-----------------------------

Environment variables control settings from ``all-in-one.conf``, most important are:

* ``CW_BASE_URL`` (default http://localhost:8080) should be set to the final url the instance will be accessed
  with. Including the scheme, for example: ``https://myapp.demo.logilab.org``

* ``CW_DB_DRIVER`` (default postgres), ``CW_DB_NAME``, ``CW_DB_USER`` and
  ``CW_DB_PASSWORD`` control database settings

* ``CW_LOGIN`` and ``CW_PASSWORD`` control the admin login and password,
  default to admin:admin

Quick testing using non persistent sqlite database
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This can be used to validate your image actually works::

   docker run --rm -it -e CW_DB_NAME=db.sqlite myimage sh -c "cubicweb-ctl db-create -a instance && uwsgi --ini /etc/uwsgi/uwsgi.ini"

Then go to http://localhost:8080 to access your instance.


Create initial database
~~~~~~~~~~~~~~~~~~~~~~~

To create the initial database you will have to run ``db-create`` first:

For example::

   # Create the database on local postgresql cluster in a database named "myapp"
   docker run --rm -it -e CW_DB_NAME=myapp CW_DB_USER=me -v /var/run/postgresql:/var/run/postgresql myimage db-create

   # Create the database on remote postgresql server in a database named "myapp"
   docker run --rm -it -e CW_DB_NAME=myapp CW_DB_USER=me CW_DB_PASSWORD=secret CW_DB_HOST=dbserver myimage db-create

   # Create the database in a local /tmp/db.sqlite file
   docker run --rm -it -e CW_DB_NAME=/tmp/db.sqlite -v /tmp/db.sqlite:/tmp/db.sqlite myimage db-create

Run wsgi server
~~~~~~~~~~~~~~~

To start uwsgi server on local port 8080::

   # run foreground
   docker run --rm -it -p 8080:8080 myapp

   # run in background
   docker run -d --restart=always --name myapp myapp

Run looping tasks (aka scheduler)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To run cubicweb looping tasks, you will also have to start the ``scheduler``::

   # run foreground
   docker run --rm -it myapp cubicweb-ctl scheduler instance

   # run in background
   docker run -d --restart=always --name myapp-scheduler myapp cubicweb-ctl scheduler instance


Best practices
--------------

In case of source tree builds, add a ``.dockerignore`` to your project to avoid
copying useless files inside the docker image. For example::

   .*
   *.egg-info
   **/__pycache__
   Dockerfile
   Jenkinsfile
   Makefile
   tox.ini
   test
   debian


Always pull the base image before running the build. Base images are rebuild in
case of debian security update or new pypi releases.

buildpackage images
-------------------

These images can be used to build debian package(s) and publish them to a repo
located in /repo suitable for use within a multi-stage build.

Example, given all dependencies are available on "deb http://apt.logilab.fr buster cubicweb-3.27" repository and given you're working in the source tree of `cubicweb-blog`_::

   FROM logilab/cubicweb:buildpackage as buildpackage
   COPY . /src
   RUN buildpackage -d /src

   FROM logilab/cubicweb:3.27
   COPY --from=buildpackage /repo /repo
   RUN apt-get update && apt-get -y --no-install-recommends install cubicweb-blog
   ENV CUBE=blog
   USER cubicweb
   RUN docker-cubicweb-helper create-instance


If you need to build more packages, or build specific revisions, the
``buildpackage`` script can also build from an archive::

   FROM logilab/cubicweb:buster-buildpackage
   RUN buildpackage -u https://hg.logilab.org/master/cubes/comment/archive/tip.tar.gz
   COPY . /src
   RUN buildpackage -d /src

   [...]