README cleanup, add optionnal deployment of apache and postgresql, make cubicweb version configurable and additional apache configuration
authorArthur Lutz <arthur.lutz@logilab.fr>
Thu, 23 Oct 2014 17:05:05 +0200
changeset 10 f0ac4b0af639
parent 9 804f4c3bb3ea
child 11 25876e1446db
README cleanup, add optionnal deployment of apache and postgresql, make cubicweb version configurable and additional apache configuration
README.rst
cubicweb/apache/vhost.sls
cubicweb/apache/vhost.tmpl
cubicweb/init.sls
pillar.example
--- a/README.rst	Thu Oct 23 16:52:23 2014 +0200
+++ b/README.rst	Thu Oct 23 17:05:05 2014 +0200
@@ -9,237 +9,27 @@
     See the full `Salt Formulas installation and usage instructions
     <http://docs.saltstack.com/topics/conventions/formulas.html>`_.
 
-Available states
-================
 
-.. contents::
-    :local:
-
-``cubicweb``
-----------
-
-Install Cubicweb from the system package manager. Note, the Cubicweb version
-available varies by platform.
-
-Example usage::
-
-    include:
-      - cubicweb
-
-    mysite:
-      git:
-        - latest
-        - name: git@git.example.com/mysite
-        - target: /var/www/mysite
-        - require:
-            - pkg: cubicweb
-
-
-Full-stack App Deployment
-=========================
-
-This formula also provides an example of how Salt can be used to deploy a
-Cubicweb app in a single command, using the `OverState System`_. It installs
-Cubicweb into a virtualenv, using pip with a requirements.txt.
-
-.. _`OverState System`: http://docs.saltstack.com/ref/states/overstate.html
-
-This example makes use of the following three files:
+Quick start
+===========
 
-* `pillar.example`_ - Pillar data
-* `overstate.single`_ - Single-host OverState deployment stages 
-* `overstate.multi`_ - Multi-host OverState deployment stages
-
-.. _pillar.example: https://github.com/saltstack-formulas/cubicweb-formula/blob/master/pillar.example
-.. _overstate.single: https://github.com/saltstack-formulas/cubicweb-formula/blob/master/overstate.single
-.. _overstate.multi: https://github.com/saltstack-formulas/cubicweb-formula/blob/master/overstate.multi
-
-Deploying this example will require that the relevant files from above (the
-Pillar data and appropriate OverState config file) are copied to the Master and
-edited as necessary. The Pillar data will need to be available to all involved
-minions.
-
-Additionally, this example makes use of several other Salt formulae:
-
-* `apache-formula`_
-* `postgres-formula`_
+* set your pillars from pillar.example
+* add cubicweb state to your top.sls
+* salt 'host' saltutil.refresh_pillars
+* salt 'host' state.highstate
 
-.. _apache-formula: https://github.com/saltstack-formulas/apache-formula.git
-.. _postgres-formula: https://github.com/saltstack-formulas/postgres-formula.git
-
-To deploy the entire stack (Apache, PostgreSQL, Cubicweb, cubes) to a
-single host, run the following command:
-
-.. code-block:: bash
-
-    # salt-run state.over deployment /path/to/overstate.single
-
-To deploy using one database server, one Apache server and one or more
-Cubiweb servers, run the following command:
-
-.. code-block:: bash
-
-    # salt-run state.over deployment /path/to/overstate.multi
-
-.. note::
-
-   If you did not create a separate ``deployment`` branch, then
-   replace ``deployment`` with ``base`` in the above ``salt-run``
-   commands.
 
 
-Other Tips
-==========
-
-Create ``settings.py`` using data from Pillar
----------------------------------------------
-
-The easiest way to create Cubicweb's ``setttings.py`` file using data from Pillar
-is to simply transform a dictionary in YAML into a dictionary in Python.
-
-``/srv/salt/mysite.sls``::
-
-    include:
-      - cubicweb.pip
-
-    mysite:
-      git:
-        - latest
-        - name: git@git.example.com/mysite
-        - target: /var/www/mysite
-        - require:
-            - pip: cubicweb_pip
-
-    mysite_settings:
-      file:
-        - managed
-        - name: /var/www/mysite/settings.py
-        - contents: |
-            globals().update({{ salt['pillar.get']('mysite:settings') | python() | indent(8) }})
-        - require:
-          - git: mysite
-
-``/srv/pillar/mysite.sls``::
+Additionnal apache configuration
+--------------------------------
 
-    mysite:
-      settings:
-        ROOT_URLCONF: mysite.urls
-        SECRET_KEY: 'gith!)on!_dq0=2l(otd67%#0urmrk6_d0!zu)i9fn=!8_g5(c'
-        DATABASES:
-          default:
-            ENGINE: cubicweb.db.backends.mysql
-            NAME: mysitedb
-            USER: mysiteuser
-            PASSWORD: mysitepass
-            HOST: localhost
-            PORT: 3306
-        TEMPLATE_DIRS:
-          - /var/www/mysite/cubicweb-tutorial/templates
-        STATICFILES_DIRS:
-          - /var/www/mysite/cubicweb-tutorial/static
-        STATIC_ROOT: /var/www/mysite/cubicweb-tutorial/staticroot
+You can deploy additional configuration files to ::
 
-Create ``settings.py`` with a template file
--------------------------------------------
-
-A more traditional (and flexible) method of creating the ``settings.py`` file
-is to actually create the file as a template.
-
-``/srv/salt/mysite/mysite.sls``::
-
-    include:
-      - cubicweb.pip
-
-    mysite:
-      git:
-        - latest
-        - name: git@git.example.com/mysite
-        - target: /var/www/mysite
-        - require:
-            - pip: cubicweb_pip
+  /etc/apache2/siteconf.d/{{ vhostname }}/*.conf
 
-    mysite_settings:
-      file:
-        - managed
-        - name: /var/www/mysite/settings.py
-        - source: salt://mysite/files/settings-tmpl.py
-        - template: jinja
-        - require:
-          - git: mysite
-
-``/srv/salt/mysite/files/settings-tmpl.py``::
-
-    {# Data can be defined inline, in Grains, in Pillar, etc #}
-
-    {% set db_settings = {
-        'default': {
-            'ENGINE': 'cubicweb.db.backends.mysql',
-            'HOST': 'localhost',
-            'NAME': 'polldb',
-            'PASSWORD': 'pollpass',
-            'PORT': 3306,
-            'USER': 'polluser',
-        }
-    } %}
-
-    {% set staticfiles_dirs_settings = [
-        '/var/www/poll/cubicweb-tutorial/static',
-    ] %}
-
-    {% set template_dirs_settings = [
-        '/var/www/poll/cubicweb-tutorial/templates',
-    ] %}
-
-    ROOT_URLCONF = mysite.urls
-
-    {# Have Salt automatically generate the SECRET_KEY for this minion #}
-    SECRET_KEY = '{{ salt['grains.get_or_set_hash']('mysite:SECRET_KEY', 50) }}'
-
-    DATABASES = {{ db_settings | python() }}
-
-    TEMPLATE_DIRS = {{ template_dirs_settings | python() }}
-
-    STATICFILES_DIRS = {{ staticfiles_dirs_settings | python() }}
 
-    STATIC_ROOT = /var/www/mysite/cubicweb-tutorial/staticroot
-
-Run ``syncdb`` or ``collectstatic`` automatically
--------------------------------------------------
-
-A wait state can be used to trigger ``cubicweb-admin.py syncdb`` or
-``cubicweb-admin.py collectstatic`` automatically. The following example runs
-both commands whenever the Git repository containing the "mysite" Cubicweb
-project is updated.
-
-::
-
-    include:
-      - cubicweb.pip
+Create ``all-in-one.conf`` using data from pillars
+--------------------------------------------------
 
-    mysite:
-      git:
-        - latest
-        - name: git@git.example.com/mysite
-        - target: /var/www/mysite
-        - require:
-            - pip: cubicweb_pip
+TODO
 
-    mysite_syncdb:
-      module:
-        - wait
-        - name: cubicweb.syncdb
-        - settings_module: "mysite.settings"
-        - bin_env: /path/to/virtualenv          # optional
-        - pythonpath: /path/to/mysite_project   # optional
-        - watch:
-          - git: mysite
-
-    mysite_collectstatic:
-      module:
-        - wait
-        - name: cubicweb.collectstatic
-        - settings_module: "mysite.settings"
-        - bin_env: /path/to/virtualenv          # optional
-        - pythonpath: /path/to/mysite_project   # optional
-        - watch:
-          - git: mysite
--- a/cubicweb/apache/vhost.sls	Thu Oct 23 16:52:23 2014 +0200
+++ b/cubicweb/apache/vhost.sls	Thu Oct 23 17:05:05 2014 +0200
@@ -14,6 +14,20 @@
     - require:
       - file: /var/www/{{ vhostname }}
 
+
+/etc/apache2/siteconf.d/{{ vhostname }}/000_default.conf:
+    file.managed:
+        - pillar_contents: apache:vhost_conf
+        - makedirs: True
+        - watch_in:
+            - service: apache2
+
+{% if vhost.get('apache') %}
+/etc/apache2/siteconf.d/{{ vhostname }}/999_extra.conf:
+    file.managed:
+        - pillar_contents: cubicweb_apps:{{ vhostname }}:apache
+{% endif %}
+
 /var/www/{{vhostname}}:
   file.directory:
     - user: www-data
@@ -27,6 +41,7 @@
     - unless: '[ -a /etc/apache2/sites-enabled/{{vhostname}} ]'
     - require:
       - pkg: apache2
+      - file: /etc/apache2/sites-available/{{ vhostname }}
 
 extend:
   apache2:
@@ -35,3 +50,15 @@
         - cmd: site-{{vhostname}}
 
 {% endfor %}
+
+{% if pillar['apache'].get('disable_default', False) %}
+disable default apache:
+    cmd.run:
+        - name: 'a2dissite default'
+        - onlyif: '[ -a /etc/apache2/sites-enabled/000-default ]'
+        - require:
+            - pkg: apache2
+        - watch_in:
+            - service: apache2
+{% endif %}
+
--- a/cubicweb/apache/vhost.tmpl	Thu Oct 23 16:52:23 2014 +0200
+++ b/cubicweb/apache/vhost.tmpl	Thu Oct 23 17:05:05 2014 +0200
@@ -1,4 +1,4 @@
-<VirtualHost 1.2.3.4:80>
+<VirtualHost *:80>
     ServerName  {{ vhostname }}
     DocumentRoot /var/www/{{ vhostname }}
     ServerAdmin sysadmin@logilab.fr
@@ -6,7 +6,7 @@
     CustomLog /var/log/apache2/{{ vhostname }}-access.log combined2
     ErrorLog /var/log/apache2/{{ vhostname }}-error.log
 
-    <Directory /home/logilab/www/{{ vhostname }}>
+    <Directory /var/www/{{ vhostname }}>
         Order allow,deny
         Allow from all
         Options MultiViews FollowSymLinks
@@ -19,6 +19,7 @@
         Allow from all
     </Proxy>
 {% for name, config in vhost.iteritems() %}
+    Include /etc/apache2/siteconf.d/{{ vhostname }}/*.conf
     ProxyPass {{ config.cwapp.basepath }} http://{{config.cwapp.backend}}:{{ config.cwapp.port }}/
     ProxyPassReverse {{ config.cwapp.basepath }} http://{{ config.cwapp.backend }}:{{ config.cwapp.port }}/
 {% endfor %}
--- a/cubicweb/init.sls	Thu Oct 23 16:52:23 2014 +0200
+++ b/cubicweb/init.sls	Thu Oct 23 17:05:05 2014 +0200
@@ -1,5 +1,12 @@
 include:
   - logilab.public
+  - cubicweb.instances
+{% if salt['pillar.get']('apache:enable', False) %}
+  - cubicweb.apache.vhost
+{% endif %}
+{% if salt['pillar.get']('postgresql:enable', False) %}
+  - cubicweb.postgres
+{% endif %}
 
 cw-deps:
   pkg.installed:
@@ -22,11 +29,18 @@
 cw-core:
   pkg.installed:
     - pkgs:
+    {% if pillar.get('cubicweb_version') %}
+      - cubicweb-common: {{ pillar.get('cubicweb_version') }}
+      - cubicweb-ctl: {{ pillar.get('cubicweb_version') }}
+      - cubicweb-server: {{ pillar.get('cubicweb_version') }}
+      - cubicweb-postgresql-support: {{ pillar.get('cubicweb_version') }}
+    {% else %}
       - cubicweb-common
       - cubicweb-ctl
       - cubicweb-postgresql-support
       - cubicweb-server
     - fromrepo: {{pillar.get('cubicweb_repo', 'production')}}
+    {% endif %}
     - require:
       - pkg: cw-logilab-deps
 
@@ -49,8 +63,14 @@
 cw-web:
   pkg.installed:
     - pkgs:
+    {% if pillar.get('cubicweb_version') %}
+      - cubicweb-web : {{ pillar.get('cubicweb_version') }}
+      - cubicweb-twisted : {{ pillar.get('cubicweb_version') }} 
+    {% else %}
       - cubicweb-web
       - cubicweb-twisted
     - fromrepo: {{pillar.get('cubicweb_repo', 'production')}}
+    {% endif %}
     - require:
       - pkg: cw-core
+
--- a/pillar.example	Thu Oct 23 16:52:23 2014 +0200
+++ b/pillar.example	Thu Oct 23 17:05:05 2014 +0200
@@ -1,3 +1,15 @@
+apache: 
+  enable: False # set to True if you want to deploy an apache in proxy mode
+  disable_default: False # set to True to disable default apache configuration
+  enable_modules:
+    - rewrite #list of modules to enable
+  vhost_conf: | 
+    # insert inline configuration
+    # lines here, valid for all vhosts
+postgresql:
+  enable: False # set to True to install postgresql on same host
+cubicweb_repo: production # set to acceptance if you want more recent versions of cubicweb 
+#cubicweb_version: # enter debian package version to pin down your install, cubicweb_repo is then ignored
 cubicweb_apps:
   www.cubicweb.org:
     demo1:
@@ -7,7 +19,7 @@
         password: logidemo
         host: db1
         port: 5432
-    cwapp:
+      cwapp:
         basepath: /demo/blog
         port: 9090
         adminuser: admin
@@ -28,3 +40,5 @@
         adminpass: admin
         cubes:
           blog: cubicweb-blog
+      apache: | 
+        #additional configuration specific to demo2