[docs] split existing documentation into separate files
authorArthur Lutz <arthur.lutz@logilab.fr>
Fri, 31 May 2013 17:20:12 +0200
changeset 149 eba16c8899d4
parent 148 82e6e462810a
child 150 cd599c842067
[docs] split existing documentation into separate files
docs/how-it-works.rst
docs/how-to-build-an-ami.rst
docs/index.rst
docs/main-files.rst
docs/upload-images.rst
docs/using-on-local-machine.rst
readme.rst
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/how-it-works.rst	Fri May 31 17:20:12 2013 +0200
@@ -0,0 +1,32 @@
+
+How it works
+============
+
+All the operations are currently done in /etc/rc.local. The content of the file
+is explained in the following lines.
+
+The first step parses the user-data related to our ami
+
+::
+  /usr/bin/env python /etc/salt/srv/others/ec2_ami_parser.py
+
+The next step updates the salt states, if necessary by getting the necessary
+files from the hg repository
+
+::
+  salt-call state.highstate
+  salt-call state.highstate
+
+(doing it twice is necessary to load new state/modules from the hg repository). Next,
+
+::
+  salt-call state.sls ami_builder
+
+includes all the operations done in the image. The operations can be grouped in three steps:
+
+    1. preparation (ami_pre): download the image, uncompress it, resize the fs,
+       mount it, modify the relevant files to work in a chroot
+    2. update the image, download and execute the variation files from the mercurial variation URL
+    3. postprocessing (ami_post): unmount, resize and upload the image either
+       to amazon or to openstack depending on user_data
+
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/how-to-build-an-ami.rst	Fri May 31 17:20:12 2013 +0200
@@ -0,0 +1,85 @@
+How to build an AMI
+===================
+
+Prerequisite : having an AMI builder image on AWS or OpenStack
+
+Creating an AMI is as simple as instanciating the ami-creator image in
+your cloud infrastructure with proper user-data content.
+
+What will be installed and configured in the built AMI image will
+depend on the configuration description that will be used as SaltStack
+highstate_.
+
+.. _highstate: http://docs.saltstack.com/ref/states/highstate.html
+
+So to summarize, the process of building a customized AMI image is:
+
+1. Edit/commit/push the file "variation.sls" and all its dependencies
+   in a Mercurial repository (must be accessible from the cloud
+   instance) so that a:
+
+.. sourcecode:: bash
+     user@host:~$ salt-call state.sls variation.sls
+
+   will create the machine you want.
+
+2. Then, launch an instance of the "ami-creator" image in your
+   cloud. Use at least 5Gb of disk (add more if your sls file requires
+   more).
+
+   In the userdata form of your instance, copy something like (if you
+   do not put anything, what will be used will be the latest revision
+   from the default repo).
+
+   Warning : the file needs to un-indented (remove whitespaces at beginning of lines) ::
+
+     ====== START =======
+     #salt-grains #interpreter flag
+     ### necessary unless you want the standard image ###
+     ami.hg_address_variation: http://hg.logilab.org/users/ptonelli/salt-ami-cloud-builder/variation
+     ami.hg_rev_variation: 9ed741a55f17 #optionnal
+     ami.name: my_variation
+     ami.type: ubuntu #or debian
+
+     ami.source_address: http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64.tar.gz
+     ami.source_hash: md5=1d72ed9c56abb899be02e7cae0822f1d
+
+     ### optionnal if your image uses existing kernel/ramdisk id in openstack ###
+     kernel_id:a5b900bd-c009-40b2-a763-32b32996b1ee
+     initrd_id:b953e40a-2605-402b-8663-c8556a5899f2
+
+     ### optionnal (ami_creation code update) ###
+     ami.hg_address_server: http://hg.logilab.org/master/salt-ami-cloud-builder/
+
+     ### optionnal (necessary to upload to openstack) ###
+     keystone.user: username
+     keystone.password: password
+     keystone.tenant: tenant
+     keystone.tenant_id: 00000000000000000000000000000000
+     keystone.auth_url: 'http://control.example.com:5000/v2.0'
+     keystone.insecure: False   #(optional)
+
+     ### optionnal (necessary to upload to amazon aws) ###
+     ### remove the BEGIN and END statements found in the files ###
+     ### do not forget the ' ' at each newline for ConfigParser ###
+     aws.region: eu-west-1
+     aws.private_key: 0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+     aws.certificate: 0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+      0000000000000000000000000000000000000000000000000000000000000000
+
+     ====== END ========
+
+3. you can then connect to your instance and wait until everything in ``/mnt``
+   is unmounted. Upload to glance (openstack) or amazon aws is automatic if you
+   provided the correct user data. 
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/index.rst	Fri May 31 17:20:12 2013 +0200
@@ -0,0 +1,11 @@
+salt-ami-cloud builder documentation
+====================================
+
+
+.. toc
+
+* using-on-local-machine
+* main-files
+* how-it-works
+* how-to-build-an-ami
+* upload-images
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/main-files.rst	Fri May 31 17:20:12 2013 +0200
@@ -0,0 +1,13 @@
+
+Main Files
+----------
+
+minion: must be copied to /etc/salt, provides a masterless configuration for salt
+
+ami_builder.sls: state file regrouping all operations to create the new image 
+
+ami_pre: state folder (contains a few sls files) applied before the creation of variation
+
+ami_post: state folder applied after the creation of variation for
+cleanup
+
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/upload-images.rst	Fri May 31 17:20:12 2013 +0200
@@ -0,0 +1,69 @@
+
+Manual upload to openstack (glance)
+===================================
+
+This procedure should not be necessary, the openstack client is functionnal and
+will be run, provided you include the relevant user data.
+
+After the end of the creation process (nothing is mounted in /mnt/variation and
+you can see a .img, a kernel and a ramdisk in /mnt/output), you can upload your
+image to glance manually, provided you have credentials as a openrc.sh file
+downloaded from openstack.
+
+1. First, add your credentials to path (change ):
+
+.. sourcecode:: bash  
+  root@machine~# . /${correct_path}/openrc.sh 
+
+2. Then, if necessary, define a name for your ami:
+
+.. sourcecode:: bash
+  root@machine~# export ami_name=variation
+
+3. And change to the correct directory:
+
+.. sourcecode:: bash
+  root@machine~# cd /mnt/output 
+
+4. If necessary, upload the kernel and the ramdisk:
+
+.. sourcecode:: bash
+  root@machine~# glance add is_public=false disk_format=aki container_format=aki \
+  name=${ami_name}_kernel < kernel_variation
+
+.. sourcecode:: bash
+  root@machine~# glance add is_public=false disk_format=ari container_format=ari \
+  name=${ami_name}_ramdisk < ramdisk_variation
+
+5. Finally, upload the image after replacing the kernel_id and ramdisk_id
+   according to the previous uploads:
+
+.. sourcecode:: bash
+  root@machine~# glance add disk_format=ami container_format=ami name="${ami_name}" \
+  kernel_id=278de93c-5276-410f-a221-667cc1443cf8 \
+  ramdisk_id=b953e40a-2605-402b-8663-c8556a5899f2 < /mnt/variation.img
+
+Manual upload to ec2
+====================
+
+This procedure should not be necessary, the amazon ec2 client is functionnal and
+will be run, provided you include the relevant user data.
+
+How to upload the ami you created on amazon manually (from_):
+
+1. Create an EBS volume of the minimal size to host the disk image
+
+2. attach the created volume to salt-ami-cloud-builder: 
+
+3. transfer the content of the image to the EBS volume (replace xvdf by correct
+   device name):
+
+.. sourcecode:: bash
+  ubuntu@machine:~$ sudo dd bs=16M if=/mnt/output/variation.img of=/dev/xvdf
+
+4. Detach the volume, take a snapshot, create an ami image from the amazon ec2
+   web interface. Make sure to use the correct "aki" (kernel image) as
+   indicated on documentation_ (for eu, use aki-71665e05).
+
+::_from: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_LaunchingInstanceFromSnapshot.html
+::_documentation: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UserProvidedkernels.html
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/using-on-local-machine.rst	Fri May 31 17:20:12 2013 +0200
@@ -0,0 +1,50 @@
+
+Using salt-ami-cloud-builder on a local machine
+===============================================
+
+This should only be used is no machine is available. If an salt-ami-cloud-builder image is
+already available, juste launch it with the necessary user data:
+
+    1. ami.type: ubuntu
+    2. a correct source/md5sum for an ubuntu image
+    3. use the server repository as variation address.
+    4. provide your correct credentials
+
+This should create and upload an up to date custom AMI image
+
+On Ubuntu, the first step is to add the salt PPA to install it:
+
+.. sourcecode:: bash
+  root@machine~# add-apt-repository -y ppa:saltstack/salt
+
+The next step is to install salt-minion and mercurial:
+
+.. sourcecode:: bash
+  root@machine~# aptitude update && aptitude install salt-minion mercurial
+
+On CentOS, enable EPEL and install salt-minon & mercurial : 
+
+.. sourcecode:: bash
+  root@machine~# wget http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
+  root@machine~# rpm -Uvh epel-release-5-4.noarch.rpm
+  root@machine~# yum install salt-minion mercurial
+
+
+Then, clone the server repository:
+
+.. sourcecode:: bash
+  root@machine~# hg clone http://hg.logilab.org/master/salt-ami-cloud-builder/ /etc/salt/srv
+
+Copy the necessary scripts so the ami creation process is launched at boot:
+
+.. sourcecode:: bash
+  root@machine~# cp /etc/salt/srv/others/rc.local /etc/rc.local
+
+and copy the configuration to make salt-minion masterless:
+
+.. sourcecode:: bash
+  root@machine~# cp /etc/salt/srv/minion /etc/salt/
+
+save this machine as an ami-creator image and you can then use it by simply
+adding relevant user data at boot
+
--- a/readme.rst	Wed May 15 10:46:11 2013 +0200
+++ b/readme.rst	Fri May 31 17:20:12 2013 +0200
@@ -20,247 +20,4 @@
 .. _SaltStack: http://docs.saltstack.com/
 .. _Mercurial: http://mercurial.selenic.com/
 
-How to create an AMI
-====================
 
-Creating an AMI is as simple as instanciating the ami-creator image in
-your cloud infrastructure with proper user-data content.
-
-What will be installed and configured in the built AMI image will
-depend on the configuration description that will be used as SaltStack
-highstate_.
-
-.. _highstate: http://docs.saltstack.com/ref/states/highstate.html
-
-So to summarize, the process of building a customized AMI image is:
-
-1. Edit/commit/push the file "variation.sls" and all its dependencies
-   in a Mercurial repository (must be accessible from the cloud
-   instance) so that a:
-
-.. sourcecode:: bash
-     user@host:~$ salt-call state.sls variation.sls
-
-   will create the machine you want.
-
-2. Then, launch an instance of the "ami-creator" image in your
-   cloud. Use at least 5Gb of disk (add more if your sls file requires
-   more).
-
-   In the userdata form of your instance, copy something like (if you
-   do not put anything, what will be used will be the latest revision
-   from the default repo).
-
-   Warning : the file needs to un-indented (remove whitespaces at beginning of lines) ::
-
-     ====== START =======
-     #salt-grains #interpreter flag
-     ### necessary unless you want the standard image ###
-     ami.hg_address_variation: http://hg.logilab.org/users/ptonelli/salt-ami-cloud-builder/variation
-     ami.hg_rev_variation: 9ed741a55f17 #optionnal
-     ami.name: my_variation
-     ami.type: ubuntu #or debian
-
-     ami.source_address: http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64.tar.gz
-     ami.source_hash: md5=1d72ed9c56abb899be02e7cae0822f1d
-
-     ### optionnal if your image uses existing kernel/ramdisk id in openstack ###
-     kernel_id:a5b900bd-c009-40b2-a763-32b32996b1ee
-     initrd_id:b953e40a-2605-402b-8663-c8556a5899f2
-
-     ### optionnal (ami_creation code update) ###
-     ami.hg_address_server: http://hg.logilab.org/master/salt-ami-cloud-builder/
-
-     ### optionnal (necessary to upload to openstack) ###
-     keystone.user: username
-     keystone.password: password
-     keystone.tenant: tenant
-     keystone.tenant_id: 00000000000000000000000000000000
-     keystone.auth_url: 'http://control.example.com:5000/v2.0'
-     keystone.insecure: False   #(optional)
-
-     ### optionnal (necessary to upload to amazon aws) ###
-     ### remove the BEGIN and END statements found in the files ###
-     ### do not forget the ' ' at each newline for ConfigParser ###
-     aws.region: eu-west-1
-     aws.private_key: 0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-     aws.certificate: 0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-      0000000000000000000000000000000000000000000000000000000000000000
-
-     ====== END ========
-
-3. you can then connect to your instance and wait until everything in ``/mnt``
-   is unmounted. Upload to glance (openstack) or amazon aws is automatic if you
-   provided the correct user data. 
-
-Main Files
-----------
-
-minion: must be copied to /etc/salt, provides a masterless configuration for salt
-
-ami_builder.sls: state file regrouping all operations to create the new image 
-
-ami_pre: state folder (contains a few sls files) applied before the creation of variation
-
-ami_post: state folder applied after the creation of variation for
-cleanup
-
-Additionnal Documentation
-=========================
-
-All the operations are currently done in /etc/rc.local. The content of the file
-is explained in the following lines.
-
-The first step parses the user-data related to our ami
-
-::
-  /usr/bin/env python /etc/salt/srv/others/ec2_ami_parser.py
-
-The next step updates the salt states, if necessary by getting the necessary
-files from the hg repository
-
-::
-  salt-call state.highstate
-  salt-call state.highstate
-
-(doing it twice is necessary to load new state/modules from the hg repository). Next,
-
-::
-  salt-call state.sls ami_builder
-
-includes all the operations done in the image. The operations can be grouped in three steps:
-
-    1. preparation (ami_pre): download the image, uncompress it, resize the fs,
-       mount it, modify the relevant files to work in a chroot
-    2. update the image, download and execute the variation files from the mercurial variation URL
-    3. postprocessing (ami_post): unmount, resize and upload the image either
-       to amazon or to openstack depending on user_data
-
-Bootstrapping an salt-ami-cloud-builder machine
-===============================================
-
-This should only be used is no machine is available. If an salt-ami-cloud-builder image is
-already available, juste launch it with the necessary user data:
-
-    1. ami.type: ubuntu
-    2. a correct source/md5sum for an ubuntu image
-    3. use the server repository as variation address.
-    4. provide your correct credentials
-
-This should create and upload an up to date custom AMI image
-
-On Ubuntu, the first step is to add the salt PPA to install it:
-
-.. sourcecode:: bash
-  root@machine~# add-apt-repository -y ppa:saltstack/salt
-
-The next step is to install salt-minion and mercurial:
-
-.. sourcecode:: bash
-  root@machine~# aptitude update && aptitude install salt-minion mercurial
-
-On CentOS, enable EPEL and install salt-minon & mercurial : 
-
-.. sourcecode:: bash
-  root@machine~# wget http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
-  root@machine~# rpm -Uvh epel-release-5-4.noarch.rpm
-  root@machine~# yum install salt-minion mercurial
-
-
-Then, clone the server repository:
-
-.. sourcecode:: bash
-  root@machine~# hg clone http://hg.logilab.org/master/salt-ami-cloud-builder/ /etc/salt/srv
-
-Copy the necessary scripts so the ami creation process is launched at boot:
-
-.. sourcecode:: bash
-  root@machine~# cp /etc/salt/srv/others/rc.local /etc/rc.local
-
-and copy the configuration to make salt-minion masterless:
-
-.. sourcecode:: bash
-  root@machine~# cp /etc/salt/srv/minion /etc/salt/
-
-save this machine as an ami-creator image and you can then use it by simply
-adding relevant user data at boot
-
-Manual upload to openstack (glance)
-===================================
-
-This procedure should not be necessary, the openstack client is functionnal and
-will be run, provided you include the relevant user data.
-
-After the end of the creation process (nothing is mounted in /mnt/variation and
-you can see a .img, a kernel and a ramdisk in /mnt/output), you can upload your
-image to glance manually, provided you have credentials as a openrc.sh file
-downloaded from openstack.
-
-1. First, add your credentials to path (change ):
-
-.. sourcecode:: bash  
-  root@machine~# . /${correct_path}/openrc.sh 
-
-2. Then, if necessary, define a name for your ami:
-
-.. sourcecode:: bash
-  root@machine~# export ami_name=variation
-
-3. And change to the correct directory:
-
-.. sourcecode:: bash
-  root@machine~# cd /mnt/output 
-
-4. If necessary, upload the kernel and the ramdisk:
-
-.. sourcecode:: bash
-  root@machine~# glance add is_public=false disk_format=aki container_format=aki \
-  name=${ami_name}_kernel < kernel_variation
-
-.. sourcecode:: bash
-  root@machine~# glance add is_public=false disk_format=ari container_format=ari \
-  name=${ami_name}_ramdisk < ramdisk_variation
-
-5. Finally, upload the image after replacing the kernel_id and ramdisk_id
-   according to the previous uploads:
-
-.. sourcecode:: bash
-  root@machine~# glance add disk_format=ami container_format=ami name="${ami_name}" \
-  kernel_id=278de93c-5276-410f-a221-667cc1443cf8 \
-  ramdisk_id=b953e40a-2605-402b-8663-c8556a5899f2 < /mnt/variation.img
-
-Manual upload to ec2
-====================
-
-This procedure should not be necessary, the amazon ec2 client is functionnal and
-will be run, provided you include the relevant user data.
-
-How to upload the ami you created on amazon manually (from_):
-
-1. Create an EBS volume of the minimal size to host the disk image
-
-2. attach the created volume to salt-ami-cloud-builder: 
-
-3. transfer the content of the image to the EBS volume (replace xvdf by correct
-   device name):
-
-.. sourcecode:: bash
-  ubuntu@machine:~$ sudo dd bs=16M if=/mnt/output/variation.img of=/dev/xvdf
-
-4. Detach the volume, take a snapshot, create an ami image from the amazon ec2
-   web interface. Make sure to use the correct "aki" (kernel image) as
-   indicated on documentation_ (for eu, use aki-71665e05).
-
-::_from: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_LaunchingInstanceFromSnapshot.html
-::_documentation: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UserProvidedkernels.html