Skip to content
This repository has been archived by the owner on Jun 18, 2021. It is now read-only.

Ns web vm deployment #255

Merged
merged 14 commits into from
Apr 13, 2017
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ansible/server-state-playbooks/nightshade-web/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
roles
14 changes: 14 additions & 0 deletions ansible/server-state-playbooks/nightshade-web/README.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
### Taken from manics/ansible-public-omero-example.git
### at bc730e580e7c9ed0752a68cd4aa42397e4e58a2a
### and stripped of server components, leaving just web.

### ansible playbooks & requirements for installing basic OMERO web


- playbooks set up to run from localhost rather than remotely

- after installing ansible and ansible-galaxy,
ansible-galaxy install -r requirements.yml -p roles

- install OMERO.web server
ansible-playbook playbook.yml
14 changes: 14 additions & 0 deletions ansible/server-state-playbooks/nightshade-web/Vagrantfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Vagrant.configure("2") do |config|
config.vm.box = "centos/7"
config.vm.provider "virtualbox" do |vb|
config.vm.network "forwarded_port", guest: 80, host: 8080
config.vm.network "forwarded_port", guest: 4064, host: 4064
config.vm.network "forwarded_port", guest: 4063, host: 4063
vb.customize ["modifyvm", :id, "--memory", "2048"]
end

config.vm.provision "ansible" do |ansible|
ansible.playbook = "playbook.yml"
ansible.galaxy_role_file = "requirements.yml"
end
end
184 changes: 184 additions & 0 deletions ansible/server-state-playbooks/nightshade-web/playbook.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
# Install OMERO.web with a public user on localhost

- hosts: all

handlers:

# Handler for nginx
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The omero-common role already includes a set of useful handlers, you should be able to use that instead of redefining one here.

Copy link
Member Author

@kennethgillen kennethgillen Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 - I didn't realise I could use things defined 'underneath' the playbook when I wrote that. From programming, I wouldn't have expected global scope for defined functions like that. I subsequently used e.g. the 'restart systemd' handler too, so you can see I learned, but didn't strip the original nginx one from the playbook.

- name: restart nginx
become: yes
service:
name: nginx
state: restarted

pre_tasks:

# For OMERO.web apps
# Pythonpath must exist before omero.web.config is provisioned
# or OMERO.web won't start.
# create systemd file addition for PYTHONPATH
- name: OMERO.web apps | (pre_task) configure systemd for pythonpath - create config folder
become: yes
file:
dest: "/etc/systemd/system/omero-web.service.d"
state: directory
mode: "u=rwx,go=rx"
owner: "root"
group: "root"

# systemd web-apps folder to pythonpath to env
- name: OMERO.web apps | (pre_task) configure systemd for pythonpath
become: yes
blockinfile:
create: yes
destfile: /etc/systemd/system/omero-web.service.d/pythonpath.conf
owner: "root"
group: "root"
block: |2+
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need the fancy formatting provided by |2+? Will just plain | do?

Copy link
Member Author

@kennethgillen kennethgillen Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't make it work any other way, so leaving it as 'this works'. Other options failed completely with errors, or didn't give the indenting in the destination. I am pretty new to YAML and the whitespace significance, though.

[Service]
Environment="PYTHONPATH=$PYTHONPATH:/opt/omero/web/web-extensions"
notify:
- reload systemd

roles:

# Root LV Size
- role: openmicroscopy.lvm-partition
lvm_lvname: "{{ provision_root_lvname }}"
lvm_vgname: "{{ provision_root_vgname }}"
lvm_lvmount: /
lvm_lvsize: "{{ provision_rootsize }}"
lvm_lvfilesystem: "{{ provision_root_filesystem }}"

# OMERO.web configuration in host_vars in different repository
- role: openmicroscopy.omero-web
omero_web_release: 5.2.8

- role: openmicroscopy.system-monitor-agent
when: >
((ansible_env.COBBLER_SERVER is defined)
and (ansible_env.COBBLER_SERVER == "spacewalk.lifesci.dundee.ac.uk"))

post_tasks:
- name: Install open-vm-tools if system is a VMware vm
become: yes
yum:
name: open-vm-tools
state: latest
when: >
((ansible_virtualization_type is defined)
and (ansible_virtualization_type == "VMware"))

- name: NGINX - SSL File Deployment - prepare directory
become: yes
file:
path: "{{ nginx_ssl_files_path }}"
state: directory
owner: root
group: root
mode: "u=r,go="

- name: NGINX - SSL File Deployment
become: yes
copy:
dest="{{ item.key }}"
content="{{ item.value.content }}"
owner="{{ item.value.owner }}"
group="{{ item.value.group }}"
mode="{{ item.value.mode }}"
with_dict: "{{ nginx_ssl_cert_files }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with_dict looks a bit weird here, though I haven't seen the actual variable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, looks overly complicated since the owner/group/mode don't vary

no_log: true

# post 2.3 'destfile' should be renamed 'path'
- name: NGINX - SSL Configuration - Additional listen port
become: yes
lineinfile:
destfile: /etc/nginx/conf.d/omero-web.conf
insertafter: ' listen 80;'
line: ' listen 443 ssl;'

# post 2.3 'destfile' should be renamed 'path'
- name: NGINX - SSL Configuration - Rest of SSL section to omero-web.conf
become: yes
blockinfile:
destfile: /etc/nginx/conf.d/omero-web.conf
insertbefore: '.*sendfile.*'
block: |2+

ssl_certificate {{ nginx_ssl_files_path }}/{{ nginx_ssl_cert_filename }};
ssl_certificate_key {{ nginx_ssl_files_path }}/{{ nginx_ssl_key_filename }};
ssl_protocols {{ nginx_ssl_protocols }}

if ($ssl_protocol = "") {
rewrite ^/(.*) https://$host/$1 permanent;
}
notify:
- restart nginx

# 'manual' install of omero.web.apps for Nightshade feature parity

# note: system user var defined in openmicroscopy.omero-web as a default
- name: OMERO.web apps | top-level folder
become: yes
file:
path: "{{ omero_web_extensionsdir }}"
state: directory
owner: "{{ omero_web_system_user }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this directory need to be writeable by the web user? And similarly for the tasks below.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, but this works, and it's how it's set up elsewhere. If you've another suggestion, happy to hear it.

Copy link
Member

@manics manics Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it owned by root unless you have reason to believe OMERO.web needs write access to its code.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try it.

group: "{{ omero_web_system_user }}"
mode: "u=rwx,go=rx"

# download figure
- name: OMERO.web apps | download latest figure
become: yes
unarchive:
src: https://downloads.openmicroscopy.org/latest/figure.zip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For full reproducibility you should use the versioned zip https://downloads.openmicroscopy.org/figure/1.2.1/figure-1.2.1.zip since you've got creates: "{{ omero_web_extensionsdir }}/figure-{{omero_web_extensions_figure_ver}}" below

owner: "{{ omero_web_system_user }}"
group: "{{ omero_web_system_user }}"
mode: "u=rwx,go=rx"
dest: "{{ omero_web_extensionsdir }}"
remote_src: True
creates: "{{ omero_web_extensionsdir }}/figure-{{omero_web_extensions_figure_ver}}"

# download tagging
- name: OMERO.web apps | download latest tagging
become: yes
unarchive:
src: "http://downloads.openmicroscopy.org/webtagging/{{ omero_web_extensions_tagging_ver }}/webtagging-{{ omero_web_extensions_tagging_ver }}.zip"
owner: "{{ omero_web_system_user }}"
group: "{{ omero_web_system_user }}"
mode: "u=rwx,go=rx"
dest: "{{ omero_web_extensionsdir }}"
remote_src: True
creates: "{{ omero_web_extensionsdir }}/webtagging-{{omero_web_extensions_tagging_ver}}"

# create symlinks
- name: OMERO.web apps | app-name symlinks
become: yes
file:
src: '{{ omero_web_extensionsdir }}/{{ item.src }}'
dest: '{{ omero_web_extensionsdir }}/{{ item.dest }}'
state: link
owner: "{{ omero_web_system_user }}"
group: "{{ omero_web_system_user }}"
with_items:
- { src: 'webtagging-{{ omero_web_extensions_tagging_ver }}/autotag/', dest: 'autotag' }
- { src: 'webtagging-{{ omero_web_extensions_tagging_ver }}/tagsearch/', dest: 'tagsearch' }
- { src: 'figure-{{ omero_web_extensions_figure_ver }}', dest: 'figure' }

# put the OMERO.web config for webapps here, and then restart web.
# i.e. take it out the initial set of omero.web config
# to go into {{ omero_web_basedir }}/config/*.omero
# which should then be turned into OMERO.web config by the
# omero.web systemd-based restart.
- name:
become: yes
tags:
- indev
template:
src: templates/omero-web-config-for-webapps.j2
dest: "{{ omero_web_basedir }}/config/omero-web-config-for-webapps.omero"
owner: "{{ omero_web_system_user }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be writeable by web?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Set now to root:root.

group: "{{ omero_web_system_user }}"
mode: "u=rw,go=r"
notify:
- omero-web restart omero-web
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use the handler from omero-common restart omero-web instead of the one embedded in the omero-web role which was intended for internal use by the role?

Copy link
Member Author

@kennethgillen kennethgillen Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

¯_(ツ)_/¯ I just grep -R restart * and found this in use elsewhere and having learned it's possible to re-use them, I used it. I can try.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, seems to work

changed: [infra-testpr.openmicroscopy.org]

RUNNING HANDLER [openmicroscopy.omero-common : restart omero-web] **************
changed: [infra-testpr.openmicroscopy.org]

PLAY RECAP *********************************************************************
infra-testpr.openmicroscopy.org : ok=3    changed=2    unreachable=0    failed=0

18 changes: 18 additions & 0 deletions ansible/server-state-playbooks/nightshade-web/requirements.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---

- name: openmicroscopy.omero-common
src: https://github.com/openmicroscopy/ansible-role-omero-common.git
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remember to add versions once these have been tagged.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 see workaround in ome/pydoop-features@fec18e1 until we have tags being properly pushed to galaxy again (ansible/galaxy-issues#252)

Copy link
Member

@manics manics Apr 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively since you've got git installed on the machine that's running Ansible you can just add version: X.X.X, e.g. IDR/deployment@d74d7c3
The main advantage of ome/pydoop-features@fec18e1 is you don't need to install git.


- name: openmicroscopy.omego
src: https://github.com/openmicroscopy/ansible-role-omego.git

- name: openmicroscopy.omero-web
src: https://github.com/openmicroscopy/ansible-role-omero-web.git

- name: openmicroscopy.lvm-partition
src: https://github.com/openmicroscopy/ansible-role-lvm-partition.git

- name: openmicroscopy.system-monitor-agent
src: https://github.com/openmicroscopy/ansible-role-system-monitor-agent.git


Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# {{ ansible_managed }}
# Add web-extension OMERO.web configuration
# after web already installed and running via role

config set -- omero.web.apps '["autotag", "tagsearch", "figure"]'
config set -- omero.web.ui.center_plugins '[["Auto Tag", "autotag/auto_tag_init.js.html", "auto_tag_panel"]]'
config set -- omero.web.ui.top_links '[["Data", "webindex", {"title": "Browse Data via Projects, Tags etc"}], ["History", "history", {"title": "History"}], ["Help", "http://help.openmicroscopy.org/", {"target": "new", "title": "Open OMERO user guide in a new tab"}], ["Figure", "figure_index", {"target": "new", "title": "Open OMERO.Figure in a new tab"}], ["Tag Search", "tagsearch"]]'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor point: there's no significant templated variables in here, so it could be a plain file (task copy instead of a template)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 will consider it for next time.