From c8c0a0e118c67cdc6da728e3e6ea9ac5004e472f Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Sat, 28 Aug 2021 15:04:04 +0300 Subject: [PATCH 01/40] add rework homework --- 08-ansible-04-role/README.md | 47 ++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 08-ansible-04-role/README.md diff --git a/08-ansible-04-role/README.md b/08-ansible-04-role/README.md new file mode 100644 index 000000000..8d8948fb7 --- /dev/null +++ b/08-ansible-04-role/README.md @@ -0,0 +1,47 @@ +# Домашнее задание к занятию "08.03 Работа с Roles" + +## Подготовка к выполнению +1. Создайте два пустых публичных репозитория в любом своём проекте: elastic-role и kibana-role. +2. Скачайте [role](./roles/) из репозитория с домашним заданием и перенесите его в свой репозиторий elastic-role. +3. Скачайте дистрибутив [java](https://www.oracle.com/java/technologies/javase-jdk11-downloads.html) и положите его в директорию `playbook/files/`. +4. Установите molecule: `pip3 install molecule` +5. Добавьте публичную часть своего ключа к своему профилю в github. + +## Основная часть + +Наша основная цель - разбить наш playbook на отдельные roles. Задача: сделать roles для elastic, kibana и написать playbook для использования этих ролей. Ожидаемый результат: существуют два ваших репозитория с roles и один репозиторий с playbook. + +1. Создать в старой версии playbook файл `requirements.yml` и заполнить его следующим содержимым: + ```yaml + --- + - src: git@github.com:netology-code/mnt-homeworks-ansible.git + scm: git + version: "1.0.1" + name: java + ``` +2. При помощи `ansible-galaxy` скачать себе эту роль. Запустите `molecule test`, посмотрите на вывод команды. +3. Перейдите в каталог с ролью elastic-role и создайте сценарий тестирования по умолчаню при помощи `molecule init scenario --driver-name docker`. +4. Добавьте несколько разных дистрибутивов (centos:8, ubuntu:latest) для инстансов и протестируйте роль, исправьте найденные ошибки, если они есть. +5. Создайте новый каталог с ролью при помощи `ansible-galaxy role init kibana-role`. Можете использовать другой драйвер, который более удобен вам. +6. На основе tasks из старого playbook заполните новую role. Разнесите переменные между `vars` и `default`. Проведите тестирование на разных дистрибитивах (centos:7, centos:8, ubuntu). +7. Выложите все roles в репозитории. Проставьте тэги, используя семантическую нумерацию. +8. Добавьте roles в `requirements.yml` в playbook. +9. Переработайте playbook на использование roles. +10. Выложите playbook в репозиторий. +11. В ответ приведите ссылки на оба репозитория с roles и одну ссылку на репозиторий с playbook. + +## Необязательная часть + +1. Проделайте схожие манипуляции для создания роли logstash. +2. Создайте дополнительный набор tasks, который позволяет обновлять стек ELK. +3. В ролях добавьте тестирование в раздел `verify.yml`. Данный раздел должен проверять, что elastic запущен и возвращает успешный статус по API, web-интерфейс kibana отвечает без кодов ошибки, logstash через команду `logstash -e 'input { stdin { } } output { stdout {} }'`. +4. Убедитесь в работоспособности своего стека. Возможно, потребуется тестировать все роли одновременно. +5. Выложите свои roles в репозитории. В ответ приведите ссылки. + +--- + +### Как оформить ДЗ? + +Выполненное домашнее задание пришлите ссылкой на .md-файл в вашем репозитории. + +--- From fb74e2bfb9699a9939f01aba3a5fc349f963f968 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Sat, 28 Aug 2021 15:04:17 +0300 Subject: [PATCH 02/40] add rework homework --- 08-ansible-02-playbook/README.md | 9 ------- 08-ansible-03-yandex/README.md | 26 ++++++++++++++++++ 08-ansible-03-yandex/playbook/.gitignore | 1 + .../prod/group_vars/elasticsearch.yml | 2 ++ .../playbook/inventory/prod/hosts.yml | 11 ++++++++ 08-ansible-03-yandex/playbook/site.yml | 27 +++++++++++++++++++ .../playbook/templates/elasticsearch.yml.j2 | 7 +++++ .../roles/.travis.yml | 0 .../roles/.yamllint | 0 .../roles/README.md | 0 .../roles/defaults/main.yml | 0 .../roles/handlers/main.yml | 0 .../roles/meta/main.yml | 0 .../roles/molecule/default/converge.yml | 0 .../roles/molecule/default/molecule.yml | 0 .../roles/molecule/default/verify.yml | 0 .../roles/tasks/main.yml | 0 .../roles/templates/elk.sh.j2 | 0 .../roles/tests/inventory | 0 .../roles/tests/test.yml | 0 .../roles/vars/main.yml | 0 .../README.md | 9 +++---- .../README.md | 0 08-ansible-additional/README.md | 6 ++++- 24 files changed, 82 insertions(+), 16 deletions(-) create mode 100644 08-ansible-03-yandex/README.md create mode 100644 08-ansible-03-yandex/playbook/.gitignore create mode 100644 08-ansible-03-yandex/playbook/inventory/prod/group_vars/elasticsearch.yml create mode 100644 08-ansible-03-yandex/playbook/inventory/prod/hosts.yml create mode 100755 08-ansible-03-yandex/playbook/site.yml create mode 100755 08-ansible-03-yandex/playbook/templates/elasticsearch.yml.j2 rename {08-ansible-03-role => 08-ansible-04-role}/roles/.travis.yml (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/.yamllint (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/README.md (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/defaults/main.yml (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/handlers/main.yml (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/meta/main.yml (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/molecule/default/converge.yml (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/molecule/default/molecule.yml (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/molecule/default/verify.yml (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/tasks/main.yml (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/templates/elk.sh.j2 (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/tests/inventory (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/tests/test.yml (100%) rename {08-ansible-03-role => 08-ansible-04-role}/roles/vars/main.yml (100%) rename {08-ansible-03-role => 08-ansible-05-testing}/README.md (82%) rename {08-ansible-04-module => 08-ansible-06-module}/README.md (100%) diff --git a/08-ansible-02-playbook/README.md b/08-ansible-02-playbook/README.md index 94f53c6a3..5c6e7558b 100644 --- a/08-ansible-02-playbook/README.md +++ b/08-ansible-02-playbook/README.md @@ -18,15 +18,6 @@ 9. Подготовьте README.md файл по своему playbook. В нём должно быть описано: что делает playbook, какие у него есть параметры и теги. 10. Готовый playbook выложите в свой репозиторий, в ответ предоставьте ссылку на него. -## Необязательная часть - -1. Приготовьте дополнительный хост для установки logstash. -2. Пропишите данный хост в `prod.yml` в новую группу `logstash`. -3. Дополните playbook ещё одним play, который будет исполнять установку logstash только на выделенный для него хост. -4. Все переменные для нового play определите в отдельный файл `group_vars/logstash/vars.yml`. -5. Logstash конфиг должен конфигурироваться в части ссылки на elasticsearch (можно взять, например его IP из facts или определить через vars). -6. Дополните README.md, протестируйте playbook, выложите новую версию в github. В ответ предоставьте ссылку на репозиторий. - --- ### Как оформить ДЗ? diff --git a/08-ansible-03-yandex/README.md b/08-ansible-03-yandex/README.md new file mode 100644 index 000000000..5207d527e --- /dev/null +++ b/08-ansible-03-yandex/README.md @@ -0,0 +1,26 @@ +# Домашнее задание к занятию "08.03 Использование Yandex Cloud" + +## Подготовка к выполнению +1. Создайте свой собственный (или используйте старый) публичный репозиторий на github с произвольным именем. +2. Скачайте [playbook](./playbook/) из репозитория с домашним заданием и перенесите его в свой репозиторий. + +## Основная часть +1. Допишите playbook: нужно сделать ещё один play, который устанавливает и настраивает kibana. +2. При создании tasks рекомендую использовать модули: `get_url`, `template`, `yum`, `apt`. +3. Tasks должны: скачать нужной версии дистрибутив, выполнить распаковку в выбранную директорию, сгенерировать конфигурацию с параметрами. +4. Приготовьте свой собственный inventory файл `prod.yml`. +5. Запустите `ansible-lint site.yml` и исправьте ошибки, если они есть. +6. Попробуйте запустить playbook на этом окружении с флагом `--check`. +7. Запустите playbook на `prod.yml` окружении с флагом `--diff`. Убедитесь, что изменения на системе произведены. +8. Повторно запустите playbook с флагом `--diff` и убедитесь, что playbook идемпотентен. +9. Проделайте шаги с 1 до 8 для создания ещё одного play, который устанавливает и настраивает filebeat.\ +10. Подготовьте README.md файл по своему playbook. В нём должно быть описано: что делает playbook, какие у него есть параметры и теги. +11. Готовый playbook выложите в свой репозиторий, в ответ предоставьте ссылку на него. + +--- + +### Как оформить ДЗ? + +Выполненное домашнее задание пришлите ссылкой на .md-файл в вашем репозитории. + +--- diff --git a/08-ansible-03-yandex/playbook/.gitignore b/08-ansible-03-yandex/playbook/.gitignore new file mode 100644 index 000000000..5ed0cb64c --- /dev/null +++ b/08-ansible-03-yandex/playbook/.gitignore @@ -0,0 +1 @@ +files/* \ No newline at end of file diff --git a/08-ansible-03-yandex/playbook/inventory/prod/group_vars/elasticsearch.yml b/08-ansible-03-yandex/playbook/inventory/prod/group_vars/elasticsearch.yml new file mode 100644 index 000000000..8d0eb9eed --- /dev/null +++ b/08-ansible-03-yandex/playbook/inventory/prod/group_vars/elasticsearch.yml @@ -0,0 +1,2 @@ +--- +elk_stack_version: "7.14.0" \ No newline at end of file diff --git a/08-ansible-03-yandex/playbook/inventory/prod/hosts.yml b/08-ansible-03-yandex/playbook/inventory/prod/hosts.yml new file mode 100644 index 000000000..2781de244 --- /dev/null +++ b/08-ansible-03-yandex/playbook/inventory/prod/hosts.yml @@ -0,0 +1,11 @@ +--- +all: + hosts: + el-instance: + ansible_host: + vars: + ansible_connection: ssh + ansible_user: +elasticsearch: + hosts: + el-instance: \ No newline at end of file diff --git a/08-ansible-03-yandex/playbook/site.yml b/08-ansible-03-yandex/playbook/site.yml new file mode 100755 index 000000000..0f41cb31f --- /dev/null +++ b/08-ansible-03-yandex/playbook/site.yml @@ -0,0 +1,27 @@ +--- +- name: Install Elasticsearch + hosts: elasticsearch + handlers: + - name: restart Elasticsearch + become: true + service: + name: elasticsearch + state: restarted + tasks: + - name: "Download Elasticsearch's rpm" + get_url: + url: "https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-{{ elk_stack_version }}-x86_64.rpm" + dest: "/tmp/elasticsearch-{{ elk_stack_version }}-x86_64.rpm" + register: download_elastic + until: download_elastic is succeeded + - name: Install Elasticsearch + become: true + yum: + name: "/tmp/elasticsearch-{{ elk_stack_version }}-x86_64.rpm" + state: present + - name: Configure Elasticsearch + become: true + template: + src: elasticsearch.yml.j2 + dest: /etc/elasticsearch/elasticsearch.yml + notify: restart Elasticsearch \ No newline at end of file diff --git a/08-ansible-03-yandex/playbook/templates/elasticsearch.yml.j2 b/08-ansible-03-yandex/playbook/templates/elasticsearch.yml.j2 new file mode 100755 index 000000000..6d38cd48b --- /dev/null +++ b/08-ansible-03-yandex/playbook/templates/elasticsearch.yml.j2 @@ -0,0 +1,7 @@ +path.data: /var/lib/elasticsearch +path.logs: /var/log/elasticsearch +network.host: 0.0.0.0 +discovery.seed_hosts: ["{{ ansible_facts['default_ipv4']['address'] }}"] +node.name: node-a +cluster.initial_master_nodes: + - node-a diff --git a/08-ansible-03-role/roles/.travis.yml b/08-ansible-04-role/roles/.travis.yml similarity index 100% rename from 08-ansible-03-role/roles/.travis.yml rename to 08-ansible-04-role/roles/.travis.yml diff --git a/08-ansible-03-role/roles/.yamllint b/08-ansible-04-role/roles/.yamllint similarity index 100% rename from 08-ansible-03-role/roles/.yamllint rename to 08-ansible-04-role/roles/.yamllint diff --git a/08-ansible-03-role/roles/README.md b/08-ansible-04-role/roles/README.md similarity index 100% rename from 08-ansible-03-role/roles/README.md rename to 08-ansible-04-role/roles/README.md diff --git a/08-ansible-03-role/roles/defaults/main.yml b/08-ansible-04-role/roles/defaults/main.yml similarity index 100% rename from 08-ansible-03-role/roles/defaults/main.yml rename to 08-ansible-04-role/roles/defaults/main.yml diff --git a/08-ansible-03-role/roles/handlers/main.yml b/08-ansible-04-role/roles/handlers/main.yml similarity index 100% rename from 08-ansible-03-role/roles/handlers/main.yml rename to 08-ansible-04-role/roles/handlers/main.yml diff --git a/08-ansible-03-role/roles/meta/main.yml b/08-ansible-04-role/roles/meta/main.yml similarity index 100% rename from 08-ansible-03-role/roles/meta/main.yml rename to 08-ansible-04-role/roles/meta/main.yml diff --git a/08-ansible-03-role/roles/molecule/default/converge.yml b/08-ansible-04-role/roles/molecule/default/converge.yml similarity index 100% rename from 08-ansible-03-role/roles/molecule/default/converge.yml rename to 08-ansible-04-role/roles/molecule/default/converge.yml diff --git a/08-ansible-03-role/roles/molecule/default/molecule.yml b/08-ansible-04-role/roles/molecule/default/molecule.yml similarity index 100% rename from 08-ansible-03-role/roles/molecule/default/molecule.yml rename to 08-ansible-04-role/roles/molecule/default/molecule.yml diff --git a/08-ansible-03-role/roles/molecule/default/verify.yml b/08-ansible-04-role/roles/molecule/default/verify.yml similarity index 100% rename from 08-ansible-03-role/roles/molecule/default/verify.yml rename to 08-ansible-04-role/roles/molecule/default/verify.yml diff --git a/08-ansible-03-role/roles/tasks/main.yml b/08-ansible-04-role/roles/tasks/main.yml similarity index 100% rename from 08-ansible-03-role/roles/tasks/main.yml rename to 08-ansible-04-role/roles/tasks/main.yml diff --git a/08-ansible-03-role/roles/templates/elk.sh.j2 b/08-ansible-04-role/roles/templates/elk.sh.j2 similarity index 100% rename from 08-ansible-03-role/roles/templates/elk.sh.j2 rename to 08-ansible-04-role/roles/templates/elk.sh.j2 diff --git a/08-ansible-03-role/roles/tests/inventory b/08-ansible-04-role/roles/tests/inventory similarity index 100% rename from 08-ansible-03-role/roles/tests/inventory rename to 08-ansible-04-role/roles/tests/inventory diff --git a/08-ansible-03-role/roles/tests/test.yml b/08-ansible-04-role/roles/tests/test.yml similarity index 100% rename from 08-ansible-03-role/roles/tests/test.yml rename to 08-ansible-04-role/roles/tests/test.yml diff --git a/08-ansible-03-role/roles/vars/main.yml b/08-ansible-04-role/roles/vars/main.yml similarity index 100% rename from 08-ansible-03-role/roles/vars/main.yml rename to 08-ansible-04-role/roles/vars/main.yml diff --git a/08-ansible-03-role/README.md b/08-ansible-05-testing/README.md similarity index 82% rename from 08-ansible-03-role/README.md rename to 08-ansible-05-testing/README.md index 91ae0f4bc..48d83aa7e 100644 --- a/08-ansible-03-role/README.md +++ b/08-ansible-05-testing/README.md @@ -1,11 +1,8 @@ -# Домашнее задание к занятию "08.03 Работа с Roles" +# Домашнее задание к занятию "08.05 Тестирование Roles" ## Подготовка к выполнению -1. Создайте два пустых публичных репозитория в любом своём проекте: elastic-role и kibana-role. -2. Скачайте [role](./roles/) из репозитория с домашним заданием и перенесите его в свой репозиторий elastic-role. -3. Скачайте дистрибутив [java](https://www.oracle.com/java/technologies/javase-jdk11-downloads.html) и положите его в директорию `playbook/files/`. -4. Установите molecule: `pip3 install molecule` -5. Добавьте публичную часть своего ключа к своему профилю в github. +1. Установите molecule: `pip3 install molecule` +2. Установите tox: `pip3 install tox` ## Основная часть diff --git a/08-ansible-04-module/README.md b/08-ansible-06-module/README.md similarity index 100% rename from 08-ansible-04-module/README.md rename to 08-ansible-06-module/README.md diff --git a/08-ansible-additional/README.md b/08-ansible-additional/README.md index 9f004f401..b67497e8a 100644 --- a/08-ansible-additional/README.md +++ b/08-ansible-additional/README.md @@ -1,3 +1,7 @@ # Дополнительные материалы к лекциям по Ansible -1. [Как работать с Windows](http://onreader.mdl.ru/MasteringAnsible.3ed/content/Ch03.html) \ No newline at end of file +1. [Как работать с Windows](http://onreader.mdl.ru/MasteringAnsible.3ed/content/Ch03.html) +2. [Скачать Elasticsearch](https://www.elastic.co/downloads/elasticsearch) +3. [Скачать Kibana](https://www.elastic.co/downloads/kibana) +4. [Скачать filebeat](https://www.elastic.co/downloads/beats/filebeat) +5. [Все виды beat'ов](https://www.elastic.co/downloads/beats/) \ No newline at end of file From 2318d0a297b4f7d7c82e21498b30d05d315a9404 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Sat, 28 Aug 2021 16:23:35 +0300 Subject: [PATCH 03/40] rework 05 ansible --- 08-ansible-03-yandex/README.md | 2 +- 08-ansible-04-role/README.md | 41 +++++++------- 08-ansible-04-role/roles/.travis.yml | 29 ---------- 08-ansible-04-role/roles/.yamllint | 33 ------------ 08-ansible-04-role/roles/README.md | 33 ------------ 08-ansible-04-role/roles/defaults/main.yml | 3 -- 08-ansible-04-role/roles/handlers/main.yml | 2 - 08-ansible-04-role/roles/meta/main.yml | 52 ------------------ .../roles/molecule/default/converge.yml | 7 --- .../roles/molecule/default/molecule.yml | 13 ----- .../roles/molecule/default/verify.yml | 11 ---- 08-ansible-04-role/roles/tasks/main.yml | 33 ------------ 08-ansible-04-role/roles/templates/elk.sh.j2 | 5 -- 08-ansible-04-role/roles/tests/inventory | 2 - 08-ansible-04-role/roles/tests/test.yml | 5 -- 08-ansible-04-role/roles/vars/main.yml | 2 - 08-ansible-05-testing/Dockerfile | 0 08-ansible-05-testing/README.md | 54 ++++++++++--------- 18 files changed, 50 insertions(+), 277 deletions(-) delete mode 100644 08-ansible-04-role/roles/.travis.yml delete mode 100644 08-ansible-04-role/roles/.yamllint delete mode 100644 08-ansible-04-role/roles/README.md delete mode 100644 08-ansible-04-role/roles/defaults/main.yml delete mode 100644 08-ansible-04-role/roles/handlers/main.yml delete mode 100644 08-ansible-04-role/roles/meta/main.yml delete mode 100644 08-ansible-04-role/roles/molecule/default/converge.yml delete mode 100644 08-ansible-04-role/roles/molecule/default/molecule.yml delete mode 100644 08-ansible-04-role/roles/molecule/default/verify.yml delete mode 100644 08-ansible-04-role/roles/tasks/main.yml delete mode 100644 08-ansible-04-role/roles/templates/elk.sh.j2 delete mode 100644 08-ansible-04-role/roles/tests/inventory delete mode 100644 08-ansible-04-role/roles/tests/test.yml delete mode 100644 08-ansible-04-role/roles/vars/main.yml create mode 100644 08-ansible-05-testing/Dockerfile diff --git a/08-ansible-03-yandex/README.md b/08-ansible-03-yandex/README.md index 5207d527e..caffb826b 100644 --- a/08-ansible-03-yandex/README.md +++ b/08-ansible-03-yandex/README.md @@ -13,7 +13,7 @@ 6. Попробуйте запустить playbook на этом окружении с флагом `--check`. 7. Запустите playbook на `prod.yml` окружении с флагом `--diff`. Убедитесь, что изменения на системе произведены. 8. Повторно запустите playbook с флагом `--diff` и убедитесь, что playbook идемпотентен. -9. Проделайте шаги с 1 до 8 для создания ещё одного play, который устанавливает и настраивает filebeat.\ +9. Проделайте шаги с 1 до 8 для создания ещё одного play, который устанавливает и настраивает filebeat. 10. Подготовьте README.md файл по своему playbook. В нём должно быть описано: что делает playbook, какие у него есть параметры и теги. 11. Готовый playbook выложите в свой репозиторий, в ответ предоставьте ссылку на него. diff --git a/08-ansible-04-role/README.md b/08-ansible-04-role/README.md index 8d8948fb7..54337d6b4 100644 --- a/08-ansible-04-role/README.md +++ b/08-ansible-04-role/README.md @@ -1,42 +1,41 @@ # Домашнее задание к занятию "08.03 Работа с Roles" ## Подготовка к выполнению -1. Создайте два пустых публичных репозитория в любом своём проекте: elastic-role и kibana-role. -2. Скачайте [role](./roles/) из репозитория с домашним заданием и перенесите его в свой репозиторий elastic-role. -3. Скачайте дистрибутив [java](https://www.oracle.com/java/technologies/javase-jdk11-downloads.html) и положите его в директорию `playbook/files/`. -4. Установите molecule: `pip3 install molecule` -5. Добавьте публичную часть своего ключа к своему профилю в github. +1. Создайте два пустых публичных репозитория в любом своём проекте: kibana-role и filebeat-role. +2. Добавьте публичную часть своего ключа к своему профилю в github. ## Основная часть -Наша основная цель - разбить наш playbook на отдельные roles. Задача: сделать roles для elastic, kibana и написать playbook для использования этих ролей. Ожидаемый результат: существуют два ваших репозитория с roles и один репозиторий с playbook. +Наша основная цель - разбить наш playbook на отдельные roles. Задача: сделать roles для elastic, kibana, filebeat и написать playbook для использования этих ролей. Ожидаемый результат: существуют два ваших репозитория с roles и один репозиторий с playbook. 1. Создать в старой версии playbook файл `requirements.yml` и заполнить его следующим содержимым: ```yaml --- - src: git@github.com:netology-code/mnt-homeworks-ansible.git scm: git - version: "1.0.1" - name: java + version: "2.0.0" + name: elastic ``` -2. При помощи `ansible-galaxy` скачать себе эту роль. Запустите `molecule test`, посмотрите на вывод команды. -3. Перейдите в каталог с ролью elastic-role и создайте сценарий тестирования по умолчаню при помощи `molecule init scenario --driver-name docker`. -4. Добавьте несколько разных дистрибутивов (centos:8, ubuntu:latest) для инстансов и протестируйте роль, исправьте найденные ошибки, если они есть. -5. Создайте новый каталог с ролью при помощи `ansible-galaxy role init kibana-role`. Можете использовать другой драйвер, который более удобен вам. -6. На основе tasks из старого playbook заполните новую role. Разнесите переменные между `vars` и `default`. Проведите тестирование на разных дистрибитивах (centos:7, centos:8, ubuntu). -7. Выложите все roles в репозитории. Проставьте тэги, используя семантическую нумерацию. -8. Добавьте roles в `requirements.yml` в playbook. -9. Переработайте playbook на использование roles. -10. Выложите playbook в репозиторий. -11. В ответ приведите ссылки на оба репозитория с roles и одну ссылку на репозиторий с playbook. +2. При помощи `ansible-galaxy` скачать себе эту роль. +3. Создать новый каталог с ролью при помощи `ansible-galaxy role init kibana-role`. +4. На основе tasks из старого playbook заполните новую role. Разнесите переменные между `vars` и `default`. +5. Перенести нужные шаблоны конфигов в `templates`. +6. Создать новый каталог с ролью при помощи `ansible-galaxy role init filebeat-role`. +7. На основе tasks из старого playbook заполните новую role. Разнесите переменные между `vars` и `default`. +8. Перенести нужные шаблоны конфигов в `templates`. +9. Описать в `README.md` обе роли и их параметры. +10. Выложите все roles в репозитории. Проставьте тэги, используя семантическую нумерацию. +11. Добавьте roles в `requirements.yml` в playbook. +12. Переработайте playbook на использование roles. +13. Выложите playbook в репозиторий. +14. В ответ приведите ссылки на оба репозитория с roles и одну ссылку на репозиторий с playbook. ## Необязательная часть 1. Проделайте схожие манипуляции для создания роли logstash. 2. Создайте дополнительный набор tasks, который позволяет обновлять стек ELK. -3. В ролях добавьте тестирование в раздел `verify.yml`. Данный раздел должен проверять, что elastic запущен и возвращает успешный статус по API, web-интерфейс kibana отвечает без кодов ошибки, logstash через команду `logstash -e 'input { stdin { } } output { stdout {} }'`. -4. Убедитесь в работоспособности своего стека. Возможно, потребуется тестировать все роли одновременно. -5. Выложите свои roles в репозитории. В ответ приведите ссылки. +3. Убедитесь в работоспособности своего стека: установите logstash на свой хост с elasticsearch, настройте конфиги logstash и filebeat так, чтобы они взаимодействовали друг с другом и elasticsearch корректно. +4. Выложите logstash-role в репозиторий. В ответ приведите ссылку. --- diff --git a/08-ansible-04-role/roles/.travis.yml b/08-ansible-04-role/roles/.travis.yml deleted file mode 100644 index 36bbf6208..000000000 --- a/08-ansible-04-role/roles/.travis.yml +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: python -python: "2.7" - -# Use the new container infrastructure -sudo: false - -# Install ansible -addons: - apt: - packages: - - python-pip - -install: - # Install ansible - - pip install ansible - - # Check ansible version - - ansible --version - - # Create ansible.cfg with correct roles_path - - printf '[defaults]\nroles_path=../' >ansible.cfg - -script: - # Basic role syntax check - - ansible-playbook tests/test.yml -i tests/inventory --syntax-check - -notifications: - webhooks: https://galaxy.ansible.com/api/v1/notifications/ \ No newline at end of file diff --git a/08-ansible-04-role/roles/.yamllint b/08-ansible-04-role/roles/.yamllint deleted file mode 100644 index 882767605..000000000 --- a/08-ansible-04-role/roles/.yamllint +++ /dev/null @@ -1,33 +0,0 @@ ---- -# Based on ansible-lint config -extends: default - -rules: - braces: - max-spaces-inside: 1 - level: error - brackets: - max-spaces-inside: 1 - level: error - colons: - max-spaces-after: -1 - level: error - commas: - max-spaces-after: -1 - level: error - comments: disable - comments-indentation: disable - document-start: disable - empty-lines: - max: 3 - level: error - hyphens: - level: error - indentation: disable - key-duplicates: enable - line-length: disable - new-line-at-end-of-file: disable - new-lines: - type: unix - trailing-spaces: disable - truthy: disable diff --git a/08-ansible-04-role/roles/README.md b/08-ansible-04-role/roles/README.md deleted file mode 100644 index dc7ce017b..000000000 --- a/08-ansible-04-role/roles/README.md +++ /dev/null @@ -1,33 +0,0 @@ -Elasticsearch -========= - -Simple download binaries from official website and install elasticsearch. - -Role Variables --------------- -There is only two variables that you can redefine in your playbook. -```yaml -elastic_version: "7.10.1" # Use for download only this version of elastic -elastic_home: "/opt/elastic/{{ elastic_version }}" # Use for unpackage distro and create ES_HOME variable -``` - -Example Playbook ----------------- - -Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too: - -```yaml -- hosts: all - roles: - - elastic -``` - -License -------- - -BSD - -Author Information ------------------- - -Netology Students diff --git a/08-ansible-04-role/roles/defaults/main.yml b/08-ansible-04-role/roles/defaults/main.yml deleted file mode 100644 index 6495c674a..000000000 --- a/08-ansible-04-role/roles/defaults/main.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- - elastic_version: "7.10.1" - elastic_home: "/opt/elastic/{{ elastic_version }}" diff --git a/08-ansible-04-role/roles/handlers/main.yml b/08-ansible-04-role/roles/handlers/main.yml deleted file mode 100644 index ee253cf2f..000000000 --- a/08-ansible-04-role/roles/handlers/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -# handlers file for roles diff --git a/08-ansible-04-role/roles/meta/main.yml b/08-ansible-04-role/roles/meta/main.yml deleted file mode 100644 index a7ca90007..000000000 --- a/08-ansible-04-role/roles/meta/main.yml +++ /dev/null @@ -1,52 +0,0 @@ -galaxy_info: - author: Alexey Metlyakov - description: Install elasticsearch from remote server - company: Netology - - # If the issue tracker for your role is not on github, uncomment the - # next line and provide a value - # issue_tracker_url: http://example.com/issue/tracker - - # Choose a valid license ID from https://spdx.org - some suggested licenses: - # - BSD-3-Clause (default) - # - MIT - # - GPL-2.0-or-later - # - GPL-3.0-only - # - Apache-2.0 - # - CC-BY-4.0 - license: BSD-3-Clause - - min_ansible_version: 2.8 - - # If this a Container Enabled role, provide the minimum Ansible Container version. - # min_ansible_container_version: - - # - # Provide a list of supported platforms, and for each platform a list of versions. - # If you don't wish to enumerate all versions for a particular platform, use 'all'. - # To view available platforms and versions (or releases), visit: - # https://galaxy.ansible.com/api/v1/platforms/ - # - # platforms: - # - name: Fedora - # versions: - # - all - # - 25 - # - name: SomePlatform - # versions: - # - all - # - 1.0 - # - 7 - # - 99.99 - - galaxy_tags: [elasticsearch] - # List tags for your role here, one per line. A tag is a keyword that describes - # and categorizes the role. Users find roles by searching for tags. Be sure to - # remove the '[]' above, if you add tags to this list. - # - # NOTE: A tag is limited to a single word comprised of alphanumeric characters. - # Maximum 20 tags per role. - -dependencies: [] - # List your role dependencies here, one per line. Be sure to remove the '[]' above, - # if you add dependencies to this list. diff --git a/08-ansible-04-role/roles/molecule/default/converge.yml b/08-ansible-04-role/roles/molecule/default/converge.yml deleted file mode 100644 index aae97f57a..000000000 --- a/08-ansible-04-role/roles/molecule/default/converge.yml +++ /dev/null @@ -1,7 +0,0 @@ ---- -- name: Converge - hosts: all - tasks: - - name: "Include roles" - include_role: - name: "roles" diff --git a/08-ansible-04-role/roles/molecule/default/molecule.yml b/08-ansible-04-role/roles/molecule/default/molecule.yml deleted file mode 100644 index aafdc9d3c..000000000 --- a/08-ansible-04-role/roles/molecule/default/molecule.yml +++ /dev/null @@ -1,13 +0,0 @@ ---- -dependency: - name: galaxy -driver: - name: docker -platforms: - - name: instance - image: docker.io/pycontribs/centos:7 - pre_build_image: true -provisioner: - name: ansible -verifier: - name: ansible diff --git a/08-ansible-04-role/roles/molecule/default/verify.yml b/08-ansible-04-role/roles/molecule/default/verify.yml deleted file mode 100644 index 41078d300..000000000 --- a/08-ansible-04-role/roles/molecule/default/verify.yml +++ /dev/null @@ -1,11 +0,0 @@ ---- -# This is an example playbook to execute Ansible tests. - -- name: Verify - hosts: all - gather_facts: false - tasks: - - name: Check Elastic version - shell: - cmd: /opt/elastic/7.10.1/bin/elasticsearch --version - changed_when: false \ No newline at end of file diff --git a/08-ansible-04-role/roles/tasks/main.yml b/08-ansible-04-role/roles/tasks/main.yml deleted file mode 100644 index b3ad9bd17..000000000 --- a/08-ansible-04-role/roles/tasks/main.yml +++ /dev/null @@ -1,33 +0,0 @@ ---- - - name: Upload tar.gz Elasticsearch from remote URL - get_url: - url: "https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-{{ elastic_version }}-linux-x86_64.tar.gz" - dest: "/tmp/elasticsearch-{{ elastic_version }}-linux-x86_64.tar.gz" - mode: 0755 - timeout: 60 - force: true - register: get_elastic - until: get_elastic is succeeded - tags: elastic - - name: Create directrory for Elasticsearch - file: - state: directory - path: "{{ elastic_home }}" - tags: elastic - - name: Extract Elasticsearch in the installation directory - become: true - unarchive: - copy: false - src: "/tmp/elasticsearch-{{ elastic_version }}-linux-x86_64.tar.gz" - dest: "{{ elastic_home }}" - extra_opts: [--strip-components=1] - creates: "{{ elastic_home }}/bin/elasticsearch" - tags: - - skip_ansible_lint - - elastic - - name: Set environment Elastic - become: yes - template: - src: templates/elk.sh.j2 - dest: /etc/profile.d/elk.sh - tags: elastic \ No newline at end of file diff --git a/08-ansible-04-role/roles/templates/elk.sh.j2 b/08-ansible-04-role/roles/templates/elk.sh.j2 deleted file mode 100644 index 44de41723..000000000 --- a/08-ansible-04-role/roles/templates/elk.sh.j2 +++ /dev/null @@ -1,5 +0,0 @@ -# Warning: This file is Ansible Managed, manual changes will be overwritten on next playbook run. -#!/usr/bin/env bash - -export ES_HOME={{ elastic_home }} -export PATH=$PATH:$ES_HOME/bin \ No newline at end of file diff --git a/08-ansible-04-role/roles/tests/inventory b/08-ansible-04-role/roles/tests/inventory deleted file mode 100644 index 878877b07..000000000 --- a/08-ansible-04-role/roles/tests/inventory +++ /dev/null @@ -1,2 +0,0 @@ -localhost - diff --git a/08-ansible-04-role/roles/tests/test.yml b/08-ansible-04-role/roles/tests/test.yml deleted file mode 100644 index d1b0deaf7..000000000 --- a/08-ansible-04-role/roles/tests/test.yml +++ /dev/null @@ -1,5 +0,0 @@ ---- -- hosts: localhost - remote_user: root - roles: - - roles diff --git a/08-ansible-04-role/roles/vars/main.yml b/08-ansible-04-role/roles/vars/main.yml deleted file mode 100644 index 7914d052e..000000000 --- a/08-ansible-04-role/roles/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -# vars file for roles diff --git a/08-ansible-05-testing/Dockerfile b/08-ansible-05-testing/Dockerfile new file mode 100644 index 000000000..e69de29bb diff --git a/08-ansible-05-testing/README.md b/08-ansible-05-testing/README.md index 48d83aa7e..fba8bda54 100644 --- a/08-ansible-05-testing/README.md +++ b/08-ansible-05-testing/README.md @@ -1,39 +1,43 @@ # Домашнее задание к занятию "08.05 Тестирование Roles" ## Подготовка к выполнению -1. Установите molecule: `pip3 install molecule` -2. Установите tox: `pip3 install tox` +1. Установите molecule: `pip3 install "molecule==3.3.4"` +2. Соберите локальный образ на основе [Dockerfile](./Dockerfile) ## Основная часть -Наша основная цель - разбить наш playbook на отдельные roles. Задача: сделать roles для elastic, kibana и написать playbook для использования этих ролей. Ожидаемый результат: существуют два ваших репозитория с roles и один репозиторий с playbook. - -1. Создать в старой версии playbook файл `requirements.yml` и заполнить его следующим содержимым: - ```yaml - --- - - src: git@github.com:netology-code/mnt-homeworks-ansible.git - scm: git - version: "1.0.1" - name: java - ``` -2. При помощи `ansible-galaxy` скачать себе эту роль. Запустите `molecule test`, посмотрите на вывод команды. -3. Перейдите в каталог с ролью elastic-role и создайте сценарий тестирования по умолчаню при помощи `molecule init scenario --driver-name docker`. -4. Добавьте несколько разных дистрибутивов (centos:8, ubuntu:latest) для инстансов и протестируйте роль, исправьте найденные ошибки, если они есть. -5. Создайте новый каталог с ролью при помощи `molecule init role --driver-name docker kibana-role`. Можете использовать другой драйвер, который более удобен вам. -6. На основе tasks из старого playbook заполните новую role. Разнесите переменные между `vars` и `default`. Проведите тестирование на разных дистрибитивах (centos:7, centos:8, ubuntu). -7. Выложите все roles в репозитории. Проставьте тэги, используя семантическую нумерацию. -8. Добавьте roles в `requirements.yml` в playbook. -9. Переработайте playbook на использование roles. -10. Выложите playbook в репозиторий. -11. В ответ приведите ссылки на оба репозитория с roles и одну ссылку на репозиторий с playbook. +Наша основная цель - настроить тестирование наших ролей. Задача: сделать сценарии тестирования для kibana, logstash. Ожидаемый результат: все сценарии успешно проходят тестирование ролей. + +### Molecule + +1. Запустите `molecule test` внутри корневой директории elasticsearch-role, посмотрите на вывод команды. +2. Перейдите в каталог с ролью kibana-role и создайте сценарий тестирования по умолчаню при помощи `molecule init scenario --driver-name docker`. +3. Добавьте несколько разных дистрибутивов (centos:8, ubuntu:latest) для инстансов и протестируйте роль, исправьте найденные ошибки, если они есть. +4. Добавьте несколько assert'ов в verify.yml файл, для проверки работоспособности kibana-role (проверка, что web отвечает, проверка логов, etc). Запустите тестирование роли повторно и проверьте, что оно прошло успешно. +5. Повторите шаги 2-4 для filebeat-role. +6. Добавьте новый тег на коммит с рабочим сценарием в соответствии с семантическим версионированием. + +### Tox + +1. Запустите `docker run -it -v :/opt/elasticsearch-role -w /opt/elasticsearch-role /bin/bash`, где path_to_repo - путь до корня репозитория с elasticsearch-role на вашей файловой системе. +2. Внутри контейнера выполните команду `tox`, посмотрите на вывод. +3. Добавьте файл `tox.ini` в корень репозитория каждой своей роли. +4. Создайте облегчённый сценарий для `molecule`. Проверьте его на исполнимость. +5. Пропишите правильную команду в `tox.ini` для того чтобы запускался облегчённый сценарий. +6. Запустите `docker` контейнер так, чтобы внутри оказались обе ваши роли. +7. Зайдти поочерёдно в каждую из них и запустите команду `tox`. Убедитесь, что всё отработало успешно. +8. Добавьте новый тег на коммит с рабочим сценарием в соответствии с семантическим версионированием. + +После выполнения у вас должно получится два сценария molecule и один tox.ini файл в каждом репозитории. Ссылки на репозитории являются ответами на домашнее задание. Не забудьте указать в ответе теги решений Tox и Molecule заданий. ## Необязательная часть 1. Проделайте схожие манипуляции для создания роли logstash. 2. Создайте дополнительный набор tasks, который позволяет обновлять стек ELK. -3. В ролях добавьте тестирование в раздел `verify.yml`. Данный раздел должен проверять, что elastic запущен и возвращает успешный статус по API, web-интерфейс kibana отвечает без кодов ошибки, logstash через команду `logstash -e 'input { stdin { } } output { stdout {} }'`. -4. Убедитесь в работоспособности своего стека. Возможно, потребуется тестировать все роли одновременно. -5. Выложите свои roles в репозитории. В ответ приведите ссылки. +3. В ролях добавьте тестирование в раздел `verify.yml`. Данный раздел должен проверять, что logstash через команду `logstash -e 'input { stdin { } } output { stdout {} }'`  отвечате адекватно. +4. Создайте сценарий внутри любой из своих ролей, который умеет поднимать весь стек при помощи всех ролей. +5. Убедитесь в работоспособности своего стека. Создайте отдельный verify.yml, который будет проверять работоспособность интеграции всех инструментов между ними. +6. Выложите свои roles в репозитории. В ответ приведите ссылки. --- From 623d05cf734198f3ea5f6e0f7788adc0240825b8 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Sun, 29 Aug 2021 03:35:06 +0300 Subject: [PATCH 04/40] fix docker --- 08-ansible-05-testing/Dockerfile | 16 ++++++++++++++++ 08-ansible-06-module/README.md | 13 +++++++------ 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/08-ansible-05-testing/Dockerfile b/08-ansible-05-testing/Dockerfile index e69de29bb..8b632ae58 100644 --- a/08-ansible-05-testing/Dockerfile +++ b/08-ansible-05-testing/Dockerfile @@ -0,0 +1,16 @@ +FROM registry.redhat.io/rhel8/podman:latest +ENV MOLECULE_NO_LOG false + +RUN yum reinstall glibc-common -y +RUN yum update -y && yum install tar gcc make python3-pip zlib-devel openssl-devel yum-utils libffi-devel -y + +ADD https://www.python.org/ftp/python/3.6.13/Python-3.6.13.tgz Python-3.6.13.tgz +RUN tar xf Python-3.6.13.tgz && cd Python-3.6.13/ && ./configure && make && make altinstall +ADD https://www.python.org/ftp/python/3.7.10/Python-3.7.10.tgz Python-3.7.10.tgz +RUN tar xf Python-3.7.10.tgz && cd Python-3.7.10/ && ./configure && make && make altinstall +ADD https://www.python.org/ftp/python/3.8.8/Python-3.8.8.tgz Python-3.8.8.tgz +RUN tar xf Python-3.8.8.tgz && cd Python-3.8.8/ && ./configure && make && make altinstall +ADD https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz Python-3.9.2.tgz +RUN tar xf Python-3.9.2.tgz && cd Python-3.9.2/ && ./configure && make && make altinstall +RUN python3 -m pip install --upgrade pip && pip3 install tox selinux +RUN rm -rf Python-* \ No newline at end of file diff --git a/08-ansible-06-module/README.md b/08-ansible-06-module/README.md index e881f6ae6..259d60175 100644 --- a/08-ansible-06-module/README.md +++ b/08-ansible-06-module/README.md @@ -160,7 +160,7 @@ if __name__ == '__main__': 5. Напишите single task playbook и используйте module в нём. 6. Проверьте через playbook на идемпотентность. 7. Выйдите из виртуального окружения. -8. Инициализируйте новую collection: `ansible-galaxy collection init my_own_namespace.my_own_collection` +8. Инициализируйте новую collection: `ansible-galaxy collection init my_own_namespace.yandex_cloud_elk` 9. В данную collection перенесите свой module в соответствующую директорию. 10. Single task playbook преобразуйте в single task role и перенесите в collection. У role должны быть default всех параметров module 11. Создайте playbook для использования этой role. @@ -173,11 +173,12 @@ if __name__ == '__main__': ## Необязательная часть -1. Используйте свой полёт фантазии: Создайте свой собственный module для тех roles, что мы делали в рамках предыдущих лекций. -2. Соберите из roles и module отдельную collection. -3. Создайте новый репозиторий и выложите новую collection туда. - -Если идей нет, но очень хочется попробовать что-то реализовать: реализовать module восстановления из backup elasticsearch. +1. Реализуйте свой собственный модуль для создания хостов в Yandex Cloud. +2. Модуль может (и должен) иметь зависимость от `yc`, основной функционал: создание ВМ с нужным сайзингом на основе нужной ОС. Дополнительные модули по созданию кластеров Clickhouse, MySQL и прочего реализовывать не надо, достаточно простейшего создания ВМ. +3. Модуль может формировать динамическое inventory, но данная часть не является обязательной, достаточно, чтобы он делал хосты с указанной спецификацией в YAML. +4. Протестируйте модуль на идемпотентность, исполнимость. При успехе - добавьте данный модуль в свою коллекцию. +5. Измените playbook так, чтобы он умел создавать инфраструктуру под inventory, а после устанавливал весь ваш стек ELK на нужные хосты и настраивал его. +6. В итоге, ваша коллекция обязательно должна содержать: elastic-role, kibana-role, filebeat-role, два модуля: my_own_module и модуль управления Yandex Cloud хостами и playbook, который демонстрирует создание ELK-стека. --- From f8dd9e8b6c3362c6bf94df981a38eabed98e6e80 Mon Sep 17 00:00:00 2001 From: ElenaPalagina <77622076+ElenaPalagina@users.noreply.github.com> Date: Mon, 30 Aug 2021 10:57:37 +0300 Subject: [PATCH 05/40] Update README.md --- README.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 0e5f64f67..40356b7aa 100644 --- a/README.md +++ b/README.md @@ -10,13 +10,17 @@ ## Модуль 8. Система управления конфигурациями -8.1. [Введение в Ansible](./08-ansible-01-base/README.md) +8.1. [Введение в Ansible](https://github.com/netology-code/mnt-homeworks/tree/MNT-7/08-ansible-01-based) -8.2. [Работа с Playbook](./08-ansible-02-playbook/README.md) +8.2. [Работа с Playbook]() -8.3. [Работа с Roles](./08-ansible-03-role/README.md) +8.3. [Использование YandexCloud]() -8.4. [Создание собственных модулей](./08-ansible-04-module/README.md) +8.4. [Работа с Roles]() + +8.5. [Тестирование Roles]() + +8.6. [Создание собственных модулей]() ## Модуль 9. Непрерывная разработка и интеграция From a7415257457dfa06d3616665ae1193559172c0ed Mon Sep 17 00:00:00 2001 From: ElenaPalagina <77622076+ElenaPalagina@users.noreply.github.com> Date: Mon, 30 Aug 2021 10:58:31 +0300 Subject: [PATCH 06/40] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 40356b7aa..17e3d68d6 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ ## Модуль 8. Система управления конфигурациями -8.1. [Введение в Ansible](https://github.com/netology-code/mnt-homeworks/tree/MNT-7/08-ansible-01-based) +8.1. [Введение в Ansible](https://github.com/netology-code/mnt-homeworks/tree/MNT-7/08-ansible-01-base) 8.2. [Работа с Playbook]() From 636d3a4f2e2b6175cda75c3687d9d6dd640169f6 Mon Sep 17 00:00:00 2001 From: ElenaPalagina <77622076+ElenaPalagina@users.noreply.github.com> Date: Mon, 30 Aug 2021 11:04:47 +0300 Subject: [PATCH 07/40] Update README.md --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 17e3d68d6..a6bf8eb0e 100644 --- a/README.md +++ b/README.md @@ -12,15 +12,15 @@ 8.1. [Введение в Ansible](https://github.com/netology-code/mnt-homeworks/tree/MNT-7/08-ansible-01-base) -8.2. [Работа с Playbook]() +8.2. [Работа с Playbook](https://github.com/netology-code/mnt-homeworks/tree/MNT-7/08-ansible-02-playbook) -8.3. [Использование YandexCloud]() +8.3. [Использование YandexCloud](https://github.com/netology-code/mnt-homeworks/tree/MNT-7/08-ansible-03-yandex) -8.4. [Работа с Roles]() +8.4. [Работа с Roles](https://github.com/netology-code/mnt-homeworks/tree/MNT-7/08-ansible-04-role) -8.5. [Тестирование Roles]() +8.5. [Тестирование Roles](https://github.com/netology-code/mnt-homeworks/tree/MNT-7/08-ansible-05-testing) -8.6. [Создание собственных модулей]() +8.6. [Создание собственных модулей](https://github.com/netology-code/mnt-homeworks/tree/MNT-7/08-ansible-06-module) ## Модуль 9. Непрерывная разработка и интеграция From 762134addd6268d56793b8d2d4ca05696416bd9f Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Thu, 2 Sep 2021 21:44:28 +0300 Subject: [PATCH 08/40] bump jdk version --- 08-ansible-02-playbook/playbook/group_vars/all/vars.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/08-ansible-02-playbook/playbook/group_vars/all/vars.yml b/08-ansible-02-playbook/playbook/group_vars/all/vars.yml index 363bfae3b..4324dae50 100644 --- a/08-ansible-02-playbook/playbook/group_vars/all/vars.yml +++ b/08-ansible-02-playbook/playbook/group_vars/all/vars.yml @@ -1,3 +1,3 @@ --- -java_jdk_version: 11.0.11 -java_oracle_jdk_package: jdk-11.0.11_linux-x64_bin.tar.gz +java_jdk_version: 11.0.12 +java_oracle_jdk_package: "jdk-{{ java_jdk_version }}_linux-x64_bin.tar.gz" From d16bb16afbdfffbbcac1a665ad50ac098943680f Mon Sep 17 00:00:00 2001 From: ElenaPalagina <77622076+ElenaPalagina@users.noreply.github.com> Date: Thu, 9 Sep 2021 12:27:42 +0300 Subject: [PATCH 09/40] Update README.md --- 08-ansible-04-role/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/08-ansible-04-role/README.md b/08-ansible-04-role/README.md index 54337d6b4..adea2ca83 100644 --- a/08-ansible-04-role/README.md +++ b/08-ansible-04-role/README.md @@ -1,4 +1,4 @@ -# Домашнее задание к занятию "08.03 Работа с Roles" +# Домашнее задание к занятию "8.4 Работа с Roles" ## Подготовка к выполнению 1. Создайте два пустых публичных репозитория в любом своём проекте: kibana-role и filebeat-role. From 186876ce6fa29ac6965698378c7cb1b90053b96a Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Tue, 14 Sep 2021 23:57:53 +0300 Subject: [PATCH 10/40] minor fix molecule version --- 08-ansible-05-testing/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/08-ansible-05-testing/README.md b/08-ansible-05-testing/README.md index fba8bda54..f019a4668 100644 --- a/08-ansible-05-testing/README.md +++ b/08-ansible-05-testing/README.md @@ -1,7 +1,7 @@ # Домашнее задание к занятию "08.05 Тестирование Roles" ## Подготовка к выполнению -1. Установите molecule: `pip3 install "molecule==3.3.4"` +1. Установите molecule: `pip3 install "molecule==3.4.0"` 2. Соберите локальный образ на основе [Dockerfile](./Dockerfile) ## Основная часть From 919851550d772b31746600bccc4f7af06dce75f8 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Sat, 25 Sep 2021 00:05:08 +0300 Subject: [PATCH 11/40] add sonar play --- 09-ci-02-cicd/infrastructure/.gitignore | 2 + .../infrastructure/files/CentOS-Base.repo | 48 ++++ .../infrastructure/files/pg_hba.conf | 89 ++++++++ .../inventory/cicd/group_vars/all.yml | 8 + .../inventory/cicd/group_vars/nexus.yml | 0 .../inventory/cicd/group_vars/postgres.yml | 1 + .../inventory/cicd/group_vars/sonarqube.yml | 5 + .../infrastructure/inventory/cicd/hosts.yml | 20 ++ 09-ci-02-cicd/infrastructure/site.yml | 210 ++++++++++++++++++ .../infrastructure/templates/jdk.sh.j2 | 5 + .../infrastructure/templates/sonar.unit.j2 | 22 ++ .../infrastructure/templates/wrapper.conf.j2 | 98 ++++++++ 12 files changed, 508 insertions(+) create mode 100644 09-ci-02-cicd/infrastructure/.gitignore create mode 100644 09-ci-02-cicd/infrastructure/files/CentOS-Base.repo create mode 100644 09-ci-02-cicd/infrastructure/files/pg_hba.conf create mode 100644 09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/all.yml create mode 100644 09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml create mode 100644 09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/postgres.yml create mode 100644 09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/sonarqube.yml create mode 100644 09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml create mode 100644 09-ci-02-cicd/infrastructure/site.yml create mode 100644 09-ci-02-cicd/infrastructure/templates/jdk.sh.j2 create mode 100644 09-ci-02-cicd/infrastructure/templates/sonar.unit.j2 create mode 100644 09-ci-02-cicd/infrastructure/templates/wrapper.conf.j2 diff --git a/09-ci-02-cicd/infrastructure/.gitignore b/09-ci-02-cicd/infrastructure/.gitignore new file mode 100644 index 000000000..483815e4c --- /dev/null +++ b/09-ci-02-cicd/infrastructure/.gitignore @@ -0,0 +1,2 @@ +files/*.pub +roles/* \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/files/CentOS-Base.repo b/09-ci-02-cicd/infrastructure/files/CentOS-Base.repo new file mode 100644 index 000000000..0984327f0 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/files/CentOS-Base.repo @@ -0,0 +1,48 @@ +# CentOS-Base.repo +# +# The mirror system uses the connecting IP address of the client and the +# update status of each mirror to pick mirrors that are updated to and +# geographically close to the client. You should use this for CentOS updates +# unless you are manually picking other mirrors. +# +# If the mirrorlist= does not work for you, as a fall back you can try the +# remarked out baseurl= line instead. +# +# + +[base] +name=CentOS-$releasever - Base +mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra +#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/ +gpgcheck=1 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 + +exclude=postgresql* + +#released updates +[updates] +name=CentOS-$releasever - Updates +mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra +#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/ +gpgcheck=1 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 + +exclude=postgresql* + +#additional packages that may be useful +[extras] +name=CentOS-$releasever - Extras +mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra +#baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/ +gpgcheck=1 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 + +#additional packages that extend functionality of existing packages +[centosplus] +name=CentOS-$releasever - Plus +mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus&infra=$infra +#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/ +gpgcheck=1 +enabled=0 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 + diff --git a/09-ci-02-cicd/infrastructure/files/pg_hba.conf b/09-ci-02-cicd/infrastructure/files/pg_hba.conf new file mode 100644 index 000000000..b5c361a7b --- /dev/null +++ b/09-ci-02-cicd/infrastructure/files/pg_hba.conf @@ -0,0 +1,89 @@ +# PostgreSQL Client Authentication Configuration File +# =================================================== +# +# Refer to the "Client Authentication" section in the PostgreSQL +# documentation for a complete description of this file. A short +# synopsis follows. +# +# This file controls: which hosts are allowed to connect, how clients +# are authenticated, which PostgreSQL user names they can use, which +# databases they can access. Records take one of these forms: +# +# local DATABASE USER METHOD [OPTIONS] +# host DATABASE USER ADDRESS METHOD [OPTIONS] +# hostssl DATABASE USER ADDRESS METHOD [OPTIONS] +# hostnossl DATABASE USER ADDRESS METHOD [OPTIONS] +# +# (The uppercase items must be replaced by actual values.) +# +# The first field is the connection type: "local" is a Unix-domain +# socket, "host" is either a plain or SSL-encrypted TCP/IP socket, +# "hostssl" is an SSL-encrypted TCP/IP socket, and "hostnossl" is a +# plain TCP/IP socket. +# +# DATABASE can be "all", "sameuser", "samerole", "replication", a +# database name, or a comma-separated list thereof. The "all" +# keyword does not match "replication". Access to replication +# must be enabled in a separate record (see example below). +# +# USER can be "all", a user name, a group name prefixed with "+", or a +# comma-separated list thereof. In both the DATABASE and USER fields +# you can also write a file name prefixed with "@" to include names +# from a separate file. +# +# ADDRESS specifies the set of hosts the record matches. It can be a +# host name, or it is made up of an IP address and a CIDR mask that is +# an integer (between 0 and 32 (IPv4) or 128 (IPv6) inclusive) that +# specifies the number of significant bits in the mask. A host name +# that starts with a dot (.) matches a suffix of the actual host name. +# Alternatively, you can write an IP address and netmask in separate +# columns to specify the set of hosts. Instead of a CIDR-address, you +# can write "samehost" to match any of the server's own IP addresses, +# or "samenet" to match any address in any subnet that the server is +# directly connected to. +# +# METHOD can be "trust", "reject", "md5", "password", "scram-sha-256", +# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius" or "cert". +# Note that "password" sends passwords in clear text; "md5" or +# "scram-sha-256" are preferred since they send encrypted passwords. +# +# OPTIONS are a set of options for the authentication in the format +# NAME=VALUE. The available options depend on the different +# authentication methods -- refer to the "Client Authentication" +# section in the documentation for a list of which options are +# available for which authentication methods. +# +# Database and user names containing spaces, commas, quotes and other +# special characters must be quoted. Quoting one of the keywords +# "all", "sameuser", "samerole" or "replication" makes the name lose +# its special character, and just match a database or username with +# that name. +# +# This file is read on server startup and when the server receives a +# SIGHUP signal. If you edit the file on a running system, you have to +# SIGHUP the server for the changes to take effect, run "pg_ctl reload", +# or execute "SELECT pg_reload_conf()". +# +# Put your actual configuration here +# ---------------------------------- +# +# If you want to allow non-local connections, you need to add more +# "host" records. In that case you will also need to make PostgreSQL +# listen on a non-local interface via the listen_addresses +# configuration parameter, or via the -i or -h command line switches. + + + +# TYPE DATABASE USER ADDRESS METHOD + +# "local" is for Unix domain socket connections only +local all all trust +# IPv4 local connections: +host all all 127.0.0.1/32 md5 +# IPv6 local connections: +host all all ::1/128 md5 +# Allow replication connections from localhost, by a user with the +# replication privilege. +local replication all trust +host replication all 127.0.0.1/32 md5 +host replication all ::1/128 md5 diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/all.yml b/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/all.yml new file mode 100644 index 000000000..f6afeefb1 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/all.yml @@ -0,0 +1,8 @@ +--- +jdk_url: https://download.java.net/openjdk/jdk11/ri/openjdk-11+28_linux-x64_bin.tar.gz +jdk_distr_name: openjdk-11+28_linux-x64_bin.tar.gz +jdk_folder: "{{ jdk_distr_name.split('-')[:2] | join('-') }}" +java_home: "/opt/jdk/{{ jdk_folder }}" +sonarqube_db_name: sonar +sonarqube_db_user: sonar +sonarqube_db_password: sonar \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml b/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml new file mode 100644 index 000000000..e69de29bb diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/postgres.yml b/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/postgres.yml new file mode 100644 index 000000000..e0499e7e9 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/postgres.yml @@ -0,0 +1 @@ +postgresql_version: 11 \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/sonarqube.yml b/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/sonarqube.yml new file mode 100644 index 000000000..89b9116c4 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/sonarqube.yml @@ -0,0 +1,5 @@ +sonar_version: 9.1.0.47736 +sonar_download_url: "https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-{{ sonar_version }}.zip" +sonar_version_directory: "sonarqube-{{ sonar_version }}" +sonar_db_port: "5432" +sonar_web_context: '' \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml b/09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml new file mode 100644 index 000000000..ebc0b11d6 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml @@ -0,0 +1,20 @@ +--- +all: + hosts: + sonar-01: + ansible_host: 178.154.209.230 + ansible_user: aragast + children: + sonarqube: + hosts: + sonar-01: + nexus: + hosts: + nexus-01: + ansible_host: 178.154.233.219 + ansible_user: aragast + postgres: + hosts: + sonar-01: + vars: + ansible_connection_type: paramiko \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/site.yml b/09-ci-02-cicd/infrastructure/site.yml new file mode 100644 index 000000000..7eaebd11d --- /dev/null +++ b/09-ci-02-cicd/infrastructure/site.yml @@ -0,0 +1,210 @@ +--- +- name: Get OpenJDK installed + hosts: all + pre_tasks: + - name: install unzip + become: true + yum: + name: unzip + state: present + tasks: + - name: Upload .tar.gz file conaining binaries from remote storage + get_url: + url: "{{ jdk_url }}" + dest: "/tmp/jdk-{{ jdk_distr_name }}" + mode: 0755 + register: download_java_remote_binaries + until: download_java_remote_binaries is succeeded + - name: Ensure installation dir exists + become: true + file: + state: directory + path: "{{ java_home }}" + mode: 0755 + - name: Extract java in the installation directory + become: true + unarchive: + copy: false + src: "/tmp/jdk-{{ jdk_distr_name }}" + dest: "{{ java_home }}" + extra_opts: [--strip-components=1] + creates: "{{ java_home }}/bin/java" + tags: skip_ansible_lint + - name: Export environment variables + become: true + template: + src: jdk.sh.j2 + dest: /etc/profile.d/jdk.sh + owner: root + group: root + mode: 0644 +- name: Get PostgreSQL installed + hosts: postgres + become: true + tasks: + - name: Change repo file + copy: + src: CentOS-Base.repo + dest: /etc/yum.repos.d/CentOS-Base.repo + mode: 0644 + owner: root + group: root + - name: Install PostgreSQL repos + yum: + name: https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm + state: present + - name: Install PostgreSQL + yum: + name: "postgresql{{ postgresql_version }}-server" + state: present + - name: Init template1 DB + command: /usr/pgsql-11/bin/postgresql-11-setup initdb + failed_when: false + - name: Start pgsql service + systemd: + name: "postgresql-{{ postgresql_version }}" + state: started + enabled: true + - name: Create user in system + user: + name: "{{ sonarqube_db_user }}" + - name: Create user for Sonar in PostgreSQL + become_user: postgres + command: "createuser -s -e {{ sonarqube_db_user }}" + failed_when: false + - name: Change password for Sonar user in PostgreSQL + become_user: postgres + command: "psql -c \"ALTER USER sonar WITH ENCRYPTED password '{{ sonarqube_db_password }}';\"" + - name: Create Sonar DB + become_user: postgres + command: "createdb {{ sonarqube_db_name }}" + failed_when: false + - name: Copy pg_hba.conf + copy: + src: pg_hba.conf + dest: /var/lib/pgsql/11/data/pg_hba.conf + mode: 0600 + owner: postgres + group: postgres +- name: Prepare Sonar host + hosts: sonarqube + become: true + tasks: + - name: Create group in system + group: + name: "{{ sonarqube_db_user }}" + state: present + - name: Create user in system + user: + name: "{{ sonarqube_db_user }}" + group: "{{ sonarqube_db_user }}" + - name: "Set up ssh key to access for managed node" + authorized_key: + user: "{{ sonarqube_db_user }}" + state: present + key: "{{ lookup('file', 'id_rsa.pub') }}" + - name: "Allow group to have passwordless sudo" + lineinfile: + dest: /etc/sudoers + state: present + regexp: '^%{{ sonarqube_db_user }}' + line: '%{{ sonarqube_db_user }} ALL=(ALL) NOPASSWD: ALL' + validate: 'visudo -cf %s' + - name: Increase Virtual Memory + lineinfile: + dest: /etc/sysctl.conf + state: present + regexp: '^vm.max_map_count' + line: 'vm.max_map_count=262144' + - name: Reboot VM + reboot: + +- name: Get Sonarqube installed + hosts: sonarqube + vars: + ansible_user: "{{ sonarqube_db_user }}" + tasks: + - name: Get distrib ZIP + get_url: + url: "{{ sonar_download_url }}" + dest: "/tmp/{{ sonar_version_directory }}.zip" + validate_certs: false + - name: Unzip Sonar + become: true + unarchive: + src: "/tmp/{{ sonar_version_directory }}.zip" + dest: /usr/local/ + copy: false + owner: "{{ sonarqube_db_user }}" + group: "{{ sonarqube_db_user }}" + creates: /usr/local/sonar/COPYING + + - name: Move Sonar into place. + become: true + copy: + src: /usr/local/{{ sonar_version_directory }}/ + dest: /usr/local/sonar/ + owner: "{{ sonarqube_db_user }}" + group: "{{ sonarqube_db_user }}" + remote_src: true + + - name: Configure SonarQube JDBC settings for PostgreSQL. + lineinfile: + dest: /usr/local/sonar/conf/sonar.properties + regexp: "{{ item.regexp }}" + line: "{{ item.line }}" + with_items: + - regexp: "^sonar.jdbc.username" + line: "sonar.jdbc.username={{ sonarqube_db_user }}" + - regexp: "^sonar.jdbc.password" + line: "sonar.jdbc.password={{ sonarqube_db_password }}" + - regexp: "^sonar.jdbc.url" + line: "sonar.jdbc.url=jdbc:postgresql://localhost:{{ sonar_db_port }}/{{ sonarqube_db_name }}?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance" + - regexp: "^sonar.web.context" + line: "sonar.web.context={{ sonar_web_context }}" + + - name: Generate wrapper.conf + template: + src: wrapper.conf.j2 + dest: /usr/local/sonar/conf/wrapper.conf + mode: 0644 + + - name: Symlink sonar bin. + become: true + file: + src: /usr/local/sonar/bin/linux-x86-64/sonar.sh + dest: /usr/bin/sonar + state: link + register: sonar_symlink + + - name: Copy SonarQube systemd unit file into place (for systemd systems). + become: true + template: + src: sonar.unit.j2 + dest: /etc/systemd/system/sonar.service + owner: root + group: root + mode: 0755 + + - name: Ensure Sonar is running and set to start on boot. + become: true + service: + name: sonar + state: restarted + enabled: true + + - name: Allow Sonar time to build on first start. + pause: + seconds: 180 + when: sonar_symlink.changed + tags: ['skip_ansible_lint'] + + - name: Make sure Sonar is responding on the configured port. + wait_for: + port: 9000 + delay: 3 + timeout: 300 +# - name: Get Nexus installed +# hosts: nexus +# roles: +# - nexus_role \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/templates/jdk.sh.j2 b/09-ci-02-cicd/infrastructure/templates/jdk.sh.j2 new file mode 100644 index 000000000..39887b6ab --- /dev/null +++ b/09-ci-02-cicd/infrastructure/templates/jdk.sh.j2 @@ -0,0 +1,5 @@ +# Warning: This file is Ansible Managed, manual changes will be overwritten on next playbook run. +#!/usr/bin/env bash + +export JAVA_HOME={{ java_home }} +export PATH=$PATH:$JAVA_HOME/bin \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/templates/sonar.unit.j2 b/09-ci-02-cicd/infrastructure/templates/sonar.unit.j2 new file mode 100644 index 000000000..fb48c8067 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/templates/sonar.unit.j2 @@ -0,0 +1,22 @@ +[Unit] +Description=SonarQube +After=network.target network-online.target +Wants=network-online.target + +[Service] +LimitNOFILE=131072 +LimitNPROC=8192 +User={{ sonarqube_db_user }} +Group={{ sonarqube_db_user }} +ExecStart=/usr/bin/sonar start +ExecStop=/usr/bin/sonar stop +ExecReload=/usr/bin/sonar restart +PIDFile=/usr/local/sonar/bin/linux-x86-64/./SonarQube.pid +Type=simple +PermissionsStartOnly=true +TimeoutStartSec=5 +Restart=always +SuccessExitStatus=143 + +[Install] +WantedBy=multi-user.target diff --git a/09-ci-02-cicd/infrastructure/templates/wrapper.conf.j2 b/09-ci-02-cicd/infrastructure/templates/wrapper.conf.j2 new file mode 100644 index 000000000..bb271c188 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/templates/wrapper.conf.j2 @@ -0,0 +1,98 @@ +# Path to JVM executable. By default it must be available in PATH. +# Can be an absolute path, for example: +wrapper.java.command={{ java_home }}/bin/java + + +# +# DO NOT EDIT THE FOLLOWING SECTIONS +# + + +#******************************************************************** +# Wrapper Java +#******************************************************************** +wrapper.java.additional.1=-Dsonar.wrapped=true +wrapper.java.additional.2=-Djava.awt.headless=true +# extra args needed by hazelcast +wrapper.java.additional.3=--add-exports=java.base/jdk.internal.ref=ALL-UNNAMED +wrapper.java.additional.4=--add-opens=java.base/java.lang=ALL-UNNAMED +wrapper.java.additional.5=--add-opens=java.base/java.nio=ALL-UNNAMED +wrapper.java.additional.6=--add-opens=java.base/sun.nio.ch=ALL-UNNAMED +wrapper.java.additional.7=--add-opens=java.management/sun.management=ALL-UNNAMED +wrapper.java.additional.8=--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED + +wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp +wrapper.java.classpath.1=../../lib/sonar-application-9.1.0.47736.jar +wrapper.java.classpath.2=../../lib/jsw/wrapper-3.2.3.jar +wrapper.java.classpath.3=../../lib/sonar-shutdowner-9.1.0.47736.jar +wrapper.java.library.path.1=./lib +wrapper.app.parameter.1=org.sonar.application.App +wrapper.java.initmemory=8 +wrapper.java.maxmemory=32 + +#******************************************************************** +# Wrapper Logs +#******************************************************************** + +wrapper.console.format=PM +wrapper.console.loglevel=INFO +wrapper.logfile.format=M +wrapper.logfile.loglevel=INFO +wrapper.logfile.rollmode=DATE +wrapper.logfile=../../logs/sonar.YYYYMMDD.log + +# Maximum size that the log file will be allowed to grow to before +# the log is rolled. Size is specified in bytes. The default value +# of 0, disables log rolling. May abbreviate with the 'k' (kb) or +# 'm' (mb) suffix. For example: 10m = 10 megabytes. +#wrapper.logfile.maxsize=0 + +# Maximum number of rolled log files which will be allowed before old +# files are deleted. The default value of 0 implies no limit. +wrapper.logfile.maxfiles=7 + +# Log Level for sys/event log output. (See docs for log levels) +wrapper.syslog.loglevel=NONE + +#******************************************************************** +# Wrapper Windows Properties +#******************************************************************** +# Title to use when running as a console +wrapper.console.title=SonarQube + +# Disallow start of multiple instances of an application at the same time on Windows +wrapper.single_invocation=true + +#******************************************************************** +# Wrapper Windows NT/2000/XP Service Properties +#******************************************************************** +# WARNING - Do not modify any of these properties when an application +# using this configuration file has been installed as a service. +# Please uninstall the service before modifying this section. The +# service can then be reinstalled. + +# Name of the service +wrapper.ntservice.name=SonarQube + +# Display name of the service +wrapper.ntservice.displayname=SonarQube + +# Description of the service +wrapper.ntservice.description=SonarQube + +# Service dependencies. Add dependencies as needed starting from 1 +wrapper.ntservice.dependency.1= + +# Mode in which the service is installed. AUTO_START or DEMAND_START +wrapper.ntservice.starttype=AUTO_START + +# Allow the service to interact with the desktop. +wrapper.ntservice.interactive=false + +#******************************************************************** +# Forking Properties +#******************************************************************** +wrapper.disable_restarts=TRUE +wrapper.ping.timeout=0 +wrapper.shutdown.timeout=0 +wrapper.jvm_exit.timeout=0 From 0bf813fef28e185848067174ac990fdfe07c42bc Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Sun, 26 Sep 2021 01:51:31 +0300 Subject: [PATCH 12/40] add nexus play --- .../inventory/cicd/group_vars/nexus.yml | 22 +++ .../infrastructure/inventory/cicd/hosts.yml | 10 +- 09-ci-02-cicd/infrastructure/site.yml | 182 +++++++++++++++++- .../templates/nexus.properties.j2 | 12 ++ .../infrastructure/templates/nexus.systemd.j2 | 15 ++ .../templates/nexus.vmoptions.j2 | 16 ++ 6 files changed, 246 insertions(+), 11 deletions(-) create mode 100644 09-ci-02-cicd/infrastructure/templates/nexus.properties.j2 create mode 100644 09-ci-02-cicd/infrastructure/templates/nexus.systemd.j2 create mode 100644 09-ci-02-cicd/infrastructure/templates/nexus.vmoptions.j2 diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml b/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml index e69de29bb..b84eba748 100644 --- a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml +++ b/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml @@ -0,0 +1,22 @@ +--- +nexus_user_group: nexus +nexus_user_name: nexus +nexus_directory_data: "/home/{{ nexus_user_name }}/sonatype-work/nexus3" +nexus_directory_home: "/home/{{ nexus_user_name }}/nexus" +nexus_directory_log: "/home/{{ nexus_user_name }}/log" +nexus_directory_package: "/home/{{ nexus_user_name }}/pkg" +nexus_directory_tmp: "/home/{{ nexus_user_name }}/tmp" +nexus_version: 3.14.0-04 +nexus_download_url: https://download.sonatype.com/nexus/3 +nexus_service_enabled: true +nexus_ulimit: 65536 +nexus_context_path: / +nexus_host: 0.0.0.0 +nexus_port: 8081 +nexus_port_check_timeout: 600 +nexus_edition: nexus-oss-edition +nexus_features: nexus-oss-feature +nexus_java_heap_size: 1200M +nexus_java_max_direct_memory: 2G +nexus_service_start_on_boot: true +nexus_configuration_disk_free_space_limit: ~ \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml b/09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml index ebc0b11d6..ef7633b5a 100644 --- a/09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml +++ b/09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml @@ -2,8 +2,9 @@ all: hosts: sonar-01: - ansible_host: 178.154.209.230 - ansible_user: aragast + ansible_host: + nexus-01: + ansible_host: children: sonarqube: hosts: @@ -11,10 +12,9 @@ all: nexus: hosts: nexus-01: - ansible_host: 178.154.233.219 - ansible_user: aragast postgres: hosts: sonar-01: vars: - ansible_connection_type: paramiko \ No newline at end of file + ansible_connection_type: paramiko + ansible_user: \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/site.yml b/09-ci-02-cicd/infrastructure/site.yml index 7eaebd11d..5744806e8 100644 --- a/09-ci-02-cicd/infrastructure/site.yml +++ b/09-ci-02-cicd/infrastructure/site.yml @@ -1,6 +1,6 @@ --- - name: Get OpenJDK installed - hosts: all + hosts: sonarqube pre_tasks: - name: install unzip become: true @@ -15,12 +15,14 @@ mode: 0755 register: download_java_remote_binaries until: download_java_remote_binaries is succeeded + - name: Ensure installation dir exists become: true file: state: directory path: "{{ java_home }}" mode: 0755 + - name: Extract java in the installation directory become: true unarchive: @@ -29,7 +31,7 @@ dest: "{{ java_home }}" extra_opts: [--strip-components=1] creates: "{{ java_home }}/bin/java" - tags: skip_ansible_lint + - name: Export environment variables become: true template: @@ -38,6 +40,7 @@ owner: root group: root mode: 0644 + - name: Get PostgreSQL installed hosts: postgres become: true @@ -49,22 +52,27 @@ mode: 0644 owner: root group: root + - name: Install PostgreSQL repos yum: name: https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm state: present + - name: Install PostgreSQL yum: name: "postgresql{{ postgresql_version }}-server" state: present + - name: Init template1 DB command: /usr/pgsql-11/bin/postgresql-11-setup initdb failed_when: false + - name: Start pgsql service systemd: name: "postgresql-{{ postgresql_version }}" state: started enabled: true + - name: Create user in system user: name: "{{ sonarqube_db_user }}" @@ -72,13 +80,16 @@ become_user: postgres command: "createuser -s -e {{ sonarqube_db_user }}" failed_when: false + - name: Change password for Sonar user in PostgreSQL become_user: postgres command: "psql -c \"ALTER USER sonar WITH ENCRYPTED password '{{ sonarqube_db_password }}';\"" + - name: Create Sonar DB become_user: postgres command: "createdb {{ sonarqube_db_name }}" failed_when: false + - name: Copy pg_hba.conf copy: src: pg_hba.conf @@ -86,6 +97,7 @@ mode: 0600 owner: postgres group: postgres + - name: Prepare Sonar host hosts: sonarqube become: true @@ -94,15 +106,18 @@ group: name: "{{ sonarqube_db_user }}" state: present + - name: Create user in system user: name: "{{ sonarqube_db_user }}" group: "{{ sonarqube_db_user }}" + - name: "Set up ssh key to access for managed node" authorized_key: user: "{{ sonarqube_db_user }}" state: present key: "{{ lookup('file', 'id_rsa.pub') }}" + - name: "Allow group to have passwordless sudo" lineinfile: dest: /etc/sudoers @@ -110,6 +125,7 @@ regexp: '^%{{ sonarqube_db_user }}' line: '%{{ sonarqube_db_user }} ALL=(ALL) NOPASSWD: ALL' validate: 'visudo -cf %s' + - name: Increase Virtual Memory lineinfile: dest: /etc/sysctl.conf @@ -129,6 +145,7 @@ url: "{{ sonar_download_url }}" dest: "/tmp/{{ sonar_version_directory }}.zip" validate_certs: false + - name: Unzip Sonar become: true unarchive: @@ -204,7 +221,160 @@ port: 9000 delay: 3 timeout: 300 -# - name: Get Nexus installed -# hosts: nexus -# roles: -# - nexus_role \ No newline at end of file +- name: Get Nexus installed + hosts: nexus + pre_tasks: + - name: Create Nexus group + become: true + group: + name: "{{ nexus_user_group }}" + state: present + - name: Create Nexus user + become: true + user: + name: "{{ nexus_user_name }}" + - name: Install JDK + become: true + package: + name: [java-1.8.0-openjdk, java-1.8.0-openjdk-devel] + state: present + tasks: + - name: Create Nexus directories + become: true + file: + group: "{{ nexus_user_group }}" + owner: "{{ nexus_user_name }}" + path: "{{ item }}" + state: directory + with_items: + - "{{ nexus_directory_log }}" + - "{{ nexus_directory_data }}" + - "{{ nexus_directory_data }}/etc" + - "{{ nexus_directory_package }}" + - "{{ nexus_directory_tmp }}" + + - name: Download Nexus + become: true + become_user: "{{ nexus_user_name }}" + get_url: + dest: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}.tar.gz" + url: "{{ nexus_download_url }}/nexus-{{ nexus_version }}-unix.tar.gz" + + - name: Unpack Nexus + become: true + become_user: "{{ nexus_user_name }}" + unarchive: + copy: no + creates: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}" + dest: "{{ nexus_directory_package }}" + src: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}.tar.gz" + + - name: Link to Nexus Directory + become: true + become_user: "{{ nexus_user_name }}" + file: + dest: "{{ nexus_directory_home }}" + src: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}" + state: link + + - name: Add NEXUS_HOME for Nexus user + become: true + become_user: "{{ nexus_user_name }}" + lineinfile: + create: yes + dest: "/home/{{ nexus_user_name }}/.bashrc" + insertafter: EOF + line: "export NEXUS_HOME={{ nexus_directory_home }}" + + - name: Add run_as_user to Nexus.rc + become: true + become_user: "{{ nexus_user_name }}" + lineinfile: + create: yes + dest: "{{ nexus_directory_home }}/bin/nexus.rc" + insertafter: EOF + line: "run_as_user=\"{{ nexus_user_name }}\"" + regexp: "^run_as_user" + + - name: Raise nofile limit for Nexus user + become: true + pam_limits: + domain: "{{ nexus_user_name }}" + limit_type: "-" + limit_item: nofile + value: "{{ nexus_ulimit }}" + + - name: Create Nexus service for SystemD + become: true + template: + dest: /lib/systemd/system/nexus.service + mode: 0644 + src: nexus.systemd.j2 + + - name: Ensure Nexus service is enabled for SystemD + become: true + systemd: + daemon_reload: yes + enabled: yes + name: nexus + when: + - nexus_service_enabled + + - name: Create Nexus vmoptions + become: true + become_user: "{{ nexus_user_name }}" + template: + dest: "{{ nexus_directory_home }}/bin/nexus.vmoptions" + src: nexus.vmoptions.j2 + register: nexus_config_changed + + - name: Create Nexus properties + become: true + become_user: "{{ nexus_user_name }}" + template: + dest: "{{ nexus_directory_data }}/etc/nexus.properties" + src: nexus.properties.j2 + register: nexus_config_changed + + - name: Lower Nexus disk space threshold + become: true + become_user: "{{ nexus_user_name }}" + lineinfile: + backrefs: yes + dest: "{{ nexus_directory_home }}/etc/karaf/system.properties" + insertafter: EOF + line: "storage.diskCache.diskFreeSpaceLimit={{ nexus_configuration_disk_free_space_limit }}" + regexp: ^storage\.diskCache\.diskFreeSpaceLimit + when: nexus_configuration_disk_free_space_limit is not none + register: nexus_config_changed + + - name: Start Nexus service if enabled + become: true + service: + enabled: yes + name: nexus + state: started + when: + - nexus_service_start_on_boot + - not nexus_config_changed.changed + tags: + - skip_ansible_lint + + - name: Ensure Nexus service is restarted + become: true + service: + name: nexus + state: restarted + when: + - nexus_service_start_on_boot + - nexus_config_changed.changed + tags: + - skip_ansible_lint + + - name: Wait for Nexus port if started + wait_for: + port: "{{ nexus_port }}" + state: started + timeout: "{{ nexus_port_check_timeout }}" + when: + - nexus_service_start_on_boot diff --git a/09-ci-02-cicd/infrastructure/templates/nexus.properties.j2 b/09-ci-02-cicd/infrastructure/templates/nexus.properties.j2 new file mode 100644 index 000000000..aa2645463 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/templates/nexus.properties.j2 @@ -0,0 +1,12 @@ +# +# + +# Jetty section +application-host={{ nexus_host }} +application-port={{ nexus_port }} +nexus-context-path={{ nexus_context_path }} + +# Nexus section +nexus-edition={{ nexus_edition }} +nexus-features=\ + {{ nexus_features }} \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/templates/nexus.systemd.j2 b/09-ci-02-cicd/infrastructure/templates/nexus.systemd.j2 new file mode 100644 index 000000000..7665d96a8 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/templates/nexus.systemd.j2 @@ -0,0 +1,15 @@ +[Unit] +Description=nexus service +After=network.target + +[Service] +Type=forking +User={{ nexus_user_name }} +Group={{ nexus_user_group }} +LimitNOFILE={{ nexus_ulimit }} +ExecStart={{ nexus_directory_home }}/bin/nexus start +ExecStop={{ nexus_directory_home }}/bin/nexus stop +Restart=on-abort + +[Install] +WantedBy=multi-user.target \ No newline at end of file diff --git a/09-ci-02-cicd/infrastructure/templates/nexus.vmoptions.j2 b/09-ci-02-cicd/infrastructure/templates/nexus.vmoptions.j2 new file mode 100644 index 000000000..dea2288a7 --- /dev/null +++ b/09-ci-02-cicd/infrastructure/templates/nexus.vmoptions.j2 @@ -0,0 +1,16 @@ +-Xms{{ nexus_java_heap_size }} +-Xmx{{ nexus_java_heap_size }} +-XX:MaxDirectMemorySize={{ nexus_java_max_direct_memory }} +-XX:+UnlockDiagnosticVMOptions +-XX:+UnsyncloadClass +-XX:+LogVMOutput +-XX:LogFile={{ nexus_directory_log }}/jvm.log +-XX:-OmitStackTraceInFastThrow +-Djava.net.preferIPv4Stack=true +-Dkaraf.home=. +-Dkaraf.base=. +-Dkaraf.etc=etc/karaf +-Djava.util.logging.config.file=etc/karaf/java.util.logging.properties +-Dkaraf.data={{ nexus_directory_data }} +-Djava.io.tmpdir={{ nexus_directory_tmp }} +-Dkaraf.startLocalConsole=false \ No newline at end of file From 04032d79e28cdc6280680d255e4b37c520b5f7cb Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Sun, 26 Sep 2021 02:42:26 +0300 Subject: [PATCH 13/40] add jenkins playbook --- 09-ci-03-jenkins/docker/Dockerfile | 5 - 09-ci-03-jenkins/infrastructure/.gitignore | 2 + .../infrastructure/files/CentOS-Base.repo | 48 +++ .../infrastructure/files/pg_hba.conf | 89 ++++ .../inventory/cicd/group_vars/jenkins.yml | 8 + .../inventory/cicd/group_vars/nexus.yml | 27 ++ .../infrastructure/inventory/cicd/hosts.yml | 27 ++ 09-ci-03-jenkins/infrastructure/site.yml | 190 +++++++++ .../infrastructure/templates/jdk.sh.j2 | 5 + .../templates/nexus.properties.j2 | 12 + .../infrastructure/templates/nexus.systemd.j2 | 15 + .../templates/nexus.vmoptions.j2 | 16 + .../infrastructure/templates/sonar.unit.j2 | 22 + .../infrastructure/templates/wrapper.conf.j2 | 98 +++++ 09-ci-04-teamcity/infrastructure/.gitignore | 2 + .../inventory/cicd/group_vars/all.yml | 8 + .../inventory/cicd/group_vars/nexus.yml | 22 + .../inventory/cicd/group_vars/postgres.yml | 1 + .../inventory/cicd/group_vars/sonarqube.yml | 5 + .../infrastructure/inventory/cicd/hosts.yml | 20 + 09-ci-04-teamcity/infrastructure/site.yml | 380 ++++++++++++++++++ .../templates/nexus.properties.j2 | 12 + .../infrastructure/templates/nexus.systemd.j2 | 15 + .../templates/nexus.vmoptions.j2 | 16 + 24 files changed, 1040 insertions(+), 5 deletions(-) delete mode 100644 09-ci-03-jenkins/docker/Dockerfile create mode 100644 09-ci-03-jenkins/infrastructure/.gitignore create mode 100644 09-ci-03-jenkins/infrastructure/files/CentOS-Base.repo create mode 100644 09-ci-03-jenkins/infrastructure/files/pg_hba.conf create mode 100644 09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml create mode 100644 09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml create mode 100644 09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml create mode 100644 09-ci-03-jenkins/infrastructure/site.yml create mode 100644 09-ci-03-jenkins/infrastructure/templates/jdk.sh.j2 create mode 100644 09-ci-03-jenkins/infrastructure/templates/nexus.properties.j2 create mode 100644 09-ci-03-jenkins/infrastructure/templates/nexus.systemd.j2 create mode 100644 09-ci-03-jenkins/infrastructure/templates/nexus.vmoptions.j2 create mode 100644 09-ci-03-jenkins/infrastructure/templates/sonar.unit.j2 create mode 100644 09-ci-03-jenkins/infrastructure/templates/wrapper.conf.j2 create mode 100644 09-ci-04-teamcity/infrastructure/.gitignore create mode 100644 09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/all.yml create mode 100644 09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/nexus.yml create mode 100644 09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml create mode 100644 09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml create mode 100644 09-ci-04-teamcity/infrastructure/inventory/cicd/hosts.yml create mode 100644 09-ci-04-teamcity/infrastructure/site.yml create mode 100644 09-ci-04-teamcity/infrastructure/templates/nexus.properties.j2 create mode 100644 09-ci-04-teamcity/infrastructure/templates/nexus.systemd.j2 create mode 100644 09-ci-04-teamcity/infrastructure/templates/nexus.vmoptions.j2 diff --git a/09-ci-03-jenkins/docker/Dockerfile b/09-ci-03-jenkins/docker/Dockerfile deleted file mode 100644 index ffac7cad3..000000000 --- a/09-ci-03-jenkins/docker/Dockerfile +++ /dev/null @@ -1,5 +0,0 @@ -FROM pycontribs/centos:7 - -RUN yum update -y && yum install -y java-1.8.0-openjdk-devel python3 python3-pip -RUN pip3 install -U pip -RUN pip3 install "ansible<3.0.0" \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/.gitignore b/09-ci-03-jenkins/infrastructure/.gitignore new file mode 100644 index 000000000..483815e4c --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/.gitignore @@ -0,0 +1,2 @@ +files/*.pub +roles/* \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/files/CentOS-Base.repo b/09-ci-03-jenkins/infrastructure/files/CentOS-Base.repo new file mode 100644 index 000000000..0984327f0 --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/files/CentOS-Base.repo @@ -0,0 +1,48 @@ +# CentOS-Base.repo +# +# The mirror system uses the connecting IP address of the client and the +# update status of each mirror to pick mirrors that are updated to and +# geographically close to the client. You should use this for CentOS updates +# unless you are manually picking other mirrors. +# +# If the mirrorlist= does not work for you, as a fall back you can try the +# remarked out baseurl= line instead. +# +# + +[base] +name=CentOS-$releasever - Base +mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra +#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/ +gpgcheck=1 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 + +exclude=postgresql* + +#released updates +[updates] +name=CentOS-$releasever - Updates +mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra +#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/ +gpgcheck=1 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 + +exclude=postgresql* + +#additional packages that may be useful +[extras] +name=CentOS-$releasever - Extras +mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra +#baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/ +gpgcheck=1 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 + +#additional packages that extend functionality of existing packages +[centosplus] +name=CentOS-$releasever - Plus +mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus&infra=$infra +#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/ +gpgcheck=1 +enabled=0 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 + diff --git a/09-ci-03-jenkins/infrastructure/files/pg_hba.conf b/09-ci-03-jenkins/infrastructure/files/pg_hba.conf new file mode 100644 index 000000000..b5c361a7b --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/files/pg_hba.conf @@ -0,0 +1,89 @@ +# PostgreSQL Client Authentication Configuration File +# =================================================== +# +# Refer to the "Client Authentication" section in the PostgreSQL +# documentation for a complete description of this file. A short +# synopsis follows. +# +# This file controls: which hosts are allowed to connect, how clients +# are authenticated, which PostgreSQL user names they can use, which +# databases they can access. Records take one of these forms: +# +# local DATABASE USER METHOD [OPTIONS] +# host DATABASE USER ADDRESS METHOD [OPTIONS] +# hostssl DATABASE USER ADDRESS METHOD [OPTIONS] +# hostnossl DATABASE USER ADDRESS METHOD [OPTIONS] +# +# (The uppercase items must be replaced by actual values.) +# +# The first field is the connection type: "local" is a Unix-domain +# socket, "host" is either a plain or SSL-encrypted TCP/IP socket, +# "hostssl" is an SSL-encrypted TCP/IP socket, and "hostnossl" is a +# plain TCP/IP socket. +# +# DATABASE can be "all", "sameuser", "samerole", "replication", a +# database name, or a comma-separated list thereof. The "all" +# keyword does not match "replication". Access to replication +# must be enabled in a separate record (see example below). +# +# USER can be "all", a user name, a group name prefixed with "+", or a +# comma-separated list thereof. In both the DATABASE and USER fields +# you can also write a file name prefixed with "@" to include names +# from a separate file. +# +# ADDRESS specifies the set of hosts the record matches. It can be a +# host name, or it is made up of an IP address and a CIDR mask that is +# an integer (between 0 and 32 (IPv4) or 128 (IPv6) inclusive) that +# specifies the number of significant bits in the mask. A host name +# that starts with a dot (.) matches a suffix of the actual host name. +# Alternatively, you can write an IP address and netmask in separate +# columns to specify the set of hosts. Instead of a CIDR-address, you +# can write "samehost" to match any of the server's own IP addresses, +# or "samenet" to match any address in any subnet that the server is +# directly connected to. +# +# METHOD can be "trust", "reject", "md5", "password", "scram-sha-256", +# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius" or "cert". +# Note that "password" sends passwords in clear text; "md5" or +# "scram-sha-256" are preferred since they send encrypted passwords. +# +# OPTIONS are a set of options for the authentication in the format +# NAME=VALUE. The available options depend on the different +# authentication methods -- refer to the "Client Authentication" +# section in the documentation for a list of which options are +# available for which authentication methods. +# +# Database and user names containing spaces, commas, quotes and other +# special characters must be quoted. Quoting one of the keywords +# "all", "sameuser", "samerole" or "replication" makes the name lose +# its special character, and just match a database or username with +# that name. +# +# This file is read on server startup and when the server receives a +# SIGHUP signal. If you edit the file on a running system, you have to +# SIGHUP the server for the changes to take effect, run "pg_ctl reload", +# or execute "SELECT pg_reload_conf()". +# +# Put your actual configuration here +# ---------------------------------- +# +# If you want to allow non-local connections, you need to add more +# "host" records. In that case you will also need to make PostgreSQL +# listen on a non-local interface via the listen_addresses +# configuration parameter, or via the -i or -h command line switches. + + + +# TYPE DATABASE USER ADDRESS METHOD + +# "local" is for Unix domain socket connections only +local all all trust +# IPv4 local connections: +host all all 127.0.0.1/32 md5 +# IPv6 local connections: +host all all ::1/128 md5 +# Allow replication connections from localhost, by a user with the +# replication privilege. +local replication all trust +host replication all 127.0.0.1/32 md5 +host replication all ::1/128 md5 diff --git a/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml b/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml new file mode 100644 index 000000000..b60a73de1 --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml @@ -0,0 +1,8 @@ +--- +user_group: "{{ jenkins_user_group }}" +user_name: "{{ jenkins_user_name }}" +jenkins_user_name: jenkins +jenkins_user_group: jenkins +java_packages: + - java-11-openjdk-devel + - java-11-openjdk \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml b/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml new file mode 100644 index 000000000..2698ac96c --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml @@ -0,0 +1,27 @@ +--- +java_packages: + - java-1.8.0-openjdk + - java-1.8.0-openjdk-devel +user_group: "{{ nexus_user_group }}" +user_name: "{{ nexus_user_name }}" +nexus_user_group: nexus +nexus_user_name: nexus +nexus_directory_data: "/home/{{ nexus_user_name }}/sonatype-work/nexus3" +nexus_directory_home: "/home/{{ nexus_user_name }}/nexus" +nexus_directory_log: "/home/{{ nexus_user_name }}/log" +nexus_directory_package: "/home/{{ nexus_user_name }}/pkg" +nexus_directory_tmp: "/home/{{ nexus_user_name }}/tmp" +nexus_version: 3.14.0-04 +nexus_download_url: https://download.sonatype.com/nexus/3 +nexus_service_enabled: true +nexus_ulimit: 65536 +nexus_context_path: / +nexus_host: 0.0.0.0 +nexus_port: 8081 +nexus_port_check_timeout: 600 +nexus_edition: nexus-oss-edition +nexus_features: nexus-oss-feature +nexus_java_heap_size: 1200M +nexus_java_max_direct_memory: 2G +nexus_service_start_on_boot: true +nexus_configuration_disk_free_space_limit: ~ \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml b/09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml new file mode 100644 index 000000000..1bf111dc0 --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml @@ -0,0 +1,27 @@ +--- +all: + hosts: + nexus-01: + ansible_host: 84.201.147.2 + jenkins-master-01: + ansible_host: 84.201.150.155 + # jenkins-agent-01: + # ansible_host: + # jenkins-agent-02: + # ansible_host: + children: + jenkins: + children: + jenkins_masters: + hosts: + jenkins-master-01: + # jenkins_agents: + # hosts: + # jenkins-agent-01: + # jenkins-agent-02: + nexus: + hosts: + nexus-01: + vars: + ansible_connection_type: paramiko + ansible_user: aragast \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/site.yml b/09-ci-03-jenkins/infrastructure/site.yml new file mode 100644 index 000000000..2759f9ec3 --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/site.yml @@ -0,0 +1,190 @@ +--- +- name: Preapre all hosts + hosts: all + tasks: + - name: Create group + become: true + group: + name: "{{ user_group }}" + state: present + - name: Create user + become: true + user: + name: "{{ user_name }}" + - name: Install JDK + become: true + package: + name: "{{ java_packages }}" + state: present +- name: Get Nexus installed + hosts: nexus + tasks: + - name: Create Nexus directories + become: true + file: + group: "{{ nexus_user_group }}" + owner: "{{ nexus_user_name }}" + path: "{{ item }}" + state: directory + with_items: + - "{{ nexus_directory_log }}" + - "{{ nexus_directory_data }}" + - "{{ nexus_directory_data }}/etc" + - "{{ nexus_directory_package }}" + - "{{ nexus_directory_tmp }}" + + - name: Download Nexus + become: true + become_user: "{{ nexus_user_name }}" + get_url: + dest: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}.tar.gz" + url: "{{ nexus_download_url }}/nexus-{{ nexus_version }}-unix.tar.gz" + + - name: Unpack Nexus + become: true + become_user: "{{ nexus_user_name }}" + unarchive: + copy: no + creates: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}" + dest: "{{ nexus_directory_package }}" + src: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}.tar.gz" + + - name: Link to Nexus Directory + become: true + become_user: "{{ nexus_user_name }}" + file: + dest: "{{ nexus_directory_home }}" + src: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}" + state: link + + - name: Add NEXUS_HOME for Nexus user + become: true + become_user: "{{ nexus_user_name }}" + lineinfile: + create: yes + dest: "/home/{{ nexus_user_name }}/.bashrc" + insertafter: EOF + line: "export NEXUS_HOME={{ nexus_directory_home }}" + + - name: Add run_as_user to Nexus.rc + become: true + become_user: "{{ nexus_user_name }}" + lineinfile: + create: yes + dest: "{{ nexus_directory_home }}/bin/nexus.rc" + insertafter: EOF + line: "run_as_user=\"{{ nexus_user_name }}\"" + regexp: "^run_as_user" + + - name: Raise nofile limit for Nexus user + become: true + pam_limits: + domain: "{{ nexus_user_name }}" + limit_type: "-" + limit_item: nofile + value: "{{ nexus_ulimit }}" + + - name: Create Nexus service for SystemD + become: true + template: + dest: /lib/systemd/system/nexus.service + mode: 0644 + src: nexus.systemd.j2 + + - name: Ensure Nexus service is enabled for SystemD + become: true + systemd: + daemon_reload: yes + enabled: yes + name: nexus + when: + - nexus_service_enabled + + - name: Create Nexus vmoptions + become: true + become_user: "{{ nexus_user_name }}" + template: + dest: "{{ nexus_directory_home }}/bin/nexus.vmoptions" + src: nexus.vmoptions.j2 + register: nexus_config_changed + + - name: Create Nexus properties + become: true + become_user: "{{ nexus_user_name }}" + template: + dest: "{{ nexus_directory_data }}/etc/nexus.properties" + src: nexus.properties.j2 + register: nexus_config_changed + + - name: Lower Nexus disk space threshold + become: true + become_user: "{{ nexus_user_name }}" + lineinfile: + backrefs: yes + dest: "{{ nexus_directory_home }}/etc/karaf/system.properties" + insertafter: EOF + line: "storage.diskCache.diskFreeSpaceLimit={{ nexus_configuration_disk_free_space_limit }}" + regexp: ^storage\.diskCache\.diskFreeSpaceLimit + when: nexus_configuration_disk_free_space_limit is not none + register: nexus_config_changed + + - name: Start Nexus service if enabled + become: true + service: + enabled: yes + name: nexus + state: started + when: + - nexus_service_start_on_boot + - not nexus_config_changed.changed + tags: + - skip_ansible_lint + + - name: Ensure Nexus service is restarted + become: true + service: + name: nexus + state: restarted + when: + - nexus_service_start_on_boot + - nexus_config_changed.changed + tags: + - skip_ansible_lint + + - name: Wait for Nexus port if started + wait_for: + port: "{{ nexus_port }}" + state: started + timeout: "{{ nexus_port_check_timeout }}" + when: + - nexus_service_start_on_boot + +- name: Get Jenkins master installed + hosts: jenkins_masters + tasks: + - name: Get repo Jenkins + become: true + get_url: + url: https://pkg.jenkins.io/redhat-stable/jenkins.repo + dest: /etc/yum.repos.d/jenkins.repo + - name: Add Jenkins key + become: true + rpm_key: + key: https://pkg.jenkins.io/redhat-stable/jenkins.io.key + state: present + - name: Install epel-release + become: true + yum: + name: epel-release + state: present + - name: Install Jenkins + become: true + yum: + name: jenkins + state: present + - name: Start Jenkins + become: true + systemd: + name: jenkins + state: restarted + enabled: true diff --git a/09-ci-03-jenkins/infrastructure/templates/jdk.sh.j2 b/09-ci-03-jenkins/infrastructure/templates/jdk.sh.j2 new file mode 100644 index 000000000..39887b6ab --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/templates/jdk.sh.j2 @@ -0,0 +1,5 @@ +# Warning: This file is Ansible Managed, manual changes will be overwritten on next playbook run. +#!/usr/bin/env bash + +export JAVA_HOME={{ java_home }} +export PATH=$PATH:$JAVA_HOME/bin \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/templates/nexus.properties.j2 b/09-ci-03-jenkins/infrastructure/templates/nexus.properties.j2 new file mode 100644 index 000000000..aa2645463 --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/templates/nexus.properties.j2 @@ -0,0 +1,12 @@ +# +# + +# Jetty section +application-host={{ nexus_host }} +application-port={{ nexus_port }} +nexus-context-path={{ nexus_context_path }} + +# Nexus section +nexus-edition={{ nexus_edition }} +nexus-features=\ + {{ nexus_features }} \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/templates/nexus.systemd.j2 b/09-ci-03-jenkins/infrastructure/templates/nexus.systemd.j2 new file mode 100644 index 000000000..7665d96a8 --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/templates/nexus.systemd.j2 @@ -0,0 +1,15 @@ +[Unit] +Description=nexus service +After=network.target + +[Service] +Type=forking +User={{ nexus_user_name }} +Group={{ nexus_user_group }} +LimitNOFILE={{ nexus_ulimit }} +ExecStart={{ nexus_directory_home }}/bin/nexus start +ExecStop={{ nexus_directory_home }}/bin/nexus stop +Restart=on-abort + +[Install] +WantedBy=multi-user.target \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/templates/nexus.vmoptions.j2 b/09-ci-03-jenkins/infrastructure/templates/nexus.vmoptions.j2 new file mode 100644 index 000000000..dea2288a7 --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/templates/nexus.vmoptions.j2 @@ -0,0 +1,16 @@ +-Xms{{ nexus_java_heap_size }} +-Xmx{{ nexus_java_heap_size }} +-XX:MaxDirectMemorySize={{ nexus_java_max_direct_memory }} +-XX:+UnlockDiagnosticVMOptions +-XX:+UnsyncloadClass +-XX:+LogVMOutput +-XX:LogFile={{ nexus_directory_log }}/jvm.log +-XX:-OmitStackTraceInFastThrow +-Djava.net.preferIPv4Stack=true +-Dkaraf.home=. +-Dkaraf.base=. +-Dkaraf.etc=etc/karaf +-Djava.util.logging.config.file=etc/karaf/java.util.logging.properties +-Dkaraf.data={{ nexus_directory_data }} +-Djava.io.tmpdir={{ nexus_directory_tmp }} +-Dkaraf.startLocalConsole=false \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/templates/sonar.unit.j2 b/09-ci-03-jenkins/infrastructure/templates/sonar.unit.j2 new file mode 100644 index 000000000..fb48c8067 --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/templates/sonar.unit.j2 @@ -0,0 +1,22 @@ +[Unit] +Description=SonarQube +After=network.target network-online.target +Wants=network-online.target + +[Service] +LimitNOFILE=131072 +LimitNPROC=8192 +User={{ sonarqube_db_user }} +Group={{ sonarqube_db_user }} +ExecStart=/usr/bin/sonar start +ExecStop=/usr/bin/sonar stop +ExecReload=/usr/bin/sonar restart +PIDFile=/usr/local/sonar/bin/linux-x86-64/./SonarQube.pid +Type=simple +PermissionsStartOnly=true +TimeoutStartSec=5 +Restart=always +SuccessExitStatus=143 + +[Install] +WantedBy=multi-user.target diff --git a/09-ci-03-jenkins/infrastructure/templates/wrapper.conf.j2 b/09-ci-03-jenkins/infrastructure/templates/wrapper.conf.j2 new file mode 100644 index 000000000..bb271c188 --- /dev/null +++ b/09-ci-03-jenkins/infrastructure/templates/wrapper.conf.j2 @@ -0,0 +1,98 @@ +# Path to JVM executable. By default it must be available in PATH. +# Can be an absolute path, for example: +wrapper.java.command={{ java_home }}/bin/java + + +# +# DO NOT EDIT THE FOLLOWING SECTIONS +# + + +#******************************************************************** +# Wrapper Java +#******************************************************************** +wrapper.java.additional.1=-Dsonar.wrapped=true +wrapper.java.additional.2=-Djava.awt.headless=true +# extra args needed by hazelcast +wrapper.java.additional.3=--add-exports=java.base/jdk.internal.ref=ALL-UNNAMED +wrapper.java.additional.4=--add-opens=java.base/java.lang=ALL-UNNAMED +wrapper.java.additional.5=--add-opens=java.base/java.nio=ALL-UNNAMED +wrapper.java.additional.6=--add-opens=java.base/sun.nio.ch=ALL-UNNAMED +wrapper.java.additional.7=--add-opens=java.management/sun.management=ALL-UNNAMED +wrapper.java.additional.8=--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED + +wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp +wrapper.java.classpath.1=../../lib/sonar-application-9.1.0.47736.jar +wrapper.java.classpath.2=../../lib/jsw/wrapper-3.2.3.jar +wrapper.java.classpath.3=../../lib/sonar-shutdowner-9.1.0.47736.jar +wrapper.java.library.path.1=./lib +wrapper.app.parameter.1=org.sonar.application.App +wrapper.java.initmemory=8 +wrapper.java.maxmemory=32 + +#******************************************************************** +# Wrapper Logs +#******************************************************************** + +wrapper.console.format=PM +wrapper.console.loglevel=INFO +wrapper.logfile.format=M +wrapper.logfile.loglevel=INFO +wrapper.logfile.rollmode=DATE +wrapper.logfile=../../logs/sonar.YYYYMMDD.log + +# Maximum size that the log file will be allowed to grow to before +# the log is rolled. Size is specified in bytes. The default value +# of 0, disables log rolling. May abbreviate with the 'k' (kb) or +# 'm' (mb) suffix. For example: 10m = 10 megabytes. +#wrapper.logfile.maxsize=0 + +# Maximum number of rolled log files which will be allowed before old +# files are deleted. The default value of 0 implies no limit. +wrapper.logfile.maxfiles=7 + +# Log Level for sys/event log output. (See docs for log levels) +wrapper.syslog.loglevel=NONE + +#******************************************************************** +# Wrapper Windows Properties +#******************************************************************** +# Title to use when running as a console +wrapper.console.title=SonarQube + +# Disallow start of multiple instances of an application at the same time on Windows +wrapper.single_invocation=true + +#******************************************************************** +# Wrapper Windows NT/2000/XP Service Properties +#******************************************************************** +# WARNING - Do not modify any of these properties when an application +# using this configuration file has been installed as a service. +# Please uninstall the service before modifying this section. The +# service can then be reinstalled. + +# Name of the service +wrapper.ntservice.name=SonarQube + +# Display name of the service +wrapper.ntservice.displayname=SonarQube + +# Description of the service +wrapper.ntservice.description=SonarQube + +# Service dependencies. Add dependencies as needed starting from 1 +wrapper.ntservice.dependency.1= + +# Mode in which the service is installed. AUTO_START or DEMAND_START +wrapper.ntservice.starttype=AUTO_START + +# Allow the service to interact with the desktop. +wrapper.ntservice.interactive=false + +#******************************************************************** +# Forking Properties +#******************************************************************** +wrapper.disable_restarts=TRUE +wrapper.ping.timeout=0 +wrapper.shutdown.timeout=0 +wrapper.jvm_exit.timeout=0 diff --git a/09-ci-04-teamcity/infrastructure/.gitignore b/09-ci-04-teamcity/infrastructure/.gitignore new file mode 100644 index 000000000..483815e4c --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/.gitignore @@ -0,0 +1,2 @@ +files/*.pub +roles/* \ No newline at end of file diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/all.yml b/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/all.yml new file mode 100644 index 000000000..f6afeefb1 --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/all.yml @@ -0,0 +1,8 @@ +--- +jdk_url: https://download.java.net/openjdk/jdk11/ri/openjdk-11+28_linux-x64_bin.tar.gz +jdk_distr_name: openjdk-11+28_linux-x64_bin.tar.gz +jdk_folder: "{{ jdk_distr_name.split('-')[:2] | join('-') }}" +java_home: "/opt/jdk/{{ jdk_folder }}" +sonarqube_db_name: sonar +sonarqube_db_user: sonar +sonarqube_db_password: sonar \ No newline at end of file diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/nexus.yml b/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/nexus.yml new file mode 100644 index 000000000..b84eba748 --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/nexus.yml @@ -0,0 +1,22 @@ +--- +nexus_user_group: nexus +nexus_user_name: nexus +nexus_directory_data: "/home/{{ nexus_user_name }}/sonatype-work/nexus3" +nexus_directory_home: "/home/{{ nexus_user_name }}/nexus" +nexus_directory_log: "/home/{{ nexus_user_name }}/log" +nexus_directory_package: "/home/{{ nexus_user_name }}/pkg" +nexus_directory_tmp: "/home/{{ nexus_user_name }}/tmp" +nexus_version: 3.14.0-04 +nexus_download_url: https://download.sonatype.com/nexus/3 +nexus_service_enabled: true +nexus_ulimit: 65536 +nexus_context_path: / +nexus_host: 0.0.0.0 +nexus_port: 8081 +nexus_port_check_timeout: 600 +nexus_edition: nexus-oss-edition +nexus_features: nexus-oss-feature +nexus_java_heap_size: 1200M +nexus_java_max_direct_memory: 2G +nexus_service_start_on_boot: true +nexus_configuration_disk_free_space_limit: ~ \ No newline at end of file diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml b/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml new file mode 100644 index 000000000..e0499e7e9 --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml @@ -0,0 +1 @@ +postgresql_version: 11 \ No newline at end of file diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml b/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml new file mode 100644 index 000000000..89b9116c4 --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml @@ -0,0 +1,5 @@ +sonar_version: 9.1.0.47736 +sonar_download_url: "https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-{{ sonar_version }}.zip" +sonar_version_directory: "sonarqube-{{ sonar_version }}" +sonar_db_port: "5432" +sonar_web_context: '' \ No newline at end of file diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/hosts.yml b/09-ci-04-teamcity/infrastructure/inventory/cicd/hosts.yml new file mode 100644 index 000000000..ef7633b5a --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/inventory/cicd/hosts.yml @@ -0,0 +1,20 @@ +--- +all: + hosts: + sonar-01: + ansible_host: + nexus-01: + ansible_host: + children: + sonarqube: + hosts: + sonar-01: + nexus: + hosts: + nexus-01: + postgres: + hosts: + sonar-01: + vars: + ansible_connection_type: paramiko + ansible_user: \ No newline at end of file diff --git a/09-ci-04-teamcity/infrastructure/site.yml b/09-ci-04-teamcity/infrastructure/site.yml new file mode 100644 index 000000000..5744806e8 --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/site.yml @@ -0,0 +1,380 @@ +--- +- name: Get OpenJDK installed + hosts: sonarqube + pre_tasks: + - name: install unzip + become: true + yum: + name: unzip + state: present + tasks: + - name: Upload .tar.gz file conaining binaries from remote storage + get_url: + url: "{{ jdk_url }}" + dest: "/tmp/jdk-{{ jdk_distr_name }}" + mode: 0755 + register: download_java_remote_binaries + until: download_java_remote_binaries is succeeded + + - name: Ensure installation dir exists + become: true + file: + state: directory + path: "{{ java_home }}" + mode: 0755 + + - name: Extract java in the installation directory + become: true + unarchive: + copy: false + src: "/tmp/jdk-{{ jdk_distr_name }}" + dest: "{{ java_home }}" + extra_opts: [--strip-components=1] + creates: "{{ java_home }}/bin/java" + + - name: Export environment variables + become: true + template: + src: jdk.sh.j2 + dest: /etc/profile.d/jdk.sh + owner: root + group: root + mode: 0644 + +- name: Get PostgreSQL installed + hosts: postgres + become: true + tasks: + - name: Change repo file + copy: + src: CentOS-Base.repo + dest: /etc/yum.repos.d/CentOS-Base.repo + mode: 0644 + owner: root + group: root + + - name: Install PostgreSQL repos + yum: + name: https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm + state: present + + - name: Install PostgreSQL + yum: + name: "postgresql{{ postgresql_version }}-server" + state: present + + - name: Init template1 DB + command: /usr/pgsql-11/bin/postgresql-11-setup initdb + failed_when: false + + - name: Start pgsql service + systemd: + name: "postgresql-{{ postgresql_version }}" + state: started + enabled: true + + - name: Create user in system + user: + name: "{{ sonarqube_db_user }}" + - name: Create user for Sonar in PostgreSQL + become_user: postgres + command: "createuser -s -e {{ sonarqube_db_user }}" + failed_when: false + + - name: Change password for Sonar user in PostgreSQL + become_user: postgres + command: "psql -c \"ALTER USER sonar WITH ENCRYPTED password '{{ sonarqube_db_password }}';\"" + + - name: Create Sonar DB + become_user: postgres + command: "createdb {{ sonarqube_db_name }}" + failed_when: false + + - name: Copy pg_hba.conf + copy: + src: pg_hba.conf + dest: /var/lib/pgsql/11/data/pg_hba.conf + mode: 0600 + owner: postgres + group: postgres + +- name: Prepare Sonar host + hosts: sonarqube + become: true + tasks: + - name: Create group in system + group: + name: "{{ sonarqube_db_user }}" + state: present + + - name: Create user in system + user: + name: "{{ sonarqube_db_user }}" + group: "{{ sonarqube_db_user }}" + + - name: "Set up ssh key to access for managed node" + authorized_key: + user: "{{ sonarqube_db_user }}" + state: present + key: "{{ lookup('file', 'id_rsa.pub') }}" + + - name: "Allow group to have passwordless sudo" + lineinfile: + dest: /etc/sudoers + state: present + regexp: '^%{{ sonarqube_db_user }}' + line: '%{{ sonarqube_db_user }} ALL=(ALL) NOPASSWD: ALL' + validate: 'visudo -cf %s' + + - name: Increase Virtual Memory + lineinfile: + dest: /etc/sysctl.conf + state: present + regexp: '^vm.max_map_count' + line: 'vm.max_map_count=262144' + - name: Reboot VM + reboot: + +- name: Get Sonarqube installed + hosts: sonarqube + vars: + ansible_user: "{{ sonarqube_db_user }}" + tasks: + - name: Get distrib ZIP + get_url: + url: "{{ sonar_download_url }}" + dest: "/tmp/{{ sonar_version_directory }}.zip" + validate_certs: false + + - name: Unzip Sonar + become: true + unarchive: + src: "/tmp/{{ sonar_version_directory }}.zip" + dest: /usr/local/ + copy: false + owner: "{{ sonarqube_db_user }}" + group: "{{ sonarqube_db_user }}" + creates: /usr/local/sonar/COPYING + + - name: Move Sonar into place. + become: true + copy: + src: /usr/local/{{ sonar_version_directory }}/ + dest: /usr/local/sonar/ + owner: "{{ sonarqube_db_user }}" + group: "{{ sonarqube_db_user }}" + remote_src: true + + - name: Configure SonarQube JDBC settings for PostgreSQL. + lineinfile: + dest: /usr/local/sonar/conf/sonar.properties + regexp: "{{ item.regexp }}" + line: "{{ item.line }}" + with_items: + - regexp: "^sonar.jdbc.username" + line: "sonar.jdbc.username={{ sonarqube_db_user }}" + - regexp: "^sonar.jdbc.password" + line: "sonar.jdbc.password={{ sonarqube_db_password }}" + - regexp: "^sonar.jdbc.url" + line: "sonar.jdbc.url=jdbc:postgresql://localhost:{{ sonar_db_port }}/{{ sonarqube_db_name }}?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance" + - regexp: "^sonar.web.context" + line: "sonar.web.context={{ sonar_web_context }}" + + - name: Generate wrapper.conf + template: + src: wrapper.conf.j2 + dest: /usr/local/sonar/conf/wrapper.conf + mode: 0644 + + - name: Symlink sonar bin. + become: true + file: + src: /usr/local/sonar/bin/linux-x86-64/sonar.sh + dest: /usr/bin/sonar + state: link + register: sonar_symlink + + - name: Copy SonarQube systemd unit file into place (for systemd systems). + become: true + template: + src: sonar.unit.j2 + dest: /etc/systemd/system/sonar.service + owner: root + group: root + mode: 0755 + + - name: Ensure Sonar is running and set to start on boot. + become: true + service: + name: sonar + state: restarted + enabled: true + + - name: Allow Sonar time to build on first start. + pause: + seconds: 180 + when: sonar_symlink.changed + tags: ['skip_ansible_lint'] + + - name: Make sure Sonar is responding on the configured port. + wait_for: + port: 9000 + delay: 3 + timeout: 300 +- name: Get Nexus installed + hosts: nexus + pre_tasks: + - name: Create Nexus group + become: true + group: + name: "{{ nexus_user_group }}" + state: present + - name: Create Nexus user + become: true + user: + name: "{{ nexus_user_name }}" + - name: Install JDK + become: true + package: + name: [java-1.8.0-openjdk, java-1.8.0-openjdk-devel] + state: present + tasks: + - name: Create Nexus directories + become: true + file: + group: "{{ nexus_user_group }}" + owner: "{{ nexus_user_name }}" + path: "{{ item }}" + state: directory + with_items: + - "{{ nexus_directory_log }}" + - "{{ nexus_directory_data }}" + - "{{ nexus_directory_data }}/etc" + - "{{ nexus_directory_package }}" + - "{{ nexus_directory_tmp }}" + + - name: Download Nexus + become: true + become_user: "{{ nexus_user_name }}" + get_url: + dest: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}.tar.gz" + url: "{{ nexus_download_url }}/nexus-{{ nexus_version }}-unix.tar.gz" + + - name: Unpack Nexus + become: true + become_user: "{{ nexus_user_name }}" + unarchive: + copy: no + creates: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}" + dest: "{{ nexus_directory_package }}" + src: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}.tar.gz" + + - name: Link to Nexus Directory + become: true + become_user: "{{ nexus_user_name }}" + file: + dest: "{{ nexus_directory_home }}" + src: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}" + state: link + + - name: Add NEXUS_HOME for Nexus user + become: true + become_user: "{{ nexus_user_name }}" + lineinfile: + create: yes + dest: "/home/{{ nexus_user_name }}/.bashrc" + insertafter: EOF + line: "export NEXUS_HOME={{ nexus_directory_home }}" + + - name: Add run_as_user to Nexus.rc + become: true + become_user: "{{ nexus_user_name }}" + lineinfile: + create: yes + dest: "{{ nexus_directory_home }}/bin/nexus.rc" + insertafter: EOF + line: "run_as_user=\"{{ nexus_user_name }}\"" + regexp: "^run_as_user" + + - name: Raise nofile limit for Nexus user + become: true + pam_limits: + domain: "{{ nexus_user_name }}" + limit_type: "-" + limit_item: nofile + value: "{{ nexus_ulimit }}" + + - name: Create Nexus service for SystemD + become: true + template: + dest: /lib/systemd/system/nexus.service + mode: 0644 + src: nexus.systemd.j2 + + - name: Ensure Nexus service is enabled for SystemD + become: true + systemd: + daemon_reload: yes + enabled: yes + name: nexus + when: + - nexus_service_enabled + + - name: Create Nexus vmoptions + become: true + become_user: "{{ nexus_user_name }}" + template: + dest: "{{ nexus_directory_home }}/bin/nexus.vmoptions" + src: nexus.vmoptions.j2 + register: nexus_config_changed + + - name: Create Nexus properties + become: true + become_user: "{{ nexus_user_name }}" + template: + dest: "{{ nexus_directory_data }}/etc/nexus.properties" + src: nexus.properties.j2 + register: nexus_config_changed + + - name: Lower Nexus disk space threshold + become: true + become_user: "{{ nexus_user_name }}" + lineinfile: + backrefs: yes + dest: "{{ nexus_directory_home }}/etc/karaf/system.properties" + insertafter: EOF + line: "storage.diskCache.diskFreeSpaceLimit={{ nexus_configuration_disk_free_space_limit }}" + regexp: ^storage\.diskCache\.diskFreeSpaceLimit + when: nexus_configuration_disk_free_space_limit is not none + register: nexus_config_changed + + - name: Start Nexus service if enabled + become: true + service: + enabled: yes + name: nexus + state: started + when: + - nexus_service_start_on_boot + - not nexus_config_changed.changed + tags: + - skip_ansible_lint + + - name: Ensure Nexus service is restarted + become: true + service: + name: nexus + state: restarted + when: + - nexus_service_start_on_boot + - nexus_config_changed.changed + tags: + - skip_ansible_lint + + - name: Wait for Nexus port if started + wait_for: + port: "{{ nexus_port }}" + state: started + timeout: "{{ nexus_port_check_timeout }}" + when: + - nexus_service_start_on_boot diff --git a/09-ci-04-teamcity/infrastructure/templates/nexus.properties.j2 b/09-ci-04-teamcity/infrastructure/templates/nexus.properties.j2 new file mode 100644 index 000000000..aa2645463 --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/templates/nexus.properties.j2 @@ -0,0 +1,12 @@ +# +# + +# Jetty section +application-host={{ nexus_host }} +application-port={{ nexus_port }} +nexus-context-path={{ nexus_context_path }} + +# Nexus section +nexus-edition={{ nexus_edition }} +nexus-features=\ + {{ nexus_features }} \ No newline at end of file diff --git a/09-ci-04-teamcity/infrastructure/templates/nexus.systemd.j2 b/09-ci-04-teamcity/infrastructure/templates/nexus.systemd.j2 new file mode 100644 index 000000000..7665d96a8 --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/templates/nexus.systemd.j2 @@ -0,0 +1,15 @@ +[Unit] +Description=nexus service +After=network.target + +[Service] +Type=forking +User={{ nexus_user_name }} +Group={{ nexus_user_group }} +LimitNOFILE={{ nexus_ulimit }} +ExecStart={{ nexus_directory_home }}/bin/nexus start +ExecStop={{ nexus_directory_home }}/bin/nexus stop +Restart=on-abort + +[Install] +WantedBy=multi-user.target \ No newline at end of file diff --git a/09-ci-04-teamcity/infrastructure/templates/nexus.vmoptions.j2 b/09-ci-04-teamcity/infrastructure/templates/nexus.vmoptions.j2 new file mode 100644 index 000000000..dea2288a7 --- /dev/null +++ b/09-ci-04-teamcity/infrastructure/templates/nexus.vmoptions.j2 @@ -0,0 +1,16 @@ +-Xms{{ nexus_java_heap_size }} +-Xmx{{ nexus_java_heap_size }} +-XX:MaxDirectMemorySize={{ nexus_java_max_direct_memory }} +-XX:+UnlockDiagnosticVMOptions +-XX:+UnsyncloadClass +-XX:+LogVMOutput +-XX:LogFile={{ nexus_directory_log }}/jvm.log +-XX:-OmitStackTraceInFastThrow +-Djava.net.preferIPv4Stack=true +-Dkaraf.home=. +-Dkaraf.base=. +-Dkaraf.etc=etc/karaf +-Djava.util.logging.config.file=etc/karaf/java.util.logging.properties +-Dkaraf.data={{ nexus_directory_data }} +-Djava.io.tmpdir={{ nexus_directory_tmp }} +-Dkaraf.startLocalConsole=false \ No newline at end of file From 2aac3c3fc2b6d0e86a06c1d5cee3c213675d0c17 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Sun, 26 Sep 2021 15:33:36 +0300 Subject: [PATCH 14/40] add jenkins_agent play --- .../inventory/cicd/group_vars/jenkins.yml | 3 +- .../infrastructure/inventory/cicd/hosts.yml | 20 +- 09-ci-03-jenkins/infrastructure/site.yml | 181 ++++-------------- 3 files changed, 47 insertions(+), 157 deletions(-) diff --git a/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml b/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml index b60a73de1..e23ccd83d 100644 --- a/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml +++ b/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml @@ -5,4 +5,5 @@ jenkins_user_name: jenkins jenkins_user_group: jenkins java_packages: - java-11-openjdk-devel - - java-11-openjdk \ No newline at end of file + - java-11-openjdk +jenkins_agent_dir: /opt/jenkins_agent/ \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml b/09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml index 1bf111dc0..80c0d8196 100644 --- a/09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml +++ b/09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml @@ -1,27 +1,19 @@ --- all: hosts: - nexus-01: - ansible_host: 84.201.147.2 jenkins-master-01: - ansible_host: 84.201.150.155 - # jenkins-agent-01: - # ansible_host: - # jenkins-agent-02: - # ansible_host: + ansible_host: + jenkins-agent-01: + ansible_host: children: jenkins: children: jenkins_masters: hosts: jenkins-master-01: - # jenkins_agents: - # hosts: - # jenkins-agent-01: - # jenkins-agent-02: - nexus: - hosts: - nexus-01: + jenkins_agents: + hosts: + jenkins-agent-01: vars: ansible_connection_type: paramiko ansible_user: aragast \ No newline at end of file diff --git a/09-ci-03-jenkins/infrastructure/site.yml b/09-ci-03-jenkins/infrastructure/site.yml index 2759f9ec3..0ee1b2ee4 100644 --- a/09-ci-03-jenkins/infrastructure/site.yml +++ b/09-ci-03-jenkins/infrastructure/site.yml @@ -11,153 +11,13 @@ become: true user: name: "{{ user_name }}" + generate_ssh_key: true + register: ssh_key_info - name: Install JDK become: true package: name: "{{ java_packages }}" state: present -- name: Get Nexus installed - hosts: nexus - tasks: - - name: Create Nexus directories - become: true - file: - group: "{{ nexus_user_group }}" - owner: "{{ nexus_user_name }}" - path: "{{ item }}" - state: directory - with_items: - - "{{ nexus_directory_log }}" - - "{{ nexus_directory_data }}" - - "{{ nexus_directory_data }}/etc" - - "{{ nexus_directory_package }}" - - "{{ nexus_directory_tmp }}" - - - name: Download Nexus - become: true - become_user: "{{ nexus_user_name }}" - get_url: - dest: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}.tar.gz" - url: "{{ nexus_download_url }}/nexus-{{ nexus_version }}-unix.tar.gz" - - - name: Unpack Nexus - become: true - become_user: "{{ nexus_user_name }}" - unarchive: - copy: no - creates: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}" - dest: "{{ nexus_directory_package }}" - src: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}.tar.gz" - - - name: Link to Nexus Directory - become: true - become_user: "{{ nexus_user_name }}" - file: - dest: "{{ nexus_directory_home }}" - src: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}" - state: link - - - name: Add NEXUS_HOME for Nexus user - become: true - become_user: "{{ nexus_user_name }}" - lineinfile: - create: yes - dest: "/home/{{ nexus_user_name }}/.bashrc" - insertafter: EOF - line: "export NEXUS_HOME={{ nexus_directory_home }}" - - - name: Add run_as_user to Nexus.rc - become: true - become_user: "{{ nexus_user_name }}" - lineinfile: - create: yes - dest: "{{ nexus_directory_home }}/bin/nexus.rc" - insertafter: EOF - line: "run_as_user=\"{{ nexus_user_name }}\"" - regexp: "^run_as_user" - - - name: Raise nofile limit for Nexus user - become: true - pam_limits: - domain: "{{ nexus_user_name }}" - limit_type: "-" - limit_item: nofile - value: "{{ nexus_ulimit }}" - - - name: Create Nexus service for SystemD - become: true - template: - dest: /lib/systemd/system/nexus.service - mode: 0644 - src: nexus.systemd.j2 - - - name: Ensure Nexus service is enabled for SystemD - become: true - systemd: - daemon_reload: yes - enabled: yes - name: nexus - when: - - nexus_service_enabled - - - name: Create Nexus vmoptions - become: true - become_user: "{{ nexus_user_name }}" - template: - dest: "{{ nexus_directory_home }}/bin/nexus.vmoptions" - src: nexus.vmoptions.j2 - register: nexus_config_changed - - - name: Create Nexus properties - become: true - become_user: "{{ nexus_user_name }}" - template: - dest: "{{ nexus_directory_data }}/etc/nexus.properties" - src: nexus.properties.j2 - register: nexus_config_changed - - - name: Lower Nexus disk space threshold - become: true - become_user: "{{ nexus_user_name }}" - lineinfile: - backrefs: yes - dest: "{{ nexus_directory_home }}/etc/karaf/system.properties" - insertafter: EOF - line: "storage.diskCache.diskFreeSpaceLimit={{ nexus_configuration_disk_free_space_limit }}" - regexp: ^storage\.diskCache\.diskFreeSpaceLimit - when: nexus_configuration_disk_free_space_limit is not none - register: nexus_config_changed - - - name: Start Nexus service if enabled - become: true - service: - enabled: yes - name: nexus - state: started - when: - - nexus_service_start_on_boot - - not nexus_config_changed.changed - tags: - - skip_ansible_lint - - - name: Ensure Nexus service is restarted - become: true - service: - name: nexus - state: restarted - when: - - nexus_service_start_on_boot - - nexus_config_changed.changed - tags: - - skip_ansible_lint - - - name: Wait for Nexus port if started - wait_for: - port: "{{ nexus_port }}" - state: started - timeout: "{{ nexus_port_check_timeout }}" - when: - - nexus_service_start_on_boot - name: Get Jenkins master installed hosts: jenkins_masters @@ -182,9 +42,46 @@ yum: name: jenkins state: present + register: jenkins_installed + - name: Ensure jenkins agents are present in known_hosts file + become: true + become_user: "{{ jenkins_user_name }}" + known_hosts: + name: "{{ hostvars[item].ansible_host }}" + state: present + key: "{{ lookup('pipe', 'ssh-keyscan {{ hostvars[item].ansible_host }}') }}" + hash_host: true + with_items: "{{ groups.jenkins_agents }}" - name: Start Jenkins become: true systemd: name: jenkins state: restarted enabled: true + when: jenkins_installed is changed +- name: Prepare jenkins agent + hosts: jenkins_agents + become: true + become_user: "{{ jenkins_user_name }}" + tasks: + - name: Add master publickey into authorized_key + authorized_key: + user: "{{ jenkins_user_name }}" + state: present + key: "{{ hostvars['jenkins-master-01']['ssh_key_info']['ssh_public_key'] }}" + - name: Create agent_dir + become_user: root + file: + path: "{{ jenkins_agent_dir }}" + state: directory + owner: "{{ jenkins_user_name }}" + group: "{{ jenkins_user_group }}" + - name: Install agent.jar + get_url: + url: "http://{{ hostvars['jenkins-master-01']['ansible_host'] }}:8080/jnlpJars/agent.jar" + dest: "{{ jenkins_agent_dir }}" + register: install_agent + until: install_agent is succeeded + retries: 10 + timeout: 10 + From 15b1b6ef61d4ad2c45d2edb365311d380e432396 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Sun, 26 Sep 2021 16:09:42 +0300 Subject: [PATCH 15/40] add ansible install --- 09-ci-03-jenkins/infrastructure/site.yml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/09-ci-03-jenkins/infrastructure/site.yml b/09-ci-03-jenkins/infrastructure/site.yml index 0ee1b2ee4..f861aebaa 100644 --- a/09-ci-03-jenkins/infrastructure/site.yml +++ b/09-ci-03-jenkins/infrastructure/site.yml @@ -76,6 +76,27 @@ state: directory owner: "{{ jenkins_user_name }}" group: "{{ jenkins_user_group }}" + - name: Install some required + become_user: root + yum: + name: + - git + - python3 + state: present + - name: Update pip + become_user: root + pip: + executable: pip3 + name: pip + extra_args: --upgrade + - name: Install Ansible + pip: + executable: pip3 + name: + - setuptools_rust + - wheel + - ansible<3.0 + state: present - name: Install agent.jar get_url: url: "http://{{ hostvars['jenkins-master-01']['ansible_host'] }}:8080/jnlpJars/agent.jar" From 358727d60d269bbdeb0d12fe866845fbd8baf438 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Wed, 29 Sep 2021 13:40:35 +0300 Subject: [PATCH 16/40] add 02 homework --- 09-ci-02-devops/README.md | 35 +++++++++++++++++++ {09-ci-02-cicd => 09-ci-03-cicd}/README.md | 0 .../example/fail.py | 0 .../infrastructure/.gitignore | 0 .../infrastructure/files/CentOS-Base.repo | 0 .../infrastructure/files/pg_hba.conf | 0 .../inventory/cicd/group_vars/all.yml | 0 .../inventory/cicd/group_vars/nexus.yml | 0 .../inventory/cicd/group_vars/postgres.yml | 0 .../inventory/cicd/group_vars/sonarqube.yml | 0 .../infrastructure/inventory/cicd/hosts.yml | 0 .../infrastructure/site.yml | 0 .../infrastructure/templates/jdk.sh.j2 | 0 .../templates/nexus.properties.j2 | 0 .../infrastructure/templates/nexus.systemd.j2 | 0 .../templates/nexus.vmoptions.j2 | 0 .../infrastructure/templates/sonar.unit.j2 | 0 .../infrastructure/templates/wrapper.conf.j2 | 0 {09-ci-02-cicd => 09-ci-03-cicd}/mvn/pom.xml | 0 .../README.md | 0 .../infrastructure/.gitignore | 0 .../infrastructure/files/CentOS-Base.repo | 0 .../infrastructure/files/pg_hba.conf | 0 .../inventory/cicd/group_vars/jenkins.yml | 0 .../inventory/cicd/group_vars/nexus.yml | 0 .../infrastructure/inventory/cicd/hosts.yml | 0 .../infrastructure/site.yml | 0 .../infrastructure/templates/jdk.sh.j2 | 0 .../templates/nexus.properties.j2 | 0 .../infrastructure/templates/nexus.systemd.j2 | 0 .../templates/nexus.vmoptions.j2 | 0 .../infrastructure/templates/sonar.unit.j2 | 0 .../infrastructure/templates/wrapper.conf.j2 | 0 .../pipeline/Jenkinsfile | 0 .../README.md | 0 .../infrastructure/.gitignore | 0 .../inventory/cicd/group_vars/all.yml | 0 .../inventory/cicd/group_vars/nexus.yml | 0 .../inventory/cicd/group_vars/postgres.yml | 0 .../inventory/cicd/group_vars/sonarqube.yml | 0 .../infrastructure/inventory/cicd/hosts.yml | 0 .../infrastructure/site.yml | 0 .../templates/nexus.properties.j2 | 0 .../infrastructure/templates/nexus.systemd.j2 | 0 .../templates/nexus.vmoptions.j2 | 0 .../teamcity/docker-compose.yml | 0 .../README.md | 0 .../repository/python-api.py | 0 48 files changed, 35 insertions(+) create mode 100644 09-ci-02-devops/README.md rename {09-ci-02-cicd => 09-ci-03-cicd}/README.md (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/example/fail.py (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/.gitignore (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/files/CentOS-Base.repo (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/files/pg_hba.conf (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/inventory/cicd/group_vars/all.yml (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/inventory/cicd/group_vars/nexus.yml (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/inventory/cicd/group_vars/postgres.yml (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/inventory/cicd/group_vars/sonarqube.yml (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/inventory/cicd/hosts.yml (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/site.yml (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/templates/jdk.sh.j2 (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/templates/nexus.properties.j2 (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/templates/nexus.systemd.j2 (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/templates/nexus.vmoptions.j2 (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/templates/sonar.unit.j2 (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/infrastructure/templates/wrapper.conf.j2 (100%) rename {09-ci-02-cicd => 09-ci-03-cicd}/mvn/pom.xml (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/README.md (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/.gitignore (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/files/CentOS-Base.repo (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/files/pg_hba.conf (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/inventory/cicd/group_vars/jenkins.yml (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/inventory/cicd/group_vars/nexus.yml (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/inventory/cicd/hosts.yml (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/site.yml (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/templates/jdk.sh.j2 (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/templates/nexus.properties.j2 (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/templates/nexus.systemd.j2 (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/templates/nexus.vmoptions.j2 (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/templates/sonar.unit.j2 (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/infrastructure/templates/wrapper.conf.j2 (100%) rename {09-ci-03-jenkins => 09-ci-04-jenkins}/pipeline/Jenkinsfile (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/README.md (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/.gitignore (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/inventory/cicd/group_vars/all.yml (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/inventory/cicd/group_vars/nexus.yml (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/inventory/cicd/group_vars/postgres.yml (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/inventory/cicd/group_vars/sonarqube.yml (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/inventory/cicd/hosts.yml (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/site.yml (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/templates/nexus.properties.j2 (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/templates/nexus.systemd.j2 (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/infrastructure/templates/nexus.vmoptions.j2 (100%) rename {09-ci-04-teamcity => 09-ci-05-teamcity}/teamcity/docker-compose.yml (100%) rename {09-ci-05-gitlab => 09-ci-06-gitlab}/README.md (100%) rename {09-ci-05-gitlab => 09-ci-06-gitlab}/repository/python-api.py (100%) diff --git a/09-ci-02-devops/README.md b/09-ci-02-devops/README.md new file mode 100644 index 000000000..6eef12b69 --- /dev/null +++ b/09-ci-02-devops/README.md @@ -0,0 +1,35 @@ +# Домашнее задание к занятию "09.02 DevOps и SRE" + +## Основная часть + +1. В чём разница между структурой "Сотрудничество dev и ops" и "NoOps" + 1. В NoOps-командах не нужно сопровождение + 2. Сотрудничество dev и ops полностью разделяет две команды на отдельные + 3. NoOps распределяет все задачи ops на обе команды + 4. В NoOps dev не занимается разработкой +2. В чём различие DevOps команды с ограниченным сроком действия и антипаттерна Отдельной DevOps команды + 1. С ограниченным сроком действия нужна для временного создания DevOps, после окончания срока - DevOps заканчивается + 2. Отдельная DevOps команда способна разрулить любой вопрос Dev или Ops, а значит повышает эффективность работоспособности в долгосрочной перспективе, не нагружая лишними задачами + 3. Ответственность команды с ограниченным сроком достаточно низкая, нет необходимости продумывать долгосрочные перспективы развития процессов + 4. Команда с ограниченным сроком действия нужная для внедрения лучших DevOps практик и должна стараться не допустить возникновения антипаттерна +3. Кто такие DevOps евангелисты: + 1. Временные сотрудники, которых руководство использует для создания видимости DevOps + 2. Команда сотрудников, которые необходимы для создания благоприятной атмосферы диалога между Dev и Ops, с помощью в виде экспертных мнений и демонстрации использования DevOps практик + 3. Люди, которые приходят на собрания групп разработки и сопровождения. Их задача - наблюдать со стороны за внедрением DevOps процесса, не вмешиваясь в его построение +4. Кто такие SRE: + 1. Инженеры, которые имеют высокий уровень квалификации как в разработке, так и в сопровождении. Чаще всего вырастают из опытных разработчиков, которых отправили (по методологии SRE) исследовать работу Ops + 2. Отдельная каста сотрудников, которых набирают извне для урегулирования вопросов между Dev и Ops + 3. Архитекторы проектов, которые имеют право вето на любые изменения с любой стороны +5. Как правильно внедрять DevOps: + 1. Использовать определённый тип команды для всей индустрии + 2. Использовать определённый тип команды для всей компании + 3. Использовать определённый тип команды для конкретной команды + 4. Не существует стратегии правильного внедрения, методология выросла из постулатов гибких методологий, в каждом конкретном случае необходимо исследовать команду и пробовать внедрять разные подходы, пока не будет выбран наиболее эфективный + +--- + +### Как оформить ДЗ? + +Выполненное домашнее задание пришлите ссылкой на .md-файл в вашем репозитории. + +--- diff --git a/09-ci-02-cicd/README.md b/09-ci-03-cicd/README.md similarity index 100% rename from 09-ci-02-cicd/README.md rename to 09-ci-03-cicd/README.md diff --git a/09-ci-02-cicd/example/fail.py b/09-ci-03-cicd/example/fail.py similarity index 100% rename from 09-ci-02-cicd/example/fail.py rename to 09-ci-03-cicd/example/fail.py diff --git a/09-ci-02-cicd/infrastructure/.gitignore b/09-ci-03-cicd/infrastructure/.gitignore similarity index 100% rename from 09-ci-02-cicd/infrastructure/.gitignore rename to 09-ci-03-cicd/infrastructure/.gitignore diff --git a/09-ci-02-cicd/infrastructure/files/CentOS-Base.repo b/09-ci-03-cicd/infrastructure/files/CentOS-Base.repo similarity index 100% rename from 09-ci-02-cicd/infrastructure/files/CentOS-Base.repo rename to 09-ci-03-cicd/infrastructure/files/CentOS-Base.repo diff --git a/09-ci-02-cicd/infrastructure/files/pg_hba.conf b/09-ci-03-cicd/infrastructure/files/pg_hba.conf similarity index 100% rename from 09-ci-02-cicd/infrastructure/files/pg_hba.conf rename to 09-ci-03-cicd/infrastructure/files/pg_hba.conf diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/all.yml b/09-ci-03-cicd/infrastructure/inventory/cicd/group_vars/all.yml similarity index 100% rename from 09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/all.yml rename to 09-ci-03-cicd/infrastructure/inventory/cicd/group_vars/all.yml diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml b/09-ci-03-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml similarity index 100% rename from 09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml rename to 09-ci-03-cicd/infrastructure/inventory/cicd/group_vars/nexus.yml diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/postgres.yml b/09-ci-03-cicd/infrastructure/inventory/cicd/group_vars/postgres.yml similarity index 100% rename from 09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/postgres.yml rename to 09-ci-03-cicd/infrastructure/inventory/cicd/group_vars/postgres.yml diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/sonarqube.yml b/09-ci-03-cicd/infrastructure/inventory/cicd/group_vars/sonarqube.yml similarity index 100% rename from 09-ci-02-cicd/infrastructure/inventory/cicd/group_vars/sonarqube.yml rename to 09-ci-03-cicd/infrastructure/inventory/cicd/group_vars/sonarqube.yml diff --git a/09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml b/09-ci-03-cicd/infrastructure/inventory/cicd/hosts.yml similarity index 100% rename from 09-ci-02-cicd/infrastructure/inventory/cicd/hosts.yml rename to 09-ci-03-cicd/infrastructure/inventory/cicd/hosts.yml diff --git a/09-ci-02-cicd/infrastructure/site.yml b/09-ci-03-cicd/infrastructure/site.yml similarity index 100% rename from 09-ci-02-cicd/infrastructure/site.yml rename to 09-ci-03-cicd/infrastructure/site.yml diff --git a/09-ci-02-cicd/infrastructure/templates/jdk.sh.j2 b/09-ci-03-cicd/infrastructure/templates/jdk.sh.j2 similarity index 100% rename from 09-ci-02-cicd/infrastructure/templates/jdk.sh.j2 rename to 09-ci-03-cicd/infrastructure/templates/jdk.sh.j2 diff --git a/09-ci-02-cicd/infrastructure/templates/nexus.properties.j2 b/09-ci-03-cicd/infrastructure/templates/nexus.properties.j2 similarity index 100% rename from 09-ci-02-cicd/infrastructure/templates/nexus.properties.j2 rename to 09-ci-03-cicd/infrastructure/templates/nexus.properties.j2 diff --git a/09-ci-02-cicd/infrastructure/templates/nexus.systemd.j2 b/09-ci-03-cicd/infrastructure/templates/nexus.systemd.j2 similarity index 100% rename from 09-ci-02-cicd/infrastructure/templates/nexus.systemd.j2 rename to 09-ci-03-cicd/infrastructure/templates/nexus.systemd.j2 diff --git a/09-ci-02-cicd/infrastructure/templates/nexus.vmoptions.j2 b/09-ci-03-cicd/infrastructure/templates/nexus.vmoptions.j2 similarity index 100% rename from 09-ci-02-cicd/infrastructure/templates/nexus.vmoptions.j2 rename to 09-ci-03-cicd/infrastructure/templates/nexus.vmoptions.j2 diff --git a/09-ci-02-cicd/infrastructure/templates/sonar.unit.j2 b/09-ci-03-cicd/infrastructure/templates/sonar.unit.j2 similarity index 100% rename from 09-ci-02-cicd/infrastructure/templates/sonar.unit.j2 rename to 09-ci-03-cicd/infrastructure/templates/sonar.unit.j2 diff --git a/09-ci-02-cicd/infrastructure/templates/wrapper.conf.j2 b/09-ci-03-cicd/infrastructure/templates/wrapper.conf.j2 similarity index 100% rename from 09-ci-02-cicd/infrastructure/templates/wrapper.conf.j2 rename to 09-ci-03-cicd/infrastructure/templates/wrapper.conf.j2 diff --git a/09-ci-02-cicd/mvn/pom.xml b/09-ci-03-cicd/mvn/pom.xml similarity index 100% rename from 09-ci-02-cicd/mvn/pom.xml rename to 09-ci-03-cicd/mvn/pom.xml diff --git a/09-ci-03-jenkins/README.md b/09-ci-04-jenkins/README.md similarity index 100% rename from 09-ci-03-jenkins/README.md rename to 09-ci-04-jenkins/README.md diff --git a/09-ci-03-jenkins/infrastructure/.gitignore b/09-ci-04-jenkins/infrastructure/.gitignore similarity index 100% rename from 09-ci-03-jenkins/infrastructure/.gitignore rename to 09-ci-04-jenkins/infrastructure/.gitignore diff --git a/09-ci-03-jenkins/infrastructure/files/CentOS-Base.repo b/09-ci-04-jenkins/infrastructure/files/CentOS-Base.repo similarity index 100% rename from 09-ci-03-jenkins/infrastructure/files/CentOS-Base.repo rename to 09-ci-04-jenkins/infrastructure/files/CentOS-Base.repo diff --git a/09-ci-03-jenkins/infrastructure/files/pg_hba.conf b/09-ci-04-jenkins/infrastructure/files/pg_hba.conf similarity index 100% rename from 09-ci-03-jenkins/infrastructure/files/pg_hba.conf rename to 09-ci-04-jenkins/infrastructure/files/pg_hba.conf diff --git a/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml b/09-ci-04-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml similarity index 100% rename from 09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml rename to 09-ci-04-jenkins/infrastructure/inventory/cicd/group_vars/jenkins.yml diff --git a/09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml b/09-ci-04-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml similarity index 100% rename from 09-ci-03-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml rename to 09-ci-04-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml diff --git a/09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml b/09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml similarity index 100% rename from 09-ci-03-jenkins/infrastructure/inventory/cicd/hosts.yml rename to 09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml diff --git a/09-ci-03-jenkins/infrastructure/site.yml b/09-ci-04-jenkins/infrastructure/site.yml similarity index 100% rename from 09-ci-03-jenkins/infrastructure/site.yml rename to 09-ci-04-jenkins/infrastructure/site.yml diff --git a/09-ci-03-jenkins/infrastructure/templates/jdk.sh.j2 b/09-ci-04-jenkins/infrastructure/templates/jdk.sh.j2 similarity index 100% rename from 09-ci-03-jenkins/infrastructure/templates/jdk.sh.j2 rename to 09-ci-04-jenkins/infrastructure/templates/jdk.sh.j2 diff --git a/09-ci-03-jenkins/infrastructure/templates/nexus.properties.j2 b/09-ci-04-jenkins/infrastructure/templates/nexus.properties.j2 similarity index 100% rename from 09-ci-03-jenkins/infrastructure/templates/nexus.properties.j2 rename to 09-ci-04-jenkins/infrastructure/templates/nexus.properties.j2 diff --git a/09-ci-03-jenkins/infrastructure/templates/nexus.systemd.j2 b/09-ci-04-jenkins/infrastructure/templates/nexus.systemd.j2 similarity index 100% rename from 09-ci-03-jenkins/infrastructure/templates/nexus.systemd.j2 rename to 09-ci-04-jenkins/infrastructure/templates/nexus.systemd.j2 diff --git a/09-ci-03-jenkins/infrastructure/templates/nexus.vmoptions.j2 b/09-ci-04-jenkins/infrastructure/templates/nexus.vmoptions.j2 similarity index 100% rename from 09-ci-03-jenkins/infrastructure/templates/nexus.vmoptions.j2 rename to 09-ci-04-jenkins/infrastructure/templates/nexus.vmoptions.j2 diff --git a/09-ci-03-jenkins/infrastructure/templates/sonar.unit.j2 b/09-ci-04-jenkins/infrastructure/templates/sonar.unit.j2 similarity index 100% rename from 09-ci-03-jenkins/infrastructure/templates/sonar.unit.j2 rename to 09-ci-04-jenkins/infrastructure/templates/sonar.unit.j2 diff --git a/09-ci-03-jenkins/infrastructure/templates/wrapper.conf.j2 b/09-ci-04-jenkins/infrastructure/templates/wrapper.conf.j2 similarity index 100% rename from 09-ci-03-jenkins/infrastructure/templates/wrapper.conf.j2 rename to 09-ci-04-jenkins/infrastructure/templates/wrapper.conf.j2 diff --git a/09-ci-03-jenkins/pipeline/Jenkinsfile b/09-ci-04-jenkins/pipeline/Jenkinsfile similarity index 100% rename from 09-ci-03-jenkins/pipeline/Jenkinsfile rename to 09-ci-04-jenkins/pipeline/Jenkinsfile diff --git a/09-ci-04-teamcity/README.md b/09-ci-05-teamcity/README.md similarity index 100% rename from 09-ci-04-teamcity/README.md rename to 09-ci-05-teamcity/README.md diff --git a/09-ci-04-teamcity/infrastructure/.gitignore b/09-ci-05-teamcity/infrastructure/.gitignore similarity index 100% rename from 09-ci-04-teamcity/infrastructure/.gitignore rename to 09-ci-05-teamcity/infrastructure/.gitignore diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/all.yml b/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/all.yml similarity index 100% rename from 09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/all.yml rename to 09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/all.yml diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/nexus.yml b/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/nexus.yml similarity index 100% rename from 09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/nexus.yml rename to 09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/nexus.yml diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml b/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml similarity index 100% rename from 09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml rename to 09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml b/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml similarity index 100% rename from 09-ci-04-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml rename to 09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml diff --git a/09-ci-04-teamcity/infrastructure/inventory/cicd/hosts.yml b/09-ci-05-teamcity/infrastructure/inventory/cicd/hosts.yml similarity index 100% rename from 09-ci-04-teamcity/infrastructure/inventory/cicd/hosts.yml rename to 09-ci-05-teamcity/infrastructure/inventory/cicd/hosts.yml diff --git a/09-ci-04-teamcity/infrastructure/site.yml b/09-ci-05-teamcity/infrastructure/site.yml similarity index 100% rename from 09-ci-04-teamcity/infrastructure/site.yml rename to 09-ci-05-teamcity/infrastructure/site.yml diff --git a/09-ci-04-teamcity/infrastructure/templates/nexus.properties.j2 b/09-ci-05-teamcity/infrastructure/templates/nexus.properties.j2 similarity index 100% rename from 09-ci-04-teamcity/infrastructure/templates/nexus.properties.j2 rename to 09-ci-05-teamcity/infrastructure/templates/nexus.properties.j2 diff --git a/09-ci-04-teamcity/infrastructure/templates/nexus.systemd.j2 b/09-ci-05-teamcity/infrastructure/templates/nexus.systemd.j2 similarity index 100% rename from 09-ci-04-teamcity/infrastructure/templates/nexus.systemd.j2 rename to 09-ci-05-teamcity/infrastructure/templates/nexus.systemd.j2 diff --git a/09-ci-04-teamcity/infrastructure/templates/nexus.vmoptions.j2 b/09-ci-05-teamcity/infrastructure/templates/nexus.vmoptions.j2 similarity index 100% rename from 09-ci-04-teamcity/infrastructure/templates/nexus.vmoptions.j2 rename to 09-ci-05-teamcity/infrastructure/templates/nexus.vmoptions.j2 diff --git a/09-ci-04-teamcity/teamcity/docker-compose.yml b/09-ci-05-teamcity/teamcity/docker-compose.yml similarity index 100% rename from 09-ci-04-teamcity/teamcity/docker-compose.yml rename to 09-ci-05-teamcity/teamcity/docker-compose.yml diff --git a/09-ci-05-gitlab/README.md b/09-ci-06-gitlab/README.md similarity index 100% rename from 09-ci-05-gitlab/README.md rename to 09-ci-06-gitlab/README.md diff --git a/09-ci-05-gitlab/repository/python-api.py b/09-ci-06-gitlab/repository/python-api.py similarity index 100% rename from 09-ci-05-gitlab/repository/python-api.py rename to 09-ci-06-gitlab/repository/python-api.py From e1ff1e25341798f1b9427b5436663950e81a7eb8 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Wed, 29 Sep 2021 13:48:47 +0300 Subject: [PATCH 17/40] fix 03 homework --- 09-ci-03-cicd/README.md | 28 ++++++++++------------------ 1 file changed, 10 insertions(+), 18 deletions(-) diff --git a/09-ci-03-cicd/README.md b/09-ci-03-cicd/README.md index 0e522582e..0c22c4d80 100644 --- a/09-ci-03-cicd/README.md +++ b/09-ci-03-cicd/README.md @@ -1,17 +1,18 @@ -# Домашнее задание к занятию "09.02 CI\CD" +# Домашнее задание к занятию "09.03 CI\CD" -## Знакомоство с SonarQube -### Подготовка к выполнению -1. Выполняем `docker pull sonarqube:8.7-community` -2. Выполняем `docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:8.7-community` -3. Ждём запуск, смотрим логи через `docker logs -f sonarqube` -4. Проверяем готовность сервиса через [браузер](http://localhost:9000) -5. Заходим под admin\admin, меняем пароль на свой +## Подготовка к выполнению -В целом, в [этой статье](https://docs.sonarqube.org/latest/setup/install-server/) описаны все варианты установки, включая и docker, но так как нам он нужен разово, то достаточно того набора действий, который я указал выше. +1. Создаём 2 VM в yandex cloud со следующими параметрами: 2CPU 4RAM Centos7(остальное по минимальным требованиям) +2. Прописываем в [inventory](./infrastructure/inventory/cicd/hosts.yml) [playbook'a](./infrastructure/site.yml) созданные хосты. +3. Запускаем playbook, ожидаем успешного завершения +4. Проверяем готовность Sonarqube через [браузер](http://localhost:9000) +5. Заходим под admin\admin, меняем пароль на свой +6. Проверяем готовность Nexus через [бразуер](http://localhost:8081) +7. Подключаемся под admin\admin123, меняем пароль, сохраняем анонимный доступ +## Знакомоство с SonarQube ### Основная часть 1. Создаём новый проект, название произвольное @@ -26,15 +27,6 @@ ## Знакомство с Nexus -### Подготовка к выполнению - -1. Выполняем `docker pull sonatype/nexus3` -2. Выполняем `docker run -d -p 8081:8081 --name nexus sonatype/nexus3` -3. Ждём запуск, смотрим логи через `docker logs -f nexus` -4. Проверяем готовность сервиса через [бразуер](http://localhost:8081) -5. Узнаём пароль от admin через `docker exec -it nexus /bin/bash` -6. Подключаемся под админом, меняем пароль, сохраняем анонимный доступ - ### Основная часть 1. В репозиторий `maven-public` загружаем артефакт с GAV параметрами: From 8458c779dd0559410d76bc4c83810a22b12f3598 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Wed, 29 Sep 2021 16:37:45 +0300 Subject: [PATCH 18/40] fix space --- 09-ci-03-cicd/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/09-ci-03-cicd/README.md b/09-ci-03-cicd/README.md index 0c22c4d80..cab66e16a 100644 --- a/09-ci-03-cicd/README.md +++ b/09-ci-03-cicd/README.md @@ -1,7 +1,5 @@ # Домашнее задание к занятию "09.03 CI\CD" - - ## Подготовка к выполнению 1. Создаём 2 VM в yandex cloud со следующими параметрами: 2CPU 4RAM Centos7(остальное по минимальным требованиям) @@ -13,6 +11,7 @@ 7. Подключаемся под admin\admin123, меняем пароль, сохраняем анонимный доступ ## Знакомоство с SonarQube + ### Основная часть 1. Создаём новый проект, название произвольное From 5c4472eddb9da79a6da4e55d571a60b4007800df Mon Sep 17 00:00:00 2001 From: aragastmatb <56859548+aragastmatb@users.noreply.github.com> Date: Wed, 29 Sep 2021 18:00:02 +0300 Subject: [PATCH 19/40] =?UTF-8?q?=D1=83=D0=B1=D1=80=D0=B0=D0=BB=20=D0=BE?= =?UTF-8?q?=D0=BF=D0=B5=D1=87=D0=B0=D1=82=D0=BA=D1=83?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- 09-ci-02-devops/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/09-ci-02-devops/README.md b/09-ci-02-devops/README.md index 6eef12b69..eecb33de7 100644 --- a/09-ci-02-devops/README.md +++ b/09-ci-02-devops/README.md @@ -11,7 +11,7 @@ 1. С ограниченным сроком действия нужна для временного создания DevOps, после окончания срока - DevOps заканчивается 2. Отдельная DevOps команда способна разрулить любой вопрос Dev или Ops, а значит повышает эффективность работоспособности в долгосрочной перспективе, не нагружая лишними задачами 3. Ответственность команды с ограниченным сроком достаточно низкая, нет необходимости продумывать долгосрочные перспективы развития процессов - 4. Команда с ограниченным сроком действия нужная для внедрения лучших DevOps практик и должна стараться не допустить возникновения антипаттерна + 4. Команда с ограниченным сроком действия нужна для внедрения лучших DevOps практик и должна стараться не допустить возникновения антипаттерна 3. Кто такие DevOps евангелисты: 1. Временные сотрудники, которых руководство использует для создания видимости DevOps 2. Команда сотрудников, которые необходимы для создания благоприятной атмосферы диалога между Dev и Ops, с помощью в виде экспертных мнений и демонстрации использования DevOps практик From ad301e9fa628f41de7d907203660215b3988ebfc Mon Sep 17 00:00:00 2001 From: susanna-susanna <44302089+susanna-susanna@users.noreply.github.com> Date: Wed, 29 Sep 2021 18:15:26 +0300 Subject: [PATCH 20/40] Update README.md --- README.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index a6bf8eb0e..d4c067fd3 100644 --- a/README.md +++ b/README.md @@ -26,13 +26,15 @@ 9.1. [Жизненный цикл ПО](./09-ci-01-intro/README.md) -9.2. [Процессы CI/CD](./09-ci-02-cicd/README.md) +9.2. [DevOps и SRE](./09-ci-02-devops/README.md) -9.3. [Jenkins](./09-ci-03-jenkins/README.md) +9.3. [Процессы CI/CD](./09-ci-02-cicd/README.md) -9.4. [Teamcity](./09-ci-04-teamcity/README.md) +9.4. [Jenkins](./09-ci-03-jenkins/README.md) -9.5. [Gitlab](./09-ci-05-gitlab/README.md) +9.5. [Teamcity](./09-ci-04-teamcity/README.md) + +9.6. [Gitlab](./09-ci-05-gitlab/README.md) ## Модуль 10. Системы мониторинга From 103a59791769182f8669cbd9fc48fe9aa79b6780 Mon Sep 17 00:00:00 2001 From: susanna-susanna <44302089+susanna-susanna@users.noreply.github.com> Date: Wed, 29 Sep 2021 18:19:28 +0300 Subject: [PATCH 21/40] Update README.md --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index d4c067fd3..f4723066b 100644 --- a/README.md +++ b/README.md @@ -28,13 +28,13 @@ 9.2. [DevOps и SRE](./09-ci-02-devops/README.md) -9.3. [Процессы CI/CD](./09-ci-02-cicd/README.md) +9.3. [Процессы CI/CD](./09-ci-03-cicd/README.md) -9.4. [Jenkins](./09-ci-03-jenkins/README.md) +9.4. [Jenkins](./09-ci-04-jenkins/README.md) -9.5. [Teamcity](./09-ci-04-teamcity/README.md) +9.5. [Teamcity](./09-ci-05-teamcity/README.md) -9.6. [Gitlab](./09-ci-05-gitlab/README.md) +9.6. [Gitlab](./09-ci-06-gitlab/README.md) ## Модуль 10. Системы мониторинга From a3a61b12170611733b50f99117bee62e53160430 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Tue, 5 Oct 2021 18:09:36 +0300 Subject: [PATCH 22/40] fix certs --- 09-ci-03-cicd/infrastructure/site.yml | 2 ++ 09-ci-04-jenkins/README.md | 13 +++++-------- .../infrastructure/inventory/cicd/hosts.yml | 4 ++-- 09-ci-04-jenkins/infrastructure/site.yml | 2 ++ 4 files changed, 11 insertions(+), 10 deletions(-) diff --git a/09-ci-03-cicd/infrastructure/site.yml b/09-ci-03-cicd/infrastructure/site.yml index 5744806e8..49d31b3bb 100644 --- a/09-ci-03-cicd/infrastructure/site.yml +++ b/09-ci-03-cicd/infrastructure/site.yml @@ -57,6 +57,7 @@ yum: name: https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm state: present + validate_certs: false - name: Install PostgreSQL yum: @@ -259,6 +260,7 @@ get_url: dest: "{{ nexus_directory_package }}/nexus-{{ nexus_version }}.tar.gz" url: "{{ nexus_download_url }}/nexus-{{ nexus_version }}-unix.tar.gz" + validate_certs: false - name: Unpack Nexus become: true diff --git a/09-ci-04-jenkins/README.md b/09-ci-04-jenkins/README.md index 263e15f50..f44c0a9f9 100644 --- a/09-ci-04-jenkins/README.md +++ b/09-ci-04-jenkins/README.md @@ -2,14 +2,11 @@ ## Подготовка к выполнению -1. Установить jenkins по любой из [инструкций](https://www.jenkins.io/download/) -2. Запустить и проверить работоспособность -3. Сделать первоначальную настройку -4. Настроить под свои нужды -5. Поднять отдельный cloud -6. Для динамических агентов можно использовать [образ](https://hub.docker.com/repository/docker/aragast/agent) -7. Обязательный параметр: поставить label для динамических агентов: `ansible_docker` -8. Сделать форк репозитория с [playbook](https://github.com/aragastmatb/example-playbook) +1. Создать 2 VM: для jenkins-master и jenkins-agent +2. Установить jenkins при помощи playbook'a +3. Запустить и проверить работоспособность +4. Сделать первоначальную настройку +5. Сделать форк репозитория с [playbook](https://github.com/aragastmatb/example-playbook) ## Основная часть diff --git a/09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml b/09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml index 80c0d8196..72cc21486 100644 --- a/09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml +++ b/09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml @@ -2,9 +2,9 @@ all: hosts: jenkins-master-01: - ansible_host: + ansible_host: 130.193.44.107 jenkins-agent-01: - ansible_host: + ansible_host: 130.193.56.174 children: jenkins: children: diff --git a/09-ci-04-jenkins/infrastructure/site.yml b/09-ci-04-jenkins/infrastructure/site.yml index f861aebaa..5eff627ab 100644 --- a/09-ci-04-jenkins/infrastructure/site.yml +++ b/09-ci-04-jenkins/infrastructure/site.yml @@ -27,11 +27,13 @@ get_url: url: https://pkg.jenkins.io/redhat-stable/jenkins.repo dest: /etc/yum.repos.d/jenkins.repo + validate_certs: false - name: Add Jenkins key become: true rpm_key: key: https://pkg.jenkins.io/redhat-stable/jenkins.io.key state: present + validate_certs: false - name: Install epel-release become: true yum: From 9e1493c85aacc93e7ea06a607d7bdaee6219274f Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Wed, 6 Oct 2021 17:15:06 +0300 Subject: [PATCH 23/40] fix readme --- 09-ci-03-cicd/README.md | 18 ++-- .../infrastructure/templates/jdk.sh.j2 | 5 - .../templates/nexus.properties.j2 | 12 --- .../infrastructure/templates/nexus.systemd.j2 | 15 --- .../templates/nexus.vmoptions.j2 | 16 --- .../infrastructure/templates/sonar.unit.j2 | 22 ----- .../infrastructure/templates/wrapper.conf.j2 | 98 ------------------- 7 files changed, 10 insertions(+), 176 deletions(-) delete mode 100644 09-ci-04-jenkins/infrastructure/templates/jdk.sh.j2 delete mode 100644 09-ci-04-jenkins/infrastructure/templates/nexus.properties.j2 delete mode 100644 09-ci-04-jenkins/infrastructure/templates/nexus.systemd.j2 delete mode 100644 09-ci-04-jenkins/infrastructure/templates/nexus.vmoptions.j2 delete mode 100644 09-ci-04-jenkins/infrastructure/templates/sonar.unit.j2 delete mode 100644 09-ci-04-jenkins/infrastructure/templates/wrapper.conf.j2 diff --git a/09-ci-03-cicd/README.md b/09-ci-03-cicd/README.md index cab66e16a..b867b55aa 100644 --- a/09-ci-03-cicd/README.md +++ b/09-ci-03-cicd/README.md @@ -3,12 +3,13 @@ ## Подготовка к выполнению 1. Создаём 2 VM в yandex cloud со следующими параметрами: 2CPU 4RAM Centos7(остальное по минимальным требованиям) -2. Прописываем в [inventory](./infrastructure/inventory/cicd/hosts.yml) [playbook'a](./infrastructure/site.yml) созданные хосты. -3. Запускаем playbook, ожидаем успешного завершения -4. Проверяем готовность Sonarqube через [браузер](http://localhost:9000) -5. Заходим под admin\admin, меняем пароль на свой -6. Проверяем готовность Nexus через [бразуер](http://localhost:8081) -7. Подключаемся под admin\admin123, меняем пароль, сохраняем анонимный доступ +2. Прописываем в [inventory](./infrastructure/inventory/cicd/hosts.yml) [playbook'a](./infrastructure/site.yml) созданные хосты +3. Добавляем в [files](./infrastructure/files/) файл со своим публичным ключом (id_rsa.pub). Если ключ называется иначе - найдите таску в плейбуке, которая использует id_rsa.pub имя и исправьте на своё +4. Запускаем playbook, ожидаем успешного завершения +5. Проверяем готовность Sonarqube через [браузер](http://localhost:9000) +6. Заходим под admin\admin, меняем пароль на свой +7. Проверяем готовность Nexus через [бразуер](http://localhost:8081) +8. Подключаемся под admin\admin123, меняем пароль, сохраняем анонимный доступ ## Знакомоство с SonarQube @@ -44,8 +45,9 @@ 1. Скачиваем дистрибутив с [maven](https://maven.apache.org/download.cgi) 2. Разархивируем, делаем так, чтобы binary был доступен через вызов в shell (или меняем переменную PATH или любой другой удобный вам способ) -3. Проверяем `mvn --version` -4. Забираем директорию [mvn](./mvn) с pom +3. Удаляем из `apache-maven-/conf/settings.xml` упоминание о правиле, отвергающем http соединение( раздел mirrors->id: my-repository-http-unblocker) +4. Проверяем `mvn --version` +5. Забираем директорию [mvn](./mvn) с pom ### Основная часть diff --git a/09-ci-04-jenkins/infrastructure/templates/jdk.sh.j2 b/09-ci-04-jenkins/infrastructure/templates/jdk.sh.j2 deleted file mode 100644 index 39887b6ab..000000000 --- a/09-ci-04-jenkins/infrastructure/templates/jdk.sh.j2 +++ /dev/null @@ -1,5 +0,0 @@ -# Warning: This file is Ansible Managed, manual changes will be overwritten on next playbook run. -#!/usr/bin/env bash - -export JAVA_HOME={{ java_home }} -export PATH=$PATH:$JAVA_HOME/bin \ No newline at end of file diff --git a/09-ci-04-jenkins/infrastructure/templates/nexus.properties.j2 b/09-ci-04-jenkins/infrastructure/templates/nexus.properties.j2 deleted file mode 100644 index aa2645463..000000000 --- a/09-ci-04-jenkins/infrastructure/templates/nexus.properties.j2 +++ /dev/null @@ -1,12 +0,0 @@ -# -# - -# Jetty section -application-host={{ nexus_host }} -application-port={{ nexus_port }} -nexus-context-path={{ nexus_context_path }} - -# Nexus section -nexus-edition={{ nexus_edition }} -nexus-features=\ - {{ nexus_features }} \ No newline at end of file diff --git a/09-ci-04-jenkins/infrastructure/templates/nexus.systemd.j2 b/09-ci-04-jenkins/infrastructure/templates/nexus.systemd.j2 deleted file mode 100644 index 7665d96a8..000000000 --- a/09-ci-04-jenkins/infrastructure/templates/nexus.systemd.j2 +++ /dev/null @@ -1,15 +0,0 @@ -[Unit] -Description=nexus service -After=network.target - -[Service] -Type=forking -User={{ nexus_user_name }} -Group={{ nexus_user_group }} -LimitNOFILE={{ nexus_ulimit }} -ExecStart={{ nexus_directory_home }}/bin/nexus start -ExecStop={{ nexus_directory_home }}/bin/nexus stop -Restart=on-abort - -[Install] -WantedBy=multi-user.target \ No newline at end of file diff --git a/09-ci-04-jenkins/infrastructure/templates/nexus.vmoptions.j2 b/09-ci-04-jenkins/infrastructure/templates/nexus.vmoptions.j2 deleted file mode 100644 index dea2288a7..000000000 --- a/09-ci-04-jenkins/infrastructure/templates/nexus.vmoptions.j2 +++ /dev/null @@ -1,16 +0,0 @@ --Xms{{ nexus_java_heap_size }} --Xmx{{ nexus_java_heap_size }} --XX:MaxDirectMemorySize={{ nexus_java_max_direct_memory }} --XX:+UnlockDiagnosticVMOptions --XX:+UnsyncloadClass --XX:+LogVMOutput --XX:LogFile={{ nexus_directory_log }}/jvm.log --XX:-OmitStackTraceInFastThrow --Djava.net.preferIPv4Stack=true --Dkaraf.home=. --Dkaraf.base=. --Dkaraf.etc=etc/karaf --Djava.util.logging.config.file=etc/karaf/java.util.logging.properties --Dkaraf.data={{ nexus_directory_data }} --Djava.io.tmpdir={{ nexus_directory_tmp }} --Dkaraf.startLocalConsole=false \ No newline at end of file diff --git a/09-ci-04-jenkins/infrastructure/templates/sonar.unit.j2 b/09-ci-04-jenkins/infrastructure/templates/sonar.unit.j2 deleted file mode 100644 index fb48c8067..000000000 --- a/09-ci-04-jenkins/infrastructure/templates/sonar.unit.j2 +++ /dev/null @@ -1,22 +0,0 @@ -[Unit] -Description=SonarQube -After=network.target network-online.target -Wants=network-online.target - -[Service] -LimitNOFILE=131072 -LimitNPROC=8192 -User={{ sonarqube_db_user }} -Group={{ sonarqube_db_user }} -ExecStart=/usr/bin/sonar start -ExecStop=/usr/bin/sonar stop -ExecReload=/usr/bin/sonar restart -PIDFile=/usr/local/sonar/bin/linux-x86-64/./SonarQube.pid -Type=simple -PermissionsStartOnly=true -TimeoutStartSec=5 -Restart=always -SuccessExitStatus=143 - -[Install] -WantedBy=multi-user.target diff --git a/09-ci-04-jenkins/infrastructure/templates/wrapper.conf.j2 b/09-ci-04-jenkins/infrastructure/templates/wrapper.conf.j2 deleted file mode 100644 index bb271c188..000000000 --- a/09-ci-04-jenkins/infrastructure/templates/wrapper.conf.j2 +++ /dev/null @@ -1,98 +0,0 @@ -# Path to JVM executable. By default it must be available in PATH. -# Can be an absolute path, for example: -wrapper.java.command={{ java_home }}/bin/java - - -# -# DO NOT EDIT THE FOLLOWING SECTIONS -# - - -#******************************************************************** -# Wrapper Java -#******************************************************************** -wrapper.java.additional.1=-Dsonar.wrapped=true -wrapper.java.additional.2=-Djava.awt.headless=true -# extra args needed by hazelcast -wrapper.java.additional.3=--add-exports=java.base/jdk.internal.ref=ALL-UNNAMED -wrapper.java.additional.4=--add-opens=java.base/java.lang=ALL-UNNAMED -wrapper.java.additional.5=--add-opens=java.base/java.nio=ALL-UNNAMED -wrapper.java.additional.6=--add-opens=java.base/sun.nio.ch=ALL-UNNAMED -wrapper.java.additional.7=--add-opens=java.management/sun.management=ALL-UNNAMED -wrapper.java.additional.8=--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED - -wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp -wrapper.java.classpath.1=../../lib/sonar-application-9.1.0.47736.jar -wrapper.java.classpath.2=../../lib/jsw/wrapper-3.2.3.jar -wrapper.java.classpath.3=../../lib/sonar-shutdowner-9.1.0.47736.jar -wrapper.java.library.path.1=./lib -wrapper.app.parameter.1=org.sonar.application.App -wrapper.java.initmemory=8 -wrapper.java.maxmemory=32 - -#******************************************************************** -# Wrapper Logs -#******************************************************************** - -wrapper.console.format=PM -wrapper.console.loglevel=INFO -wrapper.logfile.format=M -wrapper.logfile.loglevel=INFO -wrapper.logfile.rollmode=DATE -wrapper.logfile=../../logs/sonar.YYYYMMDD.log - -# Maximum size that the log file will be allowed to grow to before -# the log is rolled. Size is specified in bytes. The default value -# of 0, disables log rolling. May abbreviate with the 'k' (kb) or -# 'm' (mb) suffix. For example: 10m = 10 megabytes. -#wrapper.logfile.maxsize=0 - -# Maximum number of rolled log files which will be allowed before old -# files are deleted. The default value of 0 implies no limit. -wrapper.logfile.maxfiles=7 - -# Log Level for sys/event log output. (See docs for log levels) -wrapper.syslog.loglevel=NONE - -#******************************************************************** -# Wrapper Windows Properties -#******************************************************************** -# Title to use when running as a console -wrapper.console.title=SonarQube - -# Disallow start of multiple instances of an application at the same time on Windows -wrapper.single_invocation=true - -#******************************************************************** -# Wrapper Windows NT/2000/XP Service Properties -#******************************************************************** -# WARNING - Do not modify any of these properties when an application -# using this configuration file has been installed as a service. -# Please uninstall the service before modifying this section. The -# service can then be reinstalled. - -# Name of the service -wrapper.ntservice.name=SonarQube - -# Display name of the service -wrapper.ntservice.displayname=SonarQube - -# Description of the service -wrapper.ntservice.description=SonarQube - -# Service dependencies. Add dependencies as needed starting from 1 -wrapper.ntservice.dependency.1= - -# Mode in which the service is installed. AUTO_START or DEMAND_START -wrapper.ntservice.starttype=AUTO_START - -# Allow the service to interact with the desktop. -wrapper.ntservice.interactive=false - -#******************************************************************** -# Forking Properties -#******************************************************************** -wrapper.disable_restarts=TRUE -wrapper.ping.timeout=0 -wrapper.shutdown.timeout=0 -wrapper.jvm_exit.timeout=0 From 02dfba714578186ede3b9ed066a295c394d13cad Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Thu, 7 Oct 2021 13:25:34 +0300 Subject: [PATCH 24/40] fix homework --- 09-ci-04-jenkins/README.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/09-ci-04-jenkins/README.md b/09-ci-04-jenkins/README.md index f44c0a9f9..f21cf1365 100644 --- a/09-ci-04-jenkins/README.md +++ b/09-ci-04-jenkins/README.md @@ -6,26 +6,22 @@ 2. Установить jenkins при помощи playbook'a 3. Запустить и проверить работоспособность 4. Сделать первоначальную настройку -5. Сделать форк репозитория с [playbook](https://github.com/aragastmatb/example-playbook) ## Основная часть -1. Сделать Freestyle Job, который будет запускать `ansible-playbook` из форка репозитория -2. Сделать Declarative Pipeline, который будет выкачивать репозиторий с плейбукой и запускать её +1. Сделать Freestyle Job, который будет запускать `molecule test` из любого вашего репозитория с ролью +2. Сделать Declarative Pipeline Job, который будет запускать `molecule test` из любого вашего репозитория с ролью 3. Перенести Declarative Pipeline в репозиторий в файл `Jenkinsfile` -4. Перенастроить Job на использование `Jenkinsfile` из репозитория +4. Создать Multibranch Pipeline на запуск `Jenkinsfile` из репозитория 5. Создать Scripted Pipeline, наполнить его скриптом из [pipeline](./pipeline) -6. Заменить credentialsId на свой собственный -7. Проверить работоспособность, исправить ошибки, исправленный Pipeline вложить в репозитрий в файл `ScriptedJenkinsfile` +6. Внести необходимые изменения, чтобы Pipeline запускал `ansible-playbook` без флагов `--check --diff` если не установлен параметр при запуске джобы (prod_run = True), по умолчанию параметр имеет значение False и запускает прогон с флагами `--check --diff` +7. Проверить работоспособность, исправить ошибки, исправленный Pipeline вложить в репозиторий в файл `ScriptedJenkinsfile`. Цель: получить собранный стек ELK в Ya.Cloud. 8. Отправить ссылку на репозиторий в ответе ## Необязательная часть 1. Создать скрипт на groovy, который будет собирать все Job, которые завершились хотя бы раз неуспешно. Добавить скрипт в репозиторий с решеним с названием `AllJobFailure.groovy` -2. Установить customtools plugin -3. Поднять инстанс с локальным nexus, выложить туда в анонимный доступ .tar.gz с `ansible` версии 2.9.x -4. Создать джобу, которая будет использовать `ansible` из `customtool` -5. Джоба должна просто исполнять команду `ansible --version`, в ответ прислать лог исполнения джобы +2. Дополнить Scripted Pipeline таким образом, чтобы он мог сначала запустить через Ya.Cloud CLI необходимое количество инстансов, прописать их в инвентори плейбука и после этого запускать плейбук. Тем самым, мы должны по нажатию кнопки получить готовую к использованию систему. --- From 96b2d189b117ca6bc10a62f104a2c0e062ebf68b Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Thu, 7 Oct 2021 13:25:45 +0300 Subject: [PATCH 25/40] fix homework jenkins --- .../inventory/cicd/group_vars/nexus.yml | 27 ----------- .../infrastructure/inventory/cicd/hosts.yml | 6 +-- 09-ci-04-jenkins/infrastructure/site.yml | 47 +++++++++++++++++-- 09-ci-04-jenkins/pipeline/Jenkinsfile | 6 +-- 4 files changed, 49 insertions(+), 37 deletions(-) delete mode 100644 09-ci-04-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml diff --git a/09-ci-04-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml b/09-ci-04-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml deleted file mode 100644 index 2698ac96c..000000000 --- a/09-ci-04-jenkins/infrastructure/inventory/cicd/group_vars/nexus.yml +++ /dev/null @@ -1,27 +0,0 @@ ---- -java_packages: - - java-1.8.0-openjdk - - java-1.8.0-openjdk-devel -user_group: "{{ nexus_user_group }}" -user_name: "{{ nexus_user_name }}" -nexus_user_group: nexus -nexus_user_name: nexus -nexus_directory_data: "/home/{{ nexus_user_name }}/sonatype-work/nexus3" -nexus_directory_home: "/home/{{ nexus_user_name }}/nexus" -nexus_directory_log: "/home/{{ nexus_user_name }}/log" -nexus_directory_package: "/home/{{ nexus_user_name }}/pkg" -nexus_directory_tmp: "/home/{{ nexus_user_name }}/tmp" -nexus_version: 3.14.0-04 -nexus_download_url: https://download.sonatype.com/nexus/3 -nexus_service_enabled: true -nexus_ulimit: 65536 -nexus_context_path: / -nexus_host: 0.0.0.0 -nexus_port: 8081 -nexus_port_check_timeout: 600 -nexus_edition: nexus-oss-edition -nexus_features: nexus-oss-feature -nexus_java_heap_size: 1200M -nexus_java_max_direct_memory: 2G -nexus_service_start_on_boot: true -nexus_configuration_disk_free_space_limit: ~ \ No newline at end of file diff --git a/09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml b/09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml index 72cc21486..47da9ddbd 100644 --- a/09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml +++ b/09-ci-04-jenkins/infrastructure/inventory/cicd/hosts.yml @@ -2,9 +2,9 @@ all: hosts: jenkins-master-01: - ansible_host: 130.193.44.107 + ansible_host: jenkins-agent-01: - ansible_host: 130.193.56.174 + ansible_host: children: jenkins: children: @@ -16,4 +16,4 @@ all: jenkins-agent-01: vars: ansible_connection_type: paramiko - ansible_user: aragast \ No newline at end of file + ansible_user: \ No newline at end of file diff --git a/09-ci-04-jenkins/infrastructure/site.yml b/09-ci-04-jenkins/infrastructure/site.yml index 5eff627ab..367555b8f 100644 --- a/09-ci-04-jenkins/infrastructure/site.yml +++ b/09-ci-04-jenkins/infrastructure/site.yml @@ -39,10 +39,12 @@ yum: name: epel-release state: present - - name: Install Jenkins + - name: Install Jenkins and requirements become: true yum: - name: jenkins + name: + - jenkins + - git state: present register: jenkins_installed - name: Ensure jenkins agents are present in known_hosts file @@ -78,12 +80,21 @@ state: directory owner: "{{ jenkins_user_name }}" group: "{{ jenkins_user_group }}" + - name: Add docker repo + become_user: root + get_url: + url: https://download.docker.com/linux/centos/docker-ce.repo + dest: /etc/yum.repos.d/docker-ce.repo + validate_certs: false - name: Install some required become_user: root yum: name: - git - python3 + - docker-ce + - docker-ce-cli + - containerd.io state: present - name: Update pip become_user: root @@ -92,13 +103,41 @@ name: pip extra_args: --upgrade - name: Install Ansible + become_user: root pip: executable: pip3 name: - - setuptools_rust - - wheel + - selinux - ansible<3.0 state: present + - name: Reinstall Selinux + become_user: root + pip: + executable: pip3 + name: + - selinux + state: forcereinstall + - name: Add local to PATH + lineinfile: + dest: ~/.bashrc + regexp: export PATH=$PATH:/home/jenkins/.local/bin + line: export PATH=$PATH:/home/jenkins/.local/bin + - name: Create docker group + become_user: root + group: + name: docker + state: present + - name: Add jenkinsuser to dockergroup + become_user: root + user: + name: "{{ jenkins_user_name }}" + groups: docker + - name: Restart docker + become_user: root + systemd: + name: docker + state: restarted + enabled: true - name: Install agent.jar get_url: url: "http://{{ hostvars['jenkins-master-01']['ansible_host'] }}:8080/jnlpJars/agent.jar" diff --git a/09-ci-04-jenkins/pipeline/Jenkinsfile b/09-ci-04-jenkins/pipeline/Jenkinsfile index 89bfd4121..1b12b6e96 100644 --- a/09-ci-04-jenkins/pipeline/Jenkinsfile +++ b/09-ci-04-jenkins/pipeline/Jenkinsfile @@ -1,8 +1,8 @@ -node("ansible_docker"){ +node("linux"){ stage("Git checkout"){ git credentialsId: '5ac0095d-0185-431b-94da-09a0ad9b0e2c', url: 'git@github.com:aragastmatb/example-playbook.git' } - stage("Check ssh key"){ + stage("Sample define secret_check"){ secret_check=true } stage("Run playbook"){ @@ -10,7 +10,7 @@ node("ansible_docker"){ sh 'ansible-playbook site.yml -i inventory/prod.yml' } else{ - echo 'no more keys' + echo 'need more action' } } From 9e5114acd5da3cd6f4251c03cc5543b6f8cbe579 Mon Sep 17 00:00:00 2001 From: ElenaPalagina <77622076+ElenaPalagina@users.noreply.github.com> Date: Thu, 7 Oct 2021 18:17:47 +0300 Subject: [PATCH 26/40] Update README.md --- 09-ci-04-jenkins/README.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/09-ci-04-jenkins/README.md b/09-ci-04-jenkins/README.md index f21cf1365..5b465c878 100644 --- a/09-ci-04-jenkins/README.md +++ b/09-ci-04-jenkins/README.md @@ -1,26 +1,26 @@ -# Домашнее задание к занятию "09.03 Jenkins" +# Домашнее задание к занятию "09.04 Jenkins" ## Подготовка к выполнению -1. Создать 2 VM: для jenkins-master и jenkins-agent -2. Установить jenkins при помощи playbook'a -3. Запустить и проверить работоспособность -4. Сделать первоначальную настройку +1. Создать 2 VM: для jenkins-master и jenkins-agent. +2. Установить jenkins при помощи playbook'a. +3. Запустить и проверить работоспособность. +4. Сделать первоначальную настройку. ## Основная часть -1. Сделать Freestyle Job, который будет запускать `molecule test` из любого вашего репозитория с ролью -2. Сделать Declarative Pipeline Job, который будет запускать `molecule test` из любого вашего репозитория с ролью -3. Перенести Declarative Pipeline в репозиторий в файл `Jenkinsfile` -4. Создать Multibranch Pipeline на запуск `Jenkinsfile` из репозитория -5. Создать Scripted Pipeline, наполнить его скриптом из [pipeline](./pipeline) -6. Внести необходимые изменения, чтобы Pipeline запускал `ansible-playbook` без флагов `--check --diff` если не установлен параметр при запуске джобы (prod_run = True), по умолчанию параметр имеет значение False и запускает прогон с флагами `--check --diff` +1. Сделать Freestyle Job, который будет запускать `molecule test` из любого вашего репозитория с ролью. +2. Сделать Declarative Pipeline Job, который будет запускать `molecule test` из любого вашего репозитория с ролью. +3. Перенести Declarative Pipeline в репозиторий в файл `Jenkinsfile`. +4. Создать Multibranch Pipeline на запуск `Jenkinsfile` из репозитория. +5. Создать Scripted Pipeline, наполнить его скриптом из [pipeline](./pipeline). +6. Внести необходимые изменения, чтобы Pipeline запускал `ansible-playbook` без флагов `--check --diff`, если не установлен параметр при запуске джобы (prod_run = True), по умолчанию параметр имеет значение False и запускает прогон с флагами `--check --diff`. 7. Проверить работоспособность, исправить ошибки, исправленный Pipeline вложить в репозиторий в файл `ScriptedJenkinsfile`. Цель: получить собранный стек ELK в Ya.Cloud. -8. Отправить ссылку на репозиторий в ответе +8. Отправить ссылку на репозиторий в ответе. ## Необязательная часть -1. Создать скрипт на groovy, который будет собирать все Job, которые завершились хотя бы раз неуспешно. Добавить скрипт в репозиторий с решеним с названием `AllJobFailure.groovy` +1. Создать скрипт на groovy, который будет собирать все Job, которые завершились хотя бы раз неуспешно. Добавить скрипт в репозиторий с решеним с названием `AllJobFailure.groovy`. 2. Дополнить Scripted Pipeline таким образом, чтобы он мог сначала запустить через Ya.Cloud CLI необходимое количество инстансов, прописать их в инвентори плейбука и после этого запускать плейбук. Тем самым, мы должны по нажатию кнопки получить готовую к использованию систему. --- From c778e21903ade78c41fe964c4072d5727b19002c Mon Sep 17 00:00:00 2001 From: ElenaPalagina <77622076+ElenaPalagina@users.noreply.github.com> Date: Thu, 7 Oct 2021 18:18:46 +0300 Subject: [PATCH 27/40] Update README.md --- 09-ci-05-teamcity/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/09-ci-05-teamcity/README.md b/09-ci-05-teamcity/README.md index 18e07d92f..42b8690e5 100644 --- a/09-ci-05-teamcity/README.md +++ b/09-ci-05-teamcity/README.md @@ -1,4 +1,4 @@ -# Домашнее задание к занятию "09.04 Teamcity" +# Домашнее задание к занятию "09.05 Teamcity" ## Подготовка к выполнению From edb39dfac94fb3885d202a77e31201ba56b676b6 Mon Sep 17 00:00:00 2001 From: ElenaPalagina <77622076+ElenaPalagina@users.noreply.github.com> Date: Mon, 11 Oct 2021 14:45:57 +0300 Subject: [PATCH 28/40] Update README.md --- 09-ci-05-teamcity/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/09-ci-05-teamcity/README.md b/09-ci-05-teamcity/README.md index 42b8690e5..bb97973fb 100644 --- a/09-ci-05-teamcity/README.md +++ b/09-ci-05-teamcity/README.md @@ -3,7 +3,7 @@ ## Подготовка к выполнению 1. Поднимите инфраструктуру [teamcity](./teamcity/docker-compose.yml) -2. Если хочется, можете создать свою собственную инфраструктуру на основе той технологии, которая нравится. Инструкция по установке из [документации](https://www.jetbrains.com/help/teamcity/installing-and-configuring-the-teamcity-server.html) +2. При желании можете создать свою собственную инфраструктуру на основе той технологии, которая нравится. Инструкция по установке из [документации](https://www.jetbrains.com/help/teamcity/installing-and-configuring-the-teamcity-server.html) 3. Дождитесь запуска teamcity, выполните первоначальную настройку 4. Авторизуйте агент 5. Сделайте fork [репозитория](https://github.com/aragastmatb/example-teamcity) @@ -23,7 +23,7 @@ 11. Внесите изменения из произвольной ветки `feature/add_reply` в `master` через `Merge` 12. Убедитесь, что нет собранного артефакта в сборке по ветке `master` 13. Настройте конфигурацию так, чтобы она собирала `.jar` в артефакты сборки -14. Проведите повторную сборку мастера, убедитесь, что сбора прошла успешно и артефакты собраны +14. Проведите повторную сборку мастера, убедитесь, что сборка прошла успешно и артефакты собраны 15. Проверьте, что конфигурация в репозитории содержит все настройки конфигурации из teamcity 16. В ответ предоставьте ссылку на репозиторий From 788b5a2e2d347bb70a1cb23d3ed26079273430ab Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Tue, 12 Oct 2021 13:53:05 +0300 Subject: [PATCH 29/40] fix 05 homework --- 09-ci-05-teamcity/README.md | 35 +-- .../inventory/cicd/group_vars/all.yml | 8 - .../inventory/cicd/group_vars/postgres.yml | 1 - .../inventory/cicd/group_vars/sonarqube.yml | 5 - .../infrastructure/inventory/cicd/hosts.yml | 12 +- 09-ci-05-teamcity/infrastructure/site.yml | 222 ------------------ 6 files changed, 21 insertions(+), 262 deletions(-) delete mode 100644 09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/all.yml delete mode 100644 09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml delete mode 100644 09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml diff --git a/09-ci-05-teamcity/README.md b/09-ci-05-teamcity/README.md index 18e07d92f..a8f7815be 100644 --- a/09-ci-05-teamcity/README.md +++ b/09-ci-05-teamcity/README.md @@ -2,9 +2,9 @@ ## Подготовка к выполнению -1. Поднимите инфраструктуру [teamcity](./teamcity/docker-compose.yml) -2. Если хочется, можете создать свою собственную инфраструктуру на основе той технологии, которая нравится. Инструкция по установке из [документации](https://www.jetbrains.com/help/teamcity/installing-and-configuring-the-teamcity-server.html) -3. Дождитесь запуска teamcity, выполните первоначальную настройку +1. В Ya.Cloud создайте новый инстанс (4CPU4RAM) на основе образа `jetbrains/teamcity-server` +2. Дождитесь запуска teamcity, выполните первоначальную настройку +3. Создайте ещё один инстанс(2CPU4RAM) на основе образа `jetbrains/teamcity-agent`. Пропишите к нему переменную окружения `SERVER_URL: "http://:8111"` 4. Авторизуйте агент 5. Сделайте fork [репозитория](https://github.com/aragastmatb/example-teamcity) @@ -13,19 +13,22 @@ 1. Создайте новый проект в teamcity на основе fork 2. Сделайте autodetect конфигурации 3. Сохраните необходимые шаги, запустите первую сборку master'a -4. Поменяйте условия сборки: если сборка по ветке `master`, то должен происходит `mvn clean package`, иначе `mvn clean test` -5. Мигрируйте `build configuration` в репозиторий -6. Создайте отдельную ветку `feature/add_reply` в репозитории -7. Напишите новый метод для класса Welcomer: метод должен возвращать произвольную реплику, содержащую слово `hunter` -8. Дополните тест для нового метода на поиск слова `hunter` в новой реплике -9. Сделайте push всех изменений в новую ветку в репозиторий -10. Убедитесь что сборка самостоятельно запустилась, тесты прошли успешно -11. Внесите изменения из произвольной ветки `feature/add_reply` в `master` через `Merge` -12. Убедитесь, что нет собранного артефакта в сборке по ветке `master` -13. Настройте конфигурацию так, чтобы она собирала `.jar` в артефакты сборки -14. Проведите повторную сборку мастера, убедитесь, что сбора прошла успешно и артефакты собраны -15. Проверьте, что конфигурация в репозитории содержит все настройки конфигурации из teamcity -16. В ответ предоставьте ссылку на репозиторий +4. Поменяйте условия сборки: если сборка по ветке `master`, то должен происходит `mvn clean package deploy`, иначе `mvn clean test` +5. Для deploy будет необходимо загрузить settings.xml в набор конфигураций maven у teamcity, предварительно записав туда креды для подключения к nexus +6. В pom.xml необходимо поменять ссылки на репозиторий и nexus +7. Запустите сборку по master, убедитесь что всё прошло успешно, артефакт появился в nexus +8. Мигрируйте `build configuration` в репозиторий +9. Создайте отдельную ветку `feature/add_reply` в репозитории +10. Напишите новый метод для класса Welcomer: метод должен возвращать произвольную реплику, содержащую слово `hunter` +11. Дополните тест для нового метода на поиск слова `hunter` в новой реплике +12. Сделайте push всех изменений в новую ветку в репозиторий +13. Убедитесь что сборка самостоятельно запустилась, тесты прошли успешно +14. Внесите изменения из произвольной ветки `feature/add_reply` в `master` через `Merge` +15. Убедитесь, что нет собранного артефакта в сборке по ветке `master` +16. Настройте конфигурацию так, чтобы она собирала `.jar` в артефакты сборки +17. Проведите повторную сборку мастера, убедитесь, что сбора прошла успешно и артефакты собраны +18. Проверьте, что конфигурация в репозитории содержит все настройки конфигурации из teamcity +19. В ответ предоставьте ссылку на репозиторий --- diff --git a/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/all.yml b/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/all.yml deleted file mode 100644 index f6afeefb1..000000000 --- a/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/all.yml +++ /dev/null @@ -1,8 +0,0 @@ ---- -jdk_url: https://download.java.net/openjdk/jdk11/ri/openjdk-11+28_linux-x64_bin.tar.gz -jdk_distr_name: openjdk-11+28_linux-x64_bin.tar.gz -jdk_folder: "{{ jdk_distr_name.split('-')[:2] | join('-') }}" -java_home: "/opt/jdk/{{ jdk_folder }}" -sonarqube_db_name: sonar -sonarqube_db_user: sonar -sonarqube_db_password: sonar \ No newline at end of file diff --git a/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml b/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml deleted file mode 100644 index e0499e7e9..000000000 --- a/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/postgres.yml +++ /dev/null @@ -1 +0,0 @@ -postgresql_version: 11 \ No newline at end of file diff --git a/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml b/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml deleted file mode 100644 index 89b9116c4..000000000 --- a/09-ci-05-teamcity/infrastructure/inventory/cicd/group_vars/sonarqube.yml +++ /dev/null @@ -1,5 +0,0 @@ -sonar_version: 9.1.0.47736 -sonar_download_url: "https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-{{ sonar_version }}.zip" -sonar_version_directory: "sonarqube-{{ sonar_version }}" -sonar_db_port: "5432" -sonar_web_context: '' \ No newline at end of file diff --git a/09-ci-05-teamcity/infrastructure/inventory/cicd/hosts.yml b/09-ci-05-teamcity/infrastructure/inventory/cicd/hosts.yml index ef7633b5a..8926822f0 100644 --- a/09-ci-05-teamcity/infrastructure/inventory/cicd/hosts.yml +++ b/09-ci-05-teamcity/infrastructure/inventory/cicd/hosts.yml @@ -1,20 +1,12 @@ --- all: hosts: - sonar-01: - ansible_host: nexus-01: - ansible_host: + ansible_host: children: - sonarqube: - hosts: - sonar-01: nexus: hosts: nexus-01: - postgres: - hosts: - sonar-01: vars: ansible_connection_type: paramiko - ansible_user: \ No newline at end of file + ansible_user: \ No newline at end of file diff --git a/09-ci-05-teamcity/infrastructure/site.yml b/09-ci-05-teamcity/infrastructure/site.yml index 5744806e8..4a7aab0bf 100644 --- a/09-ci-05-teamcity/infrastructure/site.yml +++ b/09-ci-05-teamcity/infrastructure/site.yml @@ -1,226 +1,4 @@ --- -- name: Get OpenJDK installed - hosts: sonarqube - pre_tasks: - - name: install unzip - become: true - yum: - name: unzip - state: present - tasks: - - name: Upload .tar.gz file conaining binaries from remote storage - get_url: - url: "{{ jdk_url }}" - dest: "/tmp/jdk-{{ jdk_distr_name }}" - mode: 0755 - register: download_java_remote_binaries - until: download_java_remote_binaries is succeeded - - - name: Ensure installation dir exists - become: true - file: - state: directory - path: "{{ java_home }}" - mode: 0755 - - - name: Extract java in the installation directory - become: true - unarchive: - copy: false - src: "/tmp/jdk-{{ jdk_distr_name }}" - dest: "{{ java_home }}" - extra_opts: [--strip-components=1] - creates: "{{ java_home }}/bin/java" - - - name: Export environment variables - become: true - template: - src: jdk.sh.j2 - dest: /etc/profile.d/jdk.sh - owner: root - group: root - mode: 0644 - -- name: Get PostgreSQL installed - hosts: postgres - become: true - tasks: - - name: Change repo file - copy: - src: CentOS-Base.repo - dest: /etc/yum.repos.d/CentOS-Base.repo - mode: 0644 - owner: root - group: root - - - name: Install PostgreSQL repos - yum: - name: https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm - state: present - - - name: Install PostgreSQL - yum: - name: "postgresql{{ postgresql_version }}-server" - state: present - - - name: Init template1 DB - command: /usr/pgsql-11/bin/postgresql-11-setup initdb - failed_when: false - - - name: Start pgsql service - systemd: - name: "postgresql-{{ postgresql_version }}" - state: started - enabled: true - - - name: Create user in system - user: - name: "{{ sonarqube_db_user }}" - - name: Create user for Sonar in PostgreSQL - become_user: postgres - command: "createuser -s -e {{ sonarqube_db_user }}" - failed_when: false - - - name: Change password for Sonar user in PostgreSQL - become_user: postgres - command: "psql -c \"ALTER USER sonar WITH ENCRYPTED password '{{ sonarqube_db_password }}';\"" - - - name: Create Sonar DB - become_user: postgres - command: "createdb {{ sonarqube_db_name }}" - failed_when: false - - - name: Copy pg_hba.conf - copy: - src: pg_hba.conf - dest: /var/lib/pgsql/11/data/pg_hba.conf - mode: 0600 - owner: postgres - group: postgres - -- name: Prepare Sonar host - hosts: sonarqube - become: true - tasks: - - name: Create group in system - group: - name: "{{ sonarqube_db_user }}" - state: present - - - name: Create user in system - user: - name: "{{ sonarqube_db_user }}" - group: "{{ sonarqube_db_user }}" - - - name: "Set up ssh key to access for managed node" - authorized_key: - user: "{{ sonarqube_db_user }}" - state: present - key: "{{ lookup('file', 'id_rsa.pub') }}" - - - name: "Allow group to have passwordless sudo" - lineinfile: - dest: /etc/sudoers - state: present - regexp: '^%{{ sonarqube_db_user }}' - line: '%{{ sonarqube_db_user }} ALL=(ALL) NOPASSWD: ALL' - validate: 'visudo -cf %s' - - - name: Increase Virtual Memory - lineinfile: - dest: /etc/sysctl.conf - state: present - regexp: '^vm.max_map_count' - line: 'vm.max_map_count=262144' - - name: Reboot VM - reboot: - -- name: Get Sonarqube installed - hosts: sonarqube - vars: - ansible_user: "{{ sonarqube_db_user }}" - tasks: - - name: Get distrib ZIP - get_url: - url: "{{ sonar_download_url }}" - dest: "/tmp/{{ sonar_version_directory }}.zip" - validate_certs: false - - - name: Unzip Sonar - become: true - unarchive: - src: "/tmp/{{ sonar_version_directory }}.zip" - dest: /usr/local/ - copy: false - owner: "{{ sonarqube_db_user }}" - group: "{{ sonarqube_db_user }}" - creates: /usr/local/sonar/COPYING - - - name: Move Sonar into place. - become: true - copy: - src: /usr/local/{{ sonar_version_directory }}/ - dest: /usr/local/sonar/ - owner: "{{ sonarqube_db_user }}" - group: "{{ sonarqube_db_user }}" - remote_src: true - - - name: Configure SonarQube JDBC settings for PostgreSQL. - lineinfile: - dest: /usr/local/sonar/conf/sonar.properties - regexp: "{{ item.regexp }}" - line: "{{ item.line }}" - with_items: - - regexp: "^sonar.jdbc.username" - line: "sonar.jdbc.username={{ sonarqube_db_user }}" - - regexp: "^sonar.jdbc.password" - line: "sonar.jdbc.password={{ sonarqube_db_password }}" - - regexp: "^sonar.jdbc.url" - line: "sonar.jdbc.url=jdbc:postgresql://localhost:{{ sonar_db_port }}/{{ sonarqube_db_name }}?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance" - - regexp: "^sonar.web.context" - line: "sonar.web.context={{ sonar_web_context }}" - - - name: Generate wrapper.conf - template: - src: wrapper.conf.j2 - dest: /usr/local/sonar/conf/wrapper.conf - mode: 0644 - - - name: Symlink sonar bin. - become: true - file: - src: /usr/local/sonar/bin/linux-x86-64/sonar.sh - dest: /usr/bin/sonar - state: link - register: sonar_symlink - - - name: Copy SonarQube systemd unit file into place (for systemd systems). - become: true - template: - src: sonar.unit.j2 - dest: /etc/systemd/system/sonar.service - owner: root - group: root - mode: 0755 - - - name: Ensure Sonar is running and set to start on boot. - become: true - service: - name: sonar - state: restarted - enabled: true - - - name: Allow Sonar time to build on first start. - pause: - seconds: 180 - when: sonar_symlink.changed - tags: ['skip_ansible_lint'] - - - name: Make sure Sonar is responding on the configured port. - wait_for: - port: 9000 - delay: 3 - timeout: 300 - name: Get Nexus installed hosts: nexus pre_tasks: From ce42f7237bca4963a3dacab69b1814eda2f560e9 Mon Sep 17 00:00:00 2001 From: Alexey Metlyakov Date: Tue, 12 Oct 2021 17:43:17 +0300 Subject: [PATCH 30/40] add settings.xml --- 09-ci-05-teamcity/README.md | 2 +- 09-ci-05-teamcity/teamcity/settings.xml | 261 ++++++++++++++++++++++++ 2 files changed, 262 insertions(+), 1 deletion(-) create mode 100644 09-ci-05-teamcity/teamcity/settings.xml diff --git a/09-ci-05-teamcity/README.md b/09-ci-05-teamcity/README.md index f11d911f9..f122e9f6f 100644 --- a/09-ci-05-teamcity/README.md +++ b/09-ci-05-teamcity/README.md @@ -14,7 +14,7 @@ 2. Сделайте autodetect конфигурации 3. Сохраните необходимые шаги, запустите первую сборку master'a 4. Поменяйте условия сборки: если сборка по ветке `master`, то должен происходит `mvn clean package deploy`, иначе `mvn clean test` -5. Для deploy будет необходимо загрузить settings.xml в набор конфигураций maven у teamcity, предварительно записав туда креды для подключения к nexus +5. Для deploy будет необходимо загрузить [settings.xml](./teamcity/settings.xml) в набор конфигураций maven у teamcity, предварительно записав туда креды для подключения к nexus 6. В pom.xml необходимо поменять ссылки на репозиторий и nexus 7. Запустите сборку по master, убедитесь что всё прошло успешно, артефакт появился в nexus 8. Мигрируйте `build configuration` в репозиторий diff --git a/09-ci-05-teamcity/teamcity/settings.xml b/09-ci-05-teamcity/teamcity/settings.xml new file mode 100644 index 000000000..7a49a5a81 --- /dev/null +++ b/09-ci-05-teamcity/teamcity/settings.xml @@ -0,0 +1,261 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + nexus + admin + admin123 + + + + + + + + + + + + + + + + + + From cf15a4ae8d77631a7a90bc60fda1108dd71afe80 Mon Sep 17 00:00:00 2001 From: ElenaPalagina <77622076+ElenaPalagina@users.noreply.github.com> Date: Thu, 14 Oct 2021 12:01:10 +0300 Subject: [PATCH 31/40] Update README.md --- 09-ci-06-gitlab/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/09-ci-06-gitlab/README.md b/09-ci-06-gitlab/README.md index 942c8924b..3fb680b22 100644 --- a/09-ci-06-gitlab/README.md +++ b/09-ci-06-gitlab/README.md @@ -1,4 +1,4 @@ -# Домашнее задание к занятию "09.05 Gitlab" +# Домашнее задание к занятию "09.06 Gitlab" ## Подготовка к выполнению From 4df2f165641e3682c13c6c5ed5ed8ee278ab1014 Mon Sep 17 00:00:00 2001 From: aragastmatb <56859548+aragastmatb@users.noreply.github.com> Date: Thu, 14 Oct 2021 17:51:49 +0300 Subject: [PATCH 32/40] Update python-api.py --- 09-ci-06-gitlab/repository/python-api.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/09-ci-06-gitlab/repository/python-api.py b/09-ci-06-gitlab/repository/python-api.py index 47c9c18f7..a2d01111f 100644 --- a/09-ci-06-gitlab/repository/python-api.py +++ b/09-ci-06-gitlab/repository/python-api.py @@ -8,9 +8,9 @@ class Info(Resource): def get(self): - return {'version': 3, 'method': 'GET', 'message': 'Already started'} # Fetches first column that is Employee ID + return {'version': 3, 'method': 'GET', 'message': 'Already started'} -api.add_resource(Info, '/get_info') # Route_1 +api.add_resource(Info, '/get_info') if __name__ == '__main__': - app.run(host='0.0.0.0', port='5290') \ No newline at end of file + app.run(host='0.0.0.0', port='5290') From fee76defa283bcf61c0f300f911f825309581f41 Mon Sep 17 00:00:00 2001 From: ElenaPalagina <77622076+ElenaPalagina@users.noreply.github.com> Date: Fri, 29 Oct 2021 12:31:59 +0300 Subject: [PATCH 33/40] Update README.md --- 09-ci-02-devops/README.md | 33 +-------------------------------- 1 file changed, 1 insertion(+), 32 deletions(-) diff --git a/09-ci-02-devops/README.md b/09-ci-02-devops/README.md index eecb33de7..e82e539b1 100644 --- a/09-ci-02-devops/README.md +++ b/09-ci-02-devops/README.md @@ -1,35 +1,4 @@ # Домашнее задание к занятию "09.02 DevOps и SRE" -## Основная часть +Домашнее задание находится у вас в личном кабинете в виде теста. -1. В чём разница между структурой "Сотрудничество dev и ops" и "NoOps" - 1. В NoOps-командах не нужно сопровождение - 2. Сотрудничество dev и ops полностью разделяет две команды на отдельные - 3. NoOps распределяет все задачи ops на обе команды - 4. В NoOps dev не занимается разработкой -2. В чём различие DevOps команды с ограниченным сроком действия и антипаттерна Отдельной DevOps команды - 1. С ограниченным сроком действия нужна для временного создания DevOps, после окончания срока - DevOps заканчивается - 2. Отдельная DevOps команда способна разрулить любой вопрос Dev или Ops, а значит повышает эффективность работоспособности в долгосрочной перспективе, не нагружая лишними задачами - 3. Ответственность команды с ограниченным сроком достаточно низкая, нет необходимости продумывать долгосрочные перспективы развития процессов - 4. Команда с ограниченным сроком действия нужна для внедрения лучших DevOps практик и должна стараться не допустить возникновения антипаттерна -3. Кто такие DevOps евангелисты: - 1. Временные сотрудники, которых руководство использует для создания видимости DevOps - 2. Команда сотрудников, которые необходимы для создания благоприятной атмосферы диалога между Dev и Ops, с помощью в виде экспертных мнений и демонстрации использования DevOps практик - 3. Люди, которые приходят на собрания групп разработки и сопровождения. Их задача - наблюдать со стороны за внедрением DevOps процесса, не вмешиваясь в его построение -4. Кто такие SRE: - 1. Инженеры, которые имеют высокий уровень квалификации как в разработке, так и в сопровождении. Чаще всего вырастают из опытных разработчиков, которых отправили (по методологии SRE) исследовать работу Ops - 2. Отдельная каста сотрудников, которых набирают извне для урегулирования вопросов между Dev и Ops - 3. Архитекторы проектов, которые имеют право вето на любые изменения с любой стороны -5. Как правильно внедрять DevOps: - 1. Использовать определённый тип команды для всей индустрии - 2. Использовать определённый тип команды для всей компании - 3. Использовать определённый тип команды для конкретной команды - 4. Не существует стратегии правильного внедрения, методология выросла из постулатов гибких методологий, в каждом конкретном случае необходимо исследовать команду и пробовать внедрять разные подходы, пока не будет выбран наиболее эфективный - ---- - -### Как оформить ДЗ? - -Выполненное домашнее задание пришлите ссылкой на .md-файл в вашем репозитории. - ---- From 927f11491c24b0acabdeac6eaa1e56e57c8d6355 Mon Sep 17 00:00:00 2001 From: aragastmatb <56859548+aragastmatb@users.noreply.github.com> Date: Mon, 29 Nov 2021 22:07:47 +0300 Subject: [PATCH 34/40] Update README.md --- 08-ansible-05-testing/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/08-ansible-05-testing/README.md b/08-ansible-05-testing/README.md index f019a4668..bddd1424f 100644 --- a/08-ansible-05-testing/README.md +++ b/08-ansible-05-testing/README.md @@ -19,7 +19,7 @@ ### Tox -1. Запустите `docker run -it -v :/opt/elasticsearch-role -w /opt/elasticsearch-role /bin/bash`, где path_to_repo - путь до корня репозитория с elasticsearch-role на вашей файловой системе. +1. Запустите `docker run --privileged=True -v :/opt/elasticsearch-role -w /opt/elasticsearch-role -it /bin/bash`, где path_to_repo - путь до корня репозитория с elasticsearch-role на вашей файловой системе. 2. Внутри контейнера выполните команду `tox`, посмотрите на вывод. 3. Добавьте файл `tox.ini` в корень репозитория каждой своей роли. 4. Создайте облегчённый сценарий для `molecule`. Проверьте его на исполнимость. From db25d306d73d30753d5bd58a3af2e1e8e6ea6917 Mon Sep 17 00:00:00 2001 From: aragastmatb <56859548+aragastmatb@users.noreply.github.com> Date: Mon, 20 Dec 2021 22:06:20 +0300 Subject: [PATCH 35/40] Update README.md --- 09-ci-04-jenkins/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/09-ci-04-jenkins/README.md b/09-ci-04-jenkins/README.md index 5b465c878..c35d78923 100644 --- a/09-ci-04-jenkins/README.md +++ b/09-ci-04-jenkins/README.md @@ -16,7 +16,7 @@ 5. Создать Scripted Pipeline, наполнить его скриптом из [pipeline](./pipeline). 6. Внести необходимые изменения, чтобы Pipeline запускал `ansible-playbook` без флагов `--check --diff`, если не установлен параметр при запуске джобы (prod_run = True), по умолчанию параметр имеет значение False и запускает прогон с флагами `--check --diff`. 7. Проверить работоспособность, исправить ошибки, исправленный Pipeline вложить в репозиторий в файл `ScriptedJenkinsfile`. Цель: получить собранный стек ELK в Ya.Cloud. -8. Отправить ссылку на репозиторий в ответе. +8. Отправить две ссылки на репозитории в ответе: с ролью и Declarative Pipeline и c плейбукой и Scripted Pipeline. ## Необязательная часть From db9e425c9807133245fc3ce70c96d05af0db650d Mon Sep 17 00:00:00 2001 From: aragastmatb <56859548+aragastmatb@users.noreply.github.com> Date: Thu, 23 Dec 2021 18:23:37 +0300 Subject: [PATCH 36/40] Update README.md --- 09-ci-05-teamcity/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/09-ci-05-teamcity/README.md b/09-ci-05-teamcity/README.md index f122e9f6f..ff931a577 100644 --- a/09-ci-05-teamcity/README.md +++ b/09-ci-05-teamcity/README.md @@ -13,7 +13,7 @@ 1. Создайте новый проект в teamcity на основе fork 2. Сделайте autodetect конфигурации 3. Сохраните необходимые шаги, запустите первую сборку master'a -4. Поменяйте условия сборки: если сборка по ветке `master`, то должен происходит `mvn clean package deploy`, иначе `mvn clean test` +4. Поменяйте условия сборки: если сборка по ветке `master`, то должен происходит `mvn clean deploy`, иначе `mvn clean test` 5. Для deploy будет необходимо загрузить [settings.xml](./teamcity/settings.xml) в набор конфигураций maven у teamcity, предварительно записав туда креды для подключения к nexus 6. В pom.xml необходимо поменять ссылки на репозиторий и nexus 7. Запустите сборку по master, убедитесь что всё прошло успешно, артефакт появился в nexus From 55bad7543b6373edebde1d8e335e875cad997ee5 Mon Sep 17 00:00:00 2001 From: aragastmatb <56859548+aragastmatb@users.noreply.github.com> Date: Mon, 14 Feb 2022 21:03:14 +0300 Subject: [PATCH 37/40] Update README.md --- 08-ansible-04-role/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/08-ansible-04-role/README.md b/08-ansible-04-role/README.md index adea2ca83..df6a03c8c 100644 --- a/08-ansible-04-role/README.md +++ b/08-ansible-04-role/README.md @@ -13,7 +13,7 @@ --- - src: git@github.com:netology-code/mnt-homeworks-ansible.git scm: git - version: "2.0.0" + version: "2.1.4" name: elastic ``` 2. При помощи `ansible-galaxy` скачать себе эту роль. From ad8c31d4e5b8641f09ff6881b800f8a9d51b79c2 Mon Sep 17 00:00:00 2001 From: aragastmatb <56859548+aragastmatb@users.noreply.github.com> Date: Thu, 17 Feb 2022 21:10:37 +0300 Subject: [PATCH 38/40] Update README.md --- 08-ansible-05-testing/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/08-ansible-05-testing/README.md b/08-ansible-05-testing/README.md index bddd1424f..de4bd43d2 100644 --- a/08-ansible-05-testing/README.md +++ b/08-ansible-05-testing/README.md @@ -6,7 +6,7 @@ ## Основная часть -Наша основная цель - настроить тестирование наших ролей. Задача: сделать сценарии тестирования для kibana, logstash. Ожидаемый результат: все сценарии успешно проходят тестирование ролей. +Наша основная цель - настроить тестирование наших ролей. Задача: сделать сценарии тестирования для kibana, filebeat. Ожидаемый результат: все сценарии успешно проходят тестирование ролей. ### Molecule From aa2bace7ca91d206053b28065e4a603d24a0bb6b Mon Sep 17 00:00:00 2001 From: aragastmatb <56859548+aragastmatb@users.noreply.github.com> Date: Mon, 28 Feb 2022 22:27:16 +0300 Subject: [PATCH 39/40] Update README.md --- 09-ci-01-intro/README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/09-ci-01-intro/README.md b/09-ci-01-intro/README.md index 64d4c4a0f..3e07430e6 100644 --- a/09-ci-01-intro/README.md +++ b/09-ci-01-intro/README.md @@ -8,19 +8,19 @@ ## Основная часть В рамках основной части необходимо создать собственные workflow для двух типов задач: bug и остальные типы задач. Задачи типа bug должны проходить следующий жизненный цикл: 1. Open -> On reproduce -2. On reproduce <-> Open, Done reproduce +2. On reproduce -> Open, Done reproduce 3. Done reproduce -> On fix -4. On fix <-> On reproduce, Done fix +4. On fix -> On reproduce, Done fix 5. Done fix -> On test -6. On test <-> On fix, Done -7. Done <-> Closed, Open +6. On test -> On fix, Done +7. Done -> Closed, Open Остальные задачи должны проходить по упрощённому workflow: 1. Open -> On develop -2. On develop <-> Open, Done develop +2. On develop -> Open, Done develop 3. Done develop -> On test -4. On test <-> On develop, Done -5. Done <-> Closed, Open +4. On test -> On develop, Done +5. Done -> Closed, Open Создать задачу с типом bug, попытаться провести его по всему workflow до Done. Создать задачу с типом epic, к ней привязать несколько задач с типом task, провести их по всему workflow до Done. При проведении обеих задач по статусам использовать kanban. Вернуть задачи в статус Open. Перейти в scrum, запланировать новый спринт, состоящий из задач эпика и одного бага, стартовать спринт, провести задачи до состояния Closed. Закрыть спринт. From 5fc6523367d68c84536466c90080fc1bedbbfdcb Mon Sep 17 00:00:00 2001 From: kocmoc1 <79825256+kocmoc1@users.noreply.github.com> Date: Thu, 3 Mar 2022 10:25:41 +0500 Subject: [PATCH 40/40] Update Dockerfile --- 08-ansible-05-testing/Dockerfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/08-ansible-05-testing/Dockerfile b/08-ansible-05-testing/Dockerfile index 8b632ae58..0e0f5d3af 100644 --- a/08-ansible-05-testing/Dockerfile +++ b/08-ansible-05-testing/Dockerfile @@ -1,4 +1,4 @@ -FROM registry.redhat.io/rhel8/podman:latest +FROM registry.redhat.io/ubi8/podman:8.5-15 ENV MOLECULE_NO_LOG false RUN yum reinstall glibc-common -y @@ -13,4 +13,4 @@ RUN tar xf Python-3.8.8.tgz && cd Python-3.8.8/ && ./configure && make && make a ADD https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz Python-3.9.2.tgz RUN tar xf Python-3.9.2.tgz && cd Python-3.9.2/ && ./configure && make && make altinstall RUN python3 -m pip install --upgrade pip && pip3 install tox selinux -RUN rm -rf Python-* \ No newline at end of file +RUN rm -rf Python-*