Added ability to create schemas only when using cloudstack-setup-data… #5187

Spaceman1984 · 2021-07-06T13:20:31Z

…bases

Description

This PR adds the ability to run the cloudstack-setup-databases script without having to know the root user credentials.

The cloud and cloud_usage databases and cloud user can be manually configured by a different admin prior to running the script.

It will allow the script to be called like this:
./cloudstack-setup-databases user:password@host:port -s
or
./cloudstack-setup-databases user:password@host:port --schema-only

Using this flag will instruct to skip the SQL files responsible for creating the databases, user and granting permissions to the user. It will still run all other SQL scripts that need to be run before starting the management server for the first time. Initial databases and user creation will be the responsibility of the administrator.

Doc PR here: apache/cloudstack-documentation#229

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

This has been tested by creating the cloud and cloud_usage databases and cloud user prior to executing the script, and then executing the script like this:

./cloudstack-setup-databases user:password@host:port -s
or
./cloudstack-setup-databases user:password@host:port --schema-only

An extra option is listed from:
./cloudstack-setup-databases -h

  -s, --schema-only     Creates the db schema without having to pass root
                        credentials - Please note: The databases (cloud,
                        cloud_usage) and user (cloud) has to be configured
                        manuallyprior to running this script when using this
                        flag.

…bases

Spaceman1984 · 2021-07-07T10:17:15Z

@blueorangutan package

blueorangutan · 2021-07-07T10:18:02Z

@Spaceman1984 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-07-07T10:42:59Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 487

Spaceman1984 · 2021-07-07T10:44:52Z

@blueorangutan test

blueorangutan · 2021-07-07T10:46:03Z

@Spaceman1984 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2021-07-07T23:18:37Z

Trillian test result (tid-1215)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43476 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5187-t1215-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 87 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	530.12	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	502.25	test_vpc_redundant.py

setup/bindir/cloud-setup-databases.in

Spaceman1984 · 2021-07-09T08:53:35Z

@blueorangutan package

blueorangutan · 2021-07-09T08:54:03Z

@Spaceman1984 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-07-09T09:19:14Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 506

sureshanaparti

changes LGTM

sureshanaparti · 2021-07-09T12:23:16Z

@blueorangutan package

blueorangutan · 2021-07-09T12:24:03Z

@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-07-09T12:49:26Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 509

sureshanaparti · 2021-07-09T12:51:32Z

@blueorangutan test

blueorangutan · 2021-07-09T12:52:03Z

@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2021-07-10T00:30:05Z

Trillian test result (tid-1226)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40205 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5187-t1226-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 87 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_02_deploy_and_upgrade_kubernetes_cluster	`Failure`	552.13	test_kubernetes_clusters.py

borisstoyanov · 2021-07-29T07:20:43Z

@blueorangutan package

blueorangutan · 2021-07-29T07:21:02Z

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-07-29T07:50:12Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 680

borisstoyanov · 2021-07-29T08:02:38Z

@blueorangutan test

blueorangutan · 2021-07-29T08:03:03Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2021-07-30T00:05:45Z

Trillian test result (tid-1404)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 55784 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5187-t1404-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_primary_storage.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 84 look OK, 5 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_primary_storage_disabled_host	`Error`	0.55	test_primary_storage.py
test_01_primary_storage_nfs	`Error`	0.10	test_primary_storage.py
ContextSuite context=TestStorageTags>:setup	`Error`	0.18	test_primary_storage.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80	`Failure`	521.22	test_internal_lb.py
test_01_invalid_upgrade_kubernetes_cluster	`Failure`	3606.28	test_kubernetes_clusters.py
test_02_deploy_and_upgrade_kubernetes_cluster	`Failure`	3614.50	test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster	`Failure`	0.05	test_kubernetes_clusters.py
test_04_basic_lifecycle_kubernetes_cluster	`Failure`	0.05	test_kubernetes_clusters.py
test_05_delete_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	`Failure`	0.05	test_kubernetes_clusters.py
test_08_deploy_and_upgrade_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	89.41	test_kubernetes_clusters.py
test_02_list_snapshots_with_removed_data_store	`Error`	1.17	test_snapshots.py
test_01_secure_vm_migration	`Error`	158.48	test_vm_life_cycle.py
test_02_unsecure_vm_migration	`Error`	271.83	test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration	`Error`	144.80	test_vm_life_cycle.py
test_08_migrate_vm	`Error`	43.66	test_vm_life_cycle.py
test_10_attachAndDetach_iso	`Failure`	1510.42	test_vm_life_cycle.py

blueorangutan · 2021-08-02T22:00:32Z

Trillian test result (tid-1448)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46832 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5187-t1448-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 87 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	490.01	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	528.37	test_vpc_redundant.py
test_disable_oobm_ha_state_ineligible	`Error`	1512.83	test_hostha_kvm.py

borisstoyanov

LGTM

alexandremattioli · 2021-08-10T12:26:40Z

LGTM

Added ability to create schemas only when using cloudstack-setup-data…

7c5bafb

…bases

Spaceman1984 requested review from harikrishna-patnala and sureshanaparti July 6, 2021 13:47

Spaceman1984 added the component:database label Jul 7, 2021

Spaceman1984 added type:enhancement python3 labels Jul 7, 2021

harikrishna-patnala reviewed Jul 8, 2021

View reviewed changes

setup/bindir/cloud-setup-databases.in Outdated Show resolved Hide resolved

setup/bindir/cloud-setup-databases.in Outdated Show resolved Hide resolved

sureshanaparti reviewed Jul 8, 2021

View reviewed changes

setup/bindir/cloud-setup-databases.in Outdated Show resolved Hide resolved

sureshanaparti reviewed Jul 8, 2021

View reviewed changes

setup/bindir/cloud-setup-databases.in Show resolved Hide resolved

Spaceman1984 added 2 commits July 8, 2021 09:29

Renamed var name

ceaf226

Added a check for passing --schema-only and --deploy-as together/.

5ee88a9

Spaceman1984 closed this Jul 9, 2021

Spaceman1984 reopened this Jul 9, 2021

sureshanaparti reviewed Jul 9, 2021

View reviewed changes

setup/bindir/cloud-setup-databases.in Outdated Show resolved Hide resolved

Moved validation to appropriate method

5ba7206

Moved description

badee0f

sureshanaparti approved these changes Jul 9, 2021

View reviewed changes

fixed whitespace

9cd8c97

Spaceman1984 closed this Jul 12, 2021

Spaceman1984 reopened this Jul 12, 2021

Spaceman1984 marked this pull request as draft July 13, 2021 08:53

sureshanaparti added this to the 4.16.0.0 milestone Jul 16, 2021

borisstoyanov approved these changes Aug 3, 2021

View reviewed changes

yadvr marked this pull request as ready for review August 26, 2021 09:55

yadvr merged commit 041b8f6 into apache:main Aug 26, 2021

nvazquez added the type:new-feature label Oct 14, 2021

Added ability to create schemas only when using cloudstack-setup-data… #5187

Added ability to create schemas only when using cloudstack-setup-data… #5187

Uh oh!

Conversation

Spaceman1984 commented Jul 6, 2021 • edited by sureshanaparti Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

Spaceman1984 commented Jul 7, 2021

Uh oh!

blueorangutan commented Jul 7, 2021

Uh oh!

blueorangutan commented Jul 7, 2021

Uh oh!

Spaceman1984 commented Jul 7, 2021

Uh oh!

blueorangutan commented Jul 7, 2021

Uh oh!

blueorangutan commented Jul 7, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Spaceman1984 commented Jul 9, 2021

Uh oh!

blueorangutan commented Jul 9, 2021

Uh oh!

blueorangutan commented Jul 9, 2021

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

sureshanaparti commented Jul 9, 2021

Uh oh!

blueorangutan commented Jul 9, 2021

Uh oh!

blueorangutan commented Jul 9, 2021

Uh oh!

sureshanaparti commented Jul 9, 2021

Uh oh!

blueorangutan commented Jul 9, 2021

Uh oh!

blueorangutan commented Jul 10, 2021

Uh oh!

borisstoyanov commented Jul 29, 2021

Uh oh!

blueorangutan commented Jul 29, 2021

Uh oh!

blueorangutan commented Jul 29, 2021

Uh oh!

borisstoyanov commented Jul 29, 2021

Uh oh!

blueorangutan commented Jul 29, 2021

Uh oh!

blueorangutan commented Jul 30, 2021

Uh oh!

blueorangutan commented Aug 2, 2021

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

alexandremattioli commented Aug 10, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Spaceman1984 commented Jul 6, 2021 •

edited by sureshanaparti

Loading