Splunk Operator: Timeout in unit tests

[ductrung-nguyen]

Please select the type of request

Enhancement

Tell us more

Describe the request
Some unit tests are not implemented properly when using real HTTP requests.
That causes timeout sometimes.
If the tests are passed, they take quite long time to finish.

Imagine the case that you have K8s clusters, but you have to use proxy to access those clusters
---> whenever you run kubectl, or access the cluster from golang code, you will get timeout.

For example:
With the fresh clone version of the operator:

=== RUN   TestPhaseManagersMsgChannels
--- PASS: TestPhaseManagersMsgChannels (76.32s)
=== RUN   TestRunPodCopyWorker
--- PASS: TestRunPodCopyWorker (150.09s)
=== RUN   TestPodCopyWorkerHandler
--- PASS: TestPodCopyWorkerHandler (754.87s)
=== RUN   TestInstallWorkerHandler
--- PASS: TestInstallWorkerHandler (151.03s)

=== RUN   TestAfwYieldWatcher
--- PASS: TestAfwYieldWatcher (10.01s)


=== RUN   TestApplyClusterManager
--- PASS: TestApplyClusterManager (10.02s)
=== RUN   TestApplyClusterManagerWithSmartstore
--- PASS: TestApplyClusterManagerWithSmartstore (90.02s)
=== RUN   TestClusterManagerWitReadyState
--- PASS: TestClusterManagerWitReadyState (75.76s)
=== RUN   TestApplyClusterMasterWithSmartstore
--- PASS: TestApplyClusterMasterWithSmartstore (90.10s)
=== RUN   TestClusterMasterWitReadyState
--- PASS: TestClusterMasterWitReadyState (92.17s)
=== RUN   TestIndexerClusterWithReadyState
--- PASS: TestIndexerClusterWithReadyState (78.07s)
=== RUN   TestLicenseMasterWithReadyState
--- PASS: TestLicenseMasterWithReadyState (86.71s)

PASS
coverage: 97.7% of statements
composite coverage: 86.0% of statements

Ginkgo ran 6 suites in 36m48.729625s
Test Suite Passed

Expected behavior
The tests can finish faster, and in more reliable way.

Splunk setup on K8S
N/A

Reproduction/Testing steps
Just run make test,

K8s environment
No local K8s cluster (kind/k3s...), only use remote K8s clusters under proxy.

Proposed changes(optional)
We should use mocked responses, or mocked object to avoid real HTTP requests, or real I/O (if possible).

In my opninion, the good practice should be:

• In unit tests: No HTTP requests, everything is mocked
• In integration tests: can have real HTTP requests, to mock/dummy servers
• In E2E tests: can have real HTTP requests to real (testing) servers

The integration tests and E2E tests can be merged together.

K8s collector data(optional)

Additional context(optional)

• Add any other context about the problem here.