fix(hadoop): Fix the JMX exporter configuration#962
Merged
siegfriedweber merged 2 commits intomainfrom Dec 20, 2024
Merged
Conversation
This was referenced Dec 19, 2024
Member
|
If we copy these files in the final stage we can reuse the layer cache better. WDYT? diff --git a/hadoop/Dockerfile b/hadoop/Dockerfile
index 0b68186..acb4e09 100644
--- a/hadoop/Dockerfile
+++ b/hadoop/Dockerfile
@@ -11,10 +11,7 @@ ARG TARGETARCH
ARG TARGETOS
ARG STACKABLE_USER_UID
-WORKDIR /stackable
-
-COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/jmx /stackable/jmx
-COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/fuse_dfs_wrapper /stackable/fuse_dfs_wrapper
+WORKDIR /stackable/jmx
# The symlink from JMX Exporter 0.16.1 to the versionless link exists because old HDFS Operators (up until and including 23.7) used to hardcode
# the version of JMX Exporter like this: "-javaagent:/stackable/jmx/jmx_prometheus_javaagent-0.16.1.jar"
@@ -27,6 +24,8 @@ RUN curl "https://repo.stackable.tech/repository/packages/jmx-exporter/jmx_prome
ln -s "/stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar" /stackable/jmx/jmx_prometheus_javaagent.jar && \
ln -s /stackable/jmx/jmx_prometheus_javaagent.jar /stackable/jmx/jmx_prometheus_javaagent-0.16.1.jar
+WORKDIR /stackable
+
RUN ARCH="${TARGETARCH/amd64/x64}" && \
curl "https://repo.stackable.tech/repository/packages/async-profiler/async-profiler-${ASYNC_PROFILER}-${TARGETOS}-${ARCH}.tar.gz" | tar -xzC . && \
ln -s "/stackable/async-profiler-${ASYNC_PROFILER}-${TARGETOS}-${ARCH}" /stackable/async-profiler
@@ -141,6 +140,7 @@ COPY --chown=${STACKABLE_USER_UID}:0 --from=hadoop-builder /stackable/jmx /stack
COPY --chown=${STACKABLE_USER_UID}:0 --from=hadoop-builder /stackable/async-profiler /stackable/async-profiler/
COPY --chown=${STACKABLE_USER_UID}:0 --from=hdfs-utils-builder /stackable/hadoop-${PRODUCT}/share/hadoop/common/lib/hdfs-utils-${HDFS_UTILS}.jar /stackable/hadoop-${PRODUCT}/share/hadoop/common/lib/hdfs-utils-${HDFS_UTILS}.jar
COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/fuse_dfs_wrapper /stackable/
+COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/jmx /stackable/jmx
# fuse is required for fusermount (called by fuse_dfs) |
razvan
previously approved these changes
Dec 19, 2024
Member
razvan
left a comment
There was a problem hiding this comment.
lgtm. tested with the patch in comments.
… builder image to the final image
razvan
approved these changes
Dec 20, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
hadoop: Fix the JMX exporter configuration for metrics suffixed with
_total,_infoand_created.The type of the metrics suffixed with
_totalis changed from GAUGE to COUNTER.The names of the metrics suffixed with
_infoand_createdare now appended with an additional_.Fixes stackabletech/hdfs-operator#634
Definition of Done Checklist
TIP: Running integration tests with a new product image
The image can be built and uploaded to the kind cluster with the following commands:
See the output of
baketo retrieve the image tag for<image-tagged-with-the-major-version>.