Skip to content

Feature/DIMS refactor GenerateViolinPlots #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ALuesink wants to merge 17 commits into
base: develop
Choose a base branch
from
Open

Feature/DIMS refactor GenerateViolinPlots #82

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
e8d0a7f
Refactored code GenerateViolinPlots
ALuesink Aug 7, 2025
9ffd606
Fixed errors
ALuesink Aug 15, 2025
aede6ee
Fixed linting
ALuesink Aug 21, 2025
8094aed
Added new package for unit testing
ALuesink Aug 21, 2025
62160b2
Added unit tests and associated files for GenerateViolinPlots
ALuesink Aug 21, 2025
cf9f349
Fixed snapshot issues GenerateViolinPlots
ALuesink Aug 21, 2025
bd128dd
Changes to snapshots
ALuesink Aug 21, 2025
624d64a
Fixed snapshot issues, second try
ALuesink Aug 21, 2025
9a56d10
Fixing snapshot issue, third try
ALuesink Aug 21, 2025
936f292
Fixed snapshot issue, fourth try
ALuesink Aug 21, 2025
a02016a
Merge branch 'develop' into feature/DIMS_refactor_GenerateViolinPlots
ALuesink Sep 5, 2025
926ad6d
Fixed issue if P1001 is present but no Z-scores
ALuesink Sep 8, 2025
b1a3858
print statement for testing
ALuesink Sep 8, 2025
ba47db5
Removed duplicated line & print statement
ALuesink Sep 11, 2025
4079ecc
Merge branch 'origin/develop' into feature/DIMS_refactor_GenerateViol…
ALuesink Sep 22, 2025
7439b26
Merge remote-tracking branch 'origin/develop' into feature/DIMS_refac…
ALuesink Oct 27, 2025
3875aaf
Fix for error dIEM plots
ALuesink Oct 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/dims_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
uses: actions/checkout@v4

- name: Install dependencies
run: Rscript -e "install.packages(c('testthat', 'withr'))"
run: Rscript -e "install.packages(c('testthat', 'withr', 'vdiffr', 'pdftools'))"

- name: Run tests
run: Rscript tests/testthat.R
1 change: 1 addition & 0 deletions DIMS/GenerateQCOutput.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,7 @@ if (length(sst_colnrs) > 0) {
} else {
sst_list_intensities <- sst_list[, intensity_col_ids]
}
sst_list_intensities <- as.data.frame(sst_list_intensities)
Copy link
Member

@rernst rernst Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole if/else/for block could use some descriptive comments.

for (col_nr in seq_len(ncol(sst_list_intensities))) {
sst_list_intensities[, col_nr] <- as.numeric(sst_list_intensities[, col_nr])
if (grepl("Zscore", colnames(sst_list_intensities)[col_nr])) {
Expand Down
649 changes: 192 additions & 457 deletions DIMS/GenerateViolinPlots.R
View file
Open in desktop
Copy link
Member

@rernst rernst Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function is still too large, in the sense that it still contains a lot of data manipulation and calculations which you ideally want to put in a function and test.

Large diffs are not rendered by default.

5 changes: 2 additions & 3 deletions DIMS/GenerateViolinPlots.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,10 @@ process GenerateViolinPlots {

script:
"""
Rscript ${baseDir}/CustomModules/DIMS/GenerateViolinPlots.R $analysis_id $params.scripts_dir $params.zscore \
Rscript ${baseDir}/CustomModules/DIMS/GenerateViolinPlots.R $analysis_id $params.export_scripts_dir \
$params.path_metabolite_groups \
$params.file_ratios_metabolites \
$params.file_expected_biomarkers_IEM \
$params.file_explanation \
$params.file_isomers
$params.file_explanation
"""
}
571 changes: 571 additions & 0 deletions DIMS/export/generate_violin_plots_functions.R
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions DIMS/tests/testthat.R
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Run all unit tests
library(testthat)
library(withr)
library(vdiffr)
library(pdftools)
# enable snapshots
local_edition(3)

Expand Down
21 changes: 21 additions & 0 deletions DIMS/tests/testthat/_snaps/generate_violin_plots.md
View file
Open in desktop
Copy link
Member

@rernst rernst Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my understanding, what does this file do? Or is it a description?

Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Create a pdf with a table of top metabolites and violin plots

Code
content_pdf_violinplots
Output
[1] "Top deviating metabolites for patient: P2025M1\n Metabolite Z.score\n Increased\n metab1 2.45\n Decreased\n metab11 −1.51\n"
[2] " Results for patient P2025M1\n test acyl carnitines\n metab1 Z=2.34\nMetabolites\n metab3 Z=0.31\n −5 0 5 10 15 20\n Z−scores\n"
[3] " Results for patient P2025M1\n test crea gua\n metab4 Z=−0.46\nMetabolites\n metab11 Z=0.84\n −5 0 5 10 15 20\n Z−scores\n"
[4] " Unit test Generate Violin Plots\nUnit test Generate Violin Plots\n"

# Saving the probability score dataframe as an Excel file

Disease P2025M1 P2025M2 P2025M3 P2025M4
1 Disease A 10.900 -10.9 49.90 -49.9
2 Disease B 0.953 0.0 2.29 0.0
3 Disease C 12.100 0.0 0.00 12.1
4 Disease D 0.000 -12.5 0.00 18.2
5 Disease E 44.300 0.0 0.00 28.1
6 Disease F 0.000 -77.4 -77.40 0.0
7 Disease G -38.700 38.7 38.70 -38.7

74 changes: 74 additions & 0 deletions DIMS/tests/testthat/_snaps/generate_violin_plots/violin-plot-p2025m1.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
290 changes: 290 additions & 0 deletions DIMS/tests/testthat/fixtures/make_test_data_GenerateViolinPlots.R
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,290 @@
### Functions used to create mock dataframes used for unit testing of GenerateViolinPlots ###

make_intensities_zscore_df <- function() {
test_intensities_zscore_df <- data.frame(
HMDB_code = c("HMDB001", "HMDB002", "HMDB003", "HMDB004", "HMDB011", "HMDB012"),
HMDB_name = c("metab1", "metab2", "metab3", "metab4", "metab5", "metab6"),
C101.1 = c(1000, 1200, 1300, 1400, 1500, 1600),
C102.1 = c(1100, 1700, 925, 1125, 1200, 1050),
C103.1 = c(1300, 750, 1000, 1220, 1100, 1200),
C104.1 = c(1650, 925, 1600, 1650, 1025, 1150),
C105.1 = c(180000, 1950, 750, 15050, 1100, 1300),
P2025M1 = c(1000, 1200, 1300, 1400, 1100, 975),
P2025M2 = c(1100, 1700, 925, 1125, 1050, 1175),
P2025M3 = c(1300, 750, 1000, 1220, 975, 1100),
P2025M4 = c(1650, 925, 1600, 1650, 1700, 1750),
P2025M5 = c(180000, 1950, 750, 15050, 10000, 1500),
mean_control = c(37010, 1305, 1115, 4089, 1185, 1260),
sd_control = c(79934.23, 508.80, 336.15, 6130.65, 186.75, 210.36),
C101.1_Zscore = c(0.45, 1.67, -1.86, 0.58, 2.47, -0.56),
C102.1_Zscore = c(2.89, 0.79, -1.88, 5.46, -0.68, 1.65),
C103.1_Zscore = c(0.54, -0.85, 1.58, 3.84, 0.84, -1.11),
C104.1_Zscore = c(0.53, 1.84, 0.35, -0.54, 1.48, 0.43),
C105.1_Zscore = c(3.46, -1.31, 0.14, -0.15, 1.48, 0.36),
P2025M1_Zscore = c(0.31, 1.84, 2.34, 0.84, -0.46, 0.14),
P2025M2_Zscore = c(2.45, 0.48, 1.45, -0.15, -1.51, 3.56),
P2025M3_Zscore = c(2.14, 0.15, -1.44, -0.78, 1.68, 0.51),
P2025M4_Zscore = c(12.18, 2.48, -0.18, 0.84, 1.48, -2.45),
P2025M5_Zscore = c(3.22, 0.48, -3.18, 0.47, 1.18, 2.14)
)
rownames(test_intensities_zscore_df) <- c("HMDB001", "HMDB002", "HMDB003", "HMDB004", "HMDB011", "HMDB012")

write.table(test_intensities_zscore_df, file = "tests/testthat/fixtures/test_intensities_zscore_df.txt", sep = "\t")
}

make_files_metabolite_groups <- function() {
# Create new directories
dir.create("tests/testthat/fixtures/test_metabolite_groups")

test_acyl_carnitines <- data.frame(
HMDB_code = c("HMDB001", "HMDB003", "HMDBAA1"),
HMDB_name = c("metab1", "metab3", "ratio1"),
Helix = c("ja", "nee", "ja"),
Helix_naam = c("Metab_1", "Metab_3", "Ratio_1"),
high_zscore = c(2, 2, 2),
low_zscore = c(-1.5, -1.5, -1.5)
)

write.table(test_acyl_carnitines, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_metabolite_groups/test_acyl_carnitines.txt")

test_crea_gua <- data.frame(
HMDB_code = c("HMDB004", "HMDB011"),
HMDB_name = c("metab4", "metab11"),
Helix = c("ja", "ja"),
Helix_naam = c("Metab_4", "Metab_11"),
high_zscore = c(2, 2),
low_zscore = c(-1.5, -1.5)
)

write.table(test_crea_gua, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_metabolite_groups/test_crea_gua.txt")
}

make_test_metab_interest_sort <- function() {
test_acyl_carnitines_patients <- data.frame(
HMDB_name = c(
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 "
Comment on lines +67 to +76
Copy link
Member

@rernst rernst Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this formatting correct?

),
Sample = c("P2025M1", "P2025M1", "P2025M2", "P2025M2", "P2025M3",
"P2025M3", "P2025M4", "P2025M4", "P2025M5", "P2025M5"),
Z_score = c(0.31, 2.34, 2.45, 1.45, 2.14, -1.44, 12.18, -0.18, 3.22, -3.18)
)

test_acyl_carnitines_controls <- data.frame(
HMDB_name = c(
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 "
),
Sample = c("C101.1", "C101.1", "C102.1", "C102.1", "C103.1", "C103.1", "C104.1", "C104.1", "C105.1", "C105.1"),
Z_score = c(0.45, -1.86, 2.89, -1.88, 0.54, 1.58, 0.53, 0.35, 3.46, 0.14)
)

write.table(test_acyl_carnitines_patients, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_acyl_carnitines_patients.txt")
write.table(test_acyl_carnitines_controls, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_acyl_carnitines_controls.txt")

test_crea_gua_patients <- data.frame(
HMDB_name = c(
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 "
),
Sample = c("P2025M1", "P2025M1", "P2025M2", "P2025M2", "P2025M3",
"P2025M3", "P2025M4", "P2025M4", "P2025M5", "P2025M5"),
Z_score = c(0.84, -0.46, -0.15, -1.51, -0.78, 1.68, 0.84, 1.48, 0.47, 1.18)
)

test_crea_gua_controls <- data.frame(
HMDB_name = c(
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 "
),
Sample = c("C101.1", "C101.1", "C102.1", "C102.1", "C103.1", "C103.1", "C104.1", "C104.1", "C105.1", "C105.1"),
Z_score = c(0.58, 2.47, 5.46, -0.68, 3.84, 0.84, -0.54, 1.48, -0.15, 1.48)
)

write.table(test_crea_gua_patients, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_crea_gua_patients.txt")
write.table(test_crea_gua_controls, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_crea_gua_controls.txt")
}

make_test_df_metabs_helix <- function() {
test_df_metabs_helix <- data.frame(
HMDB_name = c("metab1", "metab1", "metab1", "metab1", "metab1", "metab4", "metab11", "metab4",
"metab11", "metab4", "metab11", "metab4", "metab11", "metab4", "metab11"),
Sample = c("P2025M1", "P2025M2", "P2025M3", "P2025M4", "P2025M5", "P2025M1", "P2025M1", "P2025M2", "P2025M2", "P2025M3",
"P2025M3", "P2025M4", "P2025M4", "P2025M5", "P2025M5"),
Z_score = c(0.31, 2.45, 2.14, 12.18, 3.22, 0.84, -0.46, -0.15, -1.51, -0.78, 1.68, 0.84, 1.48, 0.47, 1.18),
Helix_naam = c("Metab_1", "Metab_1", "Metab_1", "Metab_1", "Metab_1", "Metab_4", "Metab_11",
"Metab_4", "Metab_11", "Metab_4", "Metab_11", "Metab_4", "Metab_11", "Metab_4", "Metab_11"),
high_zscore = rep(2, 15),
low_zscore = rep(-1.5, 15)
)

write.table(test_df_metabs_helix, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_df_metabs_helix.txt")
}

make_test_zscore_patient_df <- function() {
test_zscore_patient_df <- data.frame(
HMDB_code = c("HMDB001", "HMDB002", "HMDB003", "HMDB004", "HMDB005", "HMDB006", "HMDB007", "HMDB008", "HMDB009", "HMDB010",
"HMDB011", "HMDB012", "HMDB013", "HMDB014", "HMDB015", "HMDB016", "HMDB017", "HMDB018", "HMDB019", "HMDB020",
"HMDB021", "HMDB022", "HMDB023", "HMDB024", "HMDB025", "HMDB026", "HMDB027", "HMDB028", "HMDB029",
"HMDB030"),
HMDB_name = c("metab1", "metab2", "metab3", "metab4", "metab5", "metab6", "metab7", "metab8", "metab9", "metab10",
"metab11", "metab12", "metab13", "metab14", "metab15", "metab16", "metab17", "metab18", "metab19", "metab20",
"metab21", "metab22", "metab23", "metab24", "metab25", "metab26", "metab27", "metab28", "metab29",
"metab30"),
P2025M1 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30),
P2025M2 = c(-1, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12, -13, -14, -15,
-16, -17, -18, -19, -20, -21, -22, -23, -24, -25, -26, -27, -28, -29, -30),
P2025M3 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
-16, -17, -18, -19, -20, -21, -22, -23, -24, -25, -26, -27, -28, -29, -30),
P2025M4 = c(-1, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12, -13, -14, -15,
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30)
)

write.table(test_zscore_patient_df, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_zscore_patient_df.txt")
}

make_test_metab_perpage <- function() {
test_acyl_carnitines_df <- data.frame(
HMDB_name = c(
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 ",
"metab1 ",
"metab3 "
),
Sample = c("P2025M1", "P2025M1", "P2025M2", "P2025M2", "P2025M3",
"P2025M3", "P2025M4", "P2025M4", "P2025M5", "P2025M5",
"C101.1", "C101.1", "C102.1", "C102.1", "C103.1", "C103.1",
"C104.1", "C104.1", "C105.1", "C105.1"),
Z_score = c(0.31, 2.34, 2.45, 1.45, 2.14, -1.44, 12.18, -0.18, 3.22, -3.18,
0.45, -1.86, 2.89, -1.88, 0.54, 1.58, 0.53, 0.35, 3.46, 0.14)
)

write.table(test_acyl_carnitines_df, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_acyl_carnitines_df.txt")

test_crea_gua_df <- data.frame(
HMDB_name = c(
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 ",
"metab4 ",
"metab11 "
),
Sample = c("P2025M1", "P2025M1", "P2025M2", "P2025M2", "P2025M3",
"P2025M3", "P2025M4", "P2025M4", "P2025M5", "P2025M5",
"C101.1", "C101.1", "C102.1", "C102.1", "C103.1", "C103.1",
"C104.1", "C104.1", "C105.1", "C105.1"),
Z_score = c(0.84, -0.46, -0.15, -1.51, -0.78, 1.68, 0.84, 1.48, 0.47, 1.18,
0.58, 2.47, 5.46, -0.68, 3.84, 0.84, -0.54, 1.48, -0.15, 1.48)
)

write.table(test_crea_gua_df, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_crea_gua_df.txt")
}

make_test_expected_biomark_df <- function() {
test_expected_biomarkers_df <- data.frame(
HMDB_code = c("HMDB002", "HMDB002", "HMDB005", "HMDB005", "HMDB005", "HMDB009", "HMDB012", "HMDB012", "HMDB020", "HMDB020",
"HMDB025", "HMDB025", "HMDB025", "HMDB028", "HMDB028"),
Disease = c("Disease A", "Disease B", "Disease B", "Disease B", "Disease C", "Disease D", "Disease A", "Disease E",
"Disease F", "Disease C", "Disease F", "Disease G", "Disease D", "Disease G", "Disease E"),
M.z = c("1.2", "1.2", "2.5", "2.5", "2.5", "3.0", "4.5", "4.5", "2.5", "2.5", "5.1", "5.1", "5.1", "5.8", "5.8"),
Change = c("Increase", "Decrease", "Increase", "Increase", "Decrease", "Decrease", "Increase", "Increase", "Decrease",
"Increase", "Increase", "Decrease", "Increase", "Decrease", "Increase"),
Total_Weight = c(10.0, -1.5, 2.0, 5.0, -2.5, -3.0, 14.5, 4.0,
-7.5, 6.0, 20.0, -5.0, 3.0, -1.5, 4.0),
Absolute_Weight = c(10.0, 1.5, 2.0, 5.0, 2.5, 3.0, 14.5, 4.0,
7.5, 6.0, 20.0, 5.0, 3.0, 1.5, 4.0),
Dispensability = c("Dispensable", "Dispensable", "Indispensable", "Dispensable", "Dispensable", "Indispensable",
"Dispensable", "Dispensable", "Indispensable", "Indispensable", "Dispensable", "Dispensable",
"Dispensable", "Dispensable", "Indispensable")
)

write.table(test_expected_biomarkers_df, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_expected_biomarkers_df.txt")
}

make_test_probability_score_df <- function() {
test_probability_score_df <- data.frame(
Disease = c("Disease A", "Disease B", "Disease C", "Disease D", "Disease E", "Disease F", "Disease G"),
P2025M1 = c(10.9, 0.953, 12.1, 0, 44.3, 0, -38.7),
P2025M2 = c(-10.9, 0, 0, -12.5, 0, -77.4, 38.7),
P2025M3 = c(49.9, 2.29, 0, 0, 0, -77.4, 38.7),
P2025M4 = c(-49.9, 0, 12.1, 18.2, 28.1, 0, -38.7)
)

write.table(test_probability_score_df, sep = "\t", quote = FALSE, row.names = FALSE,
file = "tests/testthat/fixtures/test_probability_score_df.txt")
}
11 changes: 11 additions & 0 deletions DIMS/tests/testthat/fixtures/test_acyl_carnitines_controls.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
HMDB_name Sample Z_score
metab1 C101.1 0.45
metab3 C101.1 -1.86
metab1 C102.1 2.89
metab3 C102.1 -1.88
metab1 C103.1 0.54
metab3 C103.1 1.58
metab1 C104.1 0.53
metab3 C104.1 0.35
metab1 C105.1 3.46
metab3 C105.1 0.14
Loading
Loading