Skip to content

#315 - Concept Sets domain incorrectly unretires referenced concepts #316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mseaton merged 2 commits into from
Mar 4, 2026
Merged

#315 - Concept Sets domain incorrectly unretires referenced concepts #316

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5b8aebf
#315 - Concept Sets domain incorrectly unretires referenced concepts
mseaton Mar 3, 2026
6e11b41
#315 - Concept Sets domain incorrectly unretires referenced concepts
mseaton Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,8 @@ See the [documentation on Initializer's logging properties](readme/rtprops.md#lo

## Releases notes
#### version 2.12.0
* Fix conceptsets domain to prevent incorrect unretiring of associated concept

#### Version 2.11.0
* Added support for patient flags (flags, flagpriorities, flagtags) domains
* Support for 'billing' (billableservices, paymentmodes, cashpoints) for Billing V2
Expand Down
24 changes: 18 additions & 6 deletions api/src/main/java/org/openmrs/module/initializer/api/CsvParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,20 +191,21 @@ public List<String[]> getLines() {
public CsvFailingLines process(List<String[]> lines) {
// Save cached objects to avoid losing them prematurely by other Parsers
// See DisplaysCsvParser#save(OpenmrsObject)
Context.flushSession();
Context.clearSession();

CsvFailingLines result = new CsvFailingLines();
log.debug(getClass() + " processing " + lines.size() + " lines");
int saved = 0;
clearAndFlushSession(saved);

CsvFailingLines result = new CsvFailingLines();

for (String[] line : lines) {
T instance = null;
try {
instance = initialize(line);
save(instance);
saved++;
if (saved > 250) { // TODO make this configurable
Context.flushSession();
Context.clearSession();
if (saved % 250 == 0) { // TODO make this configurable
Copy link
Collaborator Author

@mseaton mseaton Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unrelated, but I found during testing. Changing this from > 250 to % 250, and adding additional logging so that more progress can be assessed during long-running loads. For example, I was loading in concepts and conceptssets csvs that each had about 10,000 rows and takes around 8-10 minutes.

clearAndFlushSession(saved);
}
}
catch (Exception e) {
Expand All @@ -218,6 +219,17 @@ public CsvFailingLines process(List<String[]> lines) {
return result;
}

/**
* Flushes and clears the hibernate session
*
* @param saved the current count of saved items to log
*/
private void clearAndFlushSession(int saved) {
Context.flushSession();
Context.clearSession();
log.debug("Number processed: " + saved);
}

/**
* "Initializes" an instance from a CSV line. It bootstraps the instance, then retires early or
* fills it. It returns an instance ready to be saved through the API service layer.
Expand Down
12 changes: 12 additions & 0 deletions api/src/main/java/org/openmrs/module/initializer/api/c/ConceptSetsCsvParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,18 @@ public Concept bootstrap(CsvLine line) throws IllegalArgumentException {
return concept;
}

@Override
public boolean setRetired(Concept instance, boolean retired) {
// The Concept Sets domain should never result in changing the retired status of the underlying concept
return false;
}

@Override
protected boolean shouldFill(Concept instance, CsvLine csvLine) {
// We need to process all set members, whether the underlying concept is retired or not
return true;
}

Copy link
Collaborator Author

@mseaton mseaton Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the primary fixes that are needed

@Override
public Concept save(Concept instance) {
return conceptService.saveConcept(instance);
Expand Down
17 changes: 17 additions & 0 deletions api/src/test/java/org/openmrs/module/initializer/api/ConceptSetsLoaderIntegrationTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;

public class ConceptSetsLoaderIntegrationTest extends DomainBaseModuleContextSensitiveTest {

Expand All @@ -49,6 +50,15 @@ public void setup() throws Exception {
@Test
public void load_shouldLoadConceptSetsAccordingToCsvFiles() {

// Test starting values

{
Concept retiredSet = cs.getConceptByUuid("bcc1796a-16f1-11f1-a7b6-da82fbe923ea");
assertNotNull(retiredSet);
assertTrue(retiredSet.getRetired());
assertEquals(0, retiredSet.getSetMembers().size());
}

// Load
conceptsLoader.load();
conceptSetsLoader.load();
Expand Down Expand Up @@ -84,5 +94,12 @@ public void load_shouldLoadConceptSetsAccordingToCsvFiles() {
assertEquals("Sight", senseAnswers.get(2).getAnswerConcept().getName().getName());
assertEquals(new Double(200), senseAnswers.get(2).getSortWeight());
}

{
Concept retiredSet = cs.getConceptByUuid("bcc1796a-16f1-11f1-a7b6-da82fbe923ea");
assertNotNull(retiredSet);
assertTrue(retiredSet.getRetired());
assertEquals(2, retiredSet.getSetMembers().size());
}
}
}
4 changes: 3 additions & 1 deletion api/src/test/java/org/openmrs/module/initializer/api/c/ConceptsCsvParserTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,9 @@ public void setup() {
ConceptClass classMisc = new ConceptClass();
classMisc.setName("Misc");
when(cs.getConceptClassByName(eq("Misc"))).thenReturn(classMisc);

ConceptClass convSet = new ConceptClass();
convSet.setName("ConvSet");
when(cs.getConceptClassByName(eq("ConvSet"))).thenReturn(convSet);
ConceptDatatype typeCoded = new ConceptDatatype();
typeCoded.setName("Coded");
when(cs.getConceptDatatypeByName(eq("Coded"))).thenReturn(typeCoded);
Expand Down
3 changes: 2 additions & 1 deletion api/src/test/resources/testAppDataDir/configuration/concepts/concepts_base.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,6 @@ foobar,,Cambodia_Lao,កម្ពុជា_ឡាវ,Lao,ឡាវ,,Lao,ឡា
,,,គំនិត_ដោយ_FSN,,ឈ្មោះខ្លីថ្មី,,,,Misc,Text,,,,,
4c93c34e-37c2-11ea-bd28-d70ffe7aa802,,MMR,,,,,,,Misc,N/A,,,,,
4d3cfdcf-1f3f-4b41-9b31-02dfd951c582,,CONCEPT_WITH_ATTRIBUTES,,,,,,,Misc,Text,,jdoe@example.com,2020-04-06,,
fda5cc2a-6245-4d91-be17-446d27aab33b,,Testing Fully Specified Name,,Testing Short Name,,,,,Misc,,,,,,
fda5cc2a-6245-4d91-be17-446d27aab33b,,Testing Fully Specified Name,,Testing Short Name,,,,,ConvSet,N/A,,,,,
1ddb8255-00d5-45e8-8830-f9567919a382,,Replaced Fully Specified Name,,Rpl SN,,Replaced Synonym,,,Misc,Text,,,,,
bcc1796a-16f1-11f1-a7b6-da82fbe923ea,true,Retired Set,,,,,,,ConvSet,N/A,,,,,
4 changes: 3 additions & 1 deletion api/src/test/resources/testAppDataDir/configuration/conceptsets/concept_sets.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,6 @@ CIEL:SENSE_SET,CIEL:TOUCH,,4.0,,
54014540-311d-11ec-8d2b-0242ac110002,Sound,,50,,
5a393db9-311d-11ec-8d2b-0242ac110002,CIEL:SMELL,Q-AND-A,"",,
How did you sense it?,CIEL:SIGHT,q-and-a,200,,
CIEL:SENSE_QUESTION,CIEL:SOUND,,100,,
CIEL:SENSE_QUESTION,CIEL:SOUND,,100,,
bcc1796a-16f1-11f1-a7b6-da82fbe923ea,SMELL,concept-set,1.0,,
bcc1796a-16f1-11f1-a7b6-da82fbe923ea,TASTE,concept-set,2.0,,
2 changes: 2 additions & 0 deletions api/src/test/resources/testdata/test-concepts.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
<concept CONCEPT_ID="5510" UUID="4cfe07b0-3061-11ec-8d2b-0242ac110002" RETIRED="false" DATE_CREATED="2019-12-11 16:59:48.0" DATE_CHANGED="2019-12-11 16:59:48.0" IS_SET="false" DATATYPE_ID="3" CLASS_ID="11" CHANGED_BY="1" CREATOR="1"/>
<concept CONCEPT_ID="5511" UUID="61214827-303f-11ec-8d2b-0242ac110002" RETIRED="false" DATE_CREATED="2019-12-11 16:59:48.0" DATE_CHANGED="2019-12-11 16:59:48.0" IS_SET="false" DATATYPE_ID="3" CLASS_ID="11" CHANGED_BY="1" CREATOR="1"/>
<concept CONCEPT_ID="5512" UUID="1ddb8255-00d5-45e8-8830-f9567919a382" RETIRED="false" DATE_CREATED="2019-12-11 16:59:48.0" DATE_CHANGED="2019-12-11 16:59:48.0" IS_SET="false" DATATYPE_ID="3" CLASS_ID="11" CHANGED_BY="1" CREATOR="1"/>
<concept CONCEPT_ID="5513" UUID="bcc1796a-16f1-11f1-a7b6-da82fbe923ea" RETIRED="true" DATE_CREATED="2019-12-11 16:59:48.0" DATE_CHANGED="2019-12-11 16:59:48.0" IS_SET="true" DATATYPE_ID="4" CLASS_ID="10" CHANGED_BY="1" CREATOR="1"/>

<concept_name CONCEPT_NAME_ID="6052" CONCEPT_ID="5498" NAME="CONCEPT_RETIRE" LOCALE="en" DATE_CREATED="2019-12-11 16:59:47.0" CREATOR="1" VOIDED="false" CONCEPT_NAME_TYPE="FULLY_SPECIFIED" LOCALE_PREFERRED="true" UUID="02f3b54f-82de-4c63-b24b-4e3c59589407"/>
<concept_name CONCEPT_NAME_ID="6053" CONCEPT_ID="5499" NAME="CONCEPT_EDIT_SHORTNAME" LOCALE="en" DATE_CREATED="2019-12-11 16:59:47.0" CREATOR="1" VOIDED="false" CONCEPT_NAME_TYPE="FULLY_SPECIFIED" LOCALE_PREFERRED="true" UUID="81ae92eb-8221-4867-bd83-30dbccf1ff4d"/>
Expand All @@ -44,6 +45,7 @@
<concept_name CONCEPT_NAME_ID="6072" CONCEPT_ID="5512" NAME="Replaced Fully Specified Name" LOCALE="en" DATE_CREATED="2019-12-11 16:59:48.0" CREATOR="1" VOIDED="false" CONCEPT_NAME_TYPE="FULLY_SPECIFIED" LOCALE_PREFERRED="true" UUID="8d09f38c-6ecb-4e3d-8a25-5cd204c43390"/>
<concept_name CONCEPT_NAME_ID="6073" CONCEPT_ID="5512" NAME="Rpl SN" LOCALE="en" DATE_CREATED="2019-12-11 16:59:48.0" CREATOR="1" VOIDED="false" CONCEPT_NAME_TYPE="SHORT" LOCALE_PREFERRED="false" UUID="ff688089-4385-4d6c-9435-523506eda50e"/>
<concept_name CONCEPT_NAME_ID="6074" CONCEPT_ID="5512" NAME="Replaced Synonym" LOCALE="en" DATE_CREATED="2019-12-11 16:59:48.0" CREATOR="1" VOIDED="false" LOCALE_PREFERRED="false" UUID="05dfba65-0590-4886-ac50-8c3f69a8ea2e"/>
<concept_name CONCEPT_NAME_ID="6075" CONCEPT_ID="5513" NAME="Retired Set" LOCALE="en" DATE_CREATED="2019-12-11 16:59:48.0" CREATOR="1" VOIDED="false" CONCEPT_NAME_TYPE="FULLY_SPECIFIED" LOCALE_PREFERRED="true" UUID="208b7100-16f2-11f1-a7b6-da82fbe923ea"/>

<concept_reference_term CONCEPT_REFERENCE_TERM_ID="12" UUID="8d340303-7cda-49ad-b49d-e1618d9c3143" CONCEPT_SOURCE_ID="1089" CODE="foo12bar" NAME="" CREATOR="1" DATE_CREATED="2019-12-11 16:59:47.0" RETIRED="false"/>

Expand Down