Issue with bulk import status response

[paynejd]

When requesting the status of a bulk import, the response has two issues:

• Result parameter is broken – if the parameter is provided with any value, it always returns the “json” result – need to fix this so that it accepts the 3 possible values: summary, report, json
• JSON response provides a serialized string instead of a proper JSON object

In addition, the inconsistent content types in the responses (e.g. summary is text and everything else is json) puts an unnecessary burden on the client (Sorry: this is a leftover from when I was developing some of the bulk import logic!) This is the right moment to change this, because I am updating the PEPFAR integration script (which is the only dependency on the summary response).

Given a request:

/importers/bulk-import/?task=:task[&result=format]

There are several possible responses (@snyaggarwal can you please fill in the gaps?):

Description	Response Code	Status	Body
Task received but not yet processing	202	PENDING	depends on result param
Task processing	202	STARTED	depends on result param
Task completed	200	SUCCESS	depends on result param
Canceled task	??	??	??
Failed task	??	??	??

For some responses, the body depends on the the result param:

• result=summary (default): change to something like this?

{
  "state": "...",
  "message": "..." # the serialized text summary
}

• result=report: looks like this:

{
    "id": "c77c8e7e-2436-4f1b-a138-a63b49a179a2-datim-admin~DATIM-MOH-DEMO-DAA-FY24",
    "state": "SUCCESS",
    "name": "core.common.tasks.bulk_import_parallel_inline",
    "queue": "DATIM-MOH-DEMO-DAA-FY24",
    "username": "datim-admin",
    "task": "c77c8e7e-2436-4f1b-a138-a63b49a179a2-datim-admin~DATIM-MOH-DEMO-DAA-FY24",
    "created_at": "2024-08-05T12:21:47.345617Z",
    "started_at": "2024-08-05T12:21:47.455480Z",
    "finished_at": "2024-08-05T12:24:56.308902Z",
    "runtime": 188.853422,
    "summary": {
        "total": 1320,
        "processed": 1320
    },
    "children": [
        "c1a85f48-3e42-4199-825e-1e3854823f2f",
        "ca746e34-3769-488f-9ebc-f24dd9c06206",
        "0c123525-da13-48ba-96d7-0630ccd8bacc",
        "c0f6f00a-179a-4c8a-962d-a7a2b089ed3e",
        "c78f5ec0-c8b8-4919-95ce-2b826a093d8a",
        "0145e4e9-45f1-499e-b40a-bd49039d8167",
        "26727206-9f04-4dfa-a3f9-05fd3c573dc3",
        "0d19f3ac-6f35-4172-a6da-8166cafdb9c3",
        "33be6d2b-0a9c-46f0-a273-8b1f9848dd13",
        "65906047-7c2f-4ac9-a693-7f18208ee0c6",
        "ddfef3f6-7183-4ce0-b327-a62b654651d1",
        "83be2125-1cc7-4331-8c0a-9fae8f326224",
        "4ee9881b-0d98-4ea8-8bae-d46ce688273b",
        "40814e35-7f8c-4f2e-a227-80ceed9f6387",
        "7148714d-b084-4c48-adb8-6a00d7c9dc42",
        "acdbb581-87f3-46fd-b35a-d5406a36afa0",
        "13e02ed4-73cd-427a-84a5-b763078c25cd",
        "48d2baad-a4c4-4ee2-b37f-9cb747cd2c64",
        "44393da4-759f-478f-874c-18988955c548",
        "7ed1bfc1-6e52-4a51-ad1e-1a2ce1ff1eaa",
        "31fb19c1-5d1b-4ab4-8c3b-d9eea51b978a",
        "7b717c69-7f1d-487e-b2c4-e6fef33e0f6d",
        "2a8884b3-9c6e-4e1e-8bf6-13bd9e209a97",
        "2bbe045e-7ff7-432a-85db-a81e96a96ff5",
        "82dc1df5-8eb3-4bbd-b107-bdaf321f1ed4",
        "6dde6559-a6c8-4803-a2d5-872cd6ff07bb",
        "24fa6b94-72e3-4f47-8bc5-d3b9dd93840e",
        "19be09e8-c957-4d29-87a2-bd743aabdebb"
    ]
}

• result=json: response is the same as above, but it also includes a result attribute

Acceptance criteria:

• summary response should change to a very brief JSON object as described above
• result param should work as documented, supporting the summary, report and json formats, with summary as the default
• When response is result=json, should provide a proper JSON object (instead of serialized string)
• Update Bulk Import API documentation to reflect the current behavior (responses are all out of date)

[snyaggarwal]

@paynejd

1. Cancelled Task -- 200, REVOKED, same response with result as None
2. Failed Task -- 200, FAILED, same response with result as None

I have updated the GET API to consider result param:

1. result=summary (or default without result) will return a string summary in "result" attribute
2. result=report -- expanded summary json in counts
3. result=json -- full result

Also fixed the serialization

[paynejd]

One more round of minor changes:

Acceptance criteria:

• report and summary formats identical where:
  • report.result is in both
  • summary result is renamed to message and in both
• deprecate the report format
• summary format the default
• update documentation

[paynejd]

@snyaggarwal looks good! Please update docs

[snyaggarwal]

@paynejd Raised a docs PR - OpenConceptLab/ocl-docs#57

[snyaggarwal]

@paynejd This is deployed on Prod/Demo as well

[snyaggarwal]

@paynejd Can this be closed?