Output formats for complex types

[sergiusens]

Based on #278 and canonical/snapcraft#5060

Overview

Add an emitter method similar to the message method that takes a data structure and outputs it either as a table or as a data structure in a serialization format (MVP: as JSON, but make it expandable)

Acceptance criteria

• A method on the emitter to write structured data to stdout
  • Output methods (and the ability to make more later):
    • Table (possibly use tabulate, but not required)
    • JSON
• Example in examples.py that can output in various formats.
• Example of one Charmcraft command using this for its --format=json arg (does not need to be merged - needs to exist as a manual e2e test before we’ll merge the craft-cli PR)

[beliaev-maksim]

💰 Bounty – $750 USD

Bounty is sponsored by Canonical, the company behind Ubuntu.

📜 Rules:

1. You must be enrolled in the GitHub Sponsors program.
2. Contribution rules from this repository apply. Please carefully read CONTRIBUTING.md
3. The bounty will be paid when the implementation meets the Business and Technical Specifications.
4. The bounty will be paid when the solution is merged into the main or release branch.
5. Contributors working on the issue must be assigned by the repository maintainers.
6. The bounty will be awarded to the first accepted solution.
7. To keep things fair, each contributor can work on only one bounty at a time. Once it’s completed, you’re free to pick the next challenge :)

💡 Note: The bounty amount is gross. Any applicable fees, taxes, or deductions from payment providers (e.g., GitHub Sponsors, bank transfers) may apply, and Canonical is not responsible for covering these additional costs.

Happy hacking! 🚀

For any questions and payouts please reach out to community+bounty@canonical.com

[Saturn225]

@sergiusens Interesting feature to tackle. Can I work on this and assigned @beliaev-maksim @sergiusens ?

[Saturn225]

Here’s a high-level plan I am thinking of to tackle this issue :

1. Emitter API extension
   Add a data() method alongside message(), accepting any Python data structure.
   Delegate to a pluggable formatter based on a new --format flag (defaulting to table).

2. Formatter interface
   Define a BaseFormatter with format(data: Any) -> str.

   • TableFormatter (using tabulate or a minimal custom renderer)
   • JsonFormatter (via json.dumps(data, indent=2, default=str)).

3. Pluggable format registry
   Maintain a mapping { "table": TableFormatter, "json": JsonFormatter }.
   Allow future registration via Emitter.register_format("yaml", YamlFormatter).
   Validate --format against registered keys and error on unknown values.

4. CLI flag integration
   Add --format to the base command parser so all commands support it.
   Pass the chosen format into the emitter on initialization.

5. Examples & manual e2e test
   In examples.py, add a demo that prints sample data as both table and JSON.
   In a simple Charmcraft command (e.g. list-resources), include a --format=json run in CI and assert valid JSON output.

6. Testing strategy
   Unit tests for each formatter covering edge cases (empty lists, nested data).
   Integration tests running a dummy command under both --format=table and --format=json, asserting exit code 0 and expected output patterns.

This approach builds on the design discussion in #278 and delivers an extendable architecture for future formats. Let me know if you would like any adjustments

cc: @medubelko @mr-cal @sergiusens I would like to get assigned and work on this

[sergiusens]

@tigarmo can you analyze @Saturn225's proposal?

[dharapandya85]

Hi @sergiusens and team,

I would like to implement the following the plan:

Implementation Plan:

• A method data() can be added to Emitter class. This method would accept data of type Any, and a format string such as "table" or "JSON". It could render output either through a table formatting function or using json.dumps().

• In examples.py, the Emitter function can be called with structured data. The output of this example would demonstrate rendering in both JSON and table formats.

• To make formatting extensible, a registry could be introduced to delegate formatting with the emitter function. Two different classes- one for JSON and one for table output- could be inherited from a common base class. These two different classes will handle JSON formatting and "table" formatting, respectively. Furthermore, the table could be formatted with borders to increase readability. This formatting registry can be added to the Emitter class. The data() function will delegate rendering by calling the formatter class with the format(data) method.

-The command in craft_cli could be modified to generate output in a table and JSON format. By adding sample data, and accepting a --format flag, with verification of output during testing, in both table and JSON formats.

Please let me know if I could proceed to solve the issue.

Thanks!

cc: @beliaev-maksim @medubelko

[mr-cal]

Hi all,

After reviewing both proposals, we'd like to see:

• Clarification on what data are passed in. Using Any is too vague, we would like to see a proposal that defines a stricter type. It can assume that the data will be something that can be formatted as a table.
• --format should not be included in commands by default. We would need a mechanism for a command to opt into formatting.

[dharapandya85]

Hi @sergiusens and team,
Based on the original issue and feedback, the updated implementation plan is as follows:

1. Implementing strict type
   ⦁ Structured data passed to formatters is now strictly typed.
   ⦁ Replacing Any with a restricted data structure that can be rendered as a table or JSON
   ⦁ Supported types include combinations of lists or dictionaries.
   ⦁ Example type definition:
   TabularData=Union[List[Dict[str,object]],Dict[str,object]]

2. Supporting Emitter
   ⦁ Adding a data() method to craft_cli.emit which delegates formatting with user selection. The Format registry can be expanded in the future, but it is currently used internally.
   ⦁ Example Usage:
   emit.data(sample_data,format="json")

3. Designing Formatters
   ⦁ A sample class BaseFormatter can be introduced which implements JSONFormatter(using json.dumps) and TableFormatter(using formatting rows and headers into plain text)
   ⦁ These formatters can be implemented as:
   format(data:TabularData)->str

4. Opt-In Format Flag
   ⦁ The --format flag is not added globally; commands that support structured output must explicitly opt in.
   ⦁ Example parser code:

 parser.add_argument(
            "--format",
            choices=["json","table"],
            default="table",
            help="Format for structured output",
        )

5. Sample command for testing
   ⦁ A test command can be implemented for structured output with emit.data available for both -- format=json and -- format=table.
   ⦁ It can be tested with CI assertions or E2E manual testing.
   ⦁ Example data:
   [{"name":"App A","version":"1.0.1","status":"active"}]

6. Testing Methodology
   ⦁ Unit tests can be performed with each formatter(nested dicts, mixed types, empty lists)
   ⦁ Integrated tests for verification of output structure matching the selected format.

Please let me know of any refinements or follow-up items before moving forward.
Thanks again!
cc: @beliaev-maksim @medubelko @mr-cal

[mr-cal]

@lengau, can you review @dharapandya85's proposal?

[mr-cal]

@tigarmo is going to review these proposals too

[tigarmo]

@dharapandya85 's proposal seems reasonable to me, but I would like to see some consideration regarding headers for the table output. For example, for data=[{'item_name': 'my item', 'created_at': '2024-08-01T14:38:32.499588'}] we likely want a table with header values more meaningful than item_name | created_at

[dharapandya85]

Hi all! I want to add these updates to my proposal. Thanks for the review.

Create a function prettify_header which will replace "_" to " " in header titles and use .title. We can create this function inside TableFormatter. A dictionary raw_headers collects the header titles, for example: item_name. Then, a display_headers dictionary converts all the headers into more user-friendly headers by calling prettify_headers.

Secondly, we can explicitly pass headers : dict[str,str] | None=None to functions like TableFormatter, or the data function. When calling emit.data(), we pass JSON or table formats, and we can define custom headers, for example: emit.data(data,format="table", headers={"item_name" : "Product","created_at": "Created On"}). This might override the beautified headers.

Example output, with prettify_header

Item Name | Created At
My Item   |   Date

[tigarmo]

@dharapandya85 thank you for the update. I like the second point (passing headers), but I'm not sure about the automatic "prettification" done by the first point. I fear it might look surprising in some cases, and possibly bad for keys like idx, cum_sum, etc.

So I think we should stick to the user explicitly passing headers for now, and we could add a pretty_headers=True option in the future if needed.