Feature Request: Add method(s) to `FailureInfo` to get a dataframe, which show more explicitly what rule(s) each row broke

[DeflateAwning]

The goal is to be able to write out a dataframe to a file with more details of what's broken.

I'll propose a few options. Most of them make sense to implement simultaneously imo, though feel free to pick your favourite.

Option 1: failure_info.invalid(keep_rule_columns: bool = False)

Simply make the .drop(self._rule_columns) part conditional. Not too sure what the names of those columns are, but they're probably reasonable.

Option 2: failure_info.invalid(with_rule_list: Literal['json', 'csv', 'list'] | None = None)

Add a column to the dataframe called something like dataframely_failed_rules. It would then add the column as a json list, as a comma-separated list of string, or as a pl.List(pl.String), depending on the option.

[delsner]

The goal is to be able to write out a dataframe to a file with more details of what's broken.

Could you please elaborate why a regular FailureInfo.write_parquet (properly supported since #75) doesn't suffice your needs?

I'd be hesitant to convert error details, to e.g. CSV format within dataframely. I think this is something the user should be responsible for.

[DeflateAwning]

Main reason - failure_info.write_parquet() is not included in any examples, FAQs, etc., so I didn't know about it.

After looking at it - I want the data in a dataframe. Sometimes I want to push it to a csv so I get use tools like grep to search through it.

Getting a path during a debugger session is notably more work than just printing out the dataframe too, for example.

[borchero]

• We should definitely mention write_parquet more prominently in the docs
• I don't think we want to add write_csv as we cannot store the schema in some metadata and lose type information. That goes against the core of dataframely.
• I'm not sure what exactly you're missing in the debugging session. Would you want a better __str__ representation of a failure info object?

[DeflateAwning]

I maintain that I meant what I said in the feature request. Parquets are obviously not first-class Polars datatypes. Anything that's supported in write_parquet should have a dataframe equivalent in the API.