Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 61 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,83 @@
# BrightPath
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

BrightPath is a Python library designed to convert Brightway2 LCA inventories
into a format that can be imported into Simapro 9.x.
BrightPath bridges life-cycle assessment (LCA) data between
[Brightway2](https://brightway.dev/) and [SimaPro](https://simapro.com/). It
bundles the mappings that are required to translate units, flow names and
metadata between both tools and exposes high-level helpers to perform the
conversion in either direction.

## Features

* Convert Brightway2 inventories exported as Excel spreadsheets to the SimaPro
CSV format.
* Import SimaPro CSV exports and normalise them so that they can be registered
as Brightway databases.
* Ship curated mappings for biosphere flows, technosphere exchanges,
sub-compartments and blacklist entries required during the conversion.

## Installation

Use the package manager [pip](https://pip.pypa.io/en/stable/) to install BrightPath.
Install BrightPath from PyPI using [pip](https://pip.pypa.io/):

```bash

pip install brightpath

pip install brightpath
```

## Usage

```python

import brightpath
### Convert Brightway inventories to SimaPro CSV

# Create a converter object with the path to the Brightway LCA inventory
converter = brightpath.BrightwayConverter('path_to_inventory')
```python
from brightpath import BrightwayConverter

converter = BrightwayConverter(
filepath="/path/to/brightway-export.xlsx",
metadata="/path/to/metadata.yaml", # optional
ecoinvent_version="3.9",
)

# Write the converted inventory to a CSV file (defaults to the current
# working directory unless ``export_dir`` is provided during initialisation).
output_path = converter.convert_to_simapro(database="ecoinvent")
print(output_path)
```

# Convert the inventory to a format compatible with Simapro 9.x
sima_inventory = converter.to_simapro()
The converter also accepts inventory data that has already been loaded into
memory via the ``data`` argument and can return the converted rows directly by
calling ``convert_to_simapro(format="data")``.

# Save the converted inventory to a CSV file
sima_inventory.to_file('output_path')
### Convert SimaPro CSV exports to Brightway datasets

```python
from brightpath import SimaproConverter

converter = SimaproConverter(
filepath="/path/to/simapro-export.csv",
ecoinvent_version="3.9",
db_name="my-simapro-import",
)

# Normalise exchange names, locations and metadata so that they align with
# Brightway conventions.
converter.convert_to_brightway()

# The processed data lives on ``converter.i`` (an instance of
# ``bw2io.SimaProCSVImporter``). You can now write the database to your
# Brightway project if desired:
# converter.i.write_database()
```

## License
## Development

[BSD-3-Clause](https://github.com/romainsacchi/brightpath/blob/master/LICENSE).
* Source code is formatted with standard Python tools and tested with
`pytest`.
* Data files required for the conversions live under `brightpath/data`.
* Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidance on running tests
and submitting pull requests.

## Contributing
## License

See [contributing](https://github.com/romainsacchi/brightpath/blob/master/CONTRIBUTING.md).
BrightPath is distributed under the
[BSD-3-Clause license](LICENSE).

107 changes: 96 additions & 11 deletions brightpath/bwconverter.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,52 @@


class BrightwayConverter:
"""
Convert Brightway2 inventories to Simapro CSV files.
"""Convert Brightway2 inventories to SimaPro CSV files.

The converter loads inventories exported from Brightway2 and prepares
them for the SimaPro import format. The instance keeps references to the
different lookup tables that are needed throughout the conversion.

:param filepath: Path to the Brightway inventory spreadsheet.
:type filepath: str | None
:param data: Brightway inventories provided directly instead of loading
them from ``filepath``.
:type data: list | None
:param metadata: Optional path to a YAML file containing additional
metadata to append to the export.
:type metadata: str | None
:param ecoinvent_version: Version string of the ecoinvent database the
inventories are linked to.
:type ecoinvent_version: str
:param export_dir: Directory where generated SimaPro CSV files are saved.
:type export_dir: str | None
:ivar filepath: Path to the Brightway inventory spreadsheet.
:vartype filepath: str | None
:ivar inventories: Brightway activities that are going to be converted.
:vartype inventories: list[dict] | None
:ivar simapro_blacklist: Exchanges that should not be exported.
:vartype simapro_blacklist: dict
:ivar simapro_fields: Order and structure of SimaPro sections.
:vartype simapro_fields: list[str]
:ivar simapro_units: Mapping of Brightway units to SimaPro units.
:vartype simapro_units: dict[str, str]
:ivar simapro_headers: Header rows used when writing the CSV file.
:vartype simapro_headers: list[str]
:ivar simapro_technosphere: Mapping from technosphere exchanges to
SimaPro names.
:vartype simapro_technosphere: dict[tuple[str, str], str]
:ivar simapro_biosphere: Mapping from biosphere exchanges to SimaPro
names.
:vartype simapro_biosphere: dict[str, str]
:ivar simapro_subcompartment: Mapping of biosphere subcompartments to
SimaPro names.
:vartype simapro_subcompartment: dict[str, str]
:ivar ei_version: Version of the ecoinvent database in use.
:vartype ei_version: str
:ivar metadata: Optional metadata loaded from the YAML file.
:vartype metadata: dict | None
:ivar export_dir: Output directory for converted CSV files.
:vartype export_dir: pathlib.Path
"""

def __init__(
Expand All @@ -48,8 +92,26 @@ def __init__(
ecoinvent_version: str = "3.9",
export_dir: str = None,
):
"""
:param filepath: path to the BW inventory spreadsheet file
"""Instantiate a converter that targets the SimaPro CSV format.

When ``filepath`` is provided the inventories are loaded from the
spreadsheet using :func:`brightpath.utils.import_bw_inventories`. If
``filepath`` is omitted, pre-loaded ``data`` can be supplied instead.
Optional ``metadata`` is validated and attached to the export.

:param filepath: Path to the Brightway inventory spreadsheet file.
:type filepath: str | None
:param data: Inventories loaded in memory. Used when ``filepath`` is
``None``.
:type data: list | None
:param metadata: Path to the metadata YAML file.
:type metadata: str | None
:param ecoinvent_version: Version of the linked ecoinvent database.
:type ecoinvent_version: str
:param export_dir: Directory where SimaPro exports will be written.
:type export_dir: str | None
:raises FileNotFoundError: If the metadata file does not exist.
:raises ValueError: If the metadata file is not a YAML document.
"""
self.filepath = filepath
self.inventories = import_bw_inventories(filepath) if self.filepath else data
Expand All @@ -70,10 +132,20 @@ def __init__(
self.export_dir = Path(export_dir) or Path.cwd()

def format_inventories_for_simapro(self, database: str):
"""
Format inventories to Simapro format.
:param database: name of the database to link to.
:return: list
"""Transform the Brightway inventories into the SimaPro structure.

This method orchestrates the conversion of each activity in the
Brightway dataset into the row-based structure expected by SimaPro.
The resulting structure is compatible with the CSV export performed
by :meth:`convert_to_simapro`.

:param database: Name of the target database to link to. Valid values
are ``"ecoinvent"`` and ``"uvek"``.
:type database: str
:return: Rows ready to be written to a SimaPro CSV file.
:rtype: list[list[str]]
:raises ValueError: If required information is missing from the
inventories.
"""

rows = [
Expand Down Expand Up @@ -500,9 +572,22 @@ def format_inventories_for_simapro(self, database: str):
def convert_to_simapro(
self, database: str = "ecoinvent", format: str = "csv"
) -> [str, list]:
"""
Convert the inventories to Simapro CSV files.
:param database: Name of the database to link to. Default is `ecoinvent`, but can be `uvek`.
"""Export the converted inventories.

The inventories are formatted using
:meth:`format_inventories_for_simapro` and either returned as raw data
or written to disk as a CSV file, depending on ``format``.

:param database: Database to use when resolving exchanges. Accepted
values are ``"ecoinvent"`` and ``"uvek"``.
:type database: str
:param format: Output mode. Use ``"data"`` to receive the converted
rows instead of writing a CSV file.
:type format: str
:return: The CSV filepath when ``format`` is ``"csv"`` or the raw
SimaPro data rows when ``format`` is ``"data"``.
:rtype: str | list[list[str]]
:raises ValueError: If an unsupported ``database`` value is supplied.
"""

if database not in ("ecoinvent", "uvek"):
Expand Down
79 changes: 71 additions & 8 deletions brightpath/simaproconverter.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,13 @@


def format_technosphere_exchange(txt: str):
"""Split and normalise a technosphere exchange name from SimaPro.

:param txt: Raw exchange string as found in a SimaPro CSV export.
:type txt: str
:return: Tuple containing the cleaned name, reference product and location.
:rtype: tuple[str, str, str]
"""

location_correction = {
"WECC, US only": "US-WECC",
Expand Down Expand Up @@ -151,13 +158,33 @@ def format_technosphere_exchange(txt: str):


def load_ecoinvent_activities(version: str) -> list:
"""Load the list of ecoinvent activities for the given version.

:param version: Ecoinvent version identifier, e.g. ``"3.9"``.
:type version: str
:return: Rows describing ecoinvent activities.
:rtype: list[list[str]]
"""
with open(DATA_DIR / "export" / f"list_ei{version}_cutoff_activities.csv") as f:
reader = csv.reader(f)
next(reader)
return [l for l in reader]


def format_biosphere_exchange(exc, ei_version, bio_flows, bio_mapping):
"""Normalise a biosphere exchange to match ecoinvent conventions.

:param exc: Exchange to adjust in-place.
:type exc: dict
:param ei_version: Version of ecoinvent used for interpretation.
:type ei_version: str
:param bio_flows: Known biosphere flows for the version.
:type bio_flows: list[tuple[str, str, str]]
:param bio_mapping: Mapping to resolve outdated flow names.
:type bio_mapping: dict
:return: The updated exchange dictionary.
:rtype: dict
"""
if "in ground" in exc["name"]:
if ei_version not in ["3.5", "3.6", "3.7", "3.8"]:
exc["name"] = exc["name"].replace(", in ground", "")
Expand Down Expand Up @@ -244,14 +271,48 @@ def format_biosphere_exchange(exc, ei_version, bio_flows, bio_mapping):


class SimaproConverter:
"""Convert SimaPro CSV exports into Brightway-compatible datasets.

:param filepath: Path to the SimaPro CSV file.
:type filepath: str
:param ecoinvent_version: Version of ecoinvent to align biosphere data to.
:type ecoinvent_version: str
:param db_name: Optional name of the Brightway database to create.
:type db_name: str | None
:ivar filepath: Normalised path to the validated CSV file.
:vartype filepath: pathlib.Path
:ivar i: Instance of :class:`bw2io.SimaProCSVImporter` handling the data.
:vartype i: bw2io.SimaProCSVImporter
:ivar db_name: Name of the Brightway database that will be created.
:vartype db_name: str
:ivar ecoinvent_version: Version of the ecoinvent database in use.
:vartype ecoinvent_version: str
:ivar biosphere: Mapping between SimaPro and Brightway biosphere flows.
:vartype biosphere: dict
:ivar technosphere: Mapping between SimaPro and Brightway technosphere
exchanges.
:vartype technosphere: dict
:ivar subcompartments: Mapping of sub-compartments between the databases.
:vartype subcompartments: dict
:ivar ei_biosphere_flows: Known biosphere flows for the selected version.
:vartype ei_biosphere_flows: list[tuple[str, str, str]]
:ivar biosphere_flows_correspondence: Mapping of outdated biosphere names.
:vartype biosphere_flows_correspondence: dict
"""

def __init__(
self, filepath: str, ecoinvent_version: str = "3.9", db_name: str = None
):
"""
Initialize the SimaproConverter object.

:param data: list of Simapro inventories
:param ecoinvent_version: ecoinvent version to use
"""Initialise the converter and load the SimaPro inventory.

:param filepath: Path to the SimaPro CSV export to convert.
:type filepath: str
:param ecoinvent_version: Ecoinvent version that should be used when
reconciling biosphere flows.
:type ecoinvent_version: str
:param db_name: Optional Brightway database name override.
:type db_name: str | None
:raises FileNotFoundError: If the provided CSV file cannot be found.
"""

logging.basicConfig(
Expand Down Expand Up @@ -283,7 +344,7 @@ def __init__(
self.i.db_name = self.db_name

def check_database_name(self):

"""Ensure exchanges reference the correct Brightway database name."""
for act in self.i.data:
act["database"] = self.i.db_name

Expand All @@ -294,7 +355,7 @@ def check_database_name(self):
exc["input"] = (self.i.db_name, exc["input"][1])

def convert_to_brightway(self):

"""Convert the imported SimaPro data into Brightway inventories."""
print("- format exchanges")
internal_datasets = []
for ds in self.i.data:
Expand Down Expand Up @@ -366,14 +427,16 @@ def convert_to_brightway(self):
print("Done!")

def remove_empty_datasets(self):
"""Remove datasets that contain no exchanges."""
self.i.data = [ds for ds in self.i.data if len(ds["exchanges"]) >= 1]

def remove_empty_exchanges(self):
"""Remove exchanges that have a zero amount."""
for ds in self.i.data:
ds["exchanges"] = [e for e in ds["exchanges"] if e["amount"] != 0.0]

def check_inventories(self):

"""Perform basic validation of the converted inventories."""
for ds in self.i.data:
if len([x for x in ds["exchanges"] if x["type"] == "production"]) != 1:
print(
Expand Down
Loading
Loading