Skip to content

[Specification] CDS-WG3-01 - Customer Data#26

Closed
daniel-utilityapi wants to merge 15 commits intolfe-cds:mainfrom
daniel-utilityapi:issue_24_customer_data
Closed

[Specification] CDS-WG3-01 - Customer Data#26
daniel-utilityapi wants to merge 15 commits intolfe-cds:mainfrom
daniel-utilityapi:issue_24_customer_data

Conversation

@daniel-utilityapi
Copy link
Contributor

@daniel-utilityapi daniel-utilityapi commented Mar 28, 2024

This is the initial draft of a proposed specification for utility servers to publish customer data.

Relevant Issue: #24

NOTE: This is a draft in progress (see "TODO" throughout the draft) and will continue to be developed and modified based on feedback and comments in this pull request and during meetings

See a rendered preview of this specification on the maintainer's branch:
https://daniel-utilityapi.github.io/CDS-Customer-Data/specs/cds-wg3-01

Timeline

This specification is under active development, so it will be merged when complete and reaching consensus among working group participants.

Copy link

@halliecramer1 halliecramer1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarifications and suggested changes, mainly related to customer usage data.

"cds_modified": "2024-01-01T00:00:00Z",
"meter_number": "M55555555",
"meter_type": "usage_forward_only",
"current_servicepoints": [

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a relationship between meter devices and service points?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reckon a service point can have multiple meter devices linked to it. If so, maybe it would be great that in the Service Point Endpoint, we also add the meterdevice_ids that are linked to it.

"cds_modified": "2024-01-01T00:00:00Z",
"cds_account_id": "111111111111-1",
"account_number": "22222-222-22222",
"statement_date": "2024-01-01",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also including billing cycle?

"item_type": "charge",
"for": "distribution_entity",
"name": "Electric delivery",
"start": "2023-12-01T00:00:00Z",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do some of these have start / end and other item types don't?

{
"summary_type": "demand",
"name": "Commercial Peak Demand",
"start": "2023-12-01T00:00:00Z",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if these start / end link to bill section and are all the same, then maybe can simplify and remove from here


This specification was developed as part of the global effort to combat the climate crisis. Specifically, in order to scalably measure carbon emissions of organizations and calculate the impact of deploying and operating clean energy technologies, companies need an efficient means of requesting authorization for and accessing Customer data from utilities and other entities.

There are thousands of utilities serving Customers across the world, and each have their own way of providing access to and formatting Customer data. This specification defines a way for these utilities and other entities ("Servers") to provide a standardized, structured process for external users and companies ("Clients") to access Customer data, requesting authorization from Customers if needed. By offering a standardized a Customer data access protocol, utilities and other entities can enable secure interoperability with other external systems providing climate-change fighting services such as carbon tracking, decarbonization measures, grid flexibility capabilities, and energy benchmarking.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo, "offering a standardized Customer.."

* `amount_due_date` - _ISO8601 date_ - (REQUIRED) The value shown on the Customer-facing bill statement as the date that the `amount_due` value is due, if available.
* `shutoff_prevention_minimum_due` - _decimal or `null`_ - (REQUIRED) The value shown on the Customer-facing bill statement as the minimum amount due to prevent service shutoff, if available. If there is no shutoff notice on the Customer-facing bill statement, this value is `null`.
* `shutoff_prevention_date` - _ISO8601 date or `null`_ - (REQUIRED) The value shown on the Customer-facing bill statement as the date that the `shutoff_prevention_minimum_due` value is due, if available. If `shutoff_prevention_minimum_due` is `null`, this value is also `null`.
* `program_participations` - _Array[[ProgramParticipation](#program-participation-format)]_ - (REQUIRED) A list of Account-level programs in which the Customer is participating. If the Server does not have this information or the Customer is not participating in any Account-level programs, this value is and empty list (`[]`).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does program participation relate to rate plan? e.g. could be fees related to a program that are separate / add on from the rate?


Listings of Aggregation objects MUST be ordered in reverse chronological order by `modified` timestamp, where the most recently updated relevant Aggregation MUST be first in each listing.

## 11. Usage Segments API <a id="usage-segments-api" href="#usage-segments-api" class="permalink">🔗</a>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is meant by "Segments"?


This specification requires Servers provide a set of APIs allowing Clients to retrieve a Customer's and Server's Usage Segment objects for which they are authorized to access. These APIs are authenticated using a Bearer `access_token` [granted](/specs/cdsc-wg1-02#grants-api) that provisions access for a [scope](#authorization-scopes) that allows access to the Usage Segments API.

### 11.1. Usage Segment Object Format <a id="usage-segment-format" href="#usage-segment-format" class="permalink">🔗</a>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when calling usage, can you specify as parameters: which service contracts? what time period? what data granularity (e.g. 15min, hourly, monthly)?

* `segment_end`
* `interval`
* `format`
* `values`
Copy link

@halliecramer1 halliecramer1 Apr 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extension to working group 2 - breakdown of values by fuel source and emissions e.g. (not fully compatible)
"supplier_mix": [
{
"startDatetime": "2021-06-01 00:00:00+00",
"endDatetime": "2021-06-01 01:00:00+00",
"values": [10.5, 50]
"format": ["kWh", "tCO2e"]
"valuesByFuelSource": [
{
"technology": "Thermal - Steam engine - Unspecified",
"type": "Fossil - Solid - Hard Coal - Unspecified",
"values": [5.0, 50]
},
{
"technology": "Solar - Photovoltaic - Unspecified",
"type": "Renewables - Heating and Cooling - Solar",
"values": [5.5, 0]
},
// Additional entries for other fuel sources if available
]
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The discussion indicated it's okay if it's not fully compatible with the Power Systems specs.

* `interval`
* `format`
* `values`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Customer EACs API: add list of eacs retired on customer's behalf
eac_id - string - (REQUIRED) The unique identifier for this EAC object.
eac_source - string - (REQUIRED) The registry provider where the EAC was issued and retired by the Service provider on the Customer's behalf, or by the Customer.
destination_id - string (REQUIRED) The organization responsible for the retirement of the EAC.
beneficiary_id - string (REQUIRED) The ID of the beneficiary (individual account or group of accounts) for this EAC.
beneficiary_type - [BeneficiaryType] - (REQUIRED) The type of beneficiary that the beneficiary ID represents, e.g. Account ID or Utility Program identifier
program_type - [ProgramType] - (OPTIONAL) Category of EAC pools dedicated for a certain purpose across a certain set of customers, e.g. ["CES", "Non-bypassable Clean Energy", "PPA", "XX Tariff"]
asset_id - string - (REQUIRED) The ID of the generation asset that generated the EAC.
technology_type - [TechnologyType] - (REQUIRED) The type of technology of the generating asset (should align with technology and type from "values by fuel source").
emissions_factor_direct - decimal - (REQUIRED) Direct emissions factor of the asset in gCO2/kWh
emissions_factor_lca - decimal - (REQUIRED) Lifecycle assessment emissions factor of the asset in gCO2/kWh
asset_origination - string - (REQUIRED) Location of asset (e.g. from region / grid)
asset_destination - string - (REQUIRED) Location of destination ID entity or beneficiary (e.g. to region / grid)
issuance_date - ISO8601 date - (REQUIRED) Date registry issued EAC.
retirement_date - ISO8601 date - (REQUIRED) Date that destination ID entity retired EAC.
period_start - ISO8601 datetime - (REQUIRED) Period start of generation from asset.
period_end - ISO8601 datetime - (REQUIRED) Period end of generation from asset.
volume - decimal - (REQUIRED) kWh of generation

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relevant fields in EACs/GCs cancellation statements, returned after asking for them on a certain period and contract:

period_start - ISO8601 datetime - (REQUIRED) Production period start fits within requested period
period_end - ISO8601 datetime - (REQUIRED) Production period end fits within requested period
registry_id - string - (REQUIRED) For traceability and to know the origin and registry managing the EAC/GC
beneficiary_id - string - (REQUIRED) The Customer or account identifier.
asset_id - string - (REQUIRED) The generation asset’s identifier.
asset_name - string (OPTIONAL) Sometimes, it is pretty challenging to relate the production sites just with the asset_id
technology_type - (REQUIRED) The technology used to generate the EAC.
production_country - string (REQUIRED) Needed for matching and reporting criteria
production_site_commissioning_date - ISO8601 datetime (REQUIRED) Relevant for reporting purposes (RE100 technical criteria)
quantity - integer (REQUIRED) - Volume of canceled EACs/GCs
unit - (REQUIRED) The unit of the EAC’s quantity.
certificate_ids - array - (REQUIRED). It usually comes with the following format: From Certificate ID & To Certificate ID (134567.....134599)
emissions_factor_direct - decimal - (REQUIRED) The carbon intensity of the generating asset in kg CO2 / kWh.

Would be great to discuss together if we fullfill all information/use cases and some of the listed fields.

"account_address": "123 Main St\nAnytown, TX 11111",
"account_type": "business",
"account_contacts": [
{

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would be better to add a reference to the next level of information? Meaning:

  1. I check the Accounts Endpoint -> I am able to identify the Service Contracts linked to this account
  2. I check the Service Contracts -> I am able to identify the Service Points linked to this contract
  3. I check the Service Point --> I am able to identify the meter devices linked to the service point.

Idk if I missed something. But I would go for a Top to bottom approach.

}
```

## Aggregation Endpoint <a id="aggregation-api" href="#aggregation-api" class="permalink">🔗</a>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing the aggregation endpoint in the index?

"format": ["kwh_net", "kwh_fwd", "kwh_rev"],
"values": [
[{"v": 1.2}, {"v": 1.2}, {"v": 0.0}],
[{"v": 1.1}, {"v": 1.1}, {"v": 0.0}],

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reckon we only account here for settlement/validated data. Nevertheless, depending on the utility, confirmed readings could come pretty delayed. Should we differentiate the type of reading? (e.g. estimated/confirmed?). In case readings need to be made available before the settlement period? (e.g. use cases for demand response)

Copy link

@eloifabrega eloifabrega Sep 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an example: Energinet API offers the following quantity.quality:

  • Not Available: Specifies that the grid operator has submitted a “missing indicator” to DataHub for the specific position, meaning that the energy quantity is not available. Therefore, no quantity will be returned for the specific
    position.

  • Estimated: Specifies that the grid operator has submitted the quantity to DataHub as an estimate

  • As provided: Specifies that the grid operator has submitted the quantity to DataHub as measured.

  • Incomplete: is applied to an aggregated energy quantity if at least one of the quantities included in the aggregation has been submitted to DataHub with a “missing indicator”

API data description can be found here: https://energinet.dk/media/4beb0pgg/customer-and-third-party-api-for-datahub-eloverblik-data-description_validfrom_20240626.pdf

* `rateplan_code`
* `rateplan_name`

Listings of Service Contract objects MUST be ordered in reverse chronological order by `cds_modified` timestamp, where the most recently updated relevant Service Contract MUST be first in each listing.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when we say relevant service contracts we refer to the ones active, for example? Or which other variables have an influence here on the order?

* `cds_usagesegment_id` - _string_ - (REQUIRED) The Server's unique identifier for this Usage Segment object.
* `cds_created` - _ISO8601 datetime_ - (REQUIRED) When the Server created this Usage Segment object.
* `cds_modified` - _ISO8601 datetime_ - (REQUIRED) When the Server last modified this Usage Segment object.
* `cds_aggregation_ids`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the descriptions of the following values missing? All of them will be REQUIRED, correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and what does format describe?

"contract_end": null,
"service_type": "electric",
"service_class": "commercial",
"rateplan_code": "COMM-1A",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would other rate plan modifiers like CARE and FERA be included here? Also, what about baseline territory?

"cds_created": "2024-01-01T00:00:00Z",
"cds_modified": "2024-01-01T00:00:00Z",
"meter_number": "M55555555",
"meter_type": "usage_forward_only",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this also include information like whether this is a submeter for EV charging, for example?

@daniel-utilityapi daniel-utilityapi changed the title [Specification] CDSC-WG1-03 - Customer Data [Specification] CDSC-WG3-01 - Customer Data Jul 16, 2024
"cds_created": "2024-01-01T00:00:00Z",
"cds_modified": "2024-01-01T00:00:00Z",
"eac_number": "AAAZZZ-123",
"source_id": "101010101-10aaa",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the source refering to here? is it the production device Unique ID?

{
"eacs": [
{
"cds_eac_id": "101010101-10",
Copy link

@Temporal247 Temporal247 Sep 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the EAC Unique ID from the registry or something else?

"101010101-10" is this just an example or does it fix the ID length to this number of figures, if so it may be too small

"related_servicepoints": [...],
"related_meterdevices": [...],
"related_billsections": [...],
"related_eacs": [...],

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this meant to be a string of EAC IDs for for the certificates that are retired for load matching claims?

"retired": "2024-02-01",
"start": "2024-01-01T00:00:00Z",
"end": "2024-02-01T00:00:00Z",
"unit": "kwh",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why in kWh and not in Wh? recommend Wh

Content-Type: application/json;charset=UTF-8

{
"eacs": [

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this EAC data based on a particular base standard or a specific EAC registry for example? EnergyTag has a standard list of attributes that we could use.

https://docs.google.com/spreadsheets/d/1aaNP2LLGvd2WW9iBqA5yjngRahrwnveB2t0vMFlXT7M/edit?gid=0#gid=0

and an API spec for hourly EAC registries that could also be helpful here:
https://energytag.org/api/#tag/Certificates/operation/certificate_bundle_cancellation_certificates_certificates_cancel_post

"asset_id": "AP111111",
"asset_origination": "grid",
"asset_destination": "grid",
"technology_type": "solar",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"asset_id": "AP111111",
"asset_origination": "grid",
"asset_destination": "grid",
"technology_type": "solar",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daniel-utilityapi daniel-utilityapi changed the title [Specification] CDSC-WG3-01 - Customer Data [Specification] CDS-WG3-01 - Customer Data May 28, 2025
@daniel-roesler
Copy link
Member

I am replacing this pull request with Pull Request #31, which is from the Maintainer branch on my new account.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants