Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions documentation/api/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ These endpoints enable programmatic triggering and retrieval of forecasts via th

Also:

- Schedules are no longer saved upside down.
- Support saving the aggregate power schedule by referencing a power sensor in the ``flex-context`` (new field ``aggregate-power``).
- Added ``root`` and ``depth`` fields to the `/assets` (GET) endpoint for listing assets, to allow selecting descendants of a given root asset up to a given depth.
- Added ``fields`` field to the `/assets` (GET) and `/assets/public` endpoints, to transfer less data by default (will be fully active, i.e. also returning less fields per default, in v0.32).
Expand Down
9 changes: 6 additions & 3 deletions documentation/api/notation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,10 @@ In this case, FlexMeasures will convert the data using the resolution of the tim

Signs of power values
^^^^^^^^^^^^^^^^^^^^^
In general, FlexMeasures lets you store data as you want. Negative power values to indicate production, positive consumption - or the other way around.

USEF recommends to use positive power values to indicate consumption and negative values to indicate production, i.e.
to take the perspective of the Prosumer.
If an asset has been configured as a pure producer or pure consumer, the web service will help avoid mistakes by checking the sign of posted power values.
We'd recommend to use positive power values to indicate consumption and negative values to indicate production, i.e.
-to take the perspective of the Prosumer.

Read more at :ref:`signs_of_power_beliefs` about our treatment of data, which includes data you send in, or you get from forecasts and schedules
(hint: you are free to define the sign for your data, but it might affect how you receive your schedules).
16 changes: 15 additions & 1 deletion documentation/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,19 @@ FlexMeasures Changelog
v0.31.0 | February XX, 2026
============================

.. warning:: As of this version, power schedules will no longer appear flipped (in UI charts) when they are recorded on *consumption* sensors.
Please note that:

- Schedules obtained via the dedicated API endpoints for scheduling are not affected by this change!
- The ``consumption_is_positive`` sensor attribute governs the sign of schedules saved to the database.
The sign convention of UI charts is to simply match what is in the database.
- If you inadvertently make use of the `sensor data API endpoint <https://flexmeasures.readthedocs.io/v0.30.3/api/v3_0.html#get--api-v3_0-sensors-id-data>`_ to fetch *schedules*,
or have reporters that compute reports on *schedules*, rather than on *measurements*, be advised that you may experience flipped results.
You may need to adjust your client-side code or reporter configuration accordingly.
(Updating a reporter configuration automatically leads to a fresh data source for the saved time series.)
- Finally, if you maintain a plugin with a custom ``Scheduler`` class that returns time series to be saved on power sensors, we recommend incrementing the version of the ``Scheduler`` class.
This will yield a fresh data source for new schedules and allow you to discriminate your flipped schedules, which, for instance, will make it easier to flip back the historical schedules if you later want to.

.. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).

New features
Expand All @@ -24,7 +37,7 @@ New features
* Added ``root`` and ``depth`` fields to the `[GET] /assets` endpoint for listing assets, to allow selecting descendants of a given root asset up to a given depth [see `PR #1874 <https://www.github.com/FlexMeasures/flexmeasures/pull/1874>`_]
* Give ability to edit sensor timezone from the UI [see `PR #1900 <https://www.github.com/FlexMeasures/flexmeasures/pull/1900>`_]
* Support creating schedules with only information known prior to some time, now also via the CLI (the API already supported it) [see `PR #1871 <https://www.github.com/FlexMeasures/flexmeasures/pull/1871>`_].
* Added capability to upate an asset's parent from the UI [`PR #1957 <https://www.github.com/FlexMeasures/flexmeasures/pull/1957>`_]
* Added capability to update an asset's parent from the UI [`PR #1957 <https://www.github.com/FlexMeasures/flexmeasures/pull/1957>`_]
* Add ``fields`` param to the asset-listing endpoints, to save bandwidth in response data [see `PR #1884 <https://www.github.com/FlexMeasures/flexmeasures/pull/1884>`_]

.. note:: For backwards-compatibility, the new ``fields`` parameter will only be fully active, i.e. also returning less fields per default, in v0.32. Set ``FLEXMEASURES_API_SUNSET_ACTIVE=True`` to test the full effect now.
Expand Down Expand Up @@ -61,6 +74,7 @@ Infrastructure / Support

Bugfixes
-----------
* Fix: schedules are no longer saved upside down [see `PR #1348 <https://www.github.com/FlexMeasures/flexmeasures/pull/1348>`_]
* Fix: visiting the job page for any forecasting job led to a server error ("Object of type datetime is not JSON serializable"), by storing serialized kwargs in forecasting job meta [see `PR #1990 <https://www.github.com/FlexMeasures/flexmeasures/pull/1990>`_]
* Fix: flex-context dialogue is empty when flex-context has two booleans with the same value [see `PR #1907 <https://www.github.com/FlexMeasures/flexmeasures/pull/1907>`_]
* Bring back the ability to show (timed) annotations on the sensor page, and add some color highlighting, too, while we're at it [see `PR #1967 <https://www.github.com/FlexMeasures/flexmeasures/pull/1967>`_]
Expand Down
27 changes: 26 additions & 1 deletion documentation/concepts/data-model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ Asset belong to accounts (read more on accounts below).

Assets are often represented in other systems (e.g. IoT gateways / local EMS) with another ID. To link FlexMeasures' representation of assets with such external representations, you can store those IDs as `external_id`.

About asset types:
About asset types
^^^^^^^^^^^^^^^^^

We model asset types explicitly. None are required for running FlexMeasures.
Some asset types have support in the UI (for icons, like a sun for ``"solar"``), and in the toy tutorial and test.
Expand Down Expand Up @@ -97,6 +98,30 @@ Each belief links to a sensor and a data source. Here are two examples:
- A thermal demand sensor containing forecasts (data source of type "forecast", e.g. heating usage forecast sent to FlexMeasures or made by FlexMeasures) and measurements (sent into FlexMeasures, data source type "user").


.. _signs_of_power_beliefs:

About signs of power & energy values
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In short: You can use any sign you want for power data.
But the scheduler in FlexMeasures needs to know how to apply the signs. Positive (+) means consumption, negative (-) means production.
Let us explain.

When beliefs are about power or energy, the sign of the value is important. It indicates whether the asset is consuming or producing.
However, there is no universal standard for this. Some systems use positive values for production and negative values for consumption, while others do the opposite.

FlexMeasures doesn't enforce any perspective (we have a design philosophy of letting users model the system in their own way).

For example, users can create PV power data with positive values indicating production, and they can also create building power data with positive values indicating consumption.
We allow this because we want the UI to match what is in the database, and users often desire both of these datasets to be shown as positive values.
We assume that this is what users send in.

Note that, if forecasts are created, they will have the same sign as original data.

For schedules, the sign of resulting power data (beliefs) is being switched when data is stored (assuming consumption , and you can prevent that by setting ``sensor.attributes["consumption_is_positive"] = True``.


.. note:: We will soon document better what the scheduler does in detail, and how the attribute works.


Accounts & Users
----------------
Expand Down
6 changes: 3 additions & 3 deletions documentation/tut/scripts/run-tutorial2-in-docker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ docker exec -it flexmeasures-server-1 flexmeasures add schedule --sensor 2 \
--flex-context '{"inflexible-device-sensors": [3]}' \
--flex-model '{"soc-min": "50 kWh"}'

#echo "[TUTORIAL-RUNNER] showing schedule ..."
#docker exec -it flexmeasures-server-1 flexmeasures show beliefs --sensor 2 --start ${TOMORROW}T07:00:00+01:00 --duration PT12H
#
echo "[TUTORIAL-RUNNER] showing schedule ..."
docker exec -it flexmeasures-server-1 flexmeasures show beliefs --sensor 2 --start ${TOMORROW}T07:00:00+01:00 --duration PT12H

#echo ""
#echo "[TUTORIAL-RUNNER] DEMONSTRATING CUSTOM SCHEDULING RESOLUTION ..."
#echo "[TUTORIAL-RUNNER] The previous schedule used the sensor's native 15-minute resolution (PT15M)."
Expand Down
2 changes: 1 addition & 1 deletion documentation/tut/scripts/run-tutorial4-in-docker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ docker exec -it flexmeasures-server-1 flexmeasures add schedule --sensor 6 --sch
--flex-context '{"consumption-price": {"sensor": 1}}' \
--flex-model '{"duration": "PT4H", "process-type": "SHIFTABLE", "power": 0.2, "time-restrictions": [{"start": "'"${TOMORROW}"'T15:00:00+02:00", "duration": "PT1H"}]}'

echo "Now visit http://localhost:5000/assets/5/graphs to see all three schedules."
echo "Now visit http://localhost:5000/assets/6/graphs to see all three schedules."
2 changes: 1 addition & 1 deletion documentation/tut/toy-example-reporter.rst
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Finally, we can create the report with the following command:
--start-offset DB,1D --end-offset DB,2D \
--resolution PT15M

Now we can visualize the diminished headroom in the following `link <http://localhost:5000/sensors/8/graphs>`_, which should resemble the following image:
Now we can visualize the diminished headroom in the following `link <http://localhost:5000/sensors/8>`_, which should resemble the following image:

.. image:: https://github.com/FlexMeasures/screenshots/raw/main/tut/toy-schedule/sensor-data-headroom.png
:align: center
Expand Down
6 changes: 3 additions & 3 deletions flexmeasures/api/v3_0/sensors.py
Original file line number Diff line number Diff line change
Expand Up @@ -1038,12 +1038,12 @@ def get_schedule( # noqa: C901
)

sign = 1
if sensor.measures_power and sensor.get_attribute(
"consumption_is_positive", True
if sensor.measures_power and not sensor.get_attribute(
"consumption_is_positive", False
):
sign = -1

# For consumption schedules, positive values denote consumption. For the db, consumption is negative
# For consumption schedules, positive values denote consumption. For the db, consumption is negative unless specified explicitly
consumption_schedule = sign * simplify_index(power_values)["event_value"]
if consumption_schedule.empty:
# for not in-built schedulers, we are not sure if they would store time series in the db
Expand Down
11 changes: 0 additions & 11 deletions flexmeasures/data/models/planning/battery.py

This file was deleted.

11 changes: 0 additions & 11 deletions flexmeasures/data/models/planning/charging_station.py

This file was deleted.

2 changes: 1 addition & 1 deletion flexmeasures/data/models/planning/process.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

class ProcessScheduler(Scheduler):

__version__ = "1"
__version__ = "2"
__author__ = "Seita"

def compute(self) -> pd.Series | None:
Expand Down
4 changes: 2 additions & 2 deletions flexmeasures/data/models/planning/storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -1224,7 +1224,7 @@ def _ensure_variable_quantity(


class StorageFallbackScheduler(MetaStorageScheduler):
__version__ = "2"
__version__ = "3"
__author__ = "Seita"

def compute(self, skip_validation: bool = False) -> SchedulerOutputType:
Expand Down Expand Up @@ -1293,7 +1293,7 @@ def compute(self, skip_validation: bool = False) -> SchedulerOutputType:


class StorageScheduler(MetaStorageScheduler):
__version__ = "6"
__version__ = "7"
__author__ = "Seita"

fallback_scheduler_class: Type[Scheduler] = StorageFallbackScheduler
Expand Down
4 changes: 2 additions & 2 deletions flexmeasures/data/services/scheduling.py
Original file line number Diff line number Diff line change
Expand Up @@ -637,8 +637,8 @@ def make_schedule( # noqa: C901

sign = 1

if result["sensor"].measures_power and result["sensor"].get_attribute(
"consumption_is_positive", True
if result["sensor"].measures_power and not result["sensor"].get_attribute(
"consumption_is_positive", False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find an answer to my own question: If we follow the USEF default that consumption is positive, why does this attribute "consumption_is_positive" default to False now ??

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just read in a comment that we are " setting the attribute purposely incorrectly".

Can we plan a follow-up where these things are actually name and described properly? I am spending a lot of time now trying to make sense of it, but I couldn't even if we used it the correct way around.

Let's have a session where we make this user-friendly.

On a positive note, I ran all tutorials, and all schedules look exactly as they have been documented earlier.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting the default to save consumption as negative values in the db is mentioned in the original issue description, but I agree it is not thoroughly reasoned - instead, I only mentioned the new default would agree with some inline comment, telling us that consumption is negative in the db (which isn't strictly true anymore - even before this PR, and perhaps it would be better to state this is the default for the db, which only comes into play when the consumption_is_positive attribute is not explicitly set).

So why is that the default for the db? I believe that, historically, we started out with a grid perspective for our db, where production is adding energy to the grid (positive), and consumption is taking it away (negative), and we took a USEF perspective only for our API, which flipped the sign convention.

I am not entirely against changing the default for the db, given that FlexMeasures has transitioned into placing behind-the-meter optimization first. However, we would again have to go through various (field) tests to come up with the correct revision/migration that doesn't unexpectedly flip schedules for end users. As an alternative, I could imagine only changing the default for new power sensors, by defaulting the consumption_is_positive attribute for new sensors to True.

I hope this is valuable input for the session you request.

And really glad to hear the tutorials are unaffected, too!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I forgot tot check the issue.
Good to learnore about the history.
But this seems now to be a conceptual hurdle for users. I hope we find a pathway that creates clarity.

):
sign = -1

Expand Down