Merge branch 'master' into feature/memCache

Author: jason-fox

.github/workflows/ci.yml

@@ -6,6 +6,7 @@ name: CI
   pull_request:
     branches:
       - master
+  workflow_dispatch:
 jobs:
   lint-markdown:
     name: Lint Markdown
@@ -91,3 +92,18 @@ jobs:
         uses: coverallsapp/github-action@master
         with:
           github-token: ${{ secrets.GITHUB_TOKEN }}
+
+  dispatch:
+    needs: unit-test
+    if: ${{ github.ref == 'refs/heads/master'  && github.event_name == 'push' }}
+    strategy:
+      matrix:
+        repo: ['telefonicaid/iotagent-ul', 'telefonicaid/sigfox-iotagent', 'telefonicaid/iotagent-json', 'telefonicaid/lightweightm2m-iotagent', 'telefonicaid/iotagent-manager']
+    runs-on: ubuntu-latest
+    steps:
+      - name: Repository Dispatch
+        uses: peter-evans/repository-dispatch@v1
+        with:
+          token: ${{ secrets.REPO_ACCESS_TOKEN }}
+          repository: ${{ matrix.repo }}
+          event-type: lib-update

.readthedocs.yml

@@ -0,0 +1,7 @@
+version: 2
+
+mkdocs:
+  configuration: mkdocs.yml
+
+python:
+  version: 3.6

CHANGES_NEXT_RELEASE

@@ -0,0 +1,4 @@
+- Remove: NGSI-v1 implementation (#966)
+- FIx: export fillDomains function to iotagent-node-lib available functions (needed to fix logs in iotagent-json#587 and iotagent-ul#508)
+- Fix: keep internalAttributes stored in device mongo collection when update it (#1092)
+- Add: new JEXL get TimeZone Offset (valid for summer winter schedulles) and binary shifting for bitwise operator

doc/advanced-topics.md

@@ -1,36 +1,36 @@
 ## Advanced Topics
 
-* [Secured access to the Context Broker](#secured-access-to-the-context-broker)
-* [GeoJSON support this only applies for NGSI-LD, not for NGSI-v1 and NGSI-v2](#geojson-support-this-only-applies-for-ngsi-ld-not-for-ngsi-v1-and-ngsi-v2)
-* [Metadata support](#metadata-support)
-    * [NGSI LD data and metadata considerations](#ngsi-ld-data-and-metadata-considerations)
-* [NGSI-LD Linked Data support](#ngsi-ld-linked-data-support)
-* [Autoprovision configuration (autoprovision)](#autoprovision-configuration-autoprovision)
-* [Explicitly defined attributes (explicitAttrs)](#explicitly-defined-attributes-explicitattrs)
-* [Configuring operation to persist the data in Context Broker (appendMode)](#configuring-operation-to-persist-the-data-in-context-broker-appendmode)
-* [Data mapping plugins](#data-mapping-plugins)
-    * [Development](#development)
-    * [Provided plugins](#provided-plugins)
-        * [Timestamp Compression plugin (compressTimestamp)](#timestamp-compression-plugin-compresstimestamp)
-        * [Attribute Alias plugin (attributeAlias)](#attribute-alias-plugin-attributealias)
-        * [Event plugin (addEvents)](#event-plugin-addevents)
-        * [Timestamp Processing Plugin (timestampProcess)](#timestamp-processing-plugin-timestampprocess)
-        * [Expression Translation plugin (expressionTransformation)](#expression-translation-plugin-expressiontransformation)
-        * [Multientity plugin (multiEntity)](#multientity-plugin-multientity)
-        * [Bidirectionality plugin (bidirectional)](#bidirectionality-plugin-bidirectional)
-    * [Autoprovision configuration (autoprovision)](#autoprovision-configuration-autoprovision)
-    * [Explicitly defined attributes (explicitAttrs)](#explicitly-defined-attributes-explicitattrs)
-    * [Configuring operation to persist the data in Context Broker (appendMode)](#configuring-operation-to-persist-the-data-in-context-broker-appendmode)
-* [Old IoTAgent data migration](#old-iotagent-data-migration)
-
+-   [Secured access to the Context Broker](#secured-access-to-the-context-broker)
+-   [GeoJSON support NGSI-LD only](#geojson-support-ngsi-ld-only)
+-   [Metadata support](#metadata-support)
+    -   [NGSI LD data and metadata considerations](#ngsi-ld-data-and-metadata-considerations)
+-   [NGSI-LD Linked Data support](#ngsi-ld-linked-data-support)
+-   [Autoprovision configuration (autoprovision)](#autoprovision-configuration-autoprovision)
+-   [Explicitly defined attributes (explicitAttrs)](#explicitly-defined-attributes-explicitattrs)
+-   [Configuring operation to persist the data in Context Broker (appendMode)](#configuring-operation-to-persist-the-data-in-context-broker-appendmode)
+-   [Data mapping plugins](#data-mapping-plugins)
+    -   [Development](#development)
+    -   [Provided plugins](#provided-plugins)
+        -   [Timestamp Compression plugin (compressTimestamp)](#timestamp-compression-plugin-compresstimestamp)
+        -   [Attribute Alias plugin (attributeAlias)](#attribute-alias-plugin-attributealias)
+        -   [Event plugin (addEvents)](#event-plugin-addevents)
+        -   [Timestamp Processing Plugin (timestampProcess)](#timestamp-processing-plugin-timestampprocess)
+        -   [Expression Translation plugin (expressionTransformation)](#expression-translation-plugin-expressiontransformation)
+        -   [Multientity plugin (multiEntity)](#multientity-plugin-multientity)
+        -   [Bidirectionality plugin (bidirectional)](#bidirectionality-plugin-bidirectional)
+    -   [Autoprovision configuration (autoprovision)](#autoprovision-configuration-autoprovision)
+    -   [Explicitly defined attributes (explicitAttrs)](#explicitly-defined-attributes-explicitattrs)
+    -   [Configuring operation to persist the data in Context Broker (appendMode)](#configuring-operation-to-persist-the-data-in-context-broker-appendmode)
+-   [Old IoTAgent data migration](#old-iotagent-data-migration)
 
 ### Secured access to the Context Broker
 
 For access to instances of the Context Broker secured with a
 [PEP Proxy](https://github.com/telefonicaid/fiware-orion-pep), an authentication mechanism based in Keystone Trust
-tokens is provided. A Trust token is a long-term token that can be issued by any user to give another user permissions
-to impersonate him with a given role in a given project. Such impersonation itself is in turn based on a short-term
-access token.
+tokens is provided. A trust token is a way of Keystone to allow an user delegates a role to another user for a
+subservice. It is a long-term token that can be issued by any user to give another user permissions to impersonate him
+with a given role in a given project (subservice). Such impersonation itself is in turn based on a short-term access
+token.
 
 For the authentication mechanisms to work, the `authentication` attribute in the configuration has to be fully
 configured, and the `authentication.enabled` subattribute should have the value `true`.
@@ -70,7 +70,12 @@ curl http://${KEYSTONE_HOST}/v3/OS-TRUST/trusts \
 Apart from the generation of the trust, the use of secured Context Brokers should be transparent to the user of the IoT
 Agent.
 
-### GeoJSON support (this only applies for NGSI-LD, not for NGSI-v1 and NGSI-v2)
+Complete info on Keystone trust tokens could be found at:
+
+-   [Trusts concept](https://docs.openstack.org/keystone/stein/user/trusts)
+-   [Trusts API](https://docs.openstack.org/keystone/stein/api_curl_examples.html#post-v3-os-trust-trusts)
+
+### GeoJSON support NGSI-LD only
 
 The defined `type` of any GeoJSON attribute can be any set to any of the standard NGSI-v2 GeoJSON types - (e.g.
 `geo:json`, `geo:point`). NGSI-LD formats such as `GeoProperty`, `Point` and `LineString` are also accepted `type`
@@ -263,26 +268,28 @@ updated as shown:
 
 ### Autoprovision configuration (autoprovision)
 
-By default, when a measure arrives to the IoTAgent, if the `device_id` does not match with an existing one, then, the IoTA 
-creates a new device and a new entity according to the group config. Defining the field `autoprovision` to `false` 
-when provisioning the device group, the IoTA to reject the measure at the southbound, allowing only to persist the 
-data to devices that are already provisioned. It makes no sense to use this field in device provisioning since it is 
-intended to avoid provisioning devices (and for it to be effective, it would have to be provisional).
+By default, when a measure arrives to the IoTAgent, if the `device_id` does not match with an existing one, then, the
+IoTA creates a new device and a new entity according to the group config. Defining the field `autoprovision` to `false`
+when provisioning the device group, the IoTA to reject the measure at the southbound, allowing only to persist the data
+to devices that are already provisioned. It makes no sense to use this field in device provisioning since it is intended
+to avoid provisioning devices (and for it to be effective, it would have to be provisional).
 
 ### Explicitly defined attributes (explicitAttrs)
 
-If a given measure element (object_id) is not defined in the mappings of the device or group provision, the measure is stored 
-in the Context Broker by adding a new attribute to the entity with the same name of the undefined measure element. By adding the
-field `explicitAttrs` with `true` value to device or group provision, the IoTAgent rejects the measure elements that are not defined 
-in the mappings of device or group provision, persisting only the one defined in the mappings of the provision. If `explicitAttrs`
-is provided both at device and group level, the device level takes precedence.
+If a given measure element (object_id) is not defined in the mappings of the device or group provision, the measure is
+stored in the Context Broker by adding a new attribute to the entity with the same name of the undefined measure
+element. By adding the field `explicitAttrs` with `true` value to device or group provision, the IoTAgent rejects the
+measure elements that are not defined in the mappings of device or group provision, persisting only the one defined in
+the mappings of the provision. If `explicitAttrs` is provided both at device and group level, the device level takes
+precedence.
 
 ### Configuring operation to persist the data in Context Broker (appendMode)
 
-This is a flag that can be enabled by activating the parameter `appendMode` in the configuration file or by using the `IOTA_APPEND_MODE`
-environment variable (more info [here](https://github.com/telefonicaid/iotagent-node-lib/blob/master/doc/installationguide.md)). 
-If this flag is activated, the update requests to the Context Broker will be performed always with APPEND type, instead of the 
-default UPDATE. This have implications in the use of attributes with Context Providers, so this flag should be used with care.
+This is a flag that can be enabled by activating the parameter `appendMode` in the configuration file or by using the
+`IOTA_APPEND_MODE` environment variable (more info
+[here](https://github.com/telefonicaid/iotagent-node-lib/blob/master/doc/installationguide.md)). If this flag is
+activated, the update requests to the Context Broker will be performed always with APPEND type, instead of the default
+UPDATE. This have implications in the use of attributes with Context Providers, so this flag should be used with care.
 
 ### Data mapping plugins
 
@@ -367,8 +374,8 @@ events in the IoT Agent with the configured type name will be marked as events.
 
 ##### Timestamp Processing Plugin (timestampProcess)
 
-This plugin processes the entity attributes looking for a `TimeInstant` attribute. If one is found, for NGSI-v1/NGSIv2,
-the plugin adds a `TimeInstant` attribute as metadata for every other attribute in the same request. With NGSI-LD, the
+This plugin processes the entity attributes looking for a `TimeInstant` attribute. If one is found, for NGSIv2, the
+plugin adds a `TimeInstant` attribute as metadata for every other attribute in the same request. With NGSI-LD, the
 Standard `observedAt` property-of-a-property is used instead.
 
 ##### Expression Translation plugin (expressionTransformation)
@@ -432,7 +439,8 @@ When a device is provisioned with bidirectional attributes, the IoTAgent subscri
 change notification for that attribute arrives to the IoTA, it applies the transformation defined in the device
 provisioning payload to the notification, and calls the underlying notification handler with the transformed entity.
 
-The following `attributes` section shows an example of the plugin configuration (using IOTA_AUTOCAST=false to avoid translation from geo:point to geo:json)
+The following `attributes` section shows an example of the plugin configuration (using IOTA_AUTOCAST=false to avoid
+translation from geo:point to geo:json)
 
 ```json
       "attributes": [