N5GEH CSV-Importer

• Use to integrate external measurement data into a FIWARE instance

  • Creation of devices and entities in the IoT agent
  • Creating subscriptions in the Contextbroker
  • Transfer of data to the CrateDB

• Usage

  • Example implemented in ./example/example.py
  • Use ENV-Variables:

        CB_URL = ""                 # URL of the context broker
        IOT_URL = ""                # URL of the IoT-Agent (JSON)
        FIWARE_SERVICE = ""         # Name of the FIWARE-Service
        FIWARE_SERVICE_PATH = ""    # Service-Path
        FIWARE_TOKEN = ""           # FIWARE-Token for Authentification with Baerer-Token
    
        CRATE_DB_URL = ""           # URL of the CrateDB
        CRATE_DB_USER = ""          # User for CrateDB (need read and write acces)
        CRATE_DB_PW = ""            # Password of the CrateDB User
    
        CONFIG_PATH = ""            # local Path for the config-file
    

  • Example for config in ./example/config.json with 5 parts:
    • basic: Information about the FIWARE service
    • cratedb: Information on importing data into a CrateDB - only data from one entity can be imported
      • The following configuration can be used to import data from multiple entities

        "entities": {
            "entity_1":{
                "colname_1":"attribute_name_1",
                "colname_2":"attribute_name_2"
            },
            "entity_2":{
                "colname_1":"attribute_name_1",
                "colname_2":"attribute_name_2"
            }
        }
        

      • Replaces this part from the default configuration - the entity type from device configuration is used

            "entity_type": "example",
            "entity_id": "example"
        

    • services: Configuration of the FIWARE service that is newly created for the import (Similar to the configuration for new devices)
    • devices: Devices that are newly created on the IoT agent (may differ from the data import into the CrateDB) (Similar to the configuration for new devices)
    • subscriptions: Subscription for the new devices (Similar to the configuration for new devices)

• Dependencies

  • The following libraries are required:
    • filip
    • crate
    • pandas
    • numpy
    • pydantic
  • Dependencies of filip must be respected
    • [pydantic] in Version 1.10 is recommended
  • Für eine Installation mit Poetry ist die pyproject.toml beigefügt