v0.25.80

This release brings major improvements to AliECS, including extensions to the Apricot runtime KV API and to the VarSpec workflow template metadata mechanism for UI widgets.

• Apricot runtime KV API

  • [apricot] Add ListRuntimeEntries to runtime key-value API

• Workflow template metadata for UI

  • [core] Add index and visibleif to VarSpec
  • [core] Add enabledif trait to VarSpec
  • [core] Update VarSpec structure

v0.25.1

This patch release includes fixes for two critical issues discovered since v0.25.0.

• [core] Ensure teardown can proceed from DEPLOYED state
• [core] Fix error when parsing public info for env

v0.25.0

This release brings major improvements to AliECS, including a change to the environment state machine which now includes a DEPLOY transition and a DEPLOYED state before CONFIGURE/CONFIGURED, needed for pre-CONFIGURE memory management facilities. A new component, o2-aliecs-shmcleaner is also provided for this purpose. This release also includes the concept of public/non-public environments, a runtime extension of the previous concept of public/non-public workflows; as well as an overhaul of InfoLogger levels to improve the non-expert user experience.

CONFIGURATION CHANGE NOTICE: starting with AliECS v0.24.0, the default ports for Apricot, Apricot HTTP, and AliECS core are respectively 32101, 32188 and 32102. The old values in the 471xx range should not be used any more.

• State machine improvements

  • [core] New o2-aliecs-shmcleaner script + Makefile for it
  • [core] New DEPLOY transition in env state machine

• Public environments

  • [coconut] Add flags for public environments
  • [coconut] Add public info on env show command
  • [coconut] New environments are public by default
  • [core] Environment set public based on wf
  • [core] Server support public/non-public envs
  • [core] Return public value on NewEnvironment

• Logging

  • [core] Increase shmcleaner log message severity to INFO
  • [core] Silence obsolete messages about status update to unknown task
  • [core] Adjust log levels and improve messages
  • [executor] Adjust log levels and improve messages

v0.24.80

This release brings major improvements to AliECS, including a change to the environment state machine which now includes a DEPLOY transition and a DEPLOYED state before CONFIGURE/CONFIGURED, needed for pre-CONFIGURE memory management facilities. A new component, o2-aliecs-shmcleaner is also provided for this purpose.

CONFIGURATION CHANGE NOTICE: starting with AliECS v0.24.0, the default ports for Apricot, Apricot HTTP, and AliECS core are respectively 32101, 32188 and 32102. The old values in the 471xx range should not be used any more.

• State machine improvements
  • [core] New o2-aliecs-shmcleaner script + Makefile for it
  • [core] New DEPLOY transition in env state machine

v0.24.2

Thie patch release increases the control command timeout to 90s.

• [core] Increase Mesos command timeout

v0.24.1

This patch release includes fixes for some recent repository access issues, as well as improved logging for debugging the timing of integration client operations.

• [core] Wrap ODC client handlers with TimeTrack
• [repos] Update repo add with flag and docs
• [repos] Update repo default with docs
• [repos] Update repo remove with docs
• [repos] Update repo refresh with docs
• [repos] Update repo default-revision with docs
• [repos] Allow addition of a repo without a global default revision branch
• [repos] Always fetch an existing repo on addition
• [repos] Fix revisions not populated

v0.24.0

This release brings major improvements to AliECS, including an enhanced cache mechanism for the git-backed workflow configuration versioning system and the ability to configure inbound channels with a static bind port. It also brings many fixes to integration plugins, and general reliability improvements.

CONFIGURATION CHANGE NOTICE: starting with AliECS v0.24.0, the default ports for Apricot, Apricot HTTP, and AliECS core are respectively 32101, 32188 and 32102. The old values in the 471xx range should not be used any more.

• Workflow template system improvements:

  • [core] Support for target in bind block, for static inbound ch config
  • [repos] Strongly typed fetch error check
  • [repos] Cache workflow templates on repo refresh
  • [repos] isPublic and varInfo parsing with single file handle
  • [repos] Make sure ControlWorkflows repos are case insensitive
  • [repos] Fix order of default revision check on repo add
  • [repos] Use shell to run git checkout instead of go-git
  • [repos] Populate the template cache on demand
  • [repos] Repo shouldn't be used as a map key
  • [repos] Add option to get revisions for ListRepos
  • [repos] Add protos

• Integration:

  • [core] Propagate run number to ODC partition
  • [core] Add timeout to Partition state polling
  • [core] Get DD polling timeout from a workflow var
  • [core] Hook errors should not be fatal
  • [core] Init pendingEORS map for DCS plugin

• Miscellaneous:

  • [apricot] Use Error instead of Fatal unless ready to os.Exit(1)
  • [build] Remove o2-aliecs-odc-shim, the legacy ODC client implementation
  • [executor] Ensure sequential access to state.activeTasks
  • [apricot] Change ports
  • [build] Regenerate protofiles
  • [coconut] Change aliecs ports
  • [core] Change default ports

v0.23.80

This release brings major improvements to AliECS, including an enhanced cache mechanism for the git-backed workflow configuration versioning system and the ability to configure inbound channels with a static bind port. It also brings many fixes to integration plugins, and general reliability improvements.

• Workflow template system improvements:

  • [core] Support for target in bind block, for static inbound ch config
  • [repos] Strongly typed fetch error check
  • [repos] Cache workflow templates on repo refresh
  • [repos] isPublic and varInfo parsing with single file handle
  • [repos] Make sure ControlWorkflows repos are case insensitive
  • [repos] Fix order of default revision check on repo add
  • [repos] Use shell to run git checkout instead of go-git
  • [repos] Populate the template cache on demand
  • [repos] Repo shouldn't be used as a map key
  • [repos] Add option to get revisions for ListRepos
  • [repos] Add protos

• Integration:

  • [core] Propagate run number to ODC partition
  • [core] Add timeout to Partition state polling
  • [core] Get DD polling timeout from a workflow var
  • [core] Hook errors should not be fatal
  • [core] Init pendingEORS map for DCS plugin

• Miscellaneous:

  • [apricot] Use Error instead of Fatal unless ready to os.Exit(1)
  • [build] Remove o2-aliecs-odc-shim, the legacy ODC client implementation
  • [executor] Ensure sequential access to state.activeTasks

v0.23.1

This patch release includes improvements to hook status handling and integration call logging. It also adds PrefixedOverride and related functions to the template system for multi-detector subworkflow operation.

• [common] Improve time tracking facilities
• [core] Fix handling and logging of critical/non-critical hooks
• [core] Make ODC failures of ERROR severity
• [core] PrefixedOverride, ToUpper/ToLower in template system

v0.23.0

This release brings major improvements to AliECS, including a new trivial HTTP API in Apricot for shell scripts and users to access real time inventory information via HTTP (i.e. a plain curl call from anywhere), improvements and additions to the template system, and miscellaneous bug fixes. It also introduces workflow template hiding and dynamic input widget hints, used in the GUI to greatly improve workflow discoverability and usability. A new Integrated Services API is also provided to track the status of DCS, DD scheduler and ODC, and the plugin system itself now ensures that all integration plugins promptly reconnect in case of interruption. Finally, this release adds integration facilities for the O² Bookkeeping system.

• Apricot trivial configuration endpoint (returns inventory on http://:47188/inventory/flps):

  • [apricot] Apricot's new Http service is separated from the already existing service.
  • [apricot] First uncomplete attempt at creating a Web Api for Apricot.
  • [apricot] GetHostInventory returns an error variable as well as the hosts list.
  • [apricot] GetHostInventory is called when needed, but isn't returning a result.
  • [apricot] GetHostInventory is better optimised.
  • [apricot] Host inventory is retrieved by http service, and converted to json or plain text.
  • [apricot] Http service should 'gracefully' shutdown on exit signals.
  • [apricot] Protofile takes the new GetHostInventory function into account.
  • [apricot] Result of make generate after modifying the Protofile.
  • [apricot] Temporary fix for having a working make all. No solution found to call for Service inside of ServiceHttp.
  • [apricot] Temporary setup for tests.
  • [apricot] The Http Service is linked to Apricot and is set up to be able to handle the inventory.
  • [apricot] The HTTP Server is shut down using the Shutdown() function associated to it.
  • [apricot] The function GetHostInventory is 'registered' in the configuration service.
  • [apricot] The project builds after 'registering' the function GetHostInventory.
  • [apricot] The HTTP Server returns the flps in either text or json depending on the format requested.
  • [apricot] The HTTP Service is functional for FLPs and detector FLPs, for JSON and text format.
  • [apricot] The Http Service is working after rebase.
  • [apricot] The HTTP service can return FLPs associated to a requested detector.
  • [apricot] The HTTP Service returns keys only once when it is both a key and a key prefix.
  • [apricot] service.go doesn't have any unused imports.
  • [apricot] Tab indentation is now replaced by spaces.
  • [apricot] Fix blocking startup

• Workflow template hiding and dynamic input widget hints:

  • [coconut] Update templ list with --all-workflows flag
  • [core] Document VarSpec entries and make string the default type
  • [core] Set default varspec type to string
  • [protos] Add VarSpec message extending WorkflowTemplateInfo
  • [repos] Prefer join over manual concatenation
  • [repos] Filter invalid repo dirs
  • [repos] Only list public workflows by default
  • [repos] Add missing "tasks"
  • [repos] Use path instead of filePath for repo identifier
  • [repos] Care for missing trailing slash in repolist
  • [repos] Move convenience functions to repoutils.go
  • [repos] Add functions to parse public workflow variable info
  • [repos] Remove AllowedKeyValues from VarSpec
  • [repos] Fix case whoops
  • [server] Update default widget and vartype values
  • [server|coconut] Don't return variable information for coconut requests
  • [server] Update vartype enum converter
  • [server] Fix bool number in vartype enum
  • [server] Include public variable info in GetWorkflowTemplates

• Integrated Services API:

  • [coconut] Print Integrated Services information on coconut info
  • [core] Expand plugin system with access functions for API calls
  • [core] Integrated Services API call
  • [core] Document IntegratedServices API values

• Integration plugin improvements:

  • [core] Reconnect all plugins continuously
  • [core] Use keepalive with backoff for gRPC clients
  • [core] Regenerate protofiles

• Workflow template system improvements and fixes:

  • [core] Ensure all contexts get the environment_id variable
  • [core] Report a channel mismatch error on bad channel configuration
  • [core] Add task information in channel config generator error
  • [core] CreateEnvironment avoid overwrite error on failed deployment
  • [core] Ensure task class defaults are template-processed
  • [core] Support for global key in bind channel declaration
  • [core] Support global channel addressing in template system
  • [task] Consistent TaskClassIdentifier string construction

• Bookkeeping system integration:

  • [core] Repo getDefaultRevision on refresh
  • [core] Add Bookkeeping integration
  • [core] Use Bookkeeping API to CreateRun,CreateLog and UpdateRun
  • [core] Add CreateFLP from Bookkeeping API
  • [core] Create FLPs after the CreateRun
  • [core] Add config entry for Bookkeeping and fix format
  • [core] Retrieve JWT using the request redirect
  • [core] Fix bookkeeping calls and version
  • [core] Update bookkeeping version
  • [core] Fix wrong identifiers

• Miscellaneous:

  • [coconut] Keep stream open after Environment error
  • [common] Introduce runtype.RunType enum, matches DCS Protobuf RunType
  • [core] Support multivalued Mesos attributes split by comma
  • [core] Generate strings for system.ID enum values with enumer
  • [core] Report error of failed environment through stream
  • [core] Report teardown complete at the end of operation
  • [occ] Allow building with Protobuf v3.14.0