Website  •  Documentation  •  Pypi

Welcome to dotflow

With Dotflow, you get a powerful and easy-to-use library designed to create execution pipelines without complication. Add tasks intuitively and control the entire process with just a few commands.

Our goal is to make task management faster and more secure, without overwhelming you with complexity. Simply instantiate the DotFlow class, add your tasks with the add method, and start execution with the start method.

Start with the basics here.

Table of Contents

Click to expand

• Getting Help
• Getting Started
  • Install
• A Simple Example
• First Steps
  • Import
  • Callback function
  • Task function
  • DotFlow Class
  • Add Task
  • Start
• CLI
  • Simple Start
  • With Initial Context
  • With Callback
  • With Mode
• More Examples
• Commit Style
• License

Getting Help

We use GitHub issues for tracking bugs and feature requests and have limited bandwidth to address them. If you need anything, I ask you to please follow our templates for opening issues or discussions.

• 🐛 Bug Report
• 📕 Documentation
• 🚀 Feature Request
• ⚠️ Security Issue
• 💬 General Question

Getting Started

Install

To install Dotflow, run the following command from the command line:

With Pip

pip install dotflow

With Poetry

poetry add dotflow

A Simple Example

The simplest file could look like this:

from dotflow import DotFlow, action

def my_callback(*args, **kwargs):
    print(args, kwargs)

@action
def my_task_x():
    print("task")

@action
def my_task_y():
    print("task")

workflow = DotFlow()

workflow.task.add(step=my_task_x, callback=my_callback)
workflow.task.add(step=my_task_y, callback=my_callback)

workflow.start()

First Steps

1. Import

Start with the basics, which is importing the necessary classes and methods. (DotFlow, action)

from dotflow import DotFlow, action

2. Callback function

Create a my_callback function to receive execution information of a task. It is not necessary to include this function, as you will still have a report at the end of the execution in the instantiated object of the DotFlow class. This my_callback function is only needed if you need to do something after the execution of the task, for example: sending a message to someone, making a phone call, or sending a letter. More details

def my_callback(*args, **kwargs):
    print(args, kwargs)

3. Task function

Now, create the function responsible for executing your task. It's very simple; just use the action decorator above the function, and that's it—you've created a task.

@action
def my_task_x():
    print("task")

4. DotFlow Class

Instantiate the DotFlow class in a workflow variable to be used in the following steps. More details.

workflow = DotFlow()

5. Add Task

Now, simply add the my_task_x and my_callback functions you created earlier to the workflow using the code below. This process is necessary to define which tasks will be executed and the order in which they will run. The execution order follows the sequence in which they were added to the workflow. More details

• Adding one step at a time:

workflow.task.add(step=my_task_x, callback=my_callback)
workflow.task.add(step=my_task_y, callback=my_callback)

• Adding multiple steps at the same time:

workflow.task.add(step=[my_task_x, my_task_y], callback=my_callback)

• Adding a step with the module path:

workflow.task.add(step="module.task.my_task_x", callback=my_callback)

6. Start

Finally, just execute the workflow with the following code snippet. More details

workflow.start()

CLI

Simple Start

dotflow start --step examples.cli_with_mode.simple_step

With Initial Context

dotflow start --step examples.cli_with_initial_context.simple_step --initial-context abc

With Callback

dotflow start --step examples.cli_with_callback.simple_step --callback examples.cli_with_callback.callback

With Mode

dotflow start --step examples.cli_with_mode.simple_step --mode sequential

dotflow start --step examples.cli_with_mode.simple_step --mode background

dotflow start --step examples.cli_with_mode.simple_step --mode parallel

Process Mode

Sequential

workflow.task.add(step=task_foo)
workflow.task.add(step=task_bar)

workflow.start()

Click to see diagram

flowchart TD
A[Start] -->|run| B
B[task_foo] -->|response to| C
C[task_bar] -->|response| D
D[Finish]

Loading

Background

workflow.task.add(step=task_foo)
workflow.task.add(step=task_bar)

workflow.start(mode="background")

Click to see diagram

flowchart TD
A[Start] -->|run| B
B[task_foo] -->|response to| C
C[task_bar] -->|response| D
D[Finish]

Loading

Parallel

workflow.task.add(step=task_a)
workflow.task.add(step=task_b)
workflow.task.add(step=task_c)
workflow.task.add(step=task_d)

workflow.start(mode="parallel")

Click to see diagram

flowchart TD
    S[Start] -->|run| A[task_a]
    S[Start] -->|run| B[task_b]
    S[Start] -->|run| C[task_c]
    S[Start] -->|run| D[task_d]
    A --> H[Finish]
    B --> H[Finish]
    C --> H[Finish]
    D --> H[Finish]

Loading

Parallel with Groups

workflow.task.add(step=task_foo, group_name="foo")
workflow.task.add(step=task_bar, group_name="bar")

workflow.start()

Click to see diagram

flowchart TD
    A[Start] -->|run| C(Parallel Groups)
    C -->|run| D[task_a]
    C -->|run| E[task_c]
    D -->|response| X[task_b]
    X --> H[Finish]
    E -->|response| Y[task_d]
    Y --> H[Finish]

Loading

More Examples

Example	Command
cli_with_callback	dotflow start --step examples.cli_with_callback.simple_step --callback examples.cli_with_callback.callback
cli_with_initial_context	dotflow start --step examples.cli_with_initial_context.simple_step --initial-context abc
cli_with_mode	dotflow start --step examples.cli_with_mode.simple_step --mode sequential
cli_with_output_context	dotflow start --step examples.cli_with_output_context.simple_step --storage file
cli_with_path	dotflow start --step examples.cli_with_path.simple_step --path .storage --storage file
flow	python examples/flow.py
simple_class_workflow	python examples/simple_class_workflow.py
simple_cli	dotflow start --step examples.simple_cli.simple_step
simple_function_workflow	python examples/simple_function_workflow.py
simple_function_workflow_with_error	python examples/simple_function_workflow_with_error.py
step_class_result_context	python examples/step_class_result_context.py
step_class_result_storage	python examples/step_class_result_storage.py
step_class_result_task	python examples/step_class_result_task.py
step_function_result_context	python examples/step_function_result_context.py
step_function_result_storage	python examples/step_function_result_storage.py
step_function_result_task	python examples/step_function_result_task.py
step_with_groups	python examples/step_with_groups.py
step_with_initial_context	python examples/step_with_initial_context.py
step_with_many_contexts	python examples/step_with_many_contexts.py
step_with_notify_telegram	python examples/step_with_notify_telegram.py
step_with_previous_context	python examples/step_with_previous_context.py
step_with_storage_file	python examples/step_with_storage_file.py
step_with_storage_mongodb	python examples/step_with_storage_mongodb.py
workflow_background_mode	python examples/workflow_background_mode.py
workflow_keep_going_true	python examples/workflow_keep_going_true.py
workflow_parallel_mode	python examples/workflow_parallel_mode.py
workflow_sequential_group_mode	python examples/workflow_sequential_group_mode.py
workflow_sequential_mode	python examples/workflow_sequential_mode.py
workflow_step_callback	python examples/workflow_step_callback.py
workflow_with_backoff	python examples/workflow_with_backoff.py
workflow_with_callback_failure	python examples/workflow_with_callback_failure.py
workflow_with_callback_success	python examples/workflow_with_callback_success.py
workflow_with_retry	python examples/workflow_with_retry.py
workflow_with_retry_delay	python examples/workflow_with_retry_delay.py
workflow_with_timeout	python examples/workflow_with_timeout.py

Commit Style

Icon	Type	Description
⚙️	FEATURE	New feature
📝	PEP8	Formatting fixes following PEP8
📌	ISSUE	Reference to issue
🪲	BUG	Bug fix
📘	DOCS	Documentation changes
📦	PyPI	PyPI releases
❤️️	TEST	Automated tests
⬆️	CI/CD	Changes in continuous integration/delivery
⚠️	SECURITY	Security improvements

License

This project is licensed under the terms of the MIT License.