Skip to content

Releases: cubewise-code/tm1py

2.2

18 Oct 20:03
@MariusWirtz MariusWirtz
2.2
85e6e8a

Choose a tag to compare

View all tags
2.2 Latest
Latest

What's Changed

  • Added re_connect_on_remote_disconnect parameter to TM1Service to mitigate rare PAoC disconnect error by @nicolasbisurgi in #1259
  • Added all_procedures property to Process that is convenient for TI code analysis by @bdunleavy22 in #1244
  • Update TM1Service to allow use of user or username argument by @bdunleavy22 in #1252
  • Added hierarchy_sort_order argument to hierarchy update function by @MariusWirtz in #1255
  • Added reorder_with_priority function to sort hierarchies in dimension by @wimgielis in #1260
  • Added retry_on_disconnect argument to specify if a process can be safely retried on connection/networking errors by @nicolasbisurgi in #1262
  • Added support for JSON data sources in TI processes by @bdunleavy22 in #1265
  • Fixed get_elements_dataframe issue when attribute has the same name as dimension by @cubewise-gng in #1263
  • Added argument delete_orphaned_consolidations to delete orphaned consolidations after dimension update by @Kevin-Dekker in #1238
  • Added badges to readme by @onefloid and @bdunleavy22 in #1273 and #1274
  • Added a new update_static_elements function that provides a more flexible way to update elements in static subsets by @bdunleavy22 in #1275
  • Introduced auto-formatter Black by @onefloid in #1277
  • Fixed bug in leaves_only mode in get_descendants function by @Cubewise-JoeCHK in #1284
  • Made response verification optional by @bdunleavy22 in #1281
  • Introduced linter by @onefloid in #1280
  • Fixed incorrect sandboxing_disabled property that prevented TM1py from writing to sandboxes by @MariusWirtz in #1287
  • Fixed minor compatibility issues with TM1 v12 #1292
  • Fixed updates to the task list in chores #1235

New Contributors

Full Changelog: 2.1...2.2

Contributors

MariusWirtz, wimgielis, and 6 other contributors
Assets 2
Loading
1 Join discussion

2.1

25 Feb 10:13
@MariusWirtz MariusWirtz
2.1
f9e3e5e

Choose a tag to compare

View all tags
2.1

What's Changed

New Contributors

Full Changelog: 2.0...2.1

Contributors

macsir, ivankulman, and 16 other contributors
Loading
0 Join discussion

2.0

31 Jan 17:40
@MariusWirtz MariusWirtz
2.0
e3bc795

Choose a tag to compare

View all tags
2.0

Highlights

1. Support for TM1 v12

TM1py now works with TM1 v12.

with TM1Service(
        address="us-east-2.aws.planninganalytics.ibm.com",
        api_key="AB4VfG7T8wPM-912uFKeYG5PGh0XbS80MVBAt7SEG6xn",
        iam_url="https://iam.cloud.ibm.com/identity/token",
        tenant="YA9A2T8BS2ZU",
        database="Database") as tm1:
    print(tm1.server.get_product_Version())

by @rclapp in #1000

2. CRUD operations to manage servers on v12

with ManageService(domain=domain, root_client=root_client, root_secret=root_secret) as manager:
   manager.create_database(instance_name="instance name",
                          database_name="database_name",
                          product_version="12.0.0",
                          number_replicas=1,
                          cpu_requests="1000m",
                          cpu_limits="2000m",
                          memory_limits="2G",
                          memory_requests="1G",
                          storage_size="20Gi")

   manager.scale_database(instance_name="instance name", database_name="database name", replicas=2)

   manager.create_database_backup(instance_name="instance name", 
                                 database_name="database name", 
                                 backup_set_name="my backup")

by @rclapp

3. Asynchronous execute_mdx functions

Speed up your MDX executions by assigning more than one worker-thread in TM1.

mdx = """
SELECT
{TM1SubsetAll([Big Dimension])} ON ROWS,
{TM1SubsetAll([Small Dimension])} ON COLUMNS
FROM [Big Cube]
"""

with TM1Service(**tm1params) as tm1:
    cells = tm1.cells.execute_mdx(
        mdx=mdx,
        # leverage 4 worker threads in TM1 for the extraction
        max_workers=4,
        # parallelization on rows axis
        async_axis=1)

by @vmitsenko in #935 and #1030

4. Hierarchy updates from data frames

Create and update TM1 hierarchies directly from pandas dataframes.

Stores ElementType Alias:a City:s Square Footage:n level001 level000 level001_weight level000_weight
S151 Numeric Boardwalk Games New York City 120 USA World 1 1
S143 Numeric Strategy Vault Zurich 250 Switzerland World 1 1
S811 Numeric Cardboard Castle Sydney 80 Sydney World 1 1 
columns = ["Stores", "ElementType", "Alias:a", "City:s", "Square Footage:n", "level001",
           "level000", "level001_weight", "level000_weight"]
data = [
    ['S151', "Numeric", "Boardwalk Games", "New York City", 120, "USA", "World", 1, 1],
    ['S143', 'Numeric', "Strategy Vault", "Zurich", 250, "Switzerland", "World", 1, 1],
    ['S811', 'Numeric', "Cardboard Castle", "Sydney", 80, "Sydney", "World", 1, 1],
]

with TM1Service(**tm1params) as tm1:
    tm1.hierarchies.update_or_create_hierarchy_from_dataframe(
        dimension_name="Stores",
        hierarchy_name="Stores",
        df=DataFrame(data=data, columns=columns)
    )
image

by @MariusWirtz in #944 and #1011

New Features, Improvements, and Bugfixes

Stats

  • Contributors: 15
  • Commits: 178
  • Changed Files: 72
  • Added Lines: 7,205
  • Deleted Lines: 1,643

How to upgrade TM1py

pip install tm1py --upgrade

New Contributors

Full Changelog: 1.11.1...2.0

Contributors

raeldor, hermie64, and 11 other contributors
Loading
0 Join discussion

1.11.3

23 May 09:29
@MariusWirtz MariusWirtz
1.11.3
7a0c30d

Choose a tag to compare

View all tags
1.11.3

Fixes

  • Add proxies arg to TM1Service #911
  • Show parameters on decorated functions in PyCharm e.g. execute_mdx_dataframe #913
  • Cater to breaking in change in urllib3 2.0 release #918
Loading

1.11.1

19 Apr 07:24
@MariusWirtz MariusWirtz
1.11.1
d1db539

Choose a tag to compare

View all tags
1.11.1

What's Changed

Full Changelog: 1.11...1.11.1

Contributors

macsir and MariusWirtz
Loading

1.11

14 Apr 14:14
@MariusWirtz MariusWirtz
1.11
f9d374b

Choose a tag to compare

View all tags
1.11

Highlights

1. Improved performance #882, #885

With this release TM1py makes better use of existing capabilities in the TM1 server to handle large read and write operations.

The write function

Pass use_blob=True to the write or write_dataframe function to use the new optimized write mode.

with TM1Service(**params) as tm1:
    cells = {
        ('Actual', 'Germany', 'T Series 4.0 L Sedan', 'Units', 'Jan'): 1500,
        ('Actual', 'Germany', 'T Series 4.0 L Sedan', 'Units', 'Feb'): 2100,
        ('Actual', 'Germany', 'T Series 4.0 L Sedan', 'Units', 'Mar'): 1100
    }
    tm1.cells.write("SalesCube", cells, use_blob=True)

On a sample of 1 million cell updates, this performs on par with Turbo Integrator and up to 6x faster than the previous write function. #882 (comment)

The write_async function

The write_async function now makes use of use_blob by default. The number of parallel worker threads in TM1 can be controlled through the max_workers argument. A reasonable value for slice_size needs to be provided.

E.g. if you are expecting 1 million cell updates, a reasonable choice would be 10 for max_workers and 100000 for slice_size.

with TM1Service(**params) as tm1:
    tm1.cells.write_async(
        cube_name="SalesCube",
        cells=cells,
        slice_size=100_000,
        max_workers=10)

On a sample of 1 million cell updates, this performs ~5 times better than plain Turbo Integrator and ~5 times better than the previous write_async function. #882 (comment)

The execute_mdx_dataframe, execute_view_dataframe functions

Pass use_blob=True to the execute_mdx_dataframe, and execute_view_dataframe functions to use the new optimized read mode.

with TM1Service(**sdata_params) as tm1:
    df = tm1.cells.execute_view_dataframe(cube_name="Sales Cube", view_name="Default", private=False, use_blob=True)

On large data sets this performs 20% to 40% better. It also reduces the memory footprint by ~70%. #885 (comment)

2. New shaped argument in the execute_view_dataframe function #893

Pass shaped=True to the execute_mdx_dataframe, and execute_view_dataframe functions to retrieve the data frame in the original shape of the cube view.

image

with TM1Service(**sdata_params) as tm1:
    df = tm1.cells.execute_view_dataframe(
        cube_name="Sales Cube",
        view_name="Default",
        shaped=True,
        use_blob=True)
account1 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
0 Gross Margin% 57.5155 56.4074 59.1279 59.5385 57.7353 57.1263 58.4302 59.7887 56.9305 56.9046 59.8059 56.2537
1 Price 19360 19532 19218.5 19631 19941 19870 19365 19216 19241 19099.5 19883.5 19432.5
2 Units 1 2 2 1 1 1 1 1 1 2 1 1
3 Sales 19.36 39.064 38.437 19.631 19.941 19.87 19.365 19.216 19.241 38.199 19.8835 19.4325
4 Variable Costs 8.225 17.029 15.71 7.943 8.428 8.519 8.05 7.727 8.287 16.462 7.992 8.501

Please note the months on the columns in the table above

3. Re-authenticate after session timeout #856

Now TM1py will re-authenticate and retry the operation once if the operation failed due to a session timeout.

This is helpful in situations where a TM1py script has been idle for a while and once it attempts to restart interaction with TM1 it fails due to a session timeout.

New Features, Improvements, and Bugfixes

  • Fully support TM1 sandbox functionality by @adscheevel in #844
  • Support use of alternate hierarchies in get_elements_dataframe function by @MariusWirtz in #876
  • Allow dtype to be inferred by pandas with argument infer_dtype in execute_mdx_dataframe_shaped by @Kevin-Dekker in #879

Acknowledgments

Big thanks to @cwffonseca, @Kevin-Dekker, and @adscheevel for contributing code to this release, and many others for reporting bugs and requesting new features.

How to upgrade TM1py

pip install tm1py --upgrade

Full Changelog: 1.10.2...1.11

Contributors

MariusWirtz, adscheevel, and 2 other contributors
Loading
0 Join discussion

1.10.2

19 Jan 10:53
@MariusWirtz MariusWirtz
1.10.2
a5ffae6

Choose a tag to compare

View all tags
1.10.2

What's Changed

New Contributors

Full Changelog: 1.10.1...1.10.2

Contributors

MariusWirtz, meyersrl, and 3 other contributors
Loading

1.10.1

24 Oct 10:15
@MariusWirtz MariusWirtz
1.10.1
c216936

Choose a tag to compare

View all tags
1.10.1

Fixes issue #819 that was introduced with release 1.10

What's Changed

New Contributors

Full Changelog: 1.10.0...1.10.1

Contributors

MariusWirtz, cubewise-gng, and tobiaskapser
Loading

1.10.0

10 Oct 14:00
@MariusWirtz MariusWirtz
1.10.0
3c88516

Choose a tag to compare

View all tags
1.10.0

Highlights

Performance improvements on all major TM1 I/O functions.

from TM1py import TM1Service

with TM1Service(address="", port=12354, ssl=True, user="admin", password="") as tm1:

    df = tm1.cells.execute_view_dataframe(cube_name="Sales", view_name="Default", private=False)
Time State SalesMeasure Value
0 202001 CA Gross Margin 13924.8
1 202001 CA Revenue 41330.4
2 202001 CA COGS 27405.6 
from TM1py import TM1Service

with TM1Service(base_url="https://localhost:12354", user="admin", password="") as tm1:

    cells = {
        ('Belgium', 'Actual', '2022', 'Apr', 'Revenue'): 10_000,
        ('Belgium', 'Actual', '2022', 'May', 'Revenue'): 15_000,
        ...
        ('Belgium', 'Actual', '2022', 'Jun', 'Revenue'): 20_000,
        ('Belgium', 'Actual', '2022', 'Apr', 'Revenue'): 45_000,
    }

    tm1.cells.write_async(cube_name="Sales", cells=cells, slice_size=32_000, max_workers=4,
                          measure_dimension_elements={'Revenue': 'Numeric'})

Full support for TM1's git deployment functionality. Including the TM1Project and deployment definitions.

with TM1Service(address="", port=11247, ssl=True, user="admin", password="") as tm1:

    project = TM1Project(name="Project Definition")

    dev_deployment = TM1ProjectDeployment(
        deployment_name="Dev",
        settings={"ServerName": "dev"})

    dev_deployment.add_task(TM1ProjectTask(
        task_name="Security Refresh",
        process="Bedrock.Security.Refresh"))

    dev_deployment.include_all_attribute_dimensions(tm1)
    dev_deployment.add_ignore(object_class="Cubes", object_name="*")

    project.add_deployment(deployment=dev_deployment)

    tm1.git.tm1project_put(project)

New and more efficient ways to query and control elements and hierarchies.

from TM1py import TM1Service

with TM1Service(base_url="https://localhost:12354", user="admin", password="") as tm1:

    # break edge as one tiny atomic operation
    tm1.elements.remove_edge(dimension_name="Region", hierarchy_name="Region", parent="EU", component="UK")

    # add new edge as one tiny atomic operation
    tm1.elements.add_edges(dimension_name="Region", hierarchy_name="Region", edges={("Other", "UK"): 1})
from TM1py import TM1Service

with TM1Service(base_url="https://localhost:12354", user="admin", password="") as tm1:

    is_parent = tm1.elements.element_is_parent(dimension_name="Region", hierarchy_name="Region", parent_name="Other",
                                               element_name="UK")

    is_ancestor = tm1.elements.element_is_ancestor(dimension_name="Region", hierarchy_name="Region",
                                                   element_name="Other", ancestor_name="UK")

Heaps of miscellaneous features, optimizations and improvements that make your life easier when doing TM1 with Python.

from TM1py import TM1Service

with TM1Service(base_url="https://localhost:12354", user="admin", password="") as tm1:

    process_names = tm1.processes.search_string_in_code(search_string="Sunrise", skip_control_processes=True)
from TM1py import TM1Service

with TM1Service(base_url="https://localhost:12354", user="admin", password="apple") as tm1:

    tm1.cubes.cube_save_data("Sales")

New Features and Improvements

  • Fix in set_time function in ChoreStartTime class to allow 0 values by @samuelko123 in #686
  • Add substitute_title function to MDXView by @MariusWirtz in #687
  • Add include_headers argument to extract_cellset_csv function by to @MariusWirtz in #689
  • Add remove_edge function to ElementService to break individual parent-child relationship in one operation by @MariusWirtz in #693
  • Fix bug to allow dimension creation from JSON file by @MariusWirtz in #698
  • Improve performance of critical build_cellset_from_pandas_dataframe function by @Kevin-Dekker in #695
  • Add get_parents function to ElementService. Allows retrieval of all parents for an element by @MariusWirtz in #699
  • Fix doubled double quotes issue in element names by @MariusWirtz in #704
  • Explicitly add elements to leaves hierarchy so solve #702 by @MariusWirtz in #703
  • Handle duplicates in write_dataframe appropriately by @MariusWirtz in #708 & #712
  • Accept str for rules update on Cube object by @MariusWirtz in #710
  • New search function to find substrings in Rules or Processes (e.g., search_string_in_code) by @adscheevel in #723
  • Fix write failure for element names with : by @MariusWirtz in #724
  • Add get_element_principal_name function by @MaaYuu in #731
  • Add get_ancestors, get_descendants functions on Hierarchy by @MariusWirtz in #732
  • Read 'NA' element name as a string instead of pandas NaN by @MariusWirtz in #739
  • Align float numbers intelligently when writing through unbound processes, to avoid "Number too big" TI errors during writeback, by @pbuncik in #749
  • Add functions to find views that contain a specified subset by @adscheevel in #751
  • Improve write_async performance by exposing measure_dimension_elements to write_async function by @MariusWirtz in #753
  • Introduce get_descendant_edges function in Hierarchy class by @skriptmeister42 in #760
  • Fix the update function in ApplicationService to use the update operation instead of the delete and recreate approach, to avoid breaking existing references of the application by @jrobinsonLOR in #762
  • Implement element_is_parent and element_is_ancestor by @rclapp in #767 and #771
  • Allow queries with selection on more than 3 axes in the execute_mdx function by @MariusWirtz in #777
  • Add support for trace_cell_calculation, trace_cell_feeders, and check_cell_feeders by @rclapp in #780
  • Add function create_many in AnnotationService to create many annotations in one operation by @MariusWirtz in #785
  • Add new element functions like get_consolidated_elements, get_parents_of_all_elements by @adscheevel in #792
  • Adjust TM1py to changes in mdxpy 0.4 by @MariusWirtz in #793
  • Handle TM1Project in the GitService by @nicolasbisurgi in #775 and #796
  • Refactor Breakpoints and introduce new debugging functionality by @adscheevel in #791
  • Allow alternative separators for elements in get_value and other functions by @tobiaskapser in #801 and #805
  • Use 100k max statements in write function with use_ti=True by @MariusWirtz in #808
  • Enable TCP keepalive option for long run requests by @macsir in #807
  • Additional features in ServerService : CubeSaveData and DeleteAllPersistentFeeders by @Mr-SabyasachiBose in #810

New Contributors

How to upgrade TM1py

To upgrade TM1py, just use the following command:

pip install TM1py --upgrade

Full Changelog: 1.9.0...1.10.0

Contributors

macsir, MariusWirtz, and 11 other contributors
Loading
4 Join discussion

1.9.0

07 Feb 21:39
@MariusWirtz MariusWirtz
1.9.0
90e7284

Choose a tag to compare

View all tags
1.9.0

Highlights

New optional use_iterative_json parameter in execute_mdx_dataframe/execute_view_dataframe functions #646, #612

This new feature allows TM1py to use iterative JSON parsing.

When use_iterative_json is True TM1py requires a significantly smaller memory footprint (down to ~ 20% of the original value) at an almost negligible cost of performance (single percentage digit):

from TM1py import TM1Service

with TM1Service(
        base_url="https://localhost:12354",
        user="admin",
        password="apple") as tm1:
    
    df = tm1.cells.execute_view_dataframe(cube_name="Sales", view_name="Very Large View", private=False,
                                          use_iterative_json=True)

This is a handy feature when dealing with large or very large data volumes (1M to 10M cells) in an environment with limited RAM.

New skip_non_updateable argument in write/write_dataframe functions #657

This optional argument to the write/write_dataframe functions asks TM1py to filter out cells that can not be updated before attempting to write. If not used, TM1py will fail with an error when attempting to write to rule-derived or consolidated cells.

from TM1py import TM1Service

with TM1Service(
        base_url="https://localhost:12354",
        user="admin",
        password="apple") as tm1:

    cells = {
        ('Belgium', 'Revenue', '2022', 'Apr', 'Revenue'): 10_000,
        ('Belgium', 'Revenue', '2022', 'May', 'Revenue'): 15_000,
        ('Belgium', 'Revenue', '2022', 'Jun', 'Revenue'): 20_000,
        ('Belgium', 'Revenue', '2022', 'Q1', 'Revenue'): 45_000,
    }

    tm1.cells.write("Sales", cells, skip_non_updateable=True)

This is a useful feature, as it saves the TM1py user from verifying the validity of the cell updates yourself when not working with flawless data sources.
Only errors w.r.t. updatability are suppressed! Attempts, for instance, to write to not existing elements will still raise errors.

New search functions to search through cubes, processes, and chores #660, #663, #665

from TM1py import TM1Service

with TM1Service(
        base_url="https://localhost:12354",
        user="admin",
        password="apple") as tm1:

    processes = tm1.processes.search_string_in_code(search_string="Sales")
from TM1py import TM1Service

with TM1Service(
        base_url="https://localhost:12354",
        user="admin",
        password="apple") as tm1:

    processes = tm1.processes.search_string_in_name(name_contains="Sales")
from TM1py import TM1Service

with TM1Service(
        base_url="https://localhost:12354",
        user="admin",
        password="apple") as tm1:

    cubes = tm1.cubes.search_for_dimension(dimension_name="Product", skip_control_cubes=True)

New Features

  • add write to_message_log_function #621
  • allow skip_zeros in execute_mdx_values function #659
  • use python built-in csv module to create csv strings in execute_mdx_csv function #678
  • new rename function in ApplicationService #682
  • support compact_json in some execute_mdx_ functions #650

Improvements and Bugfixes

  • get_last_message_from_processerrorlog to return text ac0d72f
  • raise exception when session creation fails a42753e
  • don't raise error if cellset is already deleted 3246391
  • improve execute_set_mdx function 61fe2ea
  • add update_or_create function for document from file 7f94f71
  • drop obsolete arguments in TM1Service constructor #652
  • support attribute type change #664
  • fix get_values function issue to work with alternate hierarchies seamlessly #680
  • improve handling of alternate hierarchies in write function #679
  • optional measure_dimension_elements dictionary argument to write and write_dataframe function to improve performance b21ac47

Acknowledgments

Big thanks to @rkvinoth, @jrobinsonAG, @gbryant-dev, @raeldor, @adscheevel, @jordanjeremy for contributing code to this release, and many others for reporting bugs and requesting new features.

How to upgrade TM1py

To upgrade TM1py, just use the following command:

pip install TM1py --upgrade

Contributors

raeldor, adscheevel, and 4 other contributors
Loading