Clean up usd publish plugins

[MustafaJafar]

Changelog Description

• Fix launch types in ayon-usd/client/ayon_usd/hooks/pre_resolver_init.py
• Fix typo in ayon-usd/client/ayon_usd/hooks/usd_pinning_root.py
• Align pinning file export location to rendermetadata in ayon-usd/client/ayon_usd/plugins/publish/extract_skeleton_pinning_json.py
• remove collect_usd_pinning_env_vars.py Remove ayon-usd related plugin. ayon-deadline#197.
• Fix setting PINNING_FILE_PATH and ENABLE_STATIC_GLOBAL_CACHE on publish Job
• Fix/refactor/remove unnecessary extractor-integrator.. Clean up usd publish plugins #93 (comment)
• Pin file is no longer published as it's an intermediate publish file like render metadata. and it's saved next to render.usd file.

Additional Information

This doesn't fully solve the farm rendering issue #91 as

• we need a new version of AYON USD Resolver with 17 ayonapiget does not check for failed httplib connections ayon-cpp-api#35 which allows the resolver to login using AYON service api key.
• fix pinning file generation (file content) which should be solved by Bugfix/94 pinning mode #100

Testing notes:

Enable

• Logging
  
• Pinning Generation
  

With pinning disabled (live mode):

• Submissions should work fine and in DL log, you'll be able to see resolver log. it'll mention it can't login though

2025-12-02 09:45:30:  0: STDOUT: resolverContextCache::resolverContextCache() 
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.337] [info] AyonApi::loadEnvVars()
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.337] [info] Loaded AYON_API_KEY and AYON_SERVER_URL
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.337] [info] Found SideId
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.337] [info] Loaded Environment Variables(funcEnd): AYON_API_KEY=veryinsecureapikey, AYON_SERVER_URL=http://192.168.1.12:5000, AYON_SITE_ID=military-mouse-of-jest
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.337] [info] Init AyonServer httplib::Client
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.337] [info] Constructor Getting Site Roots
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.337] [info] AyonApi::getSiteRoots()
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.337] [info] AyonApi::GET(/api/projects/Experiments/siteRoots)
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.346] [info] AyonApi::serialCorePost wrong status code: 401 expected: 200
2025-12-02 09:45:30:  0: STDOUT: [2025-12-02 09:45:30.346] [warning] not logged in 401 

With pinning enabled:

• You should find pinning file __render__pin.json next to __render__.usd.
  
• You should see pinning mode related variables in job environment.
  
• The Houdini DL job will mostly fail as pinning mode. Errors can happen for many reasons, such as pinning generation is buggy, resolver mostly skips Houdini internal initializations.

Resolve #92

[MustafaJafar]

This PR is made to reflect the changes I'm doing on my side when testing USD publish plugins.

[MustafaJafar]

Tagging @philippe-ynput and @tadeas-hejnic for visibility.

[MustafaJafar]

I've found out that collect_usd_pinning_env_vars.py plugin return nothing and the actual path is generated from

ayon-usd/client/ayon_usd/plugins/publish/extract_skeleton_pinning_json.py

Line 67 in 0d04be5

pin_file_path = staging_dir.joinpath(pin_file)

[MustafaJafar]

@antirotor The Current state of this PR should be working in Houdini.
However, there are some bits that should be cleaned up.

One thing is the pinning file workflow: Currently, CollectUSDPinningEnvVars doesn't work as expected.
From the code flow, I can understand the intention but it doesn't work. more info below.

Another thing, generalize the solution to work in Houdini and Maya.

What Pinning File workflow should be:

1. Get the publish path of the pinning file. and set it to PINNING_FILE_PATH env var.
2. Generate pinning file, add it as a representation and publish it.
3. Farm jobs should be using the published pinning file.

What is currently happening:

1. Generate pinning file in staging directory.
2. Set it to PINNING_FILE_PATH env var.

[MustafaJafar]

I'm not sure why all my DL jobs with pinning mode enabled are failing.
Therefore, I ended up disabling pinning mode and setting the AYON_API_KEY as outlined in the testing notes above.
Now, it's working.

[MustafaJafar]

I opened the scene in pinning mode and found out it failed because of using entity uris.
So, I disabled it. and also updated the keys in the pinning file manually as expected by my scene file and now it works.

Those are definitely not related to this PR. However, the part that is related to this PR is working as expected.
after tweaking the scene and updating the pinning file, I only needed to set AYON_API_KEY as outlined in testing notes above.

[MustafaJafar]

@antirotor I've updated the todo above and I'd appreciate your help as I'm not sure how to fix get_usd_pinning_envs and the pinning file publishing.

@tadeas-hejnic Could you please check the todo above? It may overlap with your PR #100 . This PR is dedicated for fixing another part of the issue so it shouldn't overlap as it's complementary to yours.

[tadeas-hejnic]

@antirotor I've updated the todo above and I'd appreciate your help as I'm not sure how to fix get_usd_pinning_envs and the pinning file publishing.

@tadeas-hejnic Could you please check the todo above? It may overlap with your PR #100 . This PR is dedicated for fixing another part of the issue so it shouldn't overlap as it's complementary to yours.

Based on the TODOs it is not overlapping my PR. #100 is related just to the pinning file generation.

[MustafaJafar]

It's weird that supporting staging directory in Houdini made pinning file publishing work as expected in develop branch

ynput/ayon-houdini#200

[MustafaJafar]

Note for me, pinning file extraction removes it as cleaning up via usd_file_path.unlink() It should have added it to the cleaning up paths not removing it manually. add setting for enable/disable removing cleaning the file.

ynput/ayon-houdini#329 (comment)

[MustafaJafar]

So, me and @BigRoy agreed to

• not publishing the pin file as it's best seen as intermediate file like render metadata.
• it'd be nice to have it next to __render__.usd file as __render__pinning.json

[MustafaJafar]

I tried opening a Houdini CMD with pinning env set and checking the __render__.usd but usdview crashed.

H:\AYON\projects\Experiments\random\houdini\work\generic>hython C:/PROGRA~1/SIDEEF~1/HOUDIN~1.370/bin/usdview H:\AYON\projects\Experiments\ideas\test_resolver_pinning\work\manual_pinning_file\ayon\renderManual_pinning_fileRenderCubeTest\usd\Expr_test_resolver_pinning_manual_pinning_file_v006\renderManual_pinning_fileRenderCubeTest\__render__.usd
Installing AYON ...
Setting scene FPS to 25
Updated $HOUDINI_OTLSCAN_PATH to H:/HOUDINI_LIBRARY/digital_assets
resolverContextCache::resolverContextCache()
Traceback (most recent call last):
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.370\bin\usdview", line 56, in <module>
    Usdviewq.Launcher().Run()
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.370\python311\Lib\site-packages\pxr\Usdviewq\__init__.py", line 86, in Run
    self.__LaunchProcess(arg_parse_result)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.370\python311\Lib\site-packages\pxr\Usdviewq\__init__.py", line 401, in __LaunchProcess
    (app, appController) = self.LaunchPreamble(arg_parse_result)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.370\python311\Lib\site-packages\pxr\Usdviewq\__init__.py", line 391, in LaunchPreamble
    appController = AppController(arg_parse_result, contextCreator)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.370\python311\Lib\site-packages\pxr\Usdviewq\appController.py", line 469, in __init__
    stage = self._openStage(
            ^^^^^^^^^^^^^^^^
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.370\python311\Lib\site-packages\pxr\Usdviewq\appController.py", line 1229, in _openStage
    if not Ar.GetResolver().Resolve(usdFilePath):
           ^^^^^^^^^^^^^^^^
pxr.Tf.ErrorException:
        Error in 'pxrInternal_v0_24__pxrReserved__::PlugPlugin::_Load' at line 261 in file D:\WeeklyDevTools20.5\dev_tools\src\usd\usd-24.03\USD-py3.11-qt5\pxr\base\plug\plugin.cpp : 'Failed to load plugin 'ayonUsdResolver': A dynamic link library (DLL) initialization routine failed.
 in 'e:/Ynput/ayon-usd/client/ayon_usd/downloads/Houdini205_Py311_Win_Windows_AMD64/ayonUsdResolver/lib/ayonUsdResolver.dll''
        Error in 'pxrInternal_v0_24__pxrReserved__::`anonymous-namespace'::_PluginResolver<class pxrInternal_v0_24__pxrReserved__::ArResolver,class pxrInternal_v0_24__pxrReserved__::Ar_ResolverFactoryBase>::Create' at line 377 in file D:\WeeklyDevTools20.5\dev_tools\src\usd\usd-24.03\USD-py3.11-qt5\pxr\usd\ar\resolver.cpp : 'Failed to load plugin ayonUsdResolver for AyonUsdResolver'

Also, tagging @tadeas-hejnic in case if this is useful.

But anyways, this is not related to the PR where the main focus here to fix some issue in publish plugins and make the resolver run on farm.

[MustafaJafar]

I think this PR should work. gave it some test runs.

[MustafaJafar]

TL;DR;

This PR

• Makes pinning generation work with usdrender product in Houdini. we need another plugin for adding support in Maya but hopefully it shouldn't deviate too much from ExtractSkeletonPinningJSON.
• pin file is no longer published.
• The resolver is now working on farm however,
  • in live mode: it tells not logged in which should be fixed in 17 ayonapiget does not check for failed httplib connections ayon-cpp-api#35 which adds support for logging in via AYON_API_KEY

    2025-12-11 15:32:28:  0: STDOUT: [2025-12-11 15:32:28.251] [info] AyonApi::serialCorePost wrong status code: 401 expected: 200
    2025-12-11 15:32:28:  0: STDOUT: [2025-12-11 15:32:28.251] [warning] not logged in 401 
    

  • in pinning mode: the render job will most likely fail as the pinning file doesn't work properly which should be fixed in Bugfix/94 pinning mode #100

Tagging @BigRoy @antirotor @philippe-ynput @tadeas-hejnic for visibility.

[BigRoy]

Aside of a few cosmetic notes - this looks good to me.

[MustafaJafar]

Aside of a few cosmetic notes - this looks good to me.

I wish if I were able to give it a nice test run with the new resolver. but let's merge it anyways.