Telemetry for function-testing-helpers usage tracking

[saga-dasgupta]

WHY are these changes introduced?

Fixes: https://github.com/shop/issues-shopifyvm/issues/630

We want telemetry on usage and adoption of shopify-function-test-helpers. We will do so by setting an SHOPIFY_INVOKED_BY env variable to the function run or function build command when invoking from shopify-function-test-helpers. This value will then be passing to monorail through a env_shopify_variables field of the monorail payload.

WHAT is this pull request doing?

• Adds new invoked-by flag to function run command.

How to test your changes?

I added a unit test, I dont know if we can tophat this change locally but once its out, we should be able to query this column in the sdp tables in BigQuery.

Post-release steps

Measuring impact

How do we know this change was effective? Please choose one:

• n/a - this doesn't need measurement, e.g. a linting rule or a bug-fix
• Existing analytics will cater for this addition
• PR includes analytics changes to measure impact

Checklist

• I've considered possible cross-platform impacts (Mac, Linux, Windows)
• I've considered possible documentation changes

[github-actions]

Coverage report

St.❔	Category	Percentage	Covered / Total
🟡	Statements	79.22% (+0.01% 🔼)	13551/17106
🟡	Branches	73.06% (+0% 🔼)	6617/9057
🟡	Functions	79.33% (+0.01% 🔼)	3492/4402
🟡	Lines	79.58% (+0.01% 🔼)	12801/16086

Show files with reduced coverage 🔻

St.❔	File	Statements	Branches	Functions	Lines
🟢	... / ui_extension.ts	94.83% (+0.04% 🔼)	81.25% (-1.64% 🔻)	100%	96.46% (+0.03% 🔼)

Test suite run success

3345 tests passing in 1371 suites.

Report generated by 🧪jest coverage report action from 8f7a779

[andrewhassan]

This is actually really great because in the helper, we can set this environment variable and old CLI versions will just ignore it!

[isaacroldan]

Does this need to be a full flag? You can just use the env var no?
The usage of this feels more like ENV stuff

[github-actions]

We detected some changes at packages/*/src and there are no updates in the .changeset.
If the changes are user-facing, run pnpm changeset add to track your changes and include them in the next release CHANGELOG.

Caution

DO NOT create changesets for features which you do not wish to be included in the public changelog of the next CLI release.

[isaacroldan]

this flag should have hidden: true as it is not intended to be used externally

[saga-dasgupta]

Actually @isaacroldan I realized that for developers with old version of CLI using shopify-function-test-helpers the invoked-by flag would throw an error, so maybe we do need it to just be set through an environment variable that we pass into monorail through either appending to an existing column or creating a new column.

[isaacroldan]

Just make getShopifyEnvironmentVariables return a JSON, this function can be simplified to a single line:

Object.fromEntries(Object.entries(process.env).filter(([key]) => key.startsWith('SHOPIFY_')))

[lopert]

Love the env var approach! Thanks for digging for it! 🙏

[github-actions]

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

• Some seemingly private modules might be re-exported through public modules.
• If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/analytics.d.ts

@@ -27,5 +27,6 @@ interface EnvironmentData {
 export declare function getEnvironmentData(config: Interfaces.Config): Promise<EnvironmentData>;
 export declare function getSensitiveEnvironmentData(config: Interfaces.Config): Promise<{
     env_plugin_installed_all: string;
+    env_shopify_variables: string;
 }>;
 export {};
\ No newline at end of file

packages/cli-kit/dist/public/node/metadata.d.ts

@@ -55,6 +55,7 @@ declare const coreData: RuntimeMetadataManager<CmdFieldsFromMonorail, {
     cmd_all_environment_flags?: string | null;
     cmd_dev_tunnel_custom?: string | null;
     env_plugin_installed_all?: string | null;
+    env_shopify_variables?: string | null;
 }, "store_", never>>;
 export declare const getAllPublicMetadata: () => Partial<CmdFieldsFromMonorail>, getAllSensitiveMetadata: () => Partial<{
     commandStartOptions: {
@@ -74,6 +75,7 @@ export declare const getAllPublicMetadata: () => Partial<CmdFieldsFromMonorail>,
     cmd_all_environment_flags?: string | null;
     cmd_dev_tunnel_custom?: string | null;
     env_plugin_installed_all?: string | null;
+    env_shopify_variables?: string | null;
 }, "store_", never>>, addPublicMetadata: (getData: ProvideMetadata<CmdFieldsFromMonorail>, onError?: MetadataErrorHandling) => Promise<void>, addSensitiveMetadata: (getData: ProvideMetadata<{
     commandStartOptions: {
         startTime: number;
@@ -92,6 +94,7 @@ export declare const getAllPublicMetadata: () => Partial<CmdFieldsFromMonorail>,
     cmd_all_environment_flags?: string | null;
     cmd_dev_tunnel_custom?: string | null;
     env_plugin_installed_all?: string | null;
+    env_shopify_variables?: string | null;
 }, "store_", never>>, onError?: MetadataErrorHandling) => Promise<void>, runWithTimer: (field: NumericKeyOf<CmdFieldsFromMonorail>) => <T>(fn: () => Promise<T>) => Promise<T>;
 export type Public = PublicSchema<typeof coreData>;
 export type Sensitive = SensitiveSchema<typeof coreData>;

packages/cli-kit/dist/public/node/monorail.d.ts

@@ -2,7 +2,7 @@ import { JsonMap } from '../../private/common/json.js';
 import { DeepRequired } from '../common/ts/deep-required.js';
 export { DeepRequired };
 type Optional<T> = T | null;
-export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.19";
+export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.20";
 export interface Schemas {
     [MONORAIL_COMMAND_TOPIC]: {
         sensitive: {
@@ -14,6 +14,7 @@ export interface Schemas {
             cmd_all_environment_flags?: Optional<string>;
             cmd_dev_tunnel_custom?: Optional<string>;
             env_plugin_installed_all?: Optional<string>;
+            env_shopify_variables?: Optional<string>;
         };
         public: {
             business_platform_id?: Optional<number>;