Skip to content
This repository was archived by the owner on Mar 16, 2026. It is now read-only.

OstinUA/Google-Play-Store-Scraper

Repository files navigation

Google Logging Tools for Node.js

A production-ready, structured debug logging library pattern for Node.js applications that need controllable verbosity, subsystem scoping, and backend-flexible log emission.

Build Status Version License: AGPL-3.0 TypeScript Coverage

Important

This repository currently ships as a Vite + TypeScript project and does not expose a dedicated npm-published logging package artifact. The implementation guidance in this README consolidates the existing repository setup with logging-library conventions and usage patterns.

Table of Contents

Features

  • Namespace-based logging model using system and optional system:subsystem semantics for granular log routing.
  • Runtime log activation through environment-driven filtering (GOOGLE_SDK_NODE_LOGGING) for minimal code-side toggling.
  • Wildcard-compatible namespace selection (for example, * or translate:*) to support broad or focused troubleshooting.
  • Structured metadata + printf-style message formatting support in a single logging call pattern.
  • Severity-level convenience APIs (debug, info, warn, error) to enforce consistent event classification.
  • Sub-logger derivation through subsystem scoping to avoid duplicating parent context in downstream modules.
  • Pluggable backend model (console-style backend, debug-package backend, structured JSON backend).
  • Event-hook capability (on('log')) for advanced telemetry workflows, side-channel processing, or custom sinks.
  • Schema-friendly structured payloads suitable for ingestion by centralized logging platforms.
  • TypeScript-first developer ergonomics for contract clarity and editor-assisted integration.
  • Frontend and developer tooling already integrated in this repository (Vite, TypeScript, React) for documentation and demo workflows.

Note

Some logging behaviors described here align with the google-logging-utils package conventions used by Google Node.js ecosystems and are included to provide practical implementation parity.

Tech Stack & Architecture

Core Stack

  • Language: TypeScript (~5.8.2 in this repository)
  • Runtime: Node.js (recommended LTS, 20+ for local development workflows)
  • Build Tooling: Vite (^6.2.0)
  • UI Runtime (for demo/web integration): React 19 + React DOM 19
  • External Service SDK in this repository: @google/genai (used by app code, independent from logging concepts)

Project Structure

.
├── App.tsx                     # Main application workflow and UI orchestration
├── components/
│   └── DataTable.tsx           # Tabular rendering for extracted output
├── geminiService.ts            # Service layer logic and external API interaction
├── index.tsx                   # React bootstrap entrypoint
├── index.html                  # Host shell for the web application
├── types.ts                    # Shared TypeScript domain contracts
├── vite.config.ts              # Vite build and environment mapping
├── package.json                # Scripts, dependencies, and project metadata
├── metadata.json               # Project descriptor metadata
├── tsconfig.json               # TypeScript compiler configuration
└── LICENSE                     # GNU AGPL v3 license text

Key Design Decisions

  1. Namespace-first logging model
    Logging should be enabled/disabled by logical system names rather than source file paths, making runtime diagnostics easier in poly-package applications.

  2. Backend abstraction over direct console calls
    Logging sinks are treated as interchangeable backends to support local development, debug-tool integration, and structured cloud ingestion.

  3. Structured-field compatibility
    Metadata objects are first-class citizens to support downstream queryability and observability pipelines.

  4. Environment-driven activation
    Log volume is controlled operationally (GOOGLE_SDK_NODE_LOGGING) without requiring code edits or rebuilds.

  5. Incremental adoption
    Existing repositories can use thin wrapper utilities around logger factories without invasive refactors.

flowchart TD
    A[Application Code] --> B[Logger Factory: system/subsystem]
    B --> C{Logging Enabled by Environment Filter?}
    C -- No --> D[No-op / Skip Emission]
    C -- Yes --> E[Build LogFields + format args]
    E --> F{Selected Backend}
    F --> G[Node Backend Console Output]
    F --> H[Debug Package Backend]
    F --> I[Structured JSON Backend]
    G --> J[Terminal]
    H --> J
    I --> K[Cloud Logging / SIEM / Log Pipeline]
Loading

Tip

Prefer system names that map to business domains (auth, billing, crawler) instead of technical folders (utils, helpers) to improve production troubleshooting.

Getting Started

Prerequisites

  • Node.js >=20 (recommended for consistency with modern tooling)
  • npm >=9
  • Git CLI
  • Optional for full app workflow in this repository: a valid GEMINI_API_KEY

Installation

  1. Clone the repository:
git clone https://github.com/yourusername/google-play-store-scraper.git
cd google-play-store-scraper
  1. Install dependencies:
npm install
  1. (Optional) Create a local environment file:
cat > .env <<'ENV'
GEMINI_API_KEY=your_google_genai_api_key
GOOGLE_SDK_NODE_LOGGING=*
ENV
  1. Start local development mode:
npm run dev

Warning

Never commit .env files or secrets. Keep runtime credentials in secure secret managers for CI/CD and production.

Testing

This repository does not currently define dedicated unit/integration/lint scripts in package.json. Use the following checks as practical quality gates.

  • Unit tests (not configured):
npm run test
  • Integration tests (not configured):
npm run test:integration
  • Lint checks (not configured):
npm run lint
  • Type safety and production build validation:
npx tsc --noEmit
npm run build
npm run preview

Caution

npm run test, npm run test:integration, and npm run lint are expected to fail until corresponding scripts are added to package.json.

Deployment

Production Build

npm ci
npm run build

Output artifacts are generated in dist/ and can be deployed to static hosts (for UI/demo scenarios) or bundled into broader Node service delivery pipelines.

CI/CD Integration Guidance

  • Run dependency installation with lockfile strictness (npm ci).
  • Execute compile checks (npx tsc --noEmit) and build (npm run build).
  • Enforce secret hygiene (GEMINI_API_KEY, GOOGLE_SDK_NODE_LOGGING) using CI secret stores.
  • Publish build artifacts (dist/) or package artifacts depending on your release workflow.

Example GitHub Actions pipeline:

name: ci
on:
  push:
    branches: [main]
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest
    env:
      GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
      GOOGLE_SDK_NODE_LOGGING: "*"
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx tsc --noEmit
      - run: npm run build

Usage

Below are practical logging-library usage patterns aligned with Google-style logging utilities.

Basic Logger Initialization

// Example: logger setup with system namespace.
import {log} from 'google-logging-utils';

const logger = log('crawler');

// Metadata object + format string + format args.
logger({jobId: 'job-42', source: 'scheduler'}, 'Started crawl for %s', 'com.spotify.music');

Severity Helpers + Sub-logger

import {log} from 'google-logging-utils';

const logger = log('crawler');
const playStoreLogger = logger.sublog('playstore');

logger.info('Crawl bootstrap complete for worker=%s', 'w-01');
playStoreLogger.debug('Fetching URL: %s', 'https://play.google.com/store/apps/details?id=com.spotify.music');
playStoreLogger.warn('Retrying request attempt=%d', 2);
playStoreLogger.error('Failed to parse response for appId=%s', 'com.spotify.music');

Hooking Log Events for Custom Processing

import {log} from 'google-logging-utils';

const logger = log('crawler:events');

logger.on('log', (fields, args) => {
  // Forward to observability pipeline, queue, or metrics bridge.
  console.log('Structured fields:', fields);
  console.log('Original args:', args);
});

logger.info('Event hook attached successfully');

Environment-Controlled Activation

GOOGLE_SDK_NODE_LOGGING=crawler,crawler:playstore node dist/server.js

Configuration

Environment Variables

Variable Required Default Description
GOOGLE_SDK_NODE_LOGGING No unset Comma-separated namespace filters that enable logging (*, system, system:subsystem).
GEMINI_API_KEY Repository-specific unset Used by existing app service integration in this repository, independent of logger semantics.

Runtime Flags and Startup Commands

npm run dev
npm run build
npm run preview

If using a Node entrypoint for backend services, set logging scope at process startup:

GOOGLE_SDK_NODE_LOGGING=* node dist/server.js

Configuration File Patterns (.env)

# Logging scope
GOOGLE_SDK_NODE_LOGGING=crawler,crawler:playstore

# Optional existing service integration key in this repository
GEMINI_API_KEY=your_google_genai_api_key

Backend Selection Pattern

Typical advanced setup with backend override (conceptual):

import {setBackend, getStructuredBackend} from 'google-logging-utils';

// Route all logs through structured JSON backend.
setBackend(getStructuredBackend());

Note

Exact API surface may vary by installed google-logging-utils version; verify against your lockfile and package documentation.

License

This repository is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0). See LICENSE for full terms.

Support the Project

Patreon Ko-fi Boosty YouTube Telegram

If you find this tool useful, consider leaving a star on GitHub or supporting the author directly.

About

AI-powered Play Store scraper using Gemini 3 Pro. Extract studio names, dual-type emails (Support & Business), and normalized install counts from any Google Play URL.

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors