• 👨‍💻 I’m currently working as a Database Administrator, building strong foundations in data management and reliability
• 🌱 Transitioning into Data Engineering by designing end‑to‑end batch and streaming pipelines
• 🛠️ Passionate about building scalable, reliable data pipelines that turn raw data into actionable insights
• 👯 Open to collaborating on Data Engineering & Open Source projects
• 👨‍💻 Explore my work here: My Portfolio
• 📫 Reach me at arjunmpec101@gmail.com
• ⚡ Fun fact: I debug pipelines the way I play games — with persistence and strategy

• 🗄️ YouTube Data Engineering Pipeline (Batch Processing)
  End‑to‑end batch ETL pipeline implementing the Medallion Architecture (Bronze → Silver → Gold).

  • Orchestrated with Apache Airflow (3.x)
  • Transformations with Apache Spark
  • Data lake layers on local filesystem (Bronze/Silver/Gold)
  • Serving layer in Postgres (analytics‑ready tables)
  • Interactive Streamlit + Altair dashboard via SQLAlchemy
  • Ingests raw YouTube trending data (CSV/JSON), cleans, enriches, and computes derived metrics for BI

• 📊 StockPulse (Streaming Pipeline)
  Real‑time streaming pipeline simulating stock ticks and processing them end‑to‑end.

  • Ingestion via Kafka producer publishing to stock_ticks topic
  • Processing with Spark Structured Streaming (schema enforcement + derived metrics)
  • Dual sinks: Postgres (serving layer) + Parquet (partitioned by index/date)
  • Interactive Streamlit + Altair dashboard for real‑time visualization
  • Fully orchestrated with Apache Airflow

Note: Top languages is only a metric of the languages my public code consists of and doesn't reflect experience or skill level.