A price monitoring microservice built with FastAPI that performs dynamic web scraping using Selenium, processes data with Pandas, and exposes RESTful endpoints for retrieving price history and averages.
This project is a price monitoring microservice designed to scrape product data dynamically using Selenium and process it with Pandas. Built with FastAPI and served via Uvicorn, it exposes RESTful endpoints to trigger scraping operations and retrieve historical and average price data. The service stores results in CSV format and is structured for extensibility. Planned improvements include support for multiple e-commerce sources, integration with a relational database using PostgreSQL and SQLAlchemy, implementation of dashboards with Plotly or Streamlit, and migration to asynchronous scraping with Playwright for better scalability and performance.
Starts scraping and saves data to CSV.
GET /scrape/{product}Returns historical and average prices.
GET /prices/{product}- Clone this repository:
git clone https://github.com/user/voting.git- (Optional but recommended) Create a virtual environment:
python -m venv .venv
source .venv/bin/activate- Install dependencies
pip install -r requirements.txt- Run the program
python main.pyMIT License – see the LICENSE file for details.