DataHarvest Pro - API data ingestion
DataHarvest Pro is a powerful data ingestion tool designed to seamlessly fetch data from APIs and synchronize it with a PostgreSQL database. With easy installation and configuration, this tool simplifies the process of acquiring data from various sources and efficiently storing it in a structured format within your PostgreSQL database. Whether you're working with real-time data streams or periodic API updates, DataHarvest Pro ensures a smooth and reliable pipeline for data ingestion, empowering you to focus on extracting valuable insights from your datasets.
Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
``` Clone repository to local environment ```
``` run pip install -r requirement.txt ```
Adjust the username, password and database name as per your configuration
Finally submit ingest job ```spart-submit --master local[*] --jars=Driver/postgresql-version.jar ```
And Done.
This job will help you in ingesting data from APIS to you database for analytical and forcasting purpose.
Regarding any support please raise the issue on gitlab issues
If you have ideas for releases in the future, it is a good idea to list them in the README.
We are open to contributions, pull the repository make the changes and test them locally follwed by the merge request
Show your appreciation to those who have contributed to the project.
For open source projects, say how it is licensed.
Project is still under development