Skip to content

Commit b06dd3b

Browse files
authored
changed readme and public apis
1 parent 1633722 commit b06dd3b

File tree

2 files changed

+59
-62
lines changed

2 files changed

+59
-62
lines changed

README.md

Lines changed: 53 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -1,99 +1,96 @@
11
# FireRequests 🔥
22

3-
[![PyPI version](https://badge.fury.io/py/firerequests.svg)](https://badge.fury.io/py/firerequests)
4-
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
3+
[![PyPI version](https://img.shields.io/pypi/v/firerequests.svg)](https://pypi.org/project/firerequests/)
4+
[![License](https://img.shields.io/pypi/l/firerequests.svg)](https://github.com/rishiraj/firerequests/blob/main/LICENSE)
5+
[![Python version](https://img.shields.io/pypi/pyversions/firerequests.svg)](https://pypi.org/project/firerequests/)
56

6-
FireRequests is a cutting-edge, high-performance asynchronous HTTP client library designed for blazing-fast file transfers. Leveraging advanced concurrency paradigms and innovative networking techniques, FireRequests achieves up to 5x real-world speed improvements over traditional synchronous methods.
7+
**FireRequests** is a high-performance, asynchronous HTTP client library for Python, engineered to accelerate your file transfers. By harnessing advanced concepts like semaphores, exponential backoff with jitter, concurrency, and fault tolerance, FireRequests can achieve up to a **5x real-world speedup** in file downloads and uploads compared to traditional synchronous methods.
78

8-
## Key Features
9+
## Features 🚀
910

10-
- **Asynchronous Architecture**: Utilizes `asyncio` for non-blocking I/O operations, maximizing throughput and minimizing latency.
11-
- **Concurrent Chunk Processing**: Implements parallel downloading and uploading of file chunks for optimal resource utilization.
12-
- **Adaptive Exponential Backoff**: Incorporates a sophisticated retry mechanism with jitter for robust error handling and network resilience.
13-
- **Semaphore-based Concurrency Control**: Employs fine-grained concurrency management to prevent resource exhaustion and ensure system stability.
14-
- **Progress Visualization**: Integrates `tqdm` for real-time progress tracking, enhancing user experience and operational visibility.
15-
- **Flexible API**: Supports both high-level convenience methods and low-level customization options for advanced use cases.
11+
- **Asynchronous I/O**: Non-blocking network and file operations using `asyncio`, `aiohttp`, and `aiofiles`, boosting throughput for I/O-bound tasks.
12+
- **Concurrent Transfers**: Uses `asyncio.Semaphore` to limit simultaneous tasks, optimizing performance by managing system resources effectively.
13+
- **Fault Tolerance**: Retries failed tasks with exponentially increasing wait times, adding random jitter to prevent network congestion.
14+
- **Chunked Processing**: Files are split into configurable chunks for parallel processing, significantly accelerating uploads/downloads.
15+
- **Compatibility**: Supports environments like Jupyter through `nest_asyncio`, enabling reusable `asyncio` loops for both batch and interactive Jupyter use.
1616

17-
## Installation
17+
## Installation 📦
18+
19+
Install FireRequests using pip:
1820

1921
```bash
2022
pip install firerequests
2123
```
2224

23-
## Usage Examples
25+
## Quick Start 🏁
2426

25-
### High-Speed File Download
27+
Accelerate your downloads with just a few lines of code:
2628

2729
```python
2830
from firerequests import FireRequests
2931

30-
fr = FireRequests()
31-
url = "https://example.com/large-file.iso"
32-
filename = "large-file.iso"
32+
url = "https://example.com/largefile.iso"
33+
filename = "largefile.iso"
3334

34-
# Asynchronous download with optimized parameters
35-
fr.run_download(url, filename, max_files=10, chunk_size=2 * 1024 * 1024)
35+
fr = FireRequests()
36+
fr.download(url, filename)
3637
```
3738

38-
### Accelerated File Upload
39+
## Real-World Speed Test 🏎️
3940

40-
```python
41-
from firerequests import FireRequests
41+
FireRequests delivers significant performance improvements over traditional download methods. Below is the result of a real-world speed test:
4242

43-
fr = FireRequests()
44-
file_path = "large-file.iso"
45-
parts_urls = ["https://example.com/upload/part1", "https://example.com/upload/part2", ...]
43+
```plaintext
44+
Normal Download 🐌: 100%|██████████| 3.42G/3.42G [06:16<00:00, 9.08MB/s]
45+
Downloading on 🔥: 100%|██████████| 3.42G/3.42G [01:15<00:00, 45.2MB/s]
4646
47-
# Parallel multi-part upload
48-
fr.run_upload(file_path, parts_urls, chunk_size=5 * 1024 * 1024, max_files=8)
47+
🐌 Download Time: 376.77 seconds
48+
🔥 Download Time: 75.75 seconds
4949
```
5050

51-
### Performance Comparison
51+
## Advanced Usage ⚙️
52+
53+
### Downloading Files
5254

5355
```python
54-
fr = FireRequests()
55-
url = "https://mirror.example.com/large-dataset.zip"
56-
filename = "large-dataset.zip"
56+
from firerequests import FireRequests
5757

58-
# Benchmark FireRequests against traditional methods
59-
fr.compare_speed(url, filename)
58+
url = "https://example.com/largefile.iso"
59+
filename = "largefile.iso"
60+
61+
fr = FireRequests()
62+
fr.download(url, filename, max_files=10, chunk_size=2 * 1024 * 1024)
6063
```
6164

62-
## Advanced Usage
65+
- **`url`**: The URL of the file to download.
66+
- **`filename`**: The local filename to save the downloaded file.
67+
- **`max_files`**: The maximum number of concurrent chunk downloads.
68+
- **`chunk_size`**: The size of each chunk in bytes.
6369

64-
### Custom Callback Integration
70+
### Uploading Files
6571

6672
```python
67-
import asyncio
6873
from firerequests import FireRequests
6974

70-
async def progress_callback(bytes_transferred):
71-
print(f"Transferred: {bytes_transferred / 1024 / 1024:.2f} MB")
75+
file_path = "largefile.iso"
76+
parts_urls = ["https://example.com/upload_part1", "https://example.com/upload_part2", ...]
7277

7378
fr = FireRequests()
74-
url = "https://example.com/massive-file.tar.gz"
75-
filename = "massive-file.tar.gz"
76-
77-
asyncio.run(fr.download_file(
78-
url, filename, max_files=12, chunk_size=4 * 1024 * 1024,
79-
callback=progress_callback
80-
))
79+
fr.upload(file_path, parts_urls, chunk_size=2 * 1024 * 1024, max_files=10)
8180
```
8281

83-
## Performance Metrics
84-
85-
In real-world tests, FireRequests demonstrated exceptional performance gains:
86-
87-
- **Traditional Download**: 376.77 seconds
88-
- **FireRequests Download**: 75.75 seconds
89-
- **Speed Improvement**: 4.98x faster
82+
### Comparing Download Speed
9083

91-
These results showcase the significant efficiency enhancements achievable through FireRequests' advanced asynchronous architecture and optimized networking strategies.
84+
```python
85+
from firerequests import FireRequests
9286

93-
## Contributing
87+
url = "https://example.com/largefile.iso"
88+
filename = "largefile.iso"
9489

95-
We welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for more details.
90+
fr = FireRequests()
91+
fr.compare_speed(url, filename)
92+
```
9693

97-
## License
94+
## License 📄
9895

99-
FireRequests is released under the Apache License 2.0. See the [LICENSE](LICENSE) file for more details.
96+
This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/rishiraj/firerequests/blob/main/LICENSE) file for details.

firerequests/main.py

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ async def download_file(
6464
session, url, filename, start, stop, headers, semaphore, parallel_failures, max_retries
6565
))
6666

67-
progress_bar = tqdm(total=file_size, unit="B", unit_scale=True, desc="Downloading")
67+
progress_bar = tqdm(total=file_size, unit="B", unit_scale=True, desc="Downloading on 🔥")
6868
for chunk_result in asyncio.as_completed(tasks):
6969
downloaded = await chunk_result
7070
progress_bar.update(downloaded)
@@ -103,7 +103,7 @@ async def upload_file(
103103
session, part_url, file_path, start, chunk_size, semaphore, parallel_failures, max_retries
104104
))
105105

106-
progress_bar = tqdm(total=file_size, unit="B", unit_scale=True, desc="Uploading")
106+
progress_bar = tqdm(total=file_size, unit="B", unit_scale=True, desc="Uploading on 🔥")
107107
for chunk_result in asyncio.as_completed(tasks):
108108
uploaded = await chunk_result
109109
progress_bar.update(uploaded)
@@ -146,7 +146,7 @@ def upload(self, file_path: str, parts_urls: List[str], chunk_size: int, max_fil
146146
def normal_download(self, url: str, filename: str):
147147
response = requests.get(url, stream=True)
148148
total_size = int(response.headers.get('content-length', 0))
149-
progress_bar = tqdm(total=total_size, unit="B", unit_scale=True, desc="Normal Download")
149+
progress_bar = tqdm(total=total_size, unit="B", unit_scale=True, desc="Normal Download 🐌")
150150
with open(filename, 'wb') as f:
151151
for data in response.iter_content(1024):
152152
progress_bar.update(len(data))
@@ -162,10 +162,10 @@ def compare_speed(self, url: str, filename: str):
162162

163163
start_time = time.time()
164164
asyncio.run(self.download_file(url, filename, max_files=10, chunk_size=2 * 1024 * 1024))
165-
fast_time = time.time() - start_time
165+
fire_time = time.time() - start_time
166166

167-
print(f"\nNormal Download Time: {normal_time:.2f} seconds")
168-
print(f"Fast Download Time: {fast_time:.2f} seconds\n")
167+
print(f"\n🐌 Download Time: {normal_time:.2f} seconds")
168+
print(f"🔥 Download Time: {fire_time:.2f} seconds\n")
169169

170170

171171
if __name__ == "__main__":

0 commit comments

Comments
 (0)