Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 132 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
name: Tests

on:
push:
branches: [master]
pull_request:
branches: [master]

jobs:
tests-ch21:
name: PHP ${{ matrix.php }} / ClickHouse 21.9
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
php: ['8.0', '8.1', '8.2', '8.3', '8.4']

services:
clickhouse:
image: clickhouse/clickhouse-server:21.9
ports:
- 8123:8123
options: >-
--health-cmd "curl -f http://localhost:8123/ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5

steps:
- uses: actions/checkout@v4

- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
extensions: curl, json
coverage: none

- name: Install dependencies
run: composer install --no-progress --prefer-dist

- name: Run tests (ClickHouse 21.9)
run: ./vendor/bin/phpunit -c phpunit-ch21.xml
env:
CLICKHOUSE_HOST: 127.0.0.1
CLICKHOUSE_PORT: 8123

tests-ch26:
name: PHP ${{ matrix.php }} / ClickHouse 26.3
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
php: ['8.0', '8.1', '8.2', '8.3', '8.4']

services:
clickhouse:
image: clickhouse/clickhouse-server:26.3.3.20
ports:
- 8124:8123
env:
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
CLICKHOUSE_PASSWORD: ""
options: >-
--health-cmd "curl -f http://localhost:8123/ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5

steps:
- uses: actions/checkout@v4

- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
extensions: curl, json
coverage: none

- name: Install dependencies
run: composer install --no-progress --prefer-dist

- name: Copy ClickHouse config for deprecated syntax
run: |
docker cp tests/clickhouse-latest-config/allow_deprecated.xml $(docker ps -q --filter "ancestor=clickhouse/clickhouse-server:26.3.3.20"):/etc/clickhouse-server/users.d/allow_deprecated.xml
sleep 2

- name: Run tests (ClickHouse 26.3)
run: ./vendor/bin/phpunit -c phpunit-ch26.xml
env:
CLICKHOUSE_HOST: 127.0.0.1
CLICKHOUSE_PORT: 8124

phpstan:
name: PHPStan
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.4'
extensions: curl, json
coverage: none

- name: Install dependencies
run: composer install --no-progress --prefer-dist

- name: Run PHPStan
run: ./vendor/bin/phpstan analyse --memory-limit=512M

phpcs:
name: Code Style
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.4'
extensions: curl, json
coverage: none

- name: Install dependencies
run: composer install --no-progress --prefer-dist

- name: Run PHPCS
run: ./vendor/bin/phpcs || true
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
},
"require-dev": {
"doctrine/coding-standard": "^8.2",
"phpstan/phpstan": "^0.12",
"phpstan/phpstan": "^2.0",
"phpunit/phpunit": "^9.5",
"sebastian/comparator": "^4.0"
},
Expand Down
94 changes: 94 additions & 0 deletions doc/exceptions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Structured Exceptions

The library provides detailed exception information from ClickHouse errors.

## Exception Hierarchy

```
ClickHouseException (interface)
├── QueryException (LogicException)
│ └── DatabaseException (ClickHouse server errors)
├── TransportException (RuntimeException) — curl/HTTP errors
└── ClickHouseUnavailableException — connection refused
```

## DatabaseException

Thrown when ClickHouse returns a server error (bad SQL, missing table, auth failure, etc.).

```php
use ClickHouseDB\Exception\DatabaseException;

try {
$db->select('SELECT * FROM non_existent_table');
} catch (DatabaseException $e) {
echo $e->getMessage(); // "Table default.non_existent_table doesn't exist.\nIN:SELECT ..."
echo $e->getCode(); // 60
echo $e->getClickHouseExceptionName(); // 'UNKNOWN_TABLE' (CH 22+) or null (older versions)
echo $e->getQueryId(); // 'abc-123-def' (from X-ClickHouse-Query-Id header)
}
```

### Available Methods

| Method | Returns | Description |
|--------|---------|-------------|
| `getMessage()` | string | Error message + SQL |
| `getCode()` | int | ClickHouse error code |
| `getClickHouseExceptionName()` | ?string | e.g. `UNKNOWN_TABLE`, `SYNTAX_ERROR` (CH 22+) |
| `getQueryId()` | ?string | Query ID from response header |
| `getRequestDetails()` | array | Request metadata |
| `getResponseDetails()` | array | Response metadata |

### Common Error Codes

| Code | Exception Name | Description |
|------|---------------|-------------|
| 60 | `UNKNOWN_TABLE` | Table doesn't exist |
| 62 | `SYNTAX_ERROR` | SQL syntax error |
| 81 | `DATABASE_NOT_FOUND` | Database doesn't exist |
| 115 | `UNKNOWN_SETTING` | Invalid setting |
| 192 | `UNKNOWN_USER` | User doesn't exist |
| 241 | `MEMORY_LIMIT_EXCEEDED` | Query exceeded memory limit |
| 516 | `AUTHENTICATION_FAILED` | Wrong password or user |

### Error Format Support

The library parses both old and new ClickHouse error formats:

```
# Old format (CH < 22)
Code: 60. DB::Exception: Table default.xxx doesn't exist., e.what() = DB::Exception

# New format (CH 22+)
Code: 60. DB::Exception: Table default.xxx doesn't exist. (UNKNOWN_TABLE) (version 24.3.2.23 (official build))
```

## TransportException

Thrown for HTTP/curl-level errors (timeouts, connection refused, etc.).

```php
use ClickHouseDB\Exception\TransportException;

try {
$db->select('SELECT sleep(100)');
} catch (TransportException $e) {
echo $e->getMessage(); // "Operation timed out"
echo $e->getCode(); // curl error number
}
```

## ClickHouseUnavailableException

Thrown when the server is unreachable.

```php
use ClickHouseDB\Exception\ClickHouseUnavailableException;

try {
$db->ping(true);
} catch (ClickHouseUnavailableException $e) {
echo "ClickHouse is down: " . $e->getMessage();
}
```
90 changes: 90 additions & 0 deletions doc/generators.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Generators (Memory-Efficient Queries)

For large resultsets that don't fit in memory, use generators to process rows one at a time.

## selectGenerator()

Streams results from ClickHouse using JSONEachRow format and yields one row at a time. Unlike `select()->rows()`, the full resultset is never loaded into PHP memory.

```php
foreach ($db->selectGenerator('SELECT * FROM huge_table') as $row) {
// $row is an associative array: ['column1' => value1, 'column2' => value2, ...]
processRow($row);
}
```

### With Bindings

```php
foreach ($db->selectGenerator(
'SELECT * FROM events WHERE date > :date',
['date' => '2024-01-01']
) as $row) {
echo $row['event_name'] . "\n";
}
```

### With Per-Query Settings

```php
foreach ($db->selectGenerator(
'SELECT * FROM huge_table',
[],
['max_execution_time' => 600]
) as $row) {
// process with extended timeout
}
```

### Count rows without loading all into memory

```php
$count = 0;
foreach ($db->selectGenerator('SELECT * FROM events') as $row) {
$count++;
}
echo "Processed $count rows";
```

### Write to file row by row

```php
$fp = fopen('output.csv', 'w');
foreach ($db->selectGenerator('SELECT id, name, email FROM users') as $row) {
fputcsv($fp, $row);
}
fclose($fp);
```

## rowsGenerator()

If you already have a `Statement` from `select()`, you can iterate over it with a generator instead of calling `rows()`:

```php
$st = $db->select('SELECT * FROM table LIMIT 1000');

// Instead of $st->rows() which returns the full array:
foreach ($st->rowsGenerator() as $row) {
echo $row['id'] . "\n";
}
```

Note: `rowsGenerator()` still loads data in `init()` first. For true streaming from ClickHouse, use `selectGenerator()`.

## Comparison

| Method | Memory | Speed | Use case |
|--------|--------|-------|----------|
| `select()->rows()` | All rows in memory | Fast for small results | < 100K rows |
| `select()->rowsGenerator()` | All rows in memory (init) | Iterator interface | When you need Generator type |
| `selectGenerator()` | One row at a time | Best for large results | > 100K rows, ETL, exports |

## How selectGenerator() Works

1. Opens a `php://temp` stream
2. Calls `streamRead()` with `FORMAT JSONEachRow`
3. Reads the stream line by line
4. Decodes each JSON line and yields it
5. Closes the stream when done

The key difference from `select()`: JSONEachRow format produces one JSON object per line, so each line can be decoded independently without parsing the entire response.
Loading
Loading