UNIFIED DATA PROCESSING FRAMEWORK

composer require flow-php/etl ~0.27.0

Changelog Release Cycle

Try Flow Online

Extracts

Read from various data sources.

Transforms

Shape and optimize for your needs.

Loads

Store and secure in one of many available data sinks.

Examples:

Description

Write data to a parquet file.

function to_parquet(string|Path $path) : Loader

Additional options:

withOptions(\Flow\Parquet\Options $options) - Options for the parquet writer
withCompression(Compressions $compression) - default Compressions::SNAPPY, supports one of the following Compressions
withSchema(Schema $schema) - when provided writer will use this instead of inferring schema from each rows batch

Download

▶ ▼ composer.json

{
    "name": "flow-php/examples",
    "description": "Flow PHP - Examples",
    "license": "MIT",
    "type": "library",
    "require": {
        "flow-php/etl": "1.x-dev",
        "flow-php/etl-adapter-parquet": "1.x-dev",
        "flow-php/parquet": "1.x-dev",
        "flow-php/snappy": "1.x-dev"
    },
    "minimum-stability": "dev"
}

code.php

<?php

declare(strict_types=1);

use function Flow\ETL\Adapter\Parquet\to_parquet;
use function Flow\ETL\DSL\{data_frame, from_array, overwrite};

require __DIR__ . '/vendor/autoload.php';

data_frame()
    ->read(from_array([
        ['id' => 1],
        ['id' => 2],
        ['id' => 3],
        ['id' => 4],
        ['id' => 5],
    ]))
    ->collect()
    ->saveMode(overwrite())
    ->write(to_parquet(__DIR__ . '/output.parquet'))
    ->run();

Contributors

Join us on GitHub