UNIFIED DATA PROCESSING FRAMEWORK

composer require flow-php/etl ^0.10.0

Changelog

Extracts

Read from various data sources.

Transforms

Shape and optimize for your needs.

Loads

Store and secure in one of many available data sinks.

Examples:

Description

A Data Frame is a structured collection of tabular data, similar to a spreadsheet.
It organizes information into rows and columns, making it easy to understand, filter, and transform.
Using a Data Frame, you can quickly merge, clean, or modify data for your ETL processes,
allowing developers to focus more on transformations rather than low-level data handling.

Unlike loading an entire dataset at once, a Data Frame processes information in smaller, manageable chunks.
As it moves through the data, it only keeps a limited number of rows in memory at any given time.
This approach helps avoid running out of memory, making it efficient and scalable for handling large datasets.

Simple example of reading from php array and writing to stdout.

Code

<?php

declare(strict_types=1);

use function Flow\ETL\DSL\{
    data_frame,
    from_array,
    to_stream};

data_frame()
    ->read(
        from_array(
            [
                ['id' => 1, 'array' => ['a' => 1, 'b' => 2, 'c' => 3]],
                ['id' => 2, 'array' => ['a' => 4, 'b' => 5, 'c' => 6]],
                ['id' => 3, 'array' => ['a' => 7, 'b' => 8, 'c' => 9]],
            ],
        )
    )
    ->collect()
    ->write(to_stream(__DIR__ . '/output.txt', truncate: false))
    ->run();

Output

+----+---------------------+
| id |               array |
+----+---------------------+
|  1 | {"a":1,"b":2,"c":3} |
|  2 | {"a":4,"b":5,"c":6} |
|  3 | {"a":7,"b":8,"c":9} |
+----+---------------------+
3 rows

Contributors

Join us on GitHub