Examples

Data frame

Description

Control how many rows are processed at once. Larger batch sizes can improve processing speed but require more memory. There is no universal optimal batch size—it depends on your dataset and transformations.

To process all rows at once, use collect.

Documentation

Batch Processing

Code

<?php

declare(strict_types=1);

use function Flow\ETL\DSL\{data_frame, from_array, to_output};

require __DIR__ . '/vendor/autoload.php';

data_frame()
    ->read(from_array([
        ['id' => 1, 'name' => 'John'],
        ['id' => 2, 'name' => 'Doe'],
        ['id' => 3, 'name' => 'Jane'],
        ['id' => 4, 'name' => 'Smith'],
        ['id' => 5, 'name' => 'Alice'],
    ]))
    ->batchSize(2)
    ->write(to_output(truncate: false))
    ->run();

Contributors

Built in the open.

Join us on GitHub