UNIFIED DATA PROCESSING FRAMEWORK

composer require flow-php/etl ^0.10.0

Changelog

Extracts

Read from various data sources.

Transforms

Shape and optimize for your needs.

Loads

Store and secure in one of many available data sinks.

Modules

DSL Functions

DSL stands for Domain Specific Language. In the case of Flow, the DSL is used to define simple functions that can be used to transform data. Most of those functions are initializing a new instance of a class under the hood since Flow is fully object-oriented. Please look at the examples below to get a better understanding of how to use the DSL functions.

EXTRACTOR

dbal_from_queries

source

/**
 * @param null|ParametersSet $parameters_set - each one parameters array will be evaluated as new query
 * @param array<int|string, DbalArrayType|DbalParameterType|DbalType|int|string> $types
 */
dbal_from_queries(Connection $connection, string $query, ?ParametersSet $parameters_set, array $types) : DbalQueryExtractor

dbal_from_query

source

/**
 * @param array<string, mixed>|list<mixed> $parameters - @deprecated use DbalQueryExtractor::withParameters() instead
 * @param array<int|string, DbalArrayType|DbalParameterType|DbalType|int|string> $types - @deprecated use DbalQueryExtractor::withTypes() instead
 */
dbal_from_query(Connection $connection, string $query, array $parameters, array $types) : DbalQueryExtractor

from_dbal_limit_offset

source

/**
 * @param Connection $connection
 * @param string|Table $table
 * @param array<OrderBy>|OrderBy $order_by
 * @param int $page_size
 * @param null|int $maximum
 *
 * @throws InvalidArgumentException
 */
from_dbal_limit_offset(Connection $connection, Table|string $table, OrderBy|array $order_by, int $page_size, ?int $maximum) : DbalLimitOffsetExtractor

from_dbal_limit_offset_qb

source

/**
 * @param Connection $connection
 * @param int $page_size
 * @param null|int $maximum
 */
from_dbal_limit_offset_qb(Connection $connection, QueryBuilder $queryBuilder, int $page_size, ?int $maximum) : DbalLimitOffsetExtractor

HELPER

dbal_dataframe_factory

source

/**
 * @param array<string, mixed>|Connection $connection
 * @param string $query
 * @param QueryParameter ...$parameters
 */
dbal_dataframe_factory(Connection|array $connection, string $query, QueryParameter $parameters) : DbalDataFrameFactory

LOADER

to_dbal_table_insert

source

/**
 * In order to control the size of the single insert, use DataFrame::chunkSize() method just before calling DataFrame::load().
 *
 * @param array<string, mixed>|Connection $connection
 * @param array{
 *  skip_conflicts?: boolean,
 *  constraint?: string,
 *  conflict_columns?: array<string>,
 *  update_columns?: array<string>,
 *  primary_key_columns?: array<string>
 * } $options - @deprecated use DbalLoader::withOperationOptions() instead
 *
 * @throws InvalidArgumentException
 */
to_dbal_table_insert(Connection|array $connection, string $table, array $options) : DbalLoader

to_dbal_table_update

source

/**
 *  In order to control the size of the single request, use DataFrame::chunkSize() method just before calling DataFrame::load().
 *
 * @param array<string, mixed>|Connection $connection
 * @param array{
 *  skip_conflicts?: boolean,
 *  constraint?: string,
 *  conflict_columns?: array<string>,
 *  update_columns?: array<string>,
 *  primary_key_columns?: array<string>
 * } $options - @deprecated use DbalLoader::withOperationOptions() instead
 *
 * @throws InvalidArgumentException
 */
to_dbal_table_update(Connection|array $connection, string $table, array $options) : DbalLoader

Contributors

Join us on GitHub