/**
* Extractor will automatically try to iterate over whole index using one of the two iteration methods:.
*
* - from/size
* - search_after
*
* Search after is selected when you provide define sort parameters in query, otherwise it will fallback to from/size.
*
* @param array{
* hosts?: array<string>,
* connectionParams?: array<mixed>,
* retries?: int,
* sniffOnStart?: boolean,
* sslCert?: array<string>,
* sslKey?: array<string>,
* sslVerification?: boolean|string,
* elasticMetaHeader?: boolean,
* includePortInHostHeader?: boolean
* } $config
* @param array<mixed> $parameters - https://www.elastic.co/guide/en/elasticsearch/reference/master/search-search.html
* @param ?array<mixed> $pit_params - when used extractor will create point in time to stabilize search results. Point in time is automatically closed when last element is extracted. https://www.elastic.co/guide/en/elasticsearch/reference/master/point-in-time-api.html - @deprecated use withPointInTime method instead
*/
from_es(array $config, array $parameters, ?array $pit_params) : ElasticsearchExtractor
Extracts
Read from various data sources.
Transforms
Shape and optimize for your needs.
Loads
Store and secure in one of many available data sinks.
Modules
DSL Functions
DSL stands for Domain Specific Language. In the case of Flow, the DSL is used to define simple functions that can be used to transform data. Most of those functions are initializing a new instance of a class under the hood since Flow is fully object-oriented. Please look at the examples below to get a better understanding of how to use the DSL functions.
EXTRACTOR
HELPER
entry_id_factory(string $entry_name) : IdFactory
/**
* Transforms elasticsearch results into clear Flow Rows using ['hits']['hits'][x]['_source'].
*
* @return HitsIntoRowsTransformer
*/
es_hits_to_rows(DocumentDataSource $source) : HitsIntoRowsTransformer
hash_id_factory(string $entry_names) : IdFactory
LOADER
/**
* https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-bulk.html.
*
* In order to control the size of the single request, use DataFrame::chunkSize() method just before calling DataFrame::load().
*
* @param array{
* hosts?: array<string>,
* connectionParams?: array<mixed>,
* retries?: int,
* sniffOnStart?: boolean,
* sslCert?: array<string>,
* sslKey?: array<string>,
* sslVerification?: boolean|string,
* elasticMetaHeader?: boolean,
* includePortInHostHeader?: boolean
* } $config
* @param string $index
* @param IdFactory $id_factory
* @param array<mixed> $parameters - https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-bulk.html - @deprecated use withParameters method instead
*/
to_es_bulk_index(array $config, string $index, IdFactory $id_factory, array $parameters) : ElasticsearchLoader
/**
* https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-bulk.html.
*
* In order to control the size of the single request, use DataFrame::chunkSize() method just before calling DataFrame::load().
*
* @param array{
* hosts?: array<string>,
* connectionParams?: array<mixed>,
* retries?: int,
* sniffOnStart?: boolean,
* sslCert?: array<string>,
* sslKey?: array<string>,
* sslVerification?: boolean|string,
* elasticMetaHeader?: boolean,
* includePortInHostHeader?: boolean
* } $config
* @param string $index
* @param IdFactory $id_factory
* @param array<mixed> $parameters - https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-bulk.html - @deprecated use withParameters method instead
*/
to_es_bulk_update(array $config, string $index, IdFactory $id_factory, array $parameters) : ElasticsearchLoader