Introduction

Data Frame

Data Frame
- Building Blocks
- Data Retrieval
- Data Manipulation
- Select/Drop
- Rename
- Map
- Filter
- Execution Mode
- Save Mode
- Join
- Group By
  - Aggregations
- Pivot
- Window Functions
- Sort
- Limit
- Offset
- Until
- Batch Processing
- Caching
- Partitioning
- Constraints
- Schema
- Display
- Error Handling
CLI

Filesystem Async AWS

The Filesystem Async AWS Bridge is a bridge that allows you to use the S3 as a filesystem in your application through Async AWS SDK.

Installation

For detailed installation instructions, see the installation page.

use function Flow\Filesystem\Bridge\AsyncAWS\DSL\{aws_s3_client, aws_s3_filesystem};

$aws = aws_s3_filesystem(
    $_ENV['AWS_S3_BUCKET'],
    aws_s3_client([
        'region' => $_ENV['AWS_S3_REGION'],
        'accessKeyId' => $_ENV['AWS_S3_KEY'],
        'accessKeySecret' => $_ENV['AWS_S3_SECRET'],
    ])
);

$fstab = fstab($aws);

The mount protocol — the URI scheme under which the filesystem is registered in the FilesystemTable — defaults to 'aws-s3'. Override by passing a fourth argument (e.g. aws_s3_filesystem($bucket, $client, options: new Options(), protocol: 'warehouse')) when you need to mount the same bucket twice under distinct names or pick a scheme more meaningful to your application.

Usage with Flow

To use the AWS S3 filesystem with Flow, you need to mount the filesystem to the configuration. This operation will mount the S3 filesystem to the fstab instance available in the DataFrame runtime.

$config = config_builder()
    ->mount(
        aws_s3_filesystem(
            $_ENV['AWS_S3_BUCKET'],
            aws_s3_client([
                'region' => $_ENV['AWS_S3_REGION'],
                'accessKeyId' => $_ENV['AWS_S3_KEY'],
                'accessKeySecret' => $_ENV['AWS_S3_SECRET'],
            ])
        )
    );

data_frame($config)
    ->read(from_csv(path('aws-s3://test.csv')))
    ->write(to_stream(__DIR__ . '/output.txt', truncate: false))
    ->run();

FileStatus values returned from list() and status() carry size (from S3 Size / ContentLength) and lastModifiedAt (from LastModified) populated directly from the S3 response — no extra HEAD call is issued when the CLI flow:filesystem:ls --long or flow:filesystem:stat prints them.

Adapters

Libraries

PHP Extensions

Symfony

Bridges

Contributors

Join us on GitHub