Overview

Service Overview

Data Flow is a data processing workflow tool that visually creates processing flows for extracting large volumes of data from various data sources and for transforming and transmitting stream/batch data, and it provides the open-source Apache NiFi. Data Flow can be used independently in the Kubernetes Engine cluster environment of Samsung Cloud Platform, or together with other application software.

Provided features

Data Flow provides the following functions.

• Convenient Installation and Management: Data Flow can be easily installed in a standard Kubernetes cluster environment via the web-based Samsung Cloud Platform Console. It automatically configures the architecture required for scalable clustering based on the open-source Apache NiFi, automatically installing ZooKeeper, Registry, and management modules. With Data Flow, you can configure and deploy configuration files, NiFi templates, and other assets needed for service integration.
• Easy Data Flow Management: You can easily create processing flows for stream/batch data in a GUI that matches the user environment, and by authoring data processing flows in a GUI, you can efficiently extract, transmit, and process data between systems.
• NiFi Template Gallery: You can share/distribute reference NiFi templates. Data Flow provides work files for data processing flows commonly used in the field as a gallery, and users can share the data processing flow work they have created.

Component

Data Flow consists of Manager and Service modules, and is provided packaged with Apache NiFi.

Data Flow Manager

Data Flow Manager provides various management functions to enable more efficient use of NiFi.

• You can upload the Nar file created by the customer through the Data Flow Manager for use in the Processor, and upload configuration files to share them.
• Frequently used NiFi templates are packaged as assets and offered in the Gallery, ready for use with a single click.
• Provides real-time monitoring of multiple services configured for the native NiFi service, as well as resource status monitoring.
• You can easily provision configuration information for NiFi components within the cluster.

Data Flow Service

• We provide a data flow management service based on Apache NiFi.
• Automatically configures the architecture required for scalable clustering based on Apache NiFi, and automatically installs the Nifi, ZooKeeper, and Nifi Registry modules.
• When providing Nifi, you can set the Description, required resource size, connection ID/PW, and Host Alias.
• After creating the service, you can modify the Description, required resource size, connection password, Host Alias, and other settings, and apply the changes to the service.

Server spec type

When creating a Data Flow service, check the following.

• Recommended Service Installation Specifications: CPU 21 core, Memory 57 GB, Storage at least 100 GB

Provision status by region

Data Flow is available in the environments below.

Preliminary Service

This is a list of services that must be pre‑configured before creating the service. Please refer to the guide provided for each service and prepare in advance.