Spring Cloud Data Flow for Cloud Foundry 1.12

Using the Spring Cloud Data Flow shell

Last Updated October 23, 2024

The open-source Spring Cloud Data Flow project provides a shell, which can be used to interact locally with a Data Flow service instance deployed on VMware Tanzu Application Service for VMs (TAS for VMs).

To have read and write access to a Spring Cloud Data Flow for VMware Tanzu service instance, you must have the SpaceDeveloper role in the space where the service instance was created. If you have only the SpaceAuditor role in the space where the service instance was created, you have only read (not write) access to the service instance.

You can use the shell with Spring Cloud Data Flow for VMware Tanzu service instances in either of two ways:

The Spring Cloud Data Flow for VMware Tanzu Cloud Foundry CLI plug-in was created to ease the use of the Spring Cloud Data Flow shell with Spring Cloud Data Flow service instances on TAS for VMs.

To run the Spring Cloud Data Flow shell, you must have a Java Runtime Environment (JRE) installed. You can download the JRE from the Java website.

Before connecting to a Data Flow service instance with the Spring Cloud Data Flow shell, be sure to log in to the TAS deployment using the Cloud Foundry Command Line Interface tool (cf CLI) and target the org and space of the Data Flow service instance. The Data Flow shell uses the cf CLI to authenticate to TAS.

Using the Cloud Foundry CLI plug-in

You can install the CLI plugin for the Cloud Foundry shell using the following command:

cf install-plugin -r CF-Community "spring-cloud-dataflow-for-pcf"

The plug-in adds a dataflow-shell command, which attaches the Data Flow shell to a Spring Cloud Data Flow for VMware Tanzu service instance. Given an existing service instance named data-flow, the following command will download the appropriate shell version and attach it to the service instance:

$ cf dataflow-shell data-flow
Attaching shell to dataflow service data-flow in org myorg / space dev as user...
Launching dataflow shell JAR
  ____                              ____ _                __
 / ___| _ __  _ __(_)_ __   __ _   / ___| | ___  _   _  __| |
 \___ \| '_ \| '__| | '_ \ / _` | | |   | |/ _ \| | | |/ _` |
  ___) | |_) | |  | | | | | (_| | | |___| | (_) | |_| | (_| |
 |____/| .__/|_|  |_|_| |_|\__, |  \____|_|\___/ \__,_|\__,_|
  ____ |_|    _          __|___/                 __________
 |  _ \  __ _| |_ __ _  |  ___| | _____      __  \ \ \ \ \ \
 | | | |/ _` | __/ _` | | |_  | |/ _ \ \ /\ / /   \ \ \ \ \ \
 | |_| | (_| | || (_| | |  _| | | (_) \ V  V /    / / / / / /
 |____/ \__,_|\__\__,_| |_|   |_|\___/ \_/\_/    /_/_/_/_/_/

2.1.0.RELEASE

Welcome to the Spring Cloud Data Flow shell. For assistance hit TAB or type "help".
dataflow:>

For more information about the Spring Cloud Data Flow shell, see its documentation.

Using the shell manually

If you would like to download and configure the Data Flow shell manually, download the shell JAR file from the Spring Releases Maven repository. Spring Cloud Data Flow for VMware Tanzu can be used with version 2.1.0.RELEASE or later of the Spring Cloud Data Flow shell.

To target a Data Flow service instance’s server with the shell, you must obtain the server’s URL. Run cf service SERVICE_NAME, where SERVICE_NAME is the name of the service instance:

$ cf service data-flow

Service instance: data-flow
Showing info of service data-flow in org myorg / space dev as user...

name:            data-flow
service:         p-dataflow
bound apps:
tags:
plan:            standard
description:     Deploys Spring Cloud Data Flow servers to orchestrate data pipelines
documentation:
dashboard:       https://p-dataflow.apps.example.com/instances/2f6ec0c6-c828-45bb-905a-4779ce50552b/dashboard

Showing status of last operation from service data-flow...

status:    create succeeded
message:   Created
started:   2018-05-17T14:43:38Z
updated:   2018-05-17T14:49:43Z

Visit the URL given for “Dashboard” and authenticate using your TAS credentials. When redirected to the service instance’s dashboard, copy the dashboard’s domain name (the URL minus the path following the domain).

For example, given the following URL:

https://dataflow-2f6ec0c6-c828-45bb-905a-4779ce50552b.apps.example.com/dashboard/#/apps

Copy the following:

https://dataflow-2f6ec0c6-c828-45bb-905a-4779ce50552b.apps.example.com

From the directory containing the shell JAR file, run the shell from the command line using a command as shown below.

$ java -jar JAR_NAME SKIP_VALIDATION --dataflow.uri=SERVER_URL --dataflow.credentials-provider-command="cf oauth-token" --dataflow.mode=skipper

In this command, replace the following placeholders as shown below.

  • JAR_NAME with the name of the downloaded Spring Cloud Data Flow shell JAR file
  • SKIP_VALIDATION with the flag and value --dataflow.skip-ssl-validation=true if your TAS installation is using self-signed SSL certificates; otherwise remove
  • SERVER_URL with the Data Flow service instance’s server URL (the domain copied above)

The complete command may look something like the following:

$ java -jar spring-cloud-dataflow-shell-2.1.0.RELEASE.jar \
--dataflow.skip-ssl-validation=true \
--dataflow.uri=https://dataflow-2f6ec0c6-c828-45bb-905a-4779ce50552b.apps.example.com \
--dataflow.credentials-provider-command="cf oauth-token" --dataflow.mode=skipper

When you run the java -jar command, the shell will initialize and give a dataflow:> prompt.

Welcome to the Spring Cloud Data Flow shell. For assistance hit TAB or type "help".
dataflow:>

For a list of available shell commands, run help from within the shell. For information about the flags which can be provided to the shell on startup, see the OSS Spring Cloud Data Flow project’s documentation.