Getting Started: ODB on a Local Development Environment

Last Updated October 25, 2024

This topic tells you how to create and manage an on-demand service broker (ODB) on a local development machine using PCF Dev and BOSH Lite.

For more information about the components in this topic, see PCF Dev and BOSH-Lite.

The examples in this topic are based on Kafka open source messaging. See the following sample code directories:

Prerequisites

Before you set up and use ODB on your local machine, install and configure the following components:

BOSH Lite v9000.131.0 or later. To install BOSH Lite, see Install in the BOSH Lite documentation.
PCF Dev file pcfdev-v0.19.1-rc.46. To install PCF Dev, see Installing PCF Dev. Record the PCF Dev domain for later. The default is local.pcfdev.io.

Part 1: Set Up

This section details how to prepare BOSH Lite and set up the Kafka example service, the Kakfa example service adapter, and ODB.

Step 1: Prepare BOSH Lite

To prepare BOSH Lite, do the following:

Target your BOSH Lite installation.
```
bosh alias-env lite -e 192.168.50.4
```

Upload the BOSH Lite stemcell.

bosh -e lite upload-stemcell \
https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent?v=3262.2

Step 2: Set Up the Kafka Example Service

To set up the Kafka example service, do the following:

Clone the Kafka example service into your workspace.

git clone \
https://github.com/pivotal-cf-experimental/kafka-example-service-release.git

In the kafka-example-service-release directory, create and upload the Kafka example service.
```
cd kafka-example-service-release
bosh create-release --name kafka-example-service
```
Upload the service to the BOSH director.
```
bosh -e lite upload-release
```

Step 3: Set Up the Kafka Example Service Adapter

To set up the Kafka example service adapter, do the following:

Clone the Kafka example service adapter.

git clone \
https://github.com/pivotal-cf-experimental/kafka-example-service-adapter-release.git

Update service adapter dependencies.

cd kafka-example-service-adapter-release
git submodule update --init --recursive

Create the example service adapter.

bosh create-release --name kafka-example-service-adapter

Upload the example service adapter to the BOSH director.
```
bosh -e lite upload-release
```

Step 4: Set Up ODB

To set up ODB, do the following:

Download the on-demand service broker from VMware Tanzu Network. To download, see On-Demand Services SDK.

Upload the on-demand-service-broker release.

bosh -e lite upload-release on-demand-service-broker-X.Y.Z.tgz

Where X.Y.Z is the ODB release version.

For example:

$ bosh -e lite upload-release on-demand-service-broker-0.21.1.tgz

Part 2: Create

This section describes how to create a BOSH deployment and a service broker on PCF Dev.

Step 1: Create a BOSH Deployment

To create a BOSH Lite deployment, do the following:

Create a new directory in your workspace and a cloud_config.yml for the BOSH Lite Director. For example:

vm_types:
- name: container
  cloud_properties: {}

networks:
- name: kafka
  type: manual
  subnets:
  - range: 10.244.1.0/24
    gateway: 10.244.1.1
    az: lite
    cloud_properties: {}

disk_types:
- name: ten
  disk_size: 10_000
  cloud_properties: {}

azs:
- name: lite
  cloud_properties: {}

compilation:
  workers: 2
  reuse_compilation_vms: true
  network: kafka
  az: lite
  cloud_properties: {}

Update the BOSH Lite cloud config using the deployment manifest.
```
bosh -e lite update-cloud-config cloud\_config.yml
```

Record the URL and UUID of your BOSH Lite director.

bosh environment

See the following example output:

$ bosh environment
Config
             /Users/pivotal/.bosh_config

Director
  Name       Bosh Lite Director
  URL        https://192.168.50.4:25555
  Version    1.3215.0 (00000000)
  User       admin
  UUID       17a45148-1d00-43bc-af28-9882e5a6535a
  CPI        warden\_cpi
  dns        disabled
  compiled\_package\_cache enabled (provider: local)
  snapshots  disabled

Create a BOSH Lite deployment manifest in a file called deployment_manifest.yml using the following as a base:

name: kafka-on-demand-broker

director_uuid: BOSH-LITE-UUID

releases:
- name: &broker-release on-demand-service-broker
  version: latest
- name: &service-adapter-release kafka-example-service-adapter
  version: latest
- name: &service-release kafka-example-service
  version: latest

stemcells:
- alias: trusty
  os: ubuntu-trusty
  version: STEMCELL-VERSION

instance_groups:
- name: broker
  instances: 1
  vm_type: container
  persistent_disk_type: ten
  stemcell: trusty
  azs: [lite]
  networks:
  - name: kafka
  jobs:
  - name: kafka-service-adapter
    release: *service-adapter-release
  - name: admin_tools
    release: *service-release
  - name: broker
    release: *broker-release
    properties:
      port: 8080
      username: broker    # or replace with your own
      password: password  # or replace with your own
      disable_ssl_cert_verification: true
      bosh:
        url: BOSH-LITE-URL
        authentication:
          basic:
            username: admin
            password: admin
      cf:
        url: https://api.PCF-DEV-DOMAIN
        authentication:
          url: https://uaa.PCF-DEV-DOMAIN
          user_credentials:
            username: admin
            password: admin
      service_adapter:
        path: /var/vcap/packages/odb-service-adapter/bin/service-adapter
      service_deployment:
        releases:
        - name: *service-release
          version: SERVICE-RELEASE-VERSION
          jobs: [kafka_server, zookeeper_server]
        stemcells:
          - os: ubuntu-trusty
            version: STEMCELL-VERSION
      service_catalog:
        id: D94A086D-203D-4966-A6F1-60A9E2300F72
        service_name: kafka-service-with-odb
        service_description: Kafka Service
        bindable: true
        plan_updatable: true
        tags: [kafka]
        plans:
        - name: small
          plan_id: 11789210-D743-4C65-9D38-C80B29F4D9C8
          description: A Kafka deployment with a single instance of each job and persistent disk
          instance_groups:
          - name: kafka_server
            vm_type: container
            instances: 1
            persistent_disk_type: ten
            azs: [lite]
            networks: [kafka]
          - name: zookeeper_server
            vm_type: container
            instances: 1
            persistent_disk_type: ten
            azs: [lite]
            networks: [kafka]
          properties:
            auto_create_topics: true
            default_replication_factor: 1

update:
  canaries: 1
  canary_watch_time: 30000-180000
  update_watch_time: 30000-180000
  max_in_flight: 4

Where:

BOSH-LITE-UUID is the UUID value you recorded in the BOSH environment step above.
BOSH-LITE-URL is the URL value you recorded in the BOSH environment step above.
PCF-DEV-DOMAIN is the PCF Dev domain you recorded in the Prerequisites above.

Deploy the broker.

bosh -e lite -d kafka-on-demand-broker deployment\_manifest.yml

Record the IP address of the deployed broker.

bosh -e lite -d kafka-on-demand-broker instances

See the following example output:

$ Using environment 'lite' as user 'admin' (openid, bosh.admin)

Task 54727. Done

Deployment 'redis-on-demand-broker-dev2'

Instance                                     Process State  AZ  IPs
broker/84294753-84b9-4be1-a338-37c1f3e71919  running        z1  10.244.1.2

1 instances

Succeeded

Step 2: Create a Service Broker on PCF Dev

To create a service broker on PCF Dev, do the following:

Create a service broker on PCF Dev and enable access to its service offering.
```
cf create-service-broker kafka-broker USERNAME PASSWORD http://BROKER-IP:8080
```
Where:
- USERNAME and PASSWORD are the broker’s credentials set under properties in the broker job.
- BROKER-IP is the value obtained in the step above. See the last step in Create a BOSH Deployment.
For example:
```
$ cf create-service-broker kafka-broker broker password http://10.244.1.2:8080
```

Enable access to the broker’s service plans.

cf enable-service-access kafka-service-with-odb

View the broker-offered services in the Marketplace.

cf marketplace

See the following example output:

Getting services from Marketplace in org pcfdev-org / space pcfdev-space as admin...
OK

service                  plans        description
kafka-service-with-odb   small        Kafka Service
p-mysql                  512mb, 1gb   MySQL databases on demand
p-rabbitmq               standard     RabbitMQ is a robust and scalable high-performance multi-protocol messaging broker.
p-redis                  shared-vm    Redis service to provide a key-value store

Create a service instance using the Kafka on-demand broker.
```
cf create-service kafka-service-with-odb small k1
```

Part 3: Verify and Use

To verify and use your on-demand service, do the following:

Step 1: Verify Your BOSH Deployment and On-Demand Service

Check the status of your service.
```
cf service k1
```
See the state change from create in progress to create succeeded.

Verify that the on-demand service is provisioned in the BOSH deployment.

bosh -e lite deployments

See the following example output:

Name                                                    Release(s)                              Stemcell(s)                                        Cloud Config
kafka-on-demand-broker                                  kafka-example-service-adapter/0+dev.2   bosh-warden-boshlite-ubuntu-trusty-go\_agent/3262.2   latest
                                                        on-demand-service-broker/0.2.0+dev.1
service-instance_2715262c-8564-4cd9-b629-0ae99e6aa4b9   kafka-example-service/0+dev.2           bosh-warden-boshlite-ubuntu-trusty-go\_agent/3262.2   latest

This example shows that the service instance is provisioned and the service releases are specified in the ODB deployment manifest.

Step 2: Use Your On-Demand Service

To use the service instance that you created, do the following:

Clone the Kafka example app.

git clone https://github.com/pivotal-cf-experimental/kafka-example-app.git

Push the app.
```
cd kafka-example-app
cf push --no-start
```
Bind the app to your service instance.
```
cf bind-service kafka-example-app k1
```
Start the app.
```
cf start kafka-example-app
```

Step 3: Read and write to your Service Instance

Now the app runs at https:/kafka-example-app.PCF-DEV-DOMAIN. You can use it to read and write to your on-demand Kafka service instance.

For example:

To write data, run the following.

curl -XPOST http://kafka-example-app.PCF-DEV-DOMAIN/queues/my-queue -d SOME-DATA

To read data, run the following.

curl http://kafka-example-app.PCF-DEV-DOMAIN/queues/my-queue

Content feedback and comments

On-Demand Services SDK for Tanzu 0.42

Getting Started&#58; ODB on a Local Development Environment

Prerequisites

Part 1: Set Up

Step 1: Prepare BOSH Lite

Step 2: Set Up the Kafka Example Service

Step 3: Set Up the Kafka Example Service Adapter

Step 4: Set Up ODB

Part 2: Create

Step 1: Create a BOSH Deployment

Step 2: Create a Service Broker on PCF Dev

Part 3: Verify and Use

Step 1: Verify Your BOSH Deployment and On-Demand Service

Step 2: Use Your On-Demand Service

Step 3: Read and write to your Service Instance

Getting Started: ODB on a Local Development Environment