Object Store Reference Application

Object Store reference application is built to showcase the method of developing a single code-line multi cloud application consuming Object Store Service of SAP Business Technology Platform (SAP BTP) Cloud Foundry Environment.

Description

Object Store service enables the storage and management of objects, which involves creation, upload, download, and deletion of objects. SAP provides Object Store Service on its SAP Business Technology Platform running on different IaaS providers like Amazon Web Service, Azure, Google Cloud Platform. Please click here for more information on Object Store Service.

Though the Object Store Service is provided by SAP on multiple IaaS providers, the way to connect to and use the service varies for each IaaS provider due to changes in the structure of credentials and in the configurations. Writing a single code-line application that works seamlessly on all these IaaS providers is a challenge that many developers face.

We have developed a single code line reference application that can work with Object Store Service on SAP Business Technology Platform Cloud Foundry Environment hosted on multiple IaaS providers. This application performs operations like upload, download, delete and listing of files. It is a spring boot application that uses Apache jclouds library which provides a multi-cloud toolkit that gives a developer the freedom to create applications that are portable across IaaS providers.

Features of the Application

• The application provides RESTful endpoints to upload, download, delete and list files.

• It calls jclouds library's API endpoints to perform the above operations on the files in Object Store Service. JClouds abstracts the code to perform these operation on the different providers like AWS S3, Google Cloud Storage and Azure Storage.

Architecture

A single REST controller accepts the request (GET, POST, DELETE).

Separate service implementations and configuration classes are provided for each of the Object Store Service provider. The right service implementation and configuration is loaded by spring boot based on the IaaS provider that the application is deployed on.

A single DAO (Data Access Object)/repository class calls the jclouds api’s to perform upload, download, delete operations on the Object Store.

Referenced Libraries

Following jclouds dependencies are used in the application.

    <!-- jclouds dependencies -->
    <dependency>
      <groupId>org.apache.jclouds.provider</groupId>
      <artifactId>aws-s3</artifactId>
      <version>2.3.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.jclouds.provider</groupId>
      <artifactId>google-cloud-storage</artifactId>
      <version>2.3.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.jclouds.provider</groupId>
      <artifactId>azureblob</artifactId>
      <version>2.3.0</version>
    </dependency>

The size of each of the jclouds dependencies are as follows:

    Dependency                Size
    -------------             -------
    aws-s3                    29kb
    google-cloud-storage      158kb
    azureblob                 138kb

Besides spring-boot and jclouds the other dependencies used in the application are:

• jackson-databind: to parse json file

• commons-fileupload: to upload files

For more information about the dependencies please refer pom.xml file.

Requirements

• Java 11
• Apache Maven 3.3+
• Cloud Foundry CLI
• SAP Business Technology Platform Global account
• Provision Object Store service. It is a paid service on Cloud Foundry.
• A Cloud Foundry user with SpaceDeveloper role to deploy the application

Download and Installation

Build the Application

• Clone the application cloud-objectstore-java-sample to your system

  Clone URL : https://github.com/SAP-samples/cloud-objectstore-java-sample.git

• Navigate to the root folder of the application and run the below maven command to build the application:

  mvn clean install
  

Deploy the Application on Cloud Foundry

1. Logon to the Cloud Foundry environment using the following commands on the command prompt:

   cf api <api>
   cf login
   

   api - URL of the Cloud Foundry landscape that you are trying to connect to.

   Enter username, password, org and space when prompted to. Please click here for more information.

   Check if the Cloud Foundry Space you will be deploying the application has the following entitlements:

   Landscape	Service	Plan	Number of Instances
   AWS	objectstore	s3-standard	1
   GCP	objectstore	gcs-standard	1
   Azure	objectstore	azure-standard	1

2. Create the Cloud Foundry Object Store Service Instance

   • To run the application on AWS landscape, create a service by executing the below command:

     cf create-service objectstore s3-standard objectstore-service

   • To run the application on GCP landscape, create a service by executing the below command:

     cf create-service objectstore gcs-standard objectstore-service

   • To run the application on Azure landscape, create a service by executing the below command:

     cf create-service objectstore azure-standard objectstore-service

3. Edit manifest.yml file. Replace the <unique_id> placeholder with any unique string. You can use your SAP User ID so that the application name is unique in the CF landscape. You can find your SAP User ID in your sap.com profile.

  ---
  applications:
  - name: <unique_id>-objectstore-sample-svc
    memory: 2G
    buildpack: sap_java_buildpack
    path: target/objectstore-sample-1.1.1.jar
	  env:
      JBP_CONFIG_COMPONENTS: "jres: ['com.sap.xs.java.buildpack.jdk.SAPMachineJDK']"
      JBP_CONFIG_SAP_MACHINE_JRE: '{ jre: { version: "11.+" } }'
      SERVICE_LABEL: "objectstore"
    services:
      - objectstore-service

4. To deploy the application, navigate to the root of the application and execute the below command:

   cf push
   

Test the Application

Postman Client can be used to test / access the REST API endpoints.

Replace the <application URL> placeholder in the below steps with the URL of the application you deployed.

Upload a file / object

POST

To upload a file / object set the below request body and hit the endpoint url.

EndPoint URL : <application URL>/objectstorage.svc/api/v1/storage/

Request Body : form-data with key-value pair. Pass the name of the key as file and the value is the path of the file.

For the file upload, we have provided a test file in the documents folder which you can use if needed for the upload testing.

The file upload option may differ version by version of postman. So keeping the above screenshot as generic, kindly adopt accordingly.

A successful upload operation gives the following response :

Status: 202

Response Body: <file-name> is successfully uploaded.

List all the files / objects

GET

To get the list of a files / objects set the content-type and hit the below endpoint url.

EndPoint URL : https://<application URL>/objectstorage.svc/api/v1/storage/

Content-Type : application/json

A successful GET operation gives the following response :

Status: 200

Response Body:

[
    {
        "etag": "CIjak4uDxeACEAE=",
        "bucket": "sap-cp-osaas-a78345d3-e45d-42eb-9c03-c47393d0d436",
        "name": "SampleFile.pdf",
        "url": "https://www.googleapis.com/storage/v1/b/sap-cp-osaas-a78345d3-e45d-42eb-9c03-c47393d0d436/o/SampleFile.pdf",
        "lastModified": "Mon Feb 18 15:30:22 IST 2019",
        "size": "245.7 KB",
        "contentType": "application/pdf",
        "userMetadata": {
            "description": "sample content"
        }
    },
    {
        "etag": "COf+0p7uxOACEAE=",
        "bucket": "sap-cp-osaas-a78345d3-e45d-42eb-9c03-c47393d0d436",
        "name": "SampleImage.jpg",
        "url": "https://www.googleapis.com/storage/v1/b/sap-cp-osaas-a78345d3-e45d-42eb-9c03-c47393d0d436/o/SampleImage.jpg",
        "lastModified": "Mon Feb 18 13:57:06 IST 2019",
        "size": "46.1 KB",
        "contentType": "image/jpeg",
        "userMetadata": {
            "description": "sample content"
        }
    }
  ...

]

Download a file / object

GET

Please open any browser and hit the below endpoint url to download a file / object rather than using Postman to test it.

EndPoint URL : https://<application URL>/objectstorage.svc/api/v1/storage/{file-name}

Delete a file / object

DELETE

To delete a file / object hit the below endpoint url by appending the file / object name in postman.

EndPoint URL : https://<application URL>/objectstorage.svc/api/v1/storage/{file-name}

A successful upload operation gives the following response :

Status: 200

Response Body: <file-name> is successfully deleted.

How to obtain support

In case you find a bug, or you need additional support, please open an issue here in GitHub.

Known Issues

• Using InputStream instead of ByteArray to upload large files to make the read operation faster causes issues. For more information see here.

License

Copyright (c) 2020 SAP SE or an SAP affiliate company. All rights reserved. This project is licensed under the Apache Software License, version 2.0 except as noted otherwise in the LICENSE file.