Getting Started (on Google Cloud)

OpenShot Cloud API is easy to install and configure. Launch your own private virtual machine using Google Cloud Marketplace, or by following this link: https://console.cloud.google.com/marketplace/product/openshotstudios-public/openshot-video-editing-cloud-api. The following video walks you though the configuration process.

See also Getting Started (on AWS).
See also Getting Started (on Azure).
See also API Endpoints / Schema.

This tutorial video covers the basics of how to configure an OpenShot Cloud API instance, and walks you through the process of creating your first video project. This video uses AWS, so a few steps might be different on Azure and Google Cloud, but generally, the process is very similar.

Launch an Instance

After you launch a virtual machine instance of the OpenShot Cloud API, it is initially configured to run as both a web server (server) and a video renderer (worker). Once you have started your instance, you can access the web server on port 80, however we need to configure a few things before OpenShot Cloud API is ready to use.

Access your new instance in a web browser. Use its public DNS or IP address. http://YourInstanceIP/

_images/root-ui.png

Architecture

This diagram illustrates the basic architecture of OpenShot Cloud API. A server listens for video editing commands using a HTTP REST API, and stores project data in a local PostgreSQL database. When a video needs to be exported / rendered, it then sends those tasks to an available worker. A worker processes each video project (one at a time), rendering a final video. When a worker is done, it can optionally POST to a webhook URL, upload the video to cloud storage, or the video file can be manually downloaded. All servers and workers share files using HTTP for accessing source assets (i.e. video, audio, image files) and for sharing the final rendered output video. NOTE: You can run the entire architecture on a single virtual machine instance, or optionally separate the workers (which allows for auto-scaling the pool of workers).

The server should be configured with a large enough disk / volume to handle all files you intend to store. When you are done with a project (after exporting/rendering), it is best practice to move the final rendered video to cloud storage and delete the project (which will delete all project-related source files automatically and help manage disk space).

_images/diagram-google-cloud.png

REST: Representational State Transfer

REST is a design pattern for web services modeled around HTTP requests. Each operation uses its own HTTP method:

Method

Description

GET

Get data from the API (i.e. request a project, clip, effect, or export)

POST

Create data on the API (i.e. create a project, clip, effect, or export)

PUT

Update existing data from the API (i.e. update a project, clip, effect, or export)

PATCH

Update partial existing data from the API (i.e. update only a Clip’s position)

DELETE

Delete existing data from the API (i.e. delete a project, clip, effect, or export)

Before You Begin

OpenShot Cloud API uses a Google Project ID (non-numeric project identifier), and the Service Account associated with all OpenShot Cloud API virtual machines. You will need to enable a few APIs, create a few resources, and add a few permissions before OpenShot Cloud API can be configured.

The following details will be required:
  • Google Project ID (non-numeric project identifier)

    • Locate the Service Account address for your OpenShot Cloud API VM, for example: 123456789-compute@developer.gserviceaccount.com. This will be used in the next few steps for granting permissions.

  • Google Pub/Sub Settings (queue for export & transcription):

    • Create Topic: OpenShotCloudAPI

    • Create Subscription: OpenShotCloudAPI-sub

    • Topic: Requires permission Pub/Sub Admin role for VM Service Account

    • Subscription: Requires permission Pub/Sub Subscriber role for VM Service Account

  • Google Cloud Storage (file storage):

    • Create Bucket: openshot-files

    • Bucket: Requires permission Storage Object Admin for VM Service Account

  • Project Policies (Google IAM & Admin Organization Policies):

    • Policy for Enforce uniform bucket-level access set to Do not enforce (required to use ACL per object)

  • VM Access Scopes (required for VM to access services):

    • Cloud Pub/Sub: Enabled

    • Storage: Read Write

OpenShot Cloud API will not work without this configuration, so please verify you have things configured correctly before moving forward.

Server Settings

To configure the settings of your new instance, SSH into the instance using your private key and the ubuntu user:

ssh -i private-key.pem ubuntu@xxx.xxx.xxx.xxx

OR use the “SSH” button in Google Cloud to login, and switch to the ubuntu user.

sudo su ubuntu

Run the following command to configure your server for the first time. NOTE: Please use the ubuntu user when running the following command, otherwise you will encounter errors.

config-openshot-cloud

When setting up your first instance, it is recommended to configure it as BOTH a server and worker, by choosing the first option: (B)oth.

Choose a ROLE for this instance.
  (B)oth   - Runs both the HTTP API, DB, and video processing
  (S)erver - Runs only the HTTP API and the DB (no video processing)
  (W)orker - Runs only the video processing tasks

When configuring your server, it is very important to provide the following Google Cloud settings. If these settings are invalid or your VM service account is lacking permissions, OpenShot Cloud API will fail to process video exports, and Cloud Storage features will throw errors.

# Required Google Cloud Settings
Google Project ID? (i.e. non-numeric)
Pub/Sub Topic ID? (OpenShotAPIExportQueue)
Pub/Sub Subscription ID? (OpenShotAPIExportQueue-sub)

Manage Server Files

It is also important to delete old projects and related files to prevent OpenShot Cloud API from running out of disk space. You can do this manually by issuing a HTTP DELETE request once you are done with a project, or you can schedule automatic clean-up of old projects. Enter a value for the # of days a project should remain on the server, before being automatically deleted.

Schedule Clean-Up: Automatically delete Projects older than X days?
   This will automatically delete OLD projects and related data:
   files, clips, effects, and exports. (0 = never delete)

Auto-Delete Projects After # of Days? (0)

Worker Settings

If you are only running a single instance (server and worker), you can skip this section. If you are launching additional worker instances, this section covers how to connect them back to the OpenShot Cloud API server.

To configure the settings of your new worker instance, SSH into the instance using your private key and the ubuntu user:

ssh -i private-key.pem ubuntu@xxx.xxx.xxx.xxx

Run the following command to configure your worker for the first time, and choose the second option: (W)orker.

config-openshot-cloud

For each worker instance you launch, you will be prompted to provide the Cloud API URL of the server (which can be an internal IP address) and to provide Google Cloud Settings (for Pub/Sub and Cloud Storage - same as configuring the server above). This allows the worker to watch the Google Pub/Sub Subscription for new video export tasks/jobs, download and upload files from Google Cloud Storage, and use the API URL to update the export record with progress, status, and the final rendered video file.

NOTE: The worker must be able to communicate over port 80 with the API server.

Now that you have configured your worker instance, it should be listening for new tasks from the server, and updating the Export record on the server when it completes tasks. Each Export record contains the hostname of the worker that processed it. So, create a test Project and then create an Export, and you will see the hostname of the worker added to the Export record.

Manage Worker Files

It is also important to delete old exports and related project files to prevent OpenShot Cloud API from running out of disk space on your worker instance. You can schedule automatic clean-up of these temporary files used during an export. Enter a value for the # of minutes to keep files for a completed video export (0 = never delete).

Schedule Clean-Up: [Worker-Only] Automatically delete Project-related files
   older than X minutes? This does NOT affect the server or in-progress exports,
   only the *temporary* files downloaded on this worker required for video exports.
   (0 = never delete, 60 = default)

Auto-Delete Project Files After # of Minutes? (60)

Verify Google Cloud Configuration

When config-openshot-cloud has completed, it will try and verify your Google Cloud configuration, and will let you know if anything went wrong. If you see a failure, please double check your Google Cloud settings, permissions, policies, and Pub/Sub Topic/Subscription name and run config-openshot-cloud again. You can also access http://YourInstanceIP/settings/google/validate/ at any time to verify your Google Cloud configuration.

Admin Interface

A full admin interface is available at http://YourInstanceIP/cloud-admin/, which allows you to create additional user logins, manage your video projects, and search your data.