Getting Started (on AWS)

OpenShot Cloud API is easy to install and configure. Launch your own private instance using AWS Marketplace, or by following this link: https://aws.amazon.com/marketplace/pp/B074H87FSJ/. The following video walks you though the configuration process.

See also Getting Started (on Azure).

This tutorial video covers the basics of how to configure an OpenShot Cloud API instance, and walks you through the process of creating your first video project.

Launch an Instance

After you launch an instance of the OpenShot Cloud API, it is initially configured to run as both a web server and task server (server), and as a video processor (worker). Once you have started your instance, you can access the web server without changing anything.

Access your new instance in a web browser. Use its public DNS or IP address. http://YourInstanceIP/

_static/root-ui.png

Architecture

This diagram illustrates the basic architecture of OpenShot Cloud API. A server listens for video editing tasks, and then sends those to an available worker. A worker processes each job (one at a time), rendering a final video. When a worker is done, it can optionally POST to a webhook URL, upload the video to S3, or the video file can be downloaded as needed. All servers and workers share files using HTTP, for accessing source assets (i.e. logos, videos, music files) and for sharing the final rendered output video.

The server should be configured with a large enough EBS volume to handle all files you intend to store. And if you are done with a project after exporting/rendering, it is best to delete it (and the source files will be removed automatically).

_static/diagram.png

REST: Representational State Transfer

REST is a design pattern for web services modeled around HTTP requests. Each operation uses its own HTTP method:

Method Description
GET Get data from the API (i.e. request a project, clip, or export)
POST Create data on the API (i.e. create a project, clip, or export)
PUT Update existing data from the API (i.e. update a project, clip, or export)
PATCH Update partial existing data from the API (i.e. update only a Clip’s position)
DELETE Delete existing data from the API (i.e. delete a project, clip, or export)

Before You Begin

OpenShot Cloud API uses AWS IAM user credentials to access S3 and SQS. Before you begin, you will need to create a new IAM user on AWS, and give it the following permissions: AmazonS3FullAccess & AmazonSQSFullAccess.

The following details will be required:
  • IAM User
    • Access Key
    • Secret Access Key
    • Permissions: AmazonS3FullAccess & AmazonSQSFullAccess
  • SQS Queue
    • Create a queue named OpenShotAPIExportQueue
    • Standard queue with default settings

OpenShot Cloud API will not work without this configuration, so please verify you have things configured correctly before moving forward.

Server Settings

To configure the settings of your new instance, SSH into the instance using your private key and the ubuntu user:

ssh -i private-key.pem ubuntu@xxx.xxx.xxx.xxx

Run the following command to configure your server for the first time:

config-openshot-cloud

When setting up your first instance, it is recommended to configure it as both a server and worker, by choosing the first option: (B)oth.

Choose a ROLE for this instance.
  (B)oth   - Runs both the HTTP API, DB, and video processing
  (S)erver - Runs only the HTTP API and the DB (no video processing)
  (W)orker - Runs only the video processing tasks

When configuring your server, it is very important to provide the following AWS settings. If these settings are invalid, OpenShot Cloud API will fail to process video exports, and S3 features will throw errors.

# This information is available for your AWS IAM User (mentioned above)
AWS Access Key ID?
AWS Secret Access Key?
AWS SQS Queue? (OpenShotAPIExportQueue)
AWS Region (us-east-1, us-west-2, eu-west-1, etc...)?

Worker Settings

If you are only running a single instance (server and worker), you can skip this section. If you are launching additional worker instances, this section covers how to connect them back to the OpenShot Cloud API server.

To configure the settings of your new worker instance, SSH into the instance using your private key and the ubuntu user:

ssh -i private-key.pem ubuntu@xxx.xxx.xxx.xxx

Run the following command to configure your worker for the first time, and choose the second option: (W)orker.

config-openshot-cloud

For each worker instance you launch, you will be prompted to provide the Cloud API URL of the server (which can be an internal IP address) and to provide AWS Settings (for SQS and S3 - same as configuring the server above). This allows the worker to watch the SQS queue for new video export tasks/jobs, download and upload files from S3, and use the API URL to update the export record with progress, status, and the final rendered video file.

NOTE: The worker must be able to communicate over port 80 with the API server.

Now that you have configured your worker instance, it should be listening for new tasks from the server, and updating the Export record on the server when it completes tasks. Each Export record contains the hostname of the worker that processed it. So, create a test Project and then create an Export, and you will see the hostname of the worker added to the Export record.

Verify AWS Configuration

When config-openshot-cloud has completed, it will try and verify your AWS configuration, and will let you know if anything went wrong. If you see a failure, please double check your AWS credentials and SQS queue name and run config-openshot-cloud again. You can also access http://YourInstanceIP/aws/validate/ at any time to verify your AWS configuration.

Admin Interface

A full admin interface is available at http://YourInstanceIP/cloud-admin/, which allows you to create additional user logins, manage your video projects, and search your data.

Python Example Script

Here is a simple example client script, using Python 3. It connects, creates a new project, uploads a file, creates a clip, and then exports a new video. It should give you a good sense of how things work in general, regardless of programming language. Learn more about the API Endpoints.

import os
import time
from requests import get, post
from requests.auth import HTTPBasicAuth

PATH = os.path.dirname(os.path.realpath(__file__))
CLOUD_URL = 'http://cloud.openshot.org'
CLOUD_AUTH = HTTPBasicAuth('demo-cloud', 'demo-password')


##########################################################
# Get list of projects
end_point = '/projects/'
r = get(CLOUD_URL + end_point, auth=CLOUD_AUTH)
print(r.json())


##########################################################
# Create new project
end_point = '/projects/'
project_data = {
    "name": "API Project",
    "width": 1920,
    "height": 1080,
    "fps_num": 30,
    "fps_den": 1,
    "sample_rate": 44100,
    "channels": 2,
    "channel_layout": 3,
    "json": "{}",
}
r = post(CLOUD_URL + end_point, data=project_data, auth=CLOUD_AUTH)
print(r.json())
project_id = r.json().get("id")
project_url = r.json().get("url")


##########################################################
# Upload file to project
end_point = '/projects/%s/files/' % project_id
source_path = os.path.join(PATH, "example-video.mp4")
source_name = os.path.split(source_path)[1]
file_data = {
    "media": None,
    "project": project_url,
    "json": "{}"
}
r = post(CLOUD_URL + end_point, data=file_data, files={"media": (source_name, open(source_path, "rb"))}, auth=CLOUD_AUTH)
file_url = r.json().get("url")
print(r.json())


##########################################################
# Create a clip for the previously uploaded file
end_point = '/projects/%s/clips/' % project_id
clip_data = {
    "file": file_url,
    "position": 0.0,
    "start": 0.0,
    "end": 30.0,
    "layer": 1,
    "project": project_url,
    "json": "{}"
}
r = post(CLOUD_URL + end_point, data=clip_data, auth=CLOUD_AUTH)
print(r.json())


##########################################################
# Create export for final rendered video
end_point = '/projects/%s/exports/' % project_id
export_data = {
    "video_format": "mp4",
    "video_codec": "libx264",
    "video_bitrate": 8000000,
    "audio_codec": "ac3",
    "audio_bitrate": 1920000,
    "start_frame": 1,
    "end_frame": None,
    "project": project_url,
    "json": "{}"
}
r = post(CLOUD_URL + end_point, data=export_data, auth=CLOUD_AUTH)
export_url = r.json().get("url")
print(r.json())


##########################################################
# Wait for Export to finish (give up after around 40 minutes)
export_output_url = None
is_exported = False
countdown = 500
while not is_exported and countdown > 1:
    r = get(export_url, auth=CLOUD_AUTH)
    print(r.json())
    is_exported = float(r.json().get("progress", 0.0)) == 100.0
    countdown -= 1
    time.sleep(5.0)

# Get final rendered url
r = get(export_url, auth=CLOUD_AUTH)
export_output_url = r.json().get("output")
print(r.json())
print("Export Successfully Completed: %s!" % export_output_url)

Node.js Example Script

Here is a simple example client script, using JavaScript and Node.js. It connects, creates a new project, uploads some files, creates some clips, exports a video, and then downloads the new video file. It should give you a good sense of how things work using asynchronous JavaScript. Learn more about the API Endpoints.

var request = require('request-promise');
var fs = require('fs');
var Promise = require("promise");
var promisePoller = require('promise-poller').default;

var projectId = '';
var projectUrl = '';
var exportId = '';
const projectData = {
    'json': '{}',
    'name': 'My Project Name'
};

const protocol = 'http';
const server = 'cloud.openshot.org';
const auth = { 'user': 'demo-cloud',
               'pass': 'demo-password'};

// Create a new project
post('/projects/', projectData)
    .then(function(response){
        projectUrl = JSON.parse(response).url;
        projectId = JSON.parse(response).id;
        console.log('Successfully created project: ' + projectUrl);

        // Add a couple clips (to be asynchronously processed)
        const promiseArray = [];
        promiseArray.push(
            createClip({
                "path": "media-files/MyVideo.mp4",
                "position": 0.0,
                "end": 10.0
            }));
        promiseArray.push(
            createClip({
                "path": "media-files/MyAudio.mp4",
                "position": 0.0,
                "end": 10.0
            }));
        promiseArray.push(
            createClip({
                "path": "media-files/Watermark.JPG",
                "position": 0.0,
                "end": 10.0,
                "layer": 2
            }));

        // Wait until all files and clips are uploaded
        Promise.all(promiseArray).then(function(responseArray){
            // Export as a new video
            exportData = {
                "export_type": "video",
                "video_format": "mp4",
                "video_codec": "libx264",
                "video_bitrate": 8000000,
                "audio_codec": "ac3",
                "audio_bitrate": 1920000,
                "project": projectUrl,
                "json": '{}'
            };
            post('/exports/', exportData)
                .then(function(response){
                    // Export has been created and will begin processing soon
                    var exportUrl = JSON.parse(response).url;
                    exportId = JSON.parse(response).id;
                    console.log('Successfully created export: ' + exportUrl);

                    // Poll until the export has finished
                    var poller = promisePoller({
                        taskFn: isExportCompleted,
                        interval: 1000,
                        retries: 60*60*1000,
                        timeout: 2000
                    }).then(function(exportOutputUrl) {
                        // New exported video is ready for download now
                        console.log('Download ' + exportOutputUrl);
                        request(exportOutputUrl).pipe(fs.createWriteStream('Output-' + projectId + '.mp4'));
                    });
                });
        });
    });

function isExportCompleted() {
    return new Promise(function (resolve, error) {

        get('/exports/' + exportId + '/', {})
            .then(function(response) {
                var exportStatus = JSON.parse(response).status;
                var exportOutputUrl = JSON.parse(response).output;
                if (exportStatus == 'completed') {
                    console.log('Export completed: ' + JSON.stringify(response));
                    resolve(exportOutputUrl);
                }
            });
    });
}

function createClip(clip) {
    // Create new File object (and upload file from filesystem)
    var fileData = {
        'json': '{}',
        'project': projectUrl,
        'media': fs.createReadStream(clip.path)
    };

    return new Promise(function (resolve) {
        post('/files/', fileData)
            .then(function(response) {
                // File uploaded and object created
                var fileUrl = JSON.parse(response).url;
                console.log('Successfully created file: ' + fileUrl);

                // Now we need to add a clip which references this new File object
                var clipData = {
                    'file': fileUrl,
                    'json': '{}',
                    'position': clip.position || 0.0,
                    'start': clip.start || 0.0,
                    'end': clip.end || JSON.parse(response).json.duration,
                    'layer': clip.layer || 0,
                    'project': projectUrl
                };
                return post('/clips/', clipData).then(function(response){
                    var clipUrl = JSON.parse(response).url;
                    console.log('Successfully created clip: ' + clipUrl);
                    resolve();
                });
            });
    });
}

function post(endpoint, data) {
    // Prepare request object and POST data to OpenShot API
    const r = request.post(protocol + '://' + auth.user + ':' + auth.pass + '@' + server + endpoint, function (err, response, body) {
        if (err) {
            return err;
        } else {
            console.log(response.statusCode + ': ' + body);
            return body;
        }
    });

    // Append form data to request form-data
    const form_data = r.form();
    for ( var key in data ) {
        form_data.append(key, data[key]);
    }
    return r;
}

function get(endpoint, data) {
    // Prepare request object and GET data to OpenShot API
    return request.get(protocol + '://' + auth.user + ':' + auth.pass + '@' + server + endpoint, function (err, response, body) {
        if (err) {
            return err;
        } else {
            console.log(response.statusCode + ': ' + body);
            return body;
        }
    });
}