Performance / Scaling

OpenShot Cloud API was designed with scalability in mind. When configured correctly, you can achieve huge and rapid scalability on cloud infrastructure. This topic can get very tricky though, so feel free to send an email to discuss your specific application: cloud-support@openshot.org.

Performance

The performance of video exports/renders is highly dependent on CPU and Memory. Ideally, you want as many CPUs and as much memory as you can afford. A general rule of thumb is the average render takes about as long as the video length. A 60 second 1080p video will usually take around 60 seconds to render. If you have a more powerful EC2 instance type, and/or are rendering at a lower resolution, it will be faster than real-time. On a slower instance type, it will be slower than real-time.

Instance Types

There are a large number of possible cloud instance types and sizes, and some of them perform very differently than others. Here are some recommendations that work well for OpenShot Cloud API. In general, we recommend the compute optimized instance types.

AWS Instance Types

c5.xlarge	Lower cost, but slower than real-time performance
c5.2xlarge	More expensive, but good performance (maintains real-time performance for most exports)
c5.4xlarge	Best performance and very expensive (faster than real-time performance)
c5.9xlarge	Similar performance to c5.4xlarge, but twice the price. Not recommended for that reason.

Azure Instance Types

F2s v2	Lower cost, but slower than real-time performance
F4s v2	More expensive, but good performance (maintains real-time performance for most exports)
F8s v2	Best performance and very expensive (faster than real-time performance)
F16s v2	Similar performance to c5.4xlarge, but twice the price. Not recommended for that reason.

Auto Scaling

Both AWS and Azure offer auto scaling solutions (each with slightly different features, but generally they work about the same). Configure a sever and a separate worker. Configure auto scaling on the worker instance, and base it off of average CPU > 50% for X minutes, or the number of messages in the Queue (SQS or Azure Queue Storage). As your video export jobs queue up, more and more worker instances will come online and help render jobs. Once the workload has ended and the queue is empty, the pool of worker instances can shrink back down to 1 (or even 0).

This is an effective strategy for managing costs (i.e. keeping as few workers running as possible), but quickly scaling up as needed.

AWS Auto Scaling Instructions

Step 1	Configure a new worker instance (use the same admin credentials when prompted during config-openshot-cloud)
Step 2	Create an Image (AMI) of the new worker instance (use a descriptive name, such as “OpenShot Cloud Worker Image”)
Step 3	Create a Launch Configuration (choose the AMI from step 2, choose the type of instance, and name descriptively)
Step 4	Create an Auto Scaling Group (choose the Launch Configuration from step 3 and name descriptively)
Step 5	Set the Scaling Policies to determine when to scale up and down (either # of SQS messages or average CPU % will work)
Step 6	If you choose a SQS scaling policy, choose a Policy type of Simple Scaling, which is compatible with CloudWatch ALARMS.
Step 7	Create 2 CloudWatch Alarms, and connect them to your AutoScaling Group (use SQS:ApproximateNumberOfMessagesVisible for your logic).
Step 8	Create 2 Scaling Policies: SQS Scale Up: If > 1 message over X minutes. And SQS Scale Down: If < 1 message over X minutes.

Azure Scale Set Instructions

Step 1	Configure a new worker virtual machine (use the same admin credentials when prompted during config-openshot-cloud)
Step 2	Capture Image of the new worker virtual machine (use a descriptive name, such as “openshot-cloud-worker-image”)
Step 3	Because of Azure limitations in the portal, we will need to use the az CLI for the remaining steps. Run the following script to create a scale set with the captured worker image. Replace the following place holders: {RESOURCE-GROUP-NAME}, {CAPTURED-IMAGE-URI}, {SUBSCRIPTION-ID}.
Step 4	az vmss create -n openshot-worker-set –computer-name-prefix openshot-worker -g {RESOURCE-GROUP-NAME} –instance-count 1 –image {CAPTURED-IMAGE-URI} –plan-name openshot-cloud-001 –plan-product openshot-cloud-api –plan-publisher openshotstudiosllc –subscription {SUBSCRIPTION-ID}
Step 5	This will successfully create a new scale set, based on your captured worker image. Now we can use the Azure portal to update the rules and settings of this scale set.
Step 6	Scaling: Choose Scale based on a metric. Click Add Rule: choose metric source: storage queue, select your storage account, and choose the OpenShot queue.
Step 7	Create 2 Rules: Queue Scale Up: If > 1 message over X minutes, increase by 1 instance. And Queue Scale Down: If < 1 message over X minutes, decrease by 1 instance.
Step 8	You can also set the minimum and maximum instance counts (to help manage costs)

Azure Using captured images (of a marketplace product)

Step 1	If you use Capture Image on an OpenShot Cloud API virtual machine, it requires extra steps to utilize this image correctly.
Step 3	Because of Azure limitations in the portal, we will need to use the az CLI for the remaining steps. Run the following script to create a virtual machine of the captured image. Replace the following place holders: {LOCATION-NAME}, {RESOURCE-GROUP-NAME}, {CAPTURED-IMAGE-URI}, {SUBSCRIPTION-ID}.
Step 4	az vm create –name openshot-worker-2 –location {LOCATION-NAME} –image {CAPTURED-IMAGE-URI} –plan-name openshot-cloud-001 –plan-product openshot-cloud-api –plan-publisher openshotstudiosllc -g {RESOURCE-GROUP-NAME} –subscription {SUBSCRIPTION-ID}
Step 5	This will successfully create a new virtual machine, based on your captured marketplace image.

Storage Scaling

Both AWS and Azure provide auto scaling file systems. In order to provide enough storage to meet you needs, these auto-scaling file systems are perfectly suited to work with OpenShot Cloud API. This is an optional step, and only recommended when you have very large amounts of files, very large video projects, or high numbers of videos being rendered.

AWS Instructions

Step 1	Create an EFS file system (default settings are fine) named OpenShotCloudStorage.
Step 2	Choose Customize and select Max I/O under Performance Mode (if you expect to have many Workers)
Step 3	Disable Encryption on data at rest (unless desired)
Step 4	Once created, copy the ID of your EFS (i.e. fs-2180ccd3), and install the mount (instructions below)
Step 6	Copy the existing video folders into the mount (instructions below). That should create 3 folders in /mnt/efs/.
Step 7	Verify the 3 folders are copied into the /mnt/efs (files, output, and temp)
Step 8	Remove and replace the existing folders with symbolic links (instructions below)
Step 9	Reboot the instance, and verify the symbolic links and EFS mount still work

# Install EFS Helper Utility
git clone https://github.com/aws/efs-utils
cd amazon-efs-utils
./build-deb.sh
sudo apt install ./build/amazon-efs-utils*deb

# Mount EFS (use your own file system ID)
sudo mkdir -p /mnt/efs
sudo mount -t efs fs-2180ccd3:/ /mnt/efs

# Auto Mount EFS (add this line to fstab, use your own file system ID)
sudo nano /etc/fstab
fs-2180ccd3:/ /mnt/efs efs _netdev,tls,iam 0 0

# Copy media files into EFS
cd ~/api/video/
sudo rm -r *   (be sure you are in the correct folder)

# Replace media folders with symbolic links
# Verify your media files are located at /mnt/efs/
cd ~/api/video/
ln -s /mnt/efs/files/ files
ln -s /mnt/efs/output/ output
ln -s /mnt/efs/temp/ temp

Worker Concurrency

Each worker is configured by default to process 2 video exports concurrently. While this improves speed and latency in many cases, it can also cause less capable instance types to run out of resources (Memory, CPU) causing higher failure rates. The following instructions can be used to configure a different # of concurrent export jobs:

sudo nano ~/api/video_queue/supervisor/cloud-worker.conf
# Change `numprocs=2` to the desired amount of concurrent exports
# CTRL+X, Save

sudo supervisorctl reload
sudo supervisorctl restart all

Hardware Acceleration

OpenShot Cloud API supports NVIDIA hardware encoding, if you are using a compatible instance type. You will also need to install a driver package: sudo apt-get install nvidia-driver-390 and use h264_nvenc as your video codec. However, the increase in encoding performance is often out-paced by the cost of accelerated instances. But feel free to use these instructions and guidelines to test hardware acceleration for your specific application.

g3s.xlarge	Lower cost, hardware accelerated, slower than real-time
g3.4xlarge	More expensive, hardware accelerated, but good performance (maintains real-time performance for most exports)
c5.4xlarge	NOT hardware accelerated but similar performance for almost half the price.

Instance Statistics

The following statistics were collected using an identical 60 second 1080p project, exported using a variety of instances, to compare speed and instance cost. Your specific project might not be 100% comparable, but this should give you an idea on performance and cost. After crunching the numbers, we still recommend a c5.xlarge instance type for the best budget option (but a bit slow on performance) and a c5.4xlarge for best performance (more expensive).