This guide covers how to add an Amazon Simple Storage Service (S3) bucket to Accurate.Video, and how to ingest files from this storage as assets. The guide covers the following steps:
Adding storages to Accurate.Video is a task that is performed by an administrator or someone with access to the AWS infrastructure. Due to the nature of this task, some AWS knowledge is required. You'll need access to an AWS account where you can create users in IAM and buckets in S3.
Ensure your user has the sufficient permissions required for adding storages in Accurate.Video. Refer to the roles available in the guide Authentication - Roles and permissions configurations.
If you cannot see the storage tab, or the Add storage button is missing, your user lacks the required permissions.
Storages are accessed through the Storages tab in the navigation bar at the top.
Clicking the Add storage button opens up a modal popup where you enter the required information about storage, shown below.
There are three options for supplying credentials when adding a storage:
This guide covers using custom credentials only.
The first step is to create a new user in AWS IAM (Identity and Access Management). The user will be used by the Accurate Video system to get access to the files located in the S3 bucket. By creating a new user, we can control and limit the access for this specific user, and it makes it easy to revoke or delete access in the future, should it be required.
Go to the AWS IAM page, and click on users. Click on “create user”. Give it a name that you will remember and enable the “programmatic access” checkbox.
Click to the next step of the user creation wizard.
On the second screen, specific permissions can be given to the user. In this case, no permissions should be given to the user, so make sure the user is not added to any groups or that any permissions are set.
By default, no permissions are added to the user. It should be OK to just click next to go to the third step of the user creation wizard.
Here you can add specific tags to the user if you wish. This is optional and is not required. A tag can be added to remember that this user was created for Accurate Video purposes.
Click next to proceed to the fourth step of the user creation wizard.
Review that everything looks OK before proceeding to the final step. There should be a warning showing that the user has no permissions. Make sure that the user AWS access type is set to programmatic access - with an access key.
Click on “create user” to proceed.
The user is now created!
Take note and write down the following information:
You will need the information above in subsequent steps, when the required bucket policy is added and when the storage is added to the Accurate Video interface.
In order to proceed, there is one more type of information required, the User ARN of the user that was just created.
In order to find this information, go back to the start page of AWS IAM. Click on “users”, and select the user just created by clicking on the user name.
A screen similar to the one above will be shown, at the top of the screen you will see the User ARN. This full string will need to be copied and remembered. There is a small icon next to the string which conveniently copies the string. Click on this to copy the string.
In this example the value is arn:aws:iam::010652268016:user/av-s3-user
In the following steps, an S3 bucket is created and configured. You can either choose to create a new S3 bucket, or re-use an existing. In this example, a new bucket is created.
Navigate to the AWS S3 start page. From here there will be a button to create a new bucket. Give the bucket a name, and select the region where the bucket should be created. In doesn’t matter, but recommended is to keep the bucket as close to the end-users as possible, as all content will be served from this bucket.
For the rest of the options, simply follow through the bucket creation wizard and use all default values for options and permissions.
After the bucket has been created, the required permissions will need to be added. Click on the bucket to open the details page. Click “permissions” at the top, and then “bucket policy”. You should see an empty screen where you can add text.
Copy and paste the following text into the bucket policy.
Update the policy with correct "arn" values according to your configuration. Click save to update the bucket policy.
In order to access media files from a browser, a CORS policy must be added to the bucket.
The CORS policy is found at the bottom of the bucket Permission page.
Copy and paste the following text:
Click save to set the CORS configuration.
Note 1: If you know the exact origin of the Accurate Video system, you can limit this instead of setting the AllowedOrigin to "*" (accept anything).
Note 2: If AllowedOrigins is not set to "*", then one of the allowed origins must be the literal string "null" (without quotation marks) in order for DASH and HLS playback to work. The reason for this is that all browsers set their origin to "null" when following redirects.
Here you can find more information about CORS and why this configuration is needed: https://en.wikipedia.org/wiki/Cross-origin_resource_sharing
All settings are now set in AWS. Now it’s time to add the storage to Accurate Video.
Go to the Accurate Video start page, click on “storages” at the top. You should see a screen similar to the above, empty without any storages. If you have already added storage, it will show up here.
Click the “add storage” button at the top.
A modal with a form should pop up. Start by giving the storage a name, this is purely for identification and can be anything you want.
Recommended best practice is to differentiate between input and output buckets. Input buckets are storages where files are being read and ingested from, with only read-only permissions. Output buckets are dedicated buckets where artifacts generated by Accurate.Video is stored, such as thumbnails, waveform data, and metadata.
You have full control of how you map your buckets, and can even have multiple input and multiple output buckets. For input buckets, simply use the read and list permissions (list if you want to ingest through the UI). For output buckets, tags can be used to determine where output artifacts are stored.
The permission section decides what AV can do with the storage and here you simply click the respective checkbox to activate certain permissions:
With the above permissions, you can control and easily create read-only storages where files are ingested from, and writable, output storages where files that are generated by Accurate.Video is stored, files such as transcoded proxy files, thumbnails, and waveforms.
Using tags, you can fine-tune where these files are stored. There are a number of pre-defined tags that can be used, refer to the guide on writable storage metadata fields for details.
The next step is to enter the S3 bucket name, note that this represents the name in S3 that was used before. The region should also be set, this has to be the same region as in S3.
If you are unsure about what region codes, look at the following list:
Click “use custom credentials” and enter the AWS Access key and AWS Secret key from before. The AWS access key is a string all in upper-case, and the AWS secret key is a slightly longer key mixed with both upper and lower characters.
Optionally you can set a periodic refresh interval. This reflects how often the S3 bucket is queried and files within the bucket are indexed by Accurate.Video. Note that it will poll the S3 API according to this schedule, so you don’t want to set the interval too high. In case the storage is very rarely updated, the best option could be to disable it and doing manual refreshes in the UI when required.
Another option is to use Amazon SNS to automatically index new files as they appear in S3. Please refer to the Configuring SNS and S3 event notifications in Accurate.Video guide on how to set this up.
After connecting the storage, you should see a green text which says Connected, this is an indication that everything is ok. If it comes up as Broken instead, something is wrong and Accurate.Video failed to access the storage.
The next step is to start ingesting files, which can be done either through the UI or the API.
Refer to the Ingest assets using the UI guide for more details on how to ingest files through the UI.
You can also configure automatic ingest, see Auto Ingest using the UI for more details.
If you're interested in learning more about automatic ingest in general, see Automatic Ingest.
You can also connect storages and ingest files through the REST API, see REST API Examples for some examples.