Analyzing videos with Amazon Rekognition and Laravel (Part 2)
Tom Oehlrich
In part 1 of this tutorial we set up the various AWS services that we need to analyze videos with Amazon Rekognition and Laravel.
Now let's proceed to the fun part by starting a new Laravel project that provides a video file upload to a S3 bucket and starts a label detection analysis with Amazon Rekognition. We will write a command that checks for the results of the video analysis and can be executed by a cronjob. The results will be displayed in a list of uploaded videos and the corresponding detail pages.
Hint: I am using a Homestead box to run my environment. If you are doing the same make sure to use "schedule: true" in your Homestead file in order to run the cronjob.
Let's create a new Laravel project "RekognitionTest" from the command line and modify the .env file with the credentials of a database of your choice.
Next we need to define some routes in routes/web.php.
The homepage shows a list of all videos that we have uploaded for analysis and if the analysis process has already been completed.
The results page displays the detailed results of the analysis of each video.
The upload GET route shows the upload form while the upload POST route stores info in the database and uploads the video to S3.
// routes/web.php
Route::get('/', 'RekognitionController@index');
Route::get('/results', 'RekognitionController@results');
Route::get('/upload', 'RekognitionController@upload');
Route::post('/upload', 'RekognitionController@store');
We'll use composer to install the packages that we need to get things running: The AWS PHP SDK, the drivers to handle S3 uploads in Laravel's filesystem flysystem as well as a cached adapter for flysystem to speed things up.
composer require aws/aws-sdk-php league/flysystem-aws-s3-v3 league/flysystem-cached-adapter
Next, we will add a couple of configuration values in the .env file. We should have all these values ready by now. If not please have a look at part 1 of this tutorial. Don't forget to modify the info according to your AWS setup.
AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
FILESYSTEM_CLOUD=s3
AWS_DEFAULT_REGION='eu-west-1'
AWS_BUCKET='rekbucket4711'
AWS_SNS_TOPIC_ARN='arn:aws:sns:eu-west-1:xxxxxxxxxxxx:RekognitionTopic'
AWS_IAM_ROLE_ARN='arn:aws:iam::xxxxxxxxxxxx:role/RekognitionRole'
The AWS PHP SDK will automatically make use of the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values.
The other values are being used explicitly in config/filesystems.php or when we are working with the AWS PHP SDK.
Model and migration wise we can keep things simple with just one database table. So let's create the Video model and its database migration with just one Laravel Artisan command.
php artisan make:model Video -m
In the Video migration we are using LONGTEXT as the data type for the 'results' fields since the analysis results for a video can become pretty large. The LONGTEXT offers about the same space as the JSON data type which could be an interesting alternative in a more advanced real-life application. By using the JSON datatype we would be able to query JSON directly with SQL.
<?php
use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;
class CreateVideosTable extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('videos', function (Blueprint $table) {
$table->increments('id');
$table->string('name');
$table->string('original_name');
$table->string('aws_job_id');
$table->tinyInteger('analyzed')->default(0)->index();
$table->longText('results')->nullable();
$table->timestamps();
});
}
/**
* Reverse the migrations.
*
* @return void
*/
public function down()
{
Schema::dropIfExists('videos');
}
}
php artisan migrate
In our Video model we are adding the 'results' column to the $casts array and declare its type to 'array'. That way the results we are retrieving from Amazon Rekognition are automatically serialized into JSON when we store them in our database. When we retrieve them from the database they are automatically deserialized to a PHP array.
<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class Video extends Model
{
/**
* The attributes that should be cast to native types.
*
* @var array
*/
protected $casts = [
'results' => 'array'
];
}
Next, we need a view to display our file upload form. I am using Tailwind CSS but feel free to use any "framework" or none at all.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<link href="https://cdn.jsdelivr.net/npm/tailwindcss/dist/tailwind.min.css" rel="stylesheet">
<title>Amazon Rekognition Test</title>
</head>
<body class="bg-grey-lighter h-screen font-sans">
<div class="container mx-auto h-full flex justify-center items-center">
<div class="w-1/3">
<h1 class="font-hairline mb-6 text-center">Amazon Rekognition Test</h1>
<form action="/upload" method="post" enctype="multipart/form-data">
<input type="hidden" name="_token" value="{{ csrf_token() }}">
<div class="border-teal p-8 border-t-12 bg-white mb-6 rounded-lg shadow-lg">
@if (session('success'))
<div class="border-green p-4 text-green">
{{ session('success') }}
</div>
@endif
@if(count($errors) > 0)
<div class="border-red p-4 text-red">
Something went wrong<br><br>
<ul>
@foreach ($errors->all() as $error)
<li>{{ $error }}</li>
@endforeach
</ul>
</div>
@endif
<div class="mb-4">
<label class="font-bold text-grey-darker block mb-2">Video</label>
<input type="file" name="file" class="block appearance-none w-full bg-white border border-grey-light hover:border-grey px-2 py-2 rounded shadow">
</div>
<div class="flex items-center justify-between">
<button class="bg-teal-dark hover:bg-teal text-white font-bold py-2 px-4 rounded">
Upload
</button>
</div>
</div>
</form>
</div>
</div>
</body>
</html>
To process the uploaded file we are going to need a controller as defined in our routes file.
php artisan make:controller RekognitionController
<?php
namespace App\Http\Controllers;
use Illuminate\Http\Request;
use Aws\Rekognition\RekognitionClient;
use App\Video;
class RekognitionController extends Controller
{
/**
* Show the list of uploaded videos
*
* @return View
*/
public function index() {
// to be added later
}
/**
* Shows the analysis results for a video
*
* @param Request $request
* @return View
*/
public function results($id) {
// to be added later
}
/**
* Show the upload form
*
* @return View
*/
public function upload() {
return view('upload');
}
/**
* Upload a video to S3 and store info in local DB
*
* @param Request $request
* @return Response
*/
public function store(Request $request) {
$request->validate([
'file' => 'required|file|max:5120|mimes:mp4',
]);
$originalFilename = $request->file->getClientOriginalName();
$fileExtension = $request->file->getClientOriginalExtension();
$uniqueFilename = str_random(32) . '.' . $fileExtension;
$path = $request->file->storeAs('', $uniqueFilename, 's3');
$client = new RekognitionClient([
'region' => env('AWS_DEFAULT_REGION', 'eu-west-1'),
'version' => 'latest'
]);
$result = $client->startLabelDetection([
'ClientRequestToken' => str_random(),
'JobTag' => 'rekognition-test',
'MinConfidence' => 50,
'NotificationChannel' => [
'RoleArn' => env('AWS_IAM_ROLE_ARN'),
'SNSTopicArn' => env('AWS_SNS_TOPIC_ARN'),
],
'Video' => [
'S3Object' => [
'Bucket' => env('AWS_BUCKET'),
'Name' => $uniqueFilename
],
],
]);
$video = new Video;
$video->name = $uniqueFilename;
$video->original_name = $originalFilename;
$video->aws_job_id = $result->get('JobId');
$video->save();
// dd($result);
return back()
->with('success','Video has been successfully uploaded');
}
}
We are doing a couple of things here. First we are validating the input file. For this demo the uploaded file needs to be an MP4 format (Amazon Rekognition can handle .mov too) of max 5 MB size. Next we are generating a random filename for the video and upload it to our S3 bucket. To use Amazon Rekognition the AWS PHP SDK provides some nice methods. First we need to initialize a RekognitionClient which is then used in the startLabelDetection method. The startLabelDetection method asks for the video that should be processed. So we are providing the S3 bucket name as well as the video's filename. We also need to define a NotificationChannel so that Amazon Rekognition knows who or what to notify about the results of the analysis. For our demo the ClientRequestToken will be a random otherwise Amazon Rekognition would see every request as the same job and return the same JobId over and over. The JobTag is just an identifier that Rekognition sends to SNS together with the results. We don't really need that for our small demo. MinConfidence tells AmazonRekognition how sure it has to be in its analysis in order to send us back any results.
As a result of the startLabelDetection method we are getting back some JSON. For us the only important information is the JobId which we are saving together with the original filename of the video, its current random filename and some timestamps to our database. After that we are returning to our upload form.
Next, let's write a command that tries to retrieve the results from Amazon Rekognition and that we start manually or by using a cronjob.
php artisan make:command GetRekognitionResults
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Aws\Rekognition\RekognitionClient;
use App\Video;
class GetRekognitionResults extends Command
{
/**
* The name and signature of the console command.
*
* @var string
*/
protected $signature = 'rekognition:get-results';
/**
* The console command description.
*
* @var string
*/
protected $description = 'Retrieves the Rekognition video analysis results';
/**
* Create a new command instance.
*
* @return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* @return mixed
*/
public function handle()
{
$videos = Video::where('analyzed', '<>', 1)
->orderBy('created_at', 'ASC')
->get();
if($videos->isNotEmpty()) {
$client = new RekognitionClient([
'region' => env('AWS_DEFAULT_REGION', 'eu-west-1'),
'version' => 'latest'
]);
foreach ($videos as $srcVideo) {
$result = $client->getLabelDetection([
'JobId' => $srcVideo->aws_job_id
]);
$this->info('Checking video ' . $srcVideo->aws_job_id . ' ' . $srcVideo->original_name);
if($result->get('JobStatus') == 'SUCCEEDED') {
$this->info('Video analysis results retrieved for ' . $srcVideo->aws_job_id . ' ' . $srcVideo->original_name);
$video = Video::find($srcVideo->id);
$video->results = $result->get('Labels');
$video->analyzed = 1;
$video->save();
}
}
}
}
}
This command loops through all video that have not been processed yet. As before we have to initialize RekognitionClient again. This time we are using getLabelDetection from the AWS PHP SDK and pass it the corresponding JobId for each video. If we get a JobStatus "SUCCEEDED" from the result we can update the video's database record by adding the results of the analysis and by setting the analyzed flag to TRUE.
We can now add our command to the app/Console/Kernel.php and setup a cronjob on our server.
<?php
/**
* Define the application's command schedule.
*
* @param \Illuminate\Console\Scheduling\Schedule $schedule
* @return void
*/
protected function schedule(Schedule $schedule)
{
$schedule->command('rekognition:get-results')->everyMinute();
}
All that is left now is to build two more views and modify our controller to serve these views.
<?php
/**
* Show the list of uploaded videos
*
* @return View
*/
public function index() {
$videos = Video::orderBy('created_at', 'desc')->get();
return view('index', [
'videos' => $videos
]);
}
/**
* Shows the analysis results for a video
*
* @param Request $request
* @return View
*/
public function results($id) {
$video = Video::find($id);
return view('results', [
'video' => $video
]);
}
The index view showing a list of all videos could be as follows:
@extends('layouts.app')
@section('content')
<div class="p-4">
<h1 class="font-hairline mb-4">Amazon Rekognition Test</h1>
<p class="mb-4"><a href="{{ url('/upload') }}" class="bg-teal-dark hover:bg-teal text-white font-bold py-2 px-4 rounded no-underline">Upload new video</a></p>
<div class="border-teal p-8 border-t-12 bg-white mb-6 rounded-lg shadow-lg">
@if($videos->isNotEmpty())
<table class="table-auto">
<tr>
<th class="p-4 border-teal border-solid border-b-2 text-left">Video</th>
<th class="p-4 border-teal border-solid border-b-2 text-left">Uploaded at</th>
<th class="p-4 border-teal border-solid border-b-2 text-left">Analyzed</th>
<th class="p-4 border-teal border-solid border-b-2"></th>
</tr>
@foreach($videos as $video)
<tr>
<td class="p-4 text-left">{{ $video->original_name }}</td>
<td class="p-4 text-left">{{ $video->created_at }}</td>
<td class="p-4 text-left">{{ $video->analyzed ? $video->updated_at : '—' }}</td>
<td>
@if($video->analyzed)
<a href="{{ url('/results', ['id' => $video->id ]) }}" class="bg-teal-dark hover:bg-teal text-white font-bold py-2 px-4 rounded no-underline">Result</a>
@endif
</td>
</tr>
@endforeach
</table>
@else
<p>No videos avaialble</p>
@endif
</div>
</div>
@endsection
And lastly, we need a view to display the detailed results of the video analysis. For our demo we are simply showing a table with the found labels, the timestamps at which they occurred in the video as well as the confidence level that Amazon Rekognition has regarding each label.
@extends('layouts.app')
@section('content')
<div class="p-4">
<h1 class="font-hairline mb-4">Amazon Rekognition Test</h1>
<h2 class="font-hairline mb-4">{{ $video['original_name'] }}</h2>
<p class="mb-4"><a href="{{ url('/') }}" class="bg-teal-dark hover:bg-teal text-white font-bold py-2 px-4 rounded no-underline">Back</a></p>
<div class="border-teal p-8 border-t-12 bg-white mb-6 rounded-lg shadow-lg">
@if(!empty($video['results']))
<table class="table-auto">
<tr>
<th class="p-4 border-teal border-solid border-b-2 text-left">Timestamp</th>
<th class="p-4 border-teal border-solid border-b-2 text-left">Label</th>
<th class="p-4 border-teal border-solid border-b-2 text-left">Confidence</th>
</tr>
@foreach($video['results'] as $result)
<tr>
<td class="p-4 text-left">{{ $result['Timestamp'] }}</td>
<td class="p-4 text-left">{{ $result['Label']['Name'] }}</td>
<td class="p-4 text-left">{{ $result['Label']['Confidence'] }}</td>
</tr>
@endforeach
</table>
@else
<p>No results avaialble</p>
@endif
</div>
</div>
@endsection
That's it. Of course label detection is not the only feature of Amazon Rekognition. The service also offers Facial Recognition, Unsafe Content Detection and more.
You can find the full source code in this GitHub repo.