Where Do My Pictures Upload From Dropsource

The motivation for this blog post is to take some fun and demonstrate the agility with which you can develop mobile applications when you lot combine the advantages of serverless engineering science and the Dropsource platform. You lot should exist able to reproduce what I've done in an hour by following this post.

If you're a fan of the HBO testify Silicon Valley, y'all may remember an episode in which Jian Yang builds an image recognition app with machine learning, which turns out merely able to recognize photos of hot dogs. I wanted to do the aforementioned, but take it a scrap further and recognize almost anything!

I've washed this by taking advantage of Amazon Rekognition (in coordination with some other services), besides as a simple Balance API on the Serverless framework that acts as the web service for the iOS app I congenital on Dropsource.

I won't keep you waiting, you can scan the following QR lawmaking to install (sideload) the app on your iOS device. Open and point your iPhone camera at the lawmaking, and then tap the "Open in iTunes" drop-downwards that appears. If this is your first time testing a Dropsource-built app, you'll demand to trust our developer certificate by following these simple instructions

Building your ain app for free in Dropsource and AWS

If yous'd like to learn how to build your ain app, follow the instructions below. Information technology should but accept you an hour to have a fully functioning app on your phone without having to lawmaking a single line of Swift lawmaking! You can take reward of your Dropsource free trial and the AWS free introductory tier to host your image recognition backend and API which I've provided for you below.

Deploy the Rekognition Stack on AWS

The kickoff affair we're going to get running are the services that process the images we upload from the app and give us the tags describing them. In short, this ready of services volition discover when something is uploaded to an S3 bucket, pass that through the Rekognition analysis, and then salve the results in a DynamoDB tabular array. This is done with AWS Step Functions that coordinate several Lambda functions to invoke various services.

For a visual representation, see the diagram below.

Getting all this up and running turns out to exist a lot more than simple than you'd imagine – thank you to the lambda-refarch-imagerecognition project AWS Labs has made available on Github.

Simply curlicue down to the readme , and hitting the 'Launch Stack' push button, you lot'll obviously need an AWS account to go started, but that'south easy and gratis to utilize. Follow the iii steps to get it running in the us-west-two region, and we're done with that fix of documentation. Once that executes, navigate to the 'Outputs' section of the stack nosotros just launched in the CloudFormation console. We need to make notation of a few values that are provided here, namely:

• Region
• DDBImageMetadataTable
• S3PhotoRepoBucket

Leave the tab open up for reference when setting up the next piece of our application backend, the REST API.

Create and Deploy The REST API

Dropsource works really well with REST APIs documented with Swagger (OpenAPI Spec), so we're going to quickly create ane for our app to collaborate with the image recognition stack we just deployed. Information technology will serve two major functions:

1. Upload images from our mobile device to the S3 bucket and providing us with the object fundamental for referencing later.
2. Querying the DynamoDB table to fetch the results posted by the Rekognition service, which are an array of image tags nosotros'll utilize to determine the content of the image.

Serverless Framework is my favorite tool for this chore. It'south super quick to get an API running on a serverless AWS Lambda stack (or a variety of other cloud providers), you can lawmaking in one of many languages, and y'all can take advantage of API Gateway to consign a Swagger file you'll need for importing your API directly into Dropsource.

This post won't arrive too deep with how to write services with Serverless, there's plenty of neat documentation already out in that location for that. For now, you lot can just follow the documentation on the Serverless site to quickly install and configure the framework, and so launch the Residuum API I've already written for yous, which you can find hither .

Deploying the Remainder API using Serverless is painless, but there are a couple items nosotros need to configure before doing that. Open the serverless.yml file in the project I've provided as well as the CloudFormation tab with the values I mentioned to a higher place.

Well-nigh the top of the serverless.yml file, you'll meet the 'custom' department with ii values. Copy the value for 'S3PhotoRepoBucket' from the CloudFormation console to the 'saucepan' belongings, and do the aforementioned with the  'DDBImageMetadataTable' value to the 'dynamoDbTable' property in your serverless.yml.

The last value in your serverless.yml file you need to configure, we'll need to grab from the DynamoDB console in AWS. Open the DynamoDB console, click 'Tables' on the left, click on photo-rekognition-demo-app-backend-ImageMetadataDDBTable and yous should run across the 'Overview' tab for the tabular array, curl to the lesser of that tab and copy the value of the Amazon Resources Proper name (ARN) property. Information technology should look something like… arn:aws:dynamodb:us-westward-2:000000000000:table/photo-rekognition-demo-app-backend-ImageMetadataDDBTable-ABC1ABCABCAB

Now go back to your serverless.yml file and paste that value nether provider->iamRoleStatements->Resource (the second statement, leave the first statement for S3 Resources equally is). You'll notice it'due south really but the AWS region and your business relationship number prepended to the table name we just configured.

While you have the DynamoDB panel open up, navigate to the 'Capacity' tab of each of the two tables shown. You can safely set all four of the chapters values to ane while y'all're building and testing your app; anything more is unnecessary if you're the only i using the services.

Just to be condom, scroll downward in your serverless.yml file and find the 'provider' section. Make sure that the 'region' property matches the 'Region' in your CloudFormation console. If it doesn't, fix the value in your yml to match the i in the console.

Y'all can now switch dorsum to your concluding where you installed Serverless. Kickoff, install the dependencies our project needs with the 'npm install' command, and then run the 'serverless deploy' command in the project folder to deploy your Remainder API to AWS (make sure you lot've run the commands to install Serverless before you do this). I've taken the time to certificate our API endpoints (you lot may have noticed that in the serverless.yml) file, so once information technology'southward deployed we can get a swagger .json file from the API Gateway panel in AWS. To do that, navigate to the service in AWS, click on the ' dev-aws-rekognition-mobile-app-service ' on the left, then 'Stages', then 'dev' to the right, open the 'Export' tab, and finally hover over the 'Export as Swagger' icon to click 'JSON' which volition starting time downloading the file.

Edifice the App in Dropsource

Elements and Styling

If you don't have a Dropsource business relationship notwithstanding, you tin easily sign up for a complimentary business relationship by clicking here . After creating your account, create a new projection (we'll exist using iOS, just yous tin do the aforementioned thing in Android with Dropsource).

Open the editor by clicking on the projection. If you lot've just signed up and created your commencement project you lot'll already exist there.

Create a page by clicking on 'Pages' to the left and pressing the plus icon. Don't check either checkbox, click next, and name the page whatever you'd like.

Adjacent, click on 'Elements' to the left and elevate out an 'Image' element onto the folio, you tin can search for it in the search box if you're having trouble finding it. Once you've done that, y'all tin manner information technology in the right bar to lucifer the settings depicted below.

Side by side, click 'constraints' (these define the size and positioning of the chemical element on the page) at the peak of the right bar and set the constraints for Image ane to friction match these (we want the epitome to exist full screen regardless of device screen size):

Next, elevate out a button element and identify centered near the bottom of screen – this will be our camera push. I chose to make this an image-button for purely artful reasons. To do the aforementioned, style the push past choosing an image to put on the button.

You'll take to upload a new one and select it. Here'south the image I used:

Subsequently you've done that, curl down to 'Background Color' for the button and fix it to rgba(255, 255, 255, 0.00) this volition remove the white groundwork from the prototype then we're just left with a yellow circle.

Next, we'll create two labels that will show/hibernate depending on what tags were detected in the paradigm. I've gone with two because my intention was to build the satirical 'Not Hotdog' app from the 'Silicon Valley' HBO show. Y'all could hands exercise some design freedom here and cull to bear witness but the image tags returned.

Nosotros'll ready up the commencement label and re-create it to create the 2d characterization, so elevate out a label element and drib it on the screen. Set the text to 'HOTDOG!', and then way it just like the images beneath. (Note that clicking 'Hidden' will hide the label from the page. Do this last so that you can meet what you lot're working on. We'll somewhen fix up logic to show it based on the prototype tags returned.)

Also set the constraints to lucifer this:

Now, click the label on the screen and correct click information technology to re-create, right click the page sail over again and paste information technology to create an identical label.

Important notation: it may be hard to select the right element when they're on top of each other. To make this easier you tin click on the Element Tree icon to see select them. You can also utilise the element tree to reorder how each element is positioned front-to-back. For the finished product nosotros'll desire the epitome at the elevation, followed by the button, and so labels.

Style this new label with new text 'Non HOTDOG!', set up the Color to 'yellow' and the Groundwork Colour to 'ruddy'. Finally, position it identically to the starting time one (information technology should be directly on tiptop of the start), and fix both labels to Hidden past checking the check box in each.

The final element we need is another characterization that will show the text for the image tags returned. Drag out a 3rd label from the elements drawer on the left. Style it and fix constraints to match this:

Logic and API Requests

Importing the API

Starting time by importing your swagger .json file nosotros exported after deploying our Balance API. Click the API tab in the right bar, and so the + button.

Utilise the dropdown and add together one of the PUT /saveImage and Go /getImageTags endpoints. We'll configure them later.

Setting upwards Page Variables

We'll next set up five page variables to hold some data and constructs we'll utilize in our app.

Exercise this by selecting the Page Variables tab as shown in the image beneath.

The 3 folio variables should exist named like above and have types:

• imageData: Data Types – Swift – String
• imageID: Data Types – Swift – Cord
• imageTagPollingTImer: Data Types – Foundation – Timer

Setting up actions and events

Side by side, double-click the button element on your folio and select the events tab for it.

Click the manage button next to the 'Tapped' event. Add an activity by clicking the plus push on the left. Search for 'Select Paradigm from Photos' and cull between Camera and Photo Library. If you've got a iOS device you can use to test your app, the camera will work for yous, but go with photo library if you lot're going to test your app with the in-browser simulator. Y'all tin ever come back here and edit this after.

Underneath the 'Paradigm Chosen' sub-action nosotros now have, nosotros demand to add six more deportment and configure them like the three images below.

Set Value – Hide Characterization one to reset the UI

Set Value Hide Label 2 to reset the UI

Fix Value – Hibernate Characterization 3 to reset the UI

Set Value – Unhide the prototype element Paradigm 1

Set Value – Set the epitome we chose from the camera or photo library to display in Prototype ane

Show Progress Overlay – Testify a loading state while we save the image and wait for tags describing it

Base 64 Encode Image – Encode the paradigm as a Base64 string for sending to our API

We at present need ii events nether the Epitome Encoded sub-action of Base64 Encode Prototype.

Gear up Value – Fix the imageData page variable equal to the Base64 encoded string

Run API Asking – Run the API request that saves the epitome data

Adjacent, nosotros'll configure the PUT /saveImage call.

In the API tab (far right of the right-bar) click the PUT endpoint we created earlier in the list pictured beneath.

Select the Parameters tab, and choose "Trunk" if it's non already selected in the dropdown. Set the three parameters using the following backdrop:

• albumName: Static Inputs – Cord – (enter any anthology name hither)
• imageData: Folio Variables – imageData
• userName: Static Inputs – String – (enter whatever username you'd similar hither)

Next, select the Events tab for the request, and click on the Manage button for the 200 response.

Add a 'Ready Value' activeness here that will take the imageID returned when we save the image to the folio variable we made. Information technology should look like this.

Nosotros're too going to create a timer that is going to be used to poll the API for paradigm tags and return them when they're available.

Add together a Create Timer action and configure it similar shown.

Under Tick, create a Run API Request action and specify the GET /getImageTags endpoint.

Now, outside the Create Timer activeness and below it, create a new Set Value activeness that saves the time to a page variable for referencing later similar this.

That's it for the PUT request, at present close the action modal and click the back arrow side by side to the asking name in the right bar. Select the Get asking, click the Events tab, and click Manage side by side to the 200 response.

Create a Finish Timer activeness to stop the timer we only set up upwardly so information technology doesn't continue running after nosotros receive the response we wanted.

Create a Set Value action to set the paradigm tags to the text value of Label 3

Create a Hibernate Progress Overlay activeness to cease the loading state we had running since we're washed with the request we need to make.

We'll now make up one's mind which label to prove hotdog, or not hotdog, by creating an If… Else conditional action.

Under the True event (if the API says we've got a hotdog), create a Set Value action to unhide Characterization 1 like this

Under the False event (no hotdog detected) create 2 Set Value actions to unhide the other Label ii and 3.

That'due south it for the app design and logic!

All that's left is a few settings for app proper noun and icon. Y'all'll find these in the left sidebar under Settings.

You can use this image for your app icon:

You can now test your app on your device or in-browser simulator past clicking the blue Exam push button in the peak right corner of the editor. Retrieve that to test in the simulator, you'll demand to have configured your camera push Tapped event to cull images from the photograph library, every bit the the simulator doesn't accept camera functionality.

I encourage you to be creative and build your ain image recognition apps. The API is ready for you to edit and redeploy, and Dropsource is peachy for quickly iterating on your app.

If you've got any questions or something isn't working for yous, please attain out to me personally at dperjar@dropsource.com, on Twitter at @dperjar , or use the live conversation support available with your Premium Project (and Premium trial) in the Dropsource editor.

Annotation: In this tutorial we cloned and expanded some of the basic functionality of a mock Non Hotdog app. A real Not Hotdog app (which is awesome) was built past SeeFood Technologies Inc. and is publicly available here.

thomaseaced1965.blogspot.com

Source: https://www.dropsource.com/blog/serverless-image-recognition-app/

Share this post