Object recognition

The Object recognition operation allows you to categorize and tag people, objects, and other content classes in your images. This operation uses AWS Rekognition machine learning for the analyses.

The feature is currently available on the REST API v0.7 or higher via Add-Ons API.

How it works

Object recognition works asynchronously through the REST API, unlike the image processing that is performed on the fly.

  1. Start a processing job via REST API. Send an input file UUID with necessary processing operations.
  2. Wait until the processing job status becomes done.
  3. The result will be available in the JSON response in the appdata section of the processed file.

Example

Let's take this image as an example:

Image with water, trees and houses

Once it is uploaded to a project, you can make an API call to run object recognition via Rekognition Add-On API:

curl -X POST -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     -d "{\"target\": \"$UUID\"}"
     "https://api.uploadcare.com/addons/aws_rekognition_detect_labels/execute/"bash

Get YOUR_PUBLIC_KEY and YOUR_SECRET_KEY from API keys.

You can track execution status using request_id from previous request's response:

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     "https://api.uploadcare.com/addons/aws_rekognition_detect_labels/execute/status/?request_id=$REQUEST_ID"bash

The result will be available in JSON response in appdata.

Get image info

Acquiring info about detected objects through a file info request to the API endpoint, specifying the include parameter.

The endpoint for requesting Object recognition info:

https://api.uploadcare.com/files/html

There are two methods for getting info on detected objects via GET requests, for multi-file and single-file cases:

GET /files/?include=appdatahtml

or

GET /files/:uuid/?include=appdata

Where:

  • include parameter points our API to include appdata, an object holding various application data.
  • :uuid identifies the unique image for which you are requesting info on detected objects.

Single-file request

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple publickey:secretkey" \
     "https://api.uploadcare.com/files/:uuid/?inlude=appdata"bash

Multi-file request

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple publickey:secretkey" \
     "https://api.uploadcare.com/files/?include=appdata"bash

Where:

  • Note, Accept header points at the REST API v0.7 or higher.
  • Uploadcare.Simple stands for the auth-scheme that requires your Uploadcare project Public and Secret keys.
  • In a single-file example, :uuid identifies the image we get info for.
  • In a multi-file example, no :uuid is provided; you will receive detection info for all files in your project.

Single-file response

The JSON response for requesting a single-file that is properly authenticated looks like this:

{
  "datetime_removed": null,
  "datetime_stored": "2018-08-07T05:31:47.326146Z",
  "datetime_uploaded": "2018-08-07T05:31:47.132454Z",
  "is_image": true,
  "is_ready": true,
  "mime_type": "image/jpeg",
  "original_file_url": "https://ucarecdn.com/af9b7f02-6cbe-45bd-9bb6-9c928bf8a72e/eberhardgrossgasteiger767950unsplash.jpg",
  "original_filename": "eberhard-grossgasteiger-767950-unsplash.jpg",
  "size": 2363253,
  "url": "https://api.uploadcare.com/files/af9b7f02-6cbe-45bd-9bb6-9c928bf8a72e/",
  "uuid": "af9b7f02-6cbe-45bd-9bb6-9c928bf8a72e",
  "variations": null,
  "content_info": {
    "image": {
      "dpi": [
        72,
        72
      ],
      "width": 3648,
      "format": "JPEG",
      "height": 2432,
      "sequence": false,
      "color_mode": "RGB",
      "orientation": null,
      "geo_location": null,
      "datetime_original": null
    }
  },
  "metadata": {},
  "appdata": {
    "aws_rekognition_detect_labels": {
      "data": {
        "Labels": [
          {
            "Name": "Nature",
            "Parents": [],
            "Instances": [],
            "Confidence": 99.34097290039062
          },
          {
            "Name": "Outdoors",
            "Parents": [],
            "Instances": [],
            "Confidence": 99.23014068603516
          },
          {
            "Name": "Tree",
            "Parents": [
              {
                "Name": "Plant"
              }
            ],
            "Instances": [],
            "Confidence": 98.99191284179688
          },
          {
            "Name": "Plant",
            "Parents": [],
            "Instances": [],
            "Confidence": 98.99191284179688
          },
          {
            "Name": "Fir",
            "Parents": [
              {
                "Name": "Tree"
              },
              {
                "Name": "Plant"
              }
            ],
            "Instances": [],
            "Confidence": 98.6605224609375
          },
          {
            "Name": "Building",
            "Parents": [],
            "Instances": [],
            "Confidence": 97.45043182373047
          },
          {
            "Name": "Housing",
            "Parents": [
              {
                "Name": "Building"
              }
            ],
            "Instances": [],
            "Confidence": 97.41302490234375
          },
          {
            "Name": "Countryside",
            "Parents": [
              {
                "Name": "Outdoors"
              },
              {
                "Name": "Nature"
              }
            ],
            "Instances": [],
            "Confidence": 92.5694580078125
          },
          {
            "Name": "Shelter",
            "Parents": [
              {
                "Name": "Building"
              },
              {
                "Name": "Countryside"
              },
              {
                "Name": "Outdoors"
              },
              {
                "Name": "Nature"
              }
            ],
            "Instances": [],
            "Confidence": 88.491455078125
          },
          {
            "Name": "Water",
            "Parents": [],
            "Instances": [],
            "Confidence": 87.63512420654297
          }
        ],
        "LabelModelVersion": "2.0"
      },
      "datetime_created": "2018-08-07T05:31:47.132454Z",
      "datetime_updated": "2022-09-20T05:24:09.871132Z",
      "version": "2016-06-27"
    }
  }
}json

Limitations

  • This operation is not available on the Free plan.

API integrations

You don't have to code most of the low-level API integrations. We have high-level libraries for all popular platforms: