Object Recognition

Object Recognition allows categorizing and tagging images. The feature is currently available on the REST API v0.7 or higher via Add-Ons API.

We currently use AWS Rekognition for the analyses.

{
 "appdata": {
    "aws_rekognition_detect_labels": {
      "data": {
        "Labels": [
          {
            "Name": "Nature",
            "Parents": [],
            "Instances": [],
            "Confidence": 99.34097290039062
          },
          {
            "Name": "Outdoors",
            "Parents": [],
            "Instances": [],
            "Confidence": 99.23014068603516
          },
          {
            "Name": "Tree",
            "Parents": [
              {
                "Name": "Plant"
              }
            ],
            "Instances": [],
            "Confidence": 98.99191284179688
          },
          {
            "Name": "Plant",
            "Parents": [],
            "Instances": [],
            "Confidence": 98.99191284179688
          },
          {
            "Name": "Fir",
            "Parents": [
              {
                "Name": "Tree"
              },
              {
                "Name": "Plant"
              }
            ],
            "Instances": [],
            "Confidence": 98.6605224609375
          },
          {
            "Name": "Building",
            "Parents": [],
            "Instances": [],
            "Confidence": 97.45043182373047
          },
          {
            "Name": "Housing",
            "Parents": [
              {
                "Name": "Building"
              }
            ],
            "Instances": [],
            "Confidence": 97.41302490234375
          },
          {
            "Name": "Countryside",
            "Parents": [
              {
                "Name": "Outdoors"
              },
              {
                "Name": "Nature"
              }
            ],
            "Instances": [],
            "Confidence": 92.5694580078125
          },
          {
            "Name": "Shelter",
            "Parents": [
              {
                "Name": "Building"
              },
              {
                "Name": "Countryside"
              },
              {
                "Name": "Outdoors"
              },
              {
                "Name": "Nature"
              }
            ],
            "Instances": [],
            "Confidence": 88.491455078125
          },
          {
            "Name": "Water",
            "Parents": [],
            "Instances": [],
            "Confidence": 87.63512420654297
          }
        ],
        "LabelModelVersion": "2.0"
      },
      "datetime_created": "2018-08-07T05:31:47.132454Z",
      "datetime_updated": "2022-09-20T05:24:09.871132Z",
      "version": "2016-06-27"
    }
  }
}json

Enabling Object Recognition

The feature is available on any paid plan.

How It Works

Once an image is uploaded to a project, you can make an API call to run object recognition via Add-Ons API:

curl -X POST -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple publickey:secretkey" \
     -d "{\"target\": \":file_uuid\"}"
     "https://api.uploadcare.com/addons/aws_rekognition_detect_labels/execute/"

You can track execution status using request_id from previous request's response:

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple publickey:secretkey" \
     "https://api.uploadcare.com/addons/aws_rekognition_detect_labels/execute/status/?request_id=:request_id"

The result will be available in file's appdata.

Getting Image Info

Here’s where the technical details begin ⚙️ Feel free to reference this section of the document by using this link.

Acquiring info on detected objects is performed via the Uploadcare REST API. Object Recognition will only work with the API v0.7 or higher.

To do so, you make file info requests to our API endpoint specifying the include parameter.

The endpoint for requesting Object Recognition info is:

https://api.uploadcare.com/files/html

There are two methods for getting info on detected objects via GET requests, for multi-file and single-file cases:

GET /files/?include=appdata

or

GET /files/:uuid/?include=appdatahtml

Where:

include parameter points our API to include appdata, an object holding various application data.
:uuid identifies the unique image for which you are requesting info on detected objects.

Request Example

Single-file example:

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple publickey:secretkey" \
     "https://api.uploadcare.com/files/:uuid/?inlude=appdata"bash

Multi-file example:

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple publickey:secretkey" \
     "https://api.uploadcare.com/files/?include=appdata"bash

Where:

Note, Accept header points at the REST API v0.7 or higher.
Uploadcare.Simple stands for the auth-scheme that requires your Uploadcare project Public and Secret keys.
In a single-file example, :uuid identifies the image we get info for.
In a multi-file example, no :uuid is provided; you will receive detection info for all files in your project 🧢

Response Example

Let’s request Object Recognition info for the cover image of this section. Here is how it looked like:

Once we make a single-file request that is properly authenticated, that’s a response JSON we get:

{
  "datetime_removed": null,
  "datetime_stored": "2018-08-07T05:31:47.326146Z",
  "datetime_uploaded": "2018-08-07T05:31:47.132454Z",
  "is_image": true,
  "is_ready": true,
  "mime_type": "image/jpeg",
  "original_file_url": "https://ucarecdn.com/af9b7f02-6cbe-45bd-9bb6-9c928bf8a72e/eberhardgrossgasteiger767950unsplash.jpg",
  "original_filename": "eberhard-grossgasteiger-767950-unsplash.jpg",
  "size": 2363253,
  "url": "https://api.uploadcare.com/files/af9b7f02-6cbe-45bd-9bb6-9c928bf8a72e/",
  "uuid": "af9b7f02-6cbe-45bd-9bb6-9c928bf8a72e",
  "variations": null,
  "content_info": {
    "image": {
      "dpi": [
        72,
        72
      ],
      "width": 3648,
      "format": "JPEG",
      "height": 2432,
      "sequence": false,
      "color_mode": "RGB",
      "orientation": null,
      "geo_location": null,
      "datetime_original": null
    }
  },
  "metadata": {},
  "appdata": {
    "aws_rekognition_detect_labels": {
      "data": {
        "Labels": [
          {
            "Name": "Nature",
            "Parents": [],
            "Instances": [],
            "Confidence": 99.34097290039062
          },
          {
            "Name": "Outdoors",
            "Parents": [],
            "Instances": [],
            "Confidence": 99.23014068603516
          },
          {
            "Name": "Tree",
            "Parents": [
              {
                "Name": "Plant"
              }
            ],
            "Instances": [],
            "Confidence": 98.99191284179688
          },
          {
            "Name": "Plant",
            "Parents": [],
            "Instances": [],
            "Confidence": 98.99191284179688
          },
          {
            "Name": "Fir",
            "Parents": [
              {
                "Name": "Tree"
              },
              {
                "Name": "Plant"
              }
            ],
            "Instances": [],
            "Confidence": 98.6605224609375
          },
          {
            "Name": "Building",
            "Parents": [],
            "Instances": [],
            "Confidence": 97.45043182373047
          },
          {
            "Name": "Housing",
            "Parents": [
              {
                "Name": "Building"
              }
            ],
            "Instances": [],
            "Confidence": 97.41302490234375
          },
          {
            "Name": "Countryside",
            "Parents": [
              {
                "Name": "Outdoors"
              },
              {
                "Name": "Nature"
              }
            ],
            "Instances": [],
            "Confidence": 92.5694580078125
          },
          {
            "Name": "Shelter",
            "Parents": [
              {
                "Name": "Building"
              },
              {
                "Name": "Countryside"
              },
              {
                "Name": "Outdoors"
              },
              {
                "Name": "Nature"
              }
            ],
            "Instances": [],
            "Confidence": 88.491455078125
          },
          {
            "Name": "Water",
            "Parents": [],
            "Instances": [],
            "Confidence": 87.63512420654297
          }
        ],
        "LabelModelVersion": "2.0"
      },
      "datetime_created": "2018-08-07T05:31:47.132454Z",
      "datetime_updated": "2022-09-20T05:24:09.871132Z",
      "version": "2016-06-27"
    }
  }
}json