TransformationsImage

Object recognition

The Object recognition operation allows you to categorize and tag people, objects, and other content classes in your images. This operation uses AWS Rekognition machine learning for the analyses.

The feature is currently available on the REST API v0.7 or higher via add-ons API.

How it works

Object recognition works asynchronously through the REST API, unlike the image processing operations performed on the fly.

  1. Start a processing job via REST API. Send an input file UUID with necessary processing operations.
  2. Wait until the processing job status becomes done.
  3. The result will be available in the JSON response in the appdata section of the processed file.

Example

Let’s take this image as an example:

Image with water, trees and houses

Once it is uploaded to a project, you can make an API call to run object recognition via Rekognition API:

$curl -L -X POST 'https://api.uploadcare.com/addons/aws_rekognition_detect_labels/execute/' \
>     -H "Accept: application/vnd.uploadcare-v0.7+json" \
>     -H "Content-Type: application/json" \
>     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
>     -d "{\"target\": \"$UUID\"}"

Get $YOUR_PUBLIC_KEY and $YOUR_SECRET_KEY from API keys.

Webhook event

To get the job result, you need to enable file.info_updated in the Webhook section of the dashboard.

After completing the processing job, the webhook will be sent to the endpoint you specified in the webhook settings.

File information in response:

1    "initiator": {
2        "type": "addon",
3        "detail": {
4            "addon_name": "aws_rekognition_detect_labels",
5            "request_id": ":request_id",
6        }
7    },
8    "hook": {
9        ...
10        "event": "file.info_updated",
11        "is_active": true,
12        "version": "0.7",
13    },
14        "uuid": ":UUID",
15        "is_image": true,
16        "is_ready": true,
17        "metadata": {},
18        "appdata": {
19            "aws_rekognition_detect_labels": {
20                "data": {
21                    "LabelModelVersion": "3.0",
22                    "Labels": [{
23                        "Confidence": 99.99856567382812,
24                        "Instances": [],
25                        "Name": "Food",
26                        "Parents": []
27                    }, {
28                        "Confidence": 99.99856567382812,
29                        "Instances": [],
30                        "Name": "Fruit",
31                        "Parents": [{
32                            "Name": "Food"
33                        }, {
34                            "Name": "Plant"
35                        }, {
36                            "Name": "Produce"
37                        }]
38                    }, {
39                        "Confidence": 99.99856567382812,
40                        "Instances": [],
41                        "Name": "Plant",
42                        "Parents": []
43                    }, {
44                        "Confidence": 99.99856567382812,
45                        "Instances": [],
46                        "Name": "Produce",
47                        "Parents": [{
48                            "Name": "Food"
49                        }]
50                    }, {
51                        "Confidence": 99.99794006347656,
52                        "Instances": [{
53                            "BoundingBox": {
54                                "Height": 0.7499178051948547,
55                                "Left": 0.5399686098098755,
56                                "Top": 0.11977619677782059,
57                                "Width": 0.22739684581756592
58                            },
59                            "Confidence": 98.84937286376953
60                        }],
61                        "Name": "Pineapple",
62                        "Parents": [{
63                            "Name": "Food"
64                        }, {
65                            "Name": "Fruit"
66                        }, {
67                            "Name": "Plant"
68                        }, {
69                            "Name": "Produce"
70                        }]
71                    }]
72                },
73                "version": "2016-06-27",
74                "datetime_created": "2023-11-08T12:15:22.965Z",
75                "datetime_updated": "2023-11-08T12:15:22.965Z"
76            },
77        }

Check the execution status

If your application does not have a backend or uses a mobile version, you can submit the request yourself.

You can track execution status using request_id from object recognition request’s response:

$curl -L "https://api.uploadcare.com/addons/aws_rekognition_detect_labels/execute/status/?request_id=$REQUEST_ID" \
>     -H "Accept: application/vnd.uploadcare-v0.7+json" \
>     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY"

The result will be available in JSON response in appdata.

Get image info

Acquiring info about detected objects through a file info request to the API endpoint, specifying the include parameter.

The endpoint for requesting Object recognition info:

1https://api.uploadcare.com/files/

There are two methods for getting info on detected objects via GET requests, for multi-file and single-file cases:

1GET /files/?include=appdata

or

GET /files/$UUID/?include=appdata

Where:

  • include parameter points our API to include appdata, an object holding various application data.
  • $UUID identifies the unique image for which you are requesting info on detected objects.

Single-file request

$curl -L "https://api.uploadcare.com/files/$UUID/?include=appdata" \
>     -H "Accept: application/vnd.uploadcare-v0.7+json" \
>     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY"

Multi-file request

$curl -L "https://api.uploadcare.com/files/?include=appdata" \
>     -H "Accept: application/vnd.uploadcare-v0.7+json" \
>     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY"

Where:

  • Note, Accept header points at the REST API v0.7 or higher.
  • Uploadcare.Simple stands for the auth-scheme that requires your Uploadcare project Public and Secret keys.
  • In a single-file example, $UUID identifies the image we get info for.
  • In a multi-file example, no $UUID is provided; you will receive detection info for all files in your project.

Single-file response

The JSON response for requesting a single-file that is properly authenticated looks like this:

1{
2  "datetime_removed": null,
3  "datetime_stored": "2018-08-07T05:31:47.326146Z",
4  "datetime_uploaded": "2018-08-07T05:31:47.132454Z",
5  "is_image": true,
6  "is_ready": true,
7  "mime_type": "image/jpeg",
8  "original_file_url": "https://ucarecdn.com/af9b7f02-6cbe-45bd-9bb6-9c928bf8a72e/eberhardgrossgasteiger767950unsplash.jpg",
9  "original_filename": "eberhard-grossgasteiger-767950-unsplash.jpg",
10  "size": 2363253,
11  "url": "https://api.uploadcare.com/files/af9b7f02-6cbe-45bd-9bb6-9c928bf8a72e/",
12  "uuid": "af9b7f02-6cbe-45bd-9bb6-9c928bf8a72e",
13  "variations": null,
14  "content_info": {
15    "image": {
16      "dpi": [72, 72],
17      "width": 3648,
18      "format": "JPEG",
19      "height": 2432,
20      "sequence": false,
21      "color_mode": "RGB",
22      "orientation": null,
23      "geo_location": null,
24      "datetime_original": null
25    }
26  },
27  "metadata": {},
28  "appdata": {
29    "aws_rekognition_detect_labels": {
30      "data": {
31        "Labels": [
32          {
33            "Name": "Nature",
34            "Parents": [],
35            "Instances": [],
36            "Confidence": 99.34097290039062
37          },
38          {
39            "Name": "Outdoors",
40            "Parents": [],
41            "Instances": [],
42            "Confidence": 99.23014068603516
43          },
44          {
45            "Name": "Tree",
46            "Parents": [
47              {
48                "Name": "Plant"
49              }
50            ],
51            "Instances": [],
52            "Confidence": 98.99191284179688
53          },
54          {
55            "Name": "Plant",
56            "Parents": [],
57            "Instances": [],
58            "Confidence": 98.99191284179688
59          },
60          {
61            "Name": "Fir",
62            "Parents": [
63              {
64                "Name": "Tree"
65              },
66              {
67                "Name": "Plant"
68              }
69            ],
70            "Instances": [],
71            "Confidence": 98.6605224609375
72          },
73          {
74            "Name": "Building",
75            "Parents": [],
76            "Instances": [],
77            "Confidence": 97.45043182373047
78          },
79          {
80            "Name": "Housing",
81            "Parents": [
82              {
83                "Name": "Building"
84              }
85            ],
86            "Instances": [],
87            "Confidence": 97.41302490234375
88          },
89          {
90            "Name": "Countryside",
91            "Parents": [
92              {
93                "Name": "Outdoors"
94              },
95              {
96                "Name": "Nature"
97              }
98            ],
99            "Instances": [],
100            "Confidence": 92.5694580078125
101          },
102          {
103            "Name": "Shelter",
104            "Parents": [
105              {
106                "Name": "Building"
107              },
108              {
109                "Name": "Countryside"
110              },
111              {
112                "Name": "Outdoors"
113              },
114              {
115                "Name": "Nature"
116              }
117            ],
118            "Instances": [],
119            "Confidence": 88.491455078125
120          },
121          {
122            "Name": "Water",
123            "Parents": [],
124            "Instances": [],
125            "Confidence": 87.63512420654297
126          }
127        ],
128        "LabelModelVersion": "2.0"
129      },
130      "datetime_created": "2018-08-07T05:31:47.132454Z",
131      "datetime_updated": "2022-09-20T05:24:09.871132Z",
132      "version": "2016-06-27"
133    }
134  }
135}

Limitations

  • This operation is not available on the Free plan.

API integrations

You don’t have to code most of the low-level API integrations. We have high-level libraries for all popular platforms:

Billing

  • This feature is available on paid plans.
  • Learn how we charge for this operation.