Unsafe content detection

Uploadcare allows the detection and identification of unwanted, NSFW, or offensive user-generated content in images to help prevent inappropriate images from being published. AWS Rekognition machine learning technology automatically identifies and flags such content. The feature is available only for images and accessible on the REST API v0.7 or higher via add-ons API.

How it works

Unsafe content detection works asynchronously through the REST API.

  1. Start a processing job via REST API. Send an input file UUID with the necessary processing operations.
  2. Wait until the processing job status becomes done.
  3. Detected moderation labels will be stored in the JSON response in the appdata section of the processed file.

Example

Execute unsafe content detection on a target UUID

$curl -X POST \
>     -H 'Content-Type: application/json' \
>     -H 'Accept: application/vnd.uploadcare-v0.7+json' \
>     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
>     "https://api.uploadcare.com/addons/aws_rekognition_detect_moderation_labels/execute/" \
>     -d "{\"target\": \"$UUID\"}"

Check out REST API reference to see how to execute an add-on on a target UUID.

Webhook event

To get the job result, you need to enable file.info_updated in the Webhook section of the dashboard.

After completing the processing job, the webhook will be sent to the endpoint you specified in the webhook settings.

File information in response:

1 "initiator": {
2    "type": "addon",
3    "detail": {
4      "addon_name": "aws_rekognition_detect_moderation_labels",
5    }
6  },
7  "hook": {
8    ...
9    "event": "file.info_updated",
10    "is_active": true,
11    "version": "0.7",
12    ...
13  },
14    "uuid": ":UUID",
15    "is_image": true,
16    "is_ready": true,
17    "metadata": {},
18    "appdata": {
19      "aws_rekognition_detect_moderation_labels": {
20        "data": {
21          "ModerationLabels": [
22            {
23              "Confidence": 99.87886810302734,
24              "Name": "Weapons",
25              "ParentName": "Violence"
26            },
27            {
28              "Confidence": 99.87886810302734,
29              "Name": "Violence",
30              "ParentName": ""
31            }
32          ],
33          "ModerationModelVersion": "6.1"
34        },
35        "version": "2016-06-27",
36        "datetime_created": "2023-11-08T09:35:08.907Z",
37        "datetime_updated": "2023-11-08T09:35:08.907Z"
38      },
39    }

Check the execution status

If your application does not have a backend or uses a mobile version, you can submit the request yourself.

Use the request ID returned by the add-on execution request described above.

$curl -X GET \
>     -H 'Content-Type: application/json' \
>     -H 'Accept: application/vnd.uploadcare-v0.7+json' \
>     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
>     "https://api.uploadcare.com/addons/aws_rekognition_detect_moderation_labels/execute/status/?request_id=$REQUEST_ID"

As a request result, you will receive a UUID of the file when the status is done.

1{
2  "result": {
3    "file_id": "22240276-2f06-41f8-9411-755c8ce926ed"
4  },
5  "status": "done"
6}

Check out REST API reference to check the execution status.

Get the result

Once the status changes to done, appdata will contain the result of the execution.

To get it, run file info request, specifying the include parameter. There are two methods for getting info on detected objects via GET requests for single-file and multi-file cases:

$curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
>     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
>     "https://api.uploadcare.com/files/?include=appdata"

or

$curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
>     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
>     "https://api.uploadcare.com/files/$UUID/?include=appdata"

You’ll get JSON with appdata:

1"appdata": {
2  "aws_rekognition_detect_moderation_labels": {
3    "data": {
4      "ModerationModelVersion": "6.0",
5      "ModerationLabels": [
6        {
7          "Confidence": 93.41645812988281,
8          "Name": "Weapons",
9          "ParentName": "Violence"
10        }
11      ]
12    },
13    "version": "2016-06-27",
14    "datetime_created": "2023-02-21T11:25:31.259763Z",
15    "datetime_updated": "2023-02-21T11:27:33.359763Z"
16  },
17}

Billing

  • This feature is available on paid plans.
  • Learn how we charge for this operation.