Unsafe content detection

Detect and identify inappropriate, unwanted, NSFW, or offensive user-generated content in order to enhance user safety, ensure brand integrity, and comply with local and global regulations.

Avoid depending solely on human moderators for reviewing content generated by third parties. Take proactive measures instead of waiting for user complaints to remove offensive or inappropriate images.

Let our system automatically identify and flag such content, allowing you to focus on more important tasks while ensuring comprehensive moderation coverage. Utilize the AWS Rekognition machine learning technology to detect moderation labels in images.

The feature is currently available on the REST API v0.7 or higher via add-ons API.

How it works

Unsafe content detection works asynchronously through the REST API.

  1. Start a processing job via REST API. Send an input file UUID with the necessary processing operations.
  2. Wait until the processing job status becomes done.
  3. Detected moderation labels will be stored in the JSON response in the appdata section of the processed file.

Example

Execute unsafe content detection on a target UUID

curl -X POST \
     -H 'Content-Type: application/json' \
     -H 'Accept: application/vnd.uploadcare-v0.7+json' \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     "https://api.uploadcare.com/addons/aws_rekognition_detect_moderation_labels/execute/" \
     -d "{\"target\": \"$UUID\"}"

Check out REST API reference to see how to execute an add-on on a target UUID.

Webhook event

To get the job result, you need to enable file.info_updated in the Webhook section of the dashboard.

After completing the processing job, the webhook will be sent to the endpoint you specified in the webhook settings.

File information in response:

"initiator": {
    "type": "addon",
    "detail": {
      "addon_name": "aws_rekognition_detect_moderation_labels",
    }
  },
  "hook": {
    ...
    "event": "file.info_updated",
    "is_active": true,
    "version": "0.7",
    ...
  },
    "uuid": ":UUID",
    "is_image": true,
    "is_ready": true,
    "metadata": {},
    "appdata": {
      "aws_rekognition_detect_moderation_labels": {
        "data": {
          "ModerationLabels": [
            {
              "Confidence": 99.87886810302734,
              "Name": "Weapons",
              "ParentName": "Violence"
            },
            {
              "Confidence": 99.87886810302734,
              "Name": "Violence",
              "ParentName": ""
            }
          ],
          "ModerationModelVersion": "6.1"
        },
        "version": "2016-06-27",
        "datetime_created": "2023-11-08T09:35:08.907Z",
        "datetime_updated": "2023-11-08T09:35:08.907Z"
      },

Check the execution status

If your application does not have a backend or uses a mobile version, you can submit the request yourself.

Use the request ID returned by the add-on execution request described above.

curl -X GET \
     -H 'Content-Type: application/json' \
     -H 'Accept: application/vnd.uploadcare-v0.7+json' \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     "https://api.uploadcare.com/addons/aws_rekognition_detect_moderation_labels/execute/status/?request_id=$REQUEST_ID"

As a request result, you will receive a UUID of the file when the status is done.

{
  "result": {
    "file_id": "22240276-2f06-41f8-9411-755c8ce926ed"
  },
  "status": "done"
}

Check out REST API reference to check the execution status.

Get the result

Once the status changes to done, appdata will contain the result of the execution.

To get it, run file info request, specifying the include parameter. There are two methods for getting info on detected objects via GET requests for single-file and multi-file cases:

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     "https://api.uploadcare.com/files/?include=appdata"

or

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     "https://api.uploadcare.com/files/$UUID/?include=appdata"

You'll get JSON with appdata:

"appdata": {
  "aws_rekognition_detect_moderation_labels": {
    "data": {
      "ModerationModelVersion": "6.0",
      "ModerationLabels": [
        {
          "Confidence": 93.41645812988281,
          "Name": "Weapons",
          "ParentName": "Violence"
        }
      ]
    },
    "version": "2016-06-27",
    "datetime_created": "2023-02-21T11:25:31.259763Z",
    "datetime_updated": "2023-02-21T11:27:33.359763Z"
  },
}

Billing

  • This feature is available on paid plans.
  • Learn how we charge for this operation.