Unsafe content detection

Detect and identify inappropriate, unwanted, NSFW, or offensive user-generated content in order to enhance user safety, ensure brand integrity, and comply with local and global regulations.

Avoid depending solely on human moderators for reviewing content generated by third parties. Take proactive measures instead of waiting for user complaints to remove offensive or inappropriate images.

Let our system automatically identify and flag such content, allowing you to focus on more important tasks while ensuring comprehensive moderation coverage. Utilize the AWS Rekognition machine learning technology to detect moderation labels in images.

The feature is currently available on the REST API v0.7 or higher via add-ons API.

How it works

Unsafe content detection works asynchronously through the REST API.

  1. Start a processing job via REST API. Send an input file UUID with the necessary processing operations.
  2. Wait until the processing job status becomes done.
  3. Detected moderation labels will be stored in the JSON response in the appdata section of the processed file.

Example

Execute unsafe content detection on a target UUID

curl -X POST \
     -H 'Content-Type: application/json' \
     -H 'Accept: application/vnd.uploadcare-v0.7+json' \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     "https://api.uploadcare.com/addons/aws_rekognition_detect_moderation_labels/execute/" \
     -d "{\"target\": \"$UUID\"}"bash

Check out REST API reference to see how to execute an add-on on a target UUID.

Check the execution status

Use the request ID returned by the add-on execution request described above.

curl -X GET \
     -H 'Content-Type: application/json' \
     -H 'Accept: application/vnd.uploadcare-v0.7+json' \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     "https://api.uploadcare.com/addons/aws_rekognition_detect_moderation_labels/execute/status/?request_id=$REQUEST_ID"bash

As a request result, you will receive a UUID of the file when the status is done.

{
  "result": {
    "file_id": "22240276-2f06-41f8-9411-755c8ce926ed"
  },
  "status": "done"
}json

Check out REST API reference to check the execution status.

Get the result

Once the status changes to done, appdata will contain the result of the execution.

To get it, run file info request, specifying the include parameter. There are two methods for getting info on detected objects via GET requests for single-file and multi-file cases:

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     "https://api.uploadcare.com/files/?include=appdata"bash

or

curl -H "Accept: application/vnd.uploadcare-v0.7+json" \
     -H "Authorization: Uploadcare.Simple $YOUR_PUBLIC_KEY:$YOUR_SECRET_KEY" \
     "https://api.uploadcare.com/files/$UUID/?include=appdata"bash

You'll get JSON with appdata:

"appdata": {
  "aws_rekognition_detect_moderation_labels": {
    "data": {
      "ModerationModelVersion": "6.0",
      "ModerationLabels": [
        {
          "Confidence": 93.41645812988281,
          "Name": "Weapons",
          "ParentName": "Violence"
        }
      ]
    },
    "version": "2016-06-27",
    "datetime_created": "2023-02-21T11:25:31.259763Z",
    "datetime_updated": "2023-02-21T11:27:33.359763Z"
  },
}json

Billing