Guides

Handling HEIC & WebP images in a face recognition pipeline

The iPhone default and the modern web default are the two formats most face APIs reject. Here's how to stop fighting them.

By SightRadar EngineeringUpdated July 18, 2026 8 min read Markdown

TL;DR

iPhones save photos as HEIC and modern CDNs serve WebP, but AWS Rekognition accepts only JPEG and PNG — so most pipelines add a server-side transcode step that costs compute and can strip or misread EXIF orientation. SightRadar decodes HEIC/HEIF, WebP, JPEG, PNG, TIFF, GIF, and BMP natively from the file's actual bytes (not its extension), so you can send the original upload. This guide covers format detection, EXIF rotation, size limits, and when you still need a conversion step.

If your users upload photos from a phone, a large share arrive as HEIC — Apple's default since iOS 11. If your images come from a CDN, many are WebP. Both are exactly the formats the hyperscaler face APIs don't take, so teams bolt on a transcode stage that adds latency, compute cost, and a whole class of orientation bugs. Let's look at the problem and how to avoid most of it.

What each API actually accepts

Format	SightRadar	AWS Rekognition	Common source
JPEG	✅	✅	Everything
PNG	✅	✅	Screenshots, graphics
WebP	✅	❌	Modern CDNs / web
HEIC / HEIF	✅	❌	iPhone default
TIFF	✅	❌	Scanners, pro cameras
GIF	✅	Azure only	Legacy
BMP	✅	Azure only	Legacy Windows

Direct decode support — formats you can send without pre-converting. Verify current limits with each vendor.

Note: The practical consequence: on a JPEG/PNG-only API, every iPhone HEIC upload and every WebP from your CDN must be transcoded before you can index or search it. That's a compute cost and a failure point on the hot path of user uploads.

Format detection: trust the bytes, not the extension

A file named photo.jpg can actually be a HEIC (common when apps rename on export). Detect format from the file's magic bytes, not its extension or the client-supplied MIME type — both lie. SightRadar detects from the actual bytes, so a mislabelled file still decodes; if you run your own pre-checks, do the same.

def sniff(data: bytes) -> str:
    if data[:3] == b"\xff\xd8\xff":                 return "jpeg"
    if data[:8] == b"\x89PNG\r\n\x1a\n":            return "png"
    if data[:4] == b"RIFF" and data[8:12] == b"WEBP":  return "webp"
    if data[4:12] in (b"ftypheic", b"ftypmif1"):       return "heic"
    return "unknown"   # let the engine decide, or reject

Sniff the real format from magic bytes before trusting the extension.

EXIF orientation — the silent face-detection killer

Phone cameras store the image upright in pixels but record the real orientation in an EXIF tag. If a downstream step drops EXIF (many naive transcodes do), the image is decoded sideways and a face detector that expects roughly upright faces can miss them. Two safe options: preserve EXIF through your pipeline, or bake the rotation into the pixels before sending.

from PIL import Image, ImageOps
def normalize(path, out):
    img = Image.open(path)
    img = ImageOps.exif_transpose(img)   # apply the orientation tag to pixels
    img.save(out)                        # now upright regardless of EXIF

Bake EXIF rotation into pixels so orientation can't be lost downstream.

Size limits and high-resolution photos

Byte cap: SightRadar accepts images up to 30 MB (raw bytes, URL fetch, or base64); larger is rejected with HTTP 413. Rekognition caps raw bytes at 5 MB (15 MB via S3).
Dimension budget: very large photos (past an internal pixel budget) are auto-downscaled before embedding rather than rejected — so full-resolution phone and DSLR photos work without you resizing first.
SVG is rejected on purpose (XML/XXE/script risk) — it's not a photo format anyway.

When you still need to convert

Native decoding removes the transcode step for ingestion into the face API. You may still convert for your own reasons — generating web thumbnails, normalising to one format in your storage, or stripping metadata for privacy. Do those for your product's needs, not because the face API forces them.

Send an original HEIC or WebP straight to the index endpoint and see it decode.

Try the quickstart

Frequently asked questions

Can I send iPhone HEIC photos to a face recognition API?

With SightRadar, yes — it decodes HEIC/HEIF natively along with WebP, JPEG, PNG, TIFF, GIF, and BMP, so you can send the original iPhone upload. AWS Rekognition accepts only JPEG and PNG, so HEIC must be transcoded first. Native decoding removes that step from the upload hot path.

Why are faces not detected in my uploaded photos?

A common cause is EXIF orientation being lost. Phones store pixels upright and record rotation in an EXIF tag; if a transcode step drops that tag, the image is decoded sideways and an upright-face detector can miss faces. Preserve EXIF through your pipeline, or bake the rotation into the pixels (for example with PIL's exif_transpose) before sending.

How large an image can I send?

SightRadar accepts images up to 30 MB of raw bytes (upload, URL fetch, or base64); larger returns HTTP 413. Very high-resolution photos beyond the internal pixel budget are auto-downscaled before embedding rather than rejected, so full-resolution phone and camera photos work without you resizing. SVG is deliberately rejected for security.

Keep reading