Snapchat is mostly a phone app, so a lot of people assume there is nothing useful on the open web. That is half true. Creator and public profiles still ship HTML you can open logged out at snapchat.com/add/<handle> or snapchat.com/@<handle>, and Spotlight pages exist as shareable URLs. The other half is the hard part: the pages are JS-heavy, fields vanish by region, and anything that looks like a private story is simply not there without an account.

This guide builds a small Python tool that:

  1. Fetches a public Snapchat profile page
  2. Pulls useful fields out of embedded JSON / page data (handle, display name, bio, subscriber count, website, avatar, Spotlight-highlight flags)
  3. Optionally lists Spotlight highlight stubs that appear on that page
  4. Stores a snapshot every run in SQLite
  5. Retries politely when Snapchat or a CDN flinches

I will not scrape private stories, Friends-only content, or anything behind a login. If a page redirects you to install the app or sign in, that is a stop sign, not a puzzle.

Legal / ethics note: Stick to publicly visible pages. Respect Snapchat's Terms of Service and rate limits. Do not harvest private users, do not automate friend adds, and do not build a dataset of minors. If your use case is your own accounts, Snapchat's official creator tools are the cleaner path.

What a public profile actually gives you

Open https://www.snapchat.com/add/<some-public-creator> in a private window. When it works, you typically get:

  • Username / handle, display title, and bio
  • Subscriber (follower) count
  • Website URL and profile picture URL
  • Flags like whether curated or Spotlight highlights exist
  • Sometimes a short list of Spotlight highlight stubs

What you do not get logged out: private Stories, chat, Friends-only tiles, or a full chronological dump of every Spotlight. Treat the web page as a marketing card, not an archive.

Setup

python -m venv .venv && source .venv/bin/activate
pip install requests beautifulsoup4

snap-scraper/
  handles.txt          # one public handle per line
  scrape_snap.py
  out/

Step 1: A polite fetch with backoff

Snapchat (and the CDNs in front of it) will return 429, soft empty pages, or app-install interstitial HTML if you hammer from a datacenter IP. Reuse one Session, send realistic headers, and back off with jitter.

# scrape_snap.py
from __future__ import annotations

import json
import random
import re
import sqlite3
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from urllib.parse import quote

import requests
from bs4 import BeautifulSoup

SESSION = requests.Session()
SESSION.headers.update({
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/129.0.0.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
    "Accept": "text/html,application/xhtml+xml;q=0.9,*/*;q=0.8",
})

HANDLE_RE = re.compile(r"^[A-Za-z0-9._-]{3,30}$")
OUT = Path("out")
OUT.mkdir(exist_ok=True)


def fetch(url: str, retries: int = 4) -> str:
    last: Exception | None = None
    for attempt in range(retries):
        try:
            resp = SESSION.get(url, timeout=30, allow_redirects=True)
            if resp.status_code in (401, 403, 429, 503):
                raise RuntimeError(f"HTTP {resp.status_code}")
            text = resp.text
            if len(text) < 800:
                raise RuntimeError("suspiciously short body")
            lower = text.lower()
            if "log in" in lower and "snapchat" in lower and "add/" not in resp.url:
                raise RuntimeError("looks like an auth / install wall")
            return text
        except Exception as e:
            last = e
            time.sleep((2 ** attempt) + random.uniform(0.3, 1.5))
    raise RuntimeError(f"failed after retries: {last}")

Two habits matter more than the retry math: reuse the session (fresh TCP every request looks like a botnet), and treat short / wall HTML as failure even when the status is 200.

Step 2: Find the embedded JSON

Snapchat does not give you a documented public JSON API. On a good day the HTML still embeds a big state blob — __NEXT_DATA__, a window.__PRELOADED_STATE__ = {...} assignment, JSON-LD, or a mix. Walk the scripts and keep the first object that looks like a profile:

def _walk(obj: Any, hits: list[dict]):
    if isinstance(obj, dict):
        keys = {k.lower() for k in obj}
        if {"username", "bio"} <= keys or {"username", "title"} <= keys:
            hits.append(obj)
        up = obj.get("userProfile") or obj.get("userprofile")
        if isinstance(up, dict):
            hits.append(up)
        for v in obj.values():
            _walk(v, hits)
    elif isinstance(obj, list):
        for v in obj:
            _walk(v, hits)


def extract_profile_blob(html: str) -> dict | None:
    soup = BeautifulSoup(html, "html.parser")
    candidates: list[dict] = []

    nextdata = soup.find("script", id="__NEXT_DATA__")
    if nextdata and nextdata.string:
        try:
            _walk(json.loads(nextdata.string), candidates)
        except json.JSONDecodeError:
            pass

    assign_re = re.compile(
        r"(?:window\.__PRELOADED_STATE__|window\.__INITIAL_STATE__)\s*=\s*(\{.*?\})\s*;?\s*</script>",
        re.DOTALL,
    )
    for m in assign_re.finditer(html):
        try:
            _walk(json.loads(m.group(1)), candidates)
        except json.JSONDecodeError:
            continue

    for tag in soup.find_all("script", attrs={"type": "application/ld+json"}):
        if not tag.string:
            continue
        try:
            _walk(json.loads(tag.string), candidates)
        except json.JSONDecodeError:
            continue

    return max(candidates, key=len) if candidates else None

This is deliberately loose. Snapchat renames nested keys; normalize what you found into a stable shape before it hits SQLite.

Step 3: Normalize fields and optional Spotlight stubs

def normalize(handle: str, blob: dict, page_url: str) -> dict:
    username = blob.get("username") or blob.get("userName") or handle
    title = blob.get("title") or blob.get("mutableName") or blob.get("displayName") or ""
    bio = blob.get("bio") or blob.get("description") or ""
    subs = blob.get("subscriberCount") or blob.get("subscriber_count") or blob.get("followers")
    website = blob.get("websiteUrl") or blob.get("website") or ""
    avatar = blob.get("profilePictureUrl") or blob.get("profile_picture_url") or ""
    has_spot = blob.get("hasSpotlightHighlights")
    if has_spot is None:
        has_spot = bool(blob.get("spotlightHighlights"))

    highlights = []
    for item in (blob.get("spotlightHighlights") or [])[:20]:
        if not isinstance(item, dict):
            continue
        thumb = item.get("thumbnailUrl")
        if isinstance(thumb, dict):
            thumb = thumb.get("value")
        highlights.append({
            "id": item.get("id") or item.get("storyId") or "",
            "title": item.get("storyTitle") or item.get("title") or "",
            "thumbnail": thumb or "",
        })

    return {
        "handle": username.lstrip("@"),
        "title": title,
        "bio": bio,
        "subscriber_count": str(subs) if subs is not None else None,
        "website_url": website,
        "profile_picture_url": avatar if isinstance(avatar, str) else "",
        "has_spotlight_highlights": bool(has_spot),
        "spotlight_highlights": highlights,
        "source_url": page_url,
        "fetched_at": datetime.now(timezone.utc).isoformat(),
        "raw_keys": sorted(blob.keys()),
    }


def scrape_handle(handle: str) -> dict:
    if not HANDLE_RE.match(handle.lstrip("@")):
        raise ValueError(f"bad handle: {handle!r}")
    handle = handle.lstrip("@")
    urls = [
        f"https://www.snapchat.com/add/{quote(handle)}",
        f"https://www.snapchat.com/@{quote(handle)}",
    ]
    last_err: Exception | None = None
    for url in urls:
        try:
            html = fetch(url)
            blob = extract_profile_blob(html)
            if not blob:
                raise RuntimeError("no profile JSON found in page")
            profile = normalize(handle, blob, url)
            if not profile["title"] and not profile["bio"] and not profile["subscriber_count"]:
                raise RuntimeError("profile fields empty after parse")
            return profile
        except Exception as e:
            last_err = e
            time.sleep(1.5 + random.random())
    raise RuntimeError(f"{handle}: {last_err}")

If both URL shapes fail, stop. Do not invent a logged-in session to “fix” it.

Step 4: SQLite snapshots

A single scrape is a curiosity. Two scrapes a week apart are a dataset — subscriber drift, bio changes, new Spotlight stubs.

def init_db(path: Path = OUT / "snap.db") -> sqlite3.Connection:
    conn = sqlite3.connect(path)
    conn.execute(
        """
        CREATE TABLE IF NOT EXISTS profile_snapshots (
            id INTEGER PRIMARY KEY,
            handle TEXT NOT NULL,
            title TEXT,
            bio TEXT,
            subscriber_count TEXT,
            website_url TEXT,
            profile_picture_url TEXT,
            has_spotlight_highlights INTEGER,
            spotlight_json TEXT,
            source_url TEXT,
            fetched_at TEXT NOT NULL
        )
        """
    )
    conn.execute(
        "CREATE INDEX IF NOT EXISTS idx_snap_handle_time "
        "ON profile_snapshots(handle, fetched_at)"
    )
    conn.commit()
    return conn


def store(conn: sqlite3.Connection, profile: dict) -> None:
    conn.execute(
        """
        INSERT INTO profile_snapshots (
            handle, title, bio, subscriber_count, website_url,
            profile_picture_url, has_spotlight_highlights, spotlight_json,
            source_url, fetched_at
        ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
        """,
        (
            profile["handle"], profile["title"], profile["bio"],
            profile["subscriber_count"], profile["website_url"],
            profile["profile_picture_url"], int(profile["has_spotlight_highlights"]),
            json.dumps(profile["spotlight_highlights"]),
            profile["source_url"], profile["fetched_at"],
        ),
    )
    conn.commit()
    (OUT / f"{profile['handle']}.json").write_text(
        json.dumps(profile, indent=2), encoding="utf-8"
    )


def main():
    handles = [
        line.strip().lstrip("@")
        for line in Path("handles.txt").read_text().splitlines()
        if line.strip() and not line.startswith("#")
    ]
    conn = init_db()
    for i, handle in enumerate(handles):
        try:
            profile = scrape_handle(handle)
            store(conn, profile)
            print(profile["handle"], profile.get("subscriber_count"), (profile.get("title") or "")[:40])
        except Exception as e:
            print(f"SKIP {handle}: {e}")
        if i < len(handles) - 1:
            time.sleep(3 + random.uniform(0, 4))


if __name__ == "__main__":
    main()

Put a couple of public creator handles in handles.txt and run python scrape_snap.py. Space requests a few seconds apart.

The walls you'll hit

Auth and install walls. From some IPs the “public” page is an app-install interstitial or a login prompt with no profile JSON. Back off, try later, or accept that handle is not scrapeable from your network.

JS-heavy rendering. A plain requests client only sees the first HTML paint. If more of the profile moves into client-side fetches, the JSON walk returns nothing even though Chrome looks fine. Playwright helps — and still loses to the same IP reputation rules.

Missing and renamed fields. subscriberCount is sometimes a string ("1200"), sometimes missing, sometimes only on badge accounts. Store NULL when absent; keep raw_keys in the JSON dump so you notice schema drift.

Spotlight stubs ≠ full feed. Profile-card highlights are curated, not every Spotlight ever posted. Deep browsing lives on /spotlight/<id>, topic pages, and search.

Regional variance and rate limits. Pin Accept-Language: en-US. Burst traffic from cloud IPs gets 429 or empty bodies fast — one handle every few seconds from a laptop is usually fine.

Private users. Handles that exist in the app but have no public web profile will never give you a clean page. Do not route around that with account cookies.

When DIY stops being worth it

The script above is a good fit for a short watchlist of public creators on a daily cron. It stops being fun when you need reliable subscriber counts across thousands of handles, Spotlight feed browsing, search, or topic collections — and you do not want to wake up because __NEXT_DATA__ moved again.

That is where a hosted API earns its keep. ScrapeAtlas* exposes Snapchat as normal HTTPS with an x-api-key header. Routes that map cleanly to the jobs above:

  • GET /v1/snapchat/profile?handle= — public profile, including userProfile fields like username, title, bio, subscriberCount, websiteUrl, profilePictureUrl, hasSpotlightHighlights
  • GET /v1/snapchat/spotlight/feed?limit= — Spotlight feed stories with creator + engagement metadata
  • GET /v1/snapchat/search?query= — public search sections (profiles, Spotlight, topics, places)
  • GET /v1/snapchat/spotlight?url= — one Spotlight URL in detail
  • GET /v1/snapchat/topic?slug= — a topic collection and its videos
import os
import requests

KEY = os.environ["SCRAPEATLAS_API_KEY"]
BASE = "https://api.scrapeatlas.com"


def sa_get(path: str, **params):
    r = requests.get(
        f"{BASE}{path}",
        params={k: v for k, v in params.items() if v is not None},
        headers={"x-api-key": KEY},
        timeout=60,
    )
    r.raise_for_status()
    return r.json()


profile = sa_get("/v1/snapchat/profile", handle="example.creator")
print(profile["status"], profile["coverage"].get("complete"))
up = profile.get("userProfile") or {}
print(up.get("username"), up.get("subscriberCount"), up.get("bio"))

feed = sa_get("/v1/snapchat/spotlight/feed", limit=20)
if not feed.get("coverage", {}).get("complete", True):
    print("partial:", feed["coverage"].get("warnings"))
for spot in feed.get("spotlights") or []:
    author = spot.get("author") or {}
    metrics = spot.get("metrics") or {}
    print(spot.get("id"), author.get("handle"), metrics.get("views"), (spot.get("caption") or "")[:60])

# one video: sa_get("/v1/snapchat/spotlight", url="https://www.snapchat.com/spotlight/...")
# search:    sa_get("/v1/snapchat/search", query="trail running")

Responses use success, status (success or partial), and a coverage object with complete and warnings. Read those every time — a partial profile is not the same as “this creator went quiet.” Each successful request costs one credit (including partials); failed requests cost nothing.

Wrapping up

  • Start from public /add/<handle> or /@<handle> only; treat auth walls as hard stops.
  • Prefer embedded JSON over brittle CSS selectors; normalize before SQLite.
  • Snapshot every run so subscriber and highlight diffs exist later.
  • Expect JS rendering, renamed fields, regional variance, and rate limits — plan for empty results, not just exceptions.
  • When you need Spotlight feeds, search, topics, or reliability without babysitting script tags, use a real Snapchat API surface instead of fighting the HTML.

If you are tracking public Snapchat creators for research, I am curious whether subscriber drift or Spotlight highlight changes ended up being the signal you cared about most.

*I build it.