Fouzan Ahmed.
All projects
Full-Stack DevelopmentAI/LLM IntegrationOnline SafetyContent Moderation

eSafety Threat Detection Platform

Hackathon-built multi-modal content safety platform scoring social media content across 8 threat categories.

Hackathon project, Grand ChallengeNov 2024 – Jan 2025
View repositoryJump to architecture

Problem

  • Content moderation tools that only look at text miss a lot of what actually constitutes online harm: a grooming conversation unfolds over many messages, a deepfake is a visual problem, and coordinated harassment shows a pattern across posts that a single-message classifier never sees.
  • Built during a hackathon for the eSafety Grand Challenge, the goal was to see how far a small team could get in a fixed window toward a genuinely multi-modal safety platform, one that looks at text, images and video together instead of treating them as separate problems.

Architecture

  1. Step 01

    Content in

    Text, images, video, metadata

  2. Step 02

    FFmpeg frame extraction

    First / middle / last frame

  3. Step 03

    GPT-4 + Gemini scoring

    0-100 across 8 threat types

  4. Step 04

    90+ detection patterns

    Grooming, deepfake, harassment

  5. Step 05

    Structured JSON report

    With recommended resources

  6. Step 06

    Docker Compose

    One-command deploy

Text, image and video content all resolve to the same structured threat report.

  • A React 18 frontend talks to a Node.js/Express backend exposing a modular REST API with 8 analysis endpoints, one per major content-safety concern.
  • GPT-4 and Gemini both score submitted content from 0 to 100 across 8 threat types, including grooming, cyberbullying, deepfakes, extremism, coordinated attacks and persistent harassment, and their outputs get combined.
  • FFmpeg extracts the first, middle and last frame from video content like Instagram Reels, so visual analysis runs alongside caption and metadata rather than the video being treated as an opaque file.
  • 90+ hand-written detection patterns cover grooming progression, cyberbullying fingerprints and deepfake indicators, split across 7 visual and 6 contextual indicator types, and layer on top of the LLM scoring rather than replacing it.
  • Cross-platform coverage extends to YouTube transcripts, TikTok metadata and multi-image batches, all resolving to the same structured JSON threat report with recommended resources attached.
  • The whole stack is containerized with Docker and Docker Compose, so it deploys with one command on Heroku, Railway or Render.

Engineering decisions

  • Two models, not one. Running GPT-4 and Gemini against the same content and combining their scores was a deliberate hedge against either model's individual blind spots or inconsistency on a genuinely subjective judgment (how harmful is this, really), rather than trusting a single model's score outright.
  • Hand-written detection patterns layered on top of LLM scoring, not instead of it. The 90+ patterns for grooming progression and cyberbullying fingerprints exist because some harm signals are pattern-based and consistent enough to catch deterministically, which is faster and more auditable than relying purely on a model's judgment call for every case.
  • Frame extraction over full video analysis. Given the hackathon's time constraints, extracting first, middle and last frames from a video gave meaningful visual coverage at a fraction of the cost and complexity of full frame-by-frame or audio-track analysis.
  • Docker Compose for one-command deploy. With judges needing to actually run the thing during a hackathon, removing every bit of manual environment setup mattered as much as the detection logic itself.

Implementation

  • Shipped a working React 18 and Node.js/Express application during the hackathon window, roughly 5,000 lines of code, with a modular REST API exposing 8 analysis endpoints.
  • Integrated GPT-4 and Gemini to score content safety from 0 to 100 across 8 threat types: grooming, cyberbullying, deepfakes, extremism, coordinated attacks and persistent harassment among them.
  • Built Instagram Reel frame extraction with FFmpeg, pulling first, middle and last frames for visual analysis alongside caption and metadata.
  • Wrote 90+ detection patterns for grooming progression, cyberbullying fingerprints and deepfake indicators, including 7 visual and 6 contextual indicator types.
  • Added cross-platform video analysis for YouTube transcripts, TikTok metadata and multi-image batches, all returning structured JSON threat reports with recommended resources.
  • Containerized the full stack with Docker and Docker Compose for one-command deployment on Heroku, Railway or Render.

Challenges

  • Balancing coverage against hackathon time constraints meant constantly deciding what to hand-code as a deterministic pattern versus what to leave to the LLM scoring, since both approaches have real gaps.
  • Getting frame extraction to run reliably across different video formats and platforms (Reels, TikTok, YouTube) within the time available, without it becoming its own multi-day project.
  • Keeping the combined GPT-4 and Gemini scoring consistent enough to be useful, since the two models don't always agree on borderline content.

Results

  • Around 5,000 lines of working code shipped inside the hackathon window, covering 8 analysis endpoints end to end.
  • 90+ detection patterns across grooming, cyberbullying and deepfake indicators, combined with multi-modal LLM scoring.
  • The full stack deploys with one command via Docker Compose, which made live demoing and judge testing straightforward.
  • Coverage extended past a single platform: text, images, Instagram Reels, YouTube transcripts and TikTok metadata all feed into the same structured threat report.

Technology

Frontend

React 18TypeScriptMaterial-UIProgressive Web App

Backend

Node.jsExpressREST APIMongoDBPython

AI & Media

OpenAI GPT-4Google GeminiFFmpegNLPMulti-modal AI

DevOps

DockerDocker Compose

More projects

APRA Data Engineering & Analytics Platform

Personal / Portfolio project

Problem

APRA publishes superannuation performance data as unversioned Excel filings, with no API and no easy way to compare funds over time.

Key result

Weekly pipeline runs unattended end to end, gated by dbt tests and CI, feeding a live 3-page Power BI dashboard.

PythonApache AirflowdbtAWS S3AWS RDSAWS EC2+8 more
ArchitectureGitHubProject details

Market Data Warehouse on Snowflake

Personal / Portfolio project

Problem

Backtesting a forecasting model on financial time series is easy to get subtly wrong: without point-in-time discipline, the model ends up training on information it wouldn't have had yet.

Key result

A clustering key fix cut partitions scanned on date filters by roughly an order of magnitude, and Time Travel snapshots kept lookahead bias out of every backtest.

SnowflakeSQLPythonAWS S3Streams & TasksWindow Functions+6 more
ArchitectureGitHubProject details