CyberCane: Neuro-Symbolic RAG for Privacy-Preserving Phishing Detection with Formal Ontology Reasoning

Abstract

System Architecture

Dual-phase neuro-symbolic pipeline with privacy-preserving RAG and formal ontology reasoning.

Healthcare Application: Domain-Specific Threat Taxonomy

Healthcare-specific threat taxonomy showing attack distribution: 34% Social Engineering, 26% AI Content, 20.5% Insurance Phishing, 19.5% Appointment Scams. Framework generalizes to finance, legal, and government domains.

Results

Table 1: Detection Performance (Precision/Recall/FPR)

Table 2: Privacy and Cost Tradeoffs vs Direct LLM Baseline

Table 3: Operating Modes (Tunable Thresholds)

Note: FPR remains constant at 0.16% across all modes via shared Phase 1 high-confidence threshold (score ≥ 5). † PHI redacted before API transmission. § Privacy-constrained fair comparison using same redaction pipeline as CyberCane. CyberCane provides 6× lower FPR (0.16% vs. 0.98%), PhishOnt verifiable reasoning chains, and tunable operating points.

Table 4: PhishOnt Ontology Coverage

Notable Findings

Layered Defense: Phase 2 Closes Phase 1 Blind Spots

79.5% of phishing emails evade Phase 1 symbolic rules (rising to 95.8% for Neutral-tone and 97.2% for Greed-tone emails). Phase 2 RAG recovers 99.0% of these Phase 1-evaded attacks, resulting in a combined two-phase miss rate of only 0.8%. This demonstrates that the dual-phase architecture is essential: neither phase alone achieves healthcare-grade performance.

PII Quantification: Redaction Scope

50.7% of DataPhish 2025 emails contain at least one PII item (mean 1.63 items/email), confirming that unredacted transmission to external APIs exposes sensitive data at scale. CyberCane's automated redaction pipeline eliminates this exposure entirely before any API call, achieving 0% PHI leakage across all test conditions.

Confidence Discriminability

Phase 2 RAG produces meaningfully separated confidence distributions: phishing emails receive a mean similarity score of 54.6% versus 52.8% for benign emails. Critically, 85.5% of benign emails fall below the phishing mean, providing a clear statistical basis for the conservative 0.40 threshold that drives the 0.16% FPR.

Methodology

Phase 1: Deterministic Symbolic Analysis

Applies rule-based heuristics without external API calls:

• DNS validation: MX record lookup, SPF/DMARC compliance checks
• URL analysis: IP literal detection, shortened URL identification, domain lookalikes
• Content patterns: Urgency cues, credential requests, freemail/brand mismatches
• Decision threshold: Score ≥ 5 → High-confidence verdict (skip Phase 2)

Phase 2: Privacy-Preserving RAG

Escalates ambiguous cases through a privacy-first pipeline:

1. PII Redaction: Regex-based entity removal (emails, phone numbers, SSNs, medical IDs)
2. Embedding: OpenAI text-embedding-3-small (1536-dim vectors)
3. HNSW Retrieval: k=8 nearest neighbors from phishing-only corpus (pgvector)
4. LLM Reasoning: GPT-4.1-mini generates explanations with retrieved context
5. Conservative Thresholds: Top similarity ≥ 0.40, Avg-Top-3 ≥ 0.35

PhishOnt: Formal Ontology Reasoning

OWL-based ontology with SWRL rules maps observed indicators to formal attack classifications:

• 8 attack types (credential phishing, BEC, malware distribution, etc.)
• 15 indicator classes (domain spoofing, urgency, missing authentication)
• Verifiable reasoning chains for interpretability
• 85.0% coverage on test split (n=1,110)

Multi-Layer Explanation System

Example Medicare phishing detection showing three explanation layers: (1) Symbolic evidence from DNS/SMTP analysis, (2) Retrieved similar phishing examples with similarity scores, (3) GPT-4.1-mini contextual reasoning. Final verdict: 94% confidence with actionable recommendations.

Reproducibility

All evaluation scripts, datasets, and threshold configurations are available in the repository:

• reports_cybercane/eval_test_split.py — Test split evaluator
• reports_cybercane/bootstrap_ci.py — 1000-sample bootstrap confidence intervals
• reports_cybercane/statistical_significance.py — McNemar χ² tests
• reports_cybercane/cost_benefit_analysis.py — ROI and cost-benefit calculations
• reports_cybercane/tune_thresholds.py — Threshold optimization (F1-maximizing grid search)
• dataset_cybercane/best_thresholds_dataphish.json — Optimal threshold configuration
• dataset_cybercane/Nazario.clean.csv — Nazario phishing corpus (n=1,110)
• dataset_cybercane/dataphish_test.jsonl — DataPhish 2025 test split (n=2,337)
• dataset_cybercane/dataphish_val.jsonl — DataPhish 2025 validation split

Quick Start

Docker Deployment (Recommended)

docker compose up --build
# Web: http://localhost:3000 | API: http://localhost:8000 | Docs: http://localhost:8000/docs

Try the API

# Phase 1: Rules-based scan (no API key required)
curl -X POST http://localhost:8000/scan \
  -H "Content-Type: application/json" \
  -d '{"sender":"alert@bank.com","subject":"Urgent: verify account","body":"Click: http://bit.ly/x","url":1}'

# Phase 2: Full RAG analysis (requires OPENAI_API_KEY)
curl -X POST http://localhost:8000/ai/analyze \
  -H "Content-Type: application/json" \
  -d '{"sender":"alert@bank.com","subject":"Urgent: verify account","body":"Click: http://bit.ly/x","url":1}'

Citation

@article{hakim2026cybercane,
  author    = {S. B. Hakim and A. Afzal and Q. Zhao and V. Majmundar and P. Sloboda and H. H. Song},
  title     = {{CyberCane}: Neuro-Symbolic {RAG} for Privacy-Preserving Phishing Detection with Formal Ontology Reasoning},
  journal   = {arXiv preprint arXiv:2604.23563},
  year      = {2026},
  note      = {[Online]. Available: \url{https://arxiv.org/abs/2604.23563}},
  primaryclass = {cs.CR}
}