Heru: 3D Facial Scanning iOS App for Cubitts Eyewear

Role: Founding 3D Engineer
Company: Cubitts
Duration: 2019–2022
Stack: iOS, Swift, Three.js, JavaScript, Node.js, Express, MongoDB, AWS, Bellus3D SDK

Related Projects:

• Tartare 3D Configurator - The custom Three.js framework powering Heru’s visualization (coming soon)
• Frame Up: Automated 3D Eyewear Modeling - Born from Heru’s manual modeling bottleneck
• Heru 2: Next Development - Accuracy Boost for made-to-measure

From Bespoke Tool to Universal Fitting Platform

I built an iOS 3D facial scanning app for Cubitts (2019-2022) that analyzes 60+ frame models in under 2 seconds to recommend perfectly fitting eyewear.

The scanning process

Heru turns eyewear fitting into a mobile experience using 3D facial scanning. It does two things:

1. Ready-to-Wear Recommendations - Analyzes facial shape across 60+ frame models × 1 to 5 sizes to find perfectly fitting options from existing inventory
2. Bespoke Adjustments - Auto-calculates precise modifications for made-to-measure frames that need custom manufacturing

Originally designed for made-to-measure automation, Heru pivoted during COVID-19 into a recommendation engine that processes all frame/size combinations in under 2 seconds.

The system was completely rebuilt in 2024-2025 for professional optician use, hitting 75% higher accuracy for in-store bespoke consultations.

The Universal Fitting Challenge

Getting perfectly fitting eyewear needs precise facial measurements and expert frame positioning knowledge. Traditional approaches require:

• In-person consultations with trained opticians for fit assessment
• Manual measurements of nose bridge, temple width, facial angles (when needed)
• Trial-and-error fitting across limited in-store inventory
• Additional weeks of production for made-to-measure adjustments (when needed)

The technical requirements were tough:

• Sub-millimeter precision for optical measurements (0.5-1mm)
• Near real-time processing across full catalog (<2 seconds)
• Automatic positioning with optimal fit and tilt per face
• Dual-purpose system for both ready-to-wear selection and bespoke customization

Automated Facial Analysis & Frame Matching System

I designed Heru as a hybrid iOS/WebGL platform, developing the collision detection and recommendation algorithms for both ready-to-wear selection and bespoke fitting. Built on Tartare , the 3D configurator framework I made in 2017, handling visualization, frame manipulation, and model management.

Herbrand is recommended based on your face shape and frame style

Dual-Purpose Workflow

For Ready-to-Wear Recommendations:

1. 3D Facial Scan - Bellus3D SDK captures facial geometry
2. Facial Analysis - Machine learning to improve and extend original landmarks
3. Catalog Simulation - Custom collision engine tests all frame/size combinations
4. Ranked Recommendations - Best-fitting ready-to-wear options sorted by fit quality
5. In-Store or Online Purchase - Direct ordering of existing inventory frames

For Made-to-Measure Orders:

1-3. (Same scanning and analysis process)

4. Bespoke Adjustments - Auto-calculate custom modifications (temple width, bridge width, tilt, etc)
5. Manufacturing Export - Production-ready specs for hand-crafted frames

Technical Architecture

Hybrid iOS + WebGL Stack

Frontend:

• iOS + Swift for native app shell, camera integration, UI
• Three.js Engine embedded via WebView for 3D visualization
• Tartare Framework handling real-time frame manipulation and configurator

Backend:

• Node.js + Express for API
• MongoDB storing scan data, frame catalog
• Recommendation Engine: built from scratch in JavaScript/Three.js
• AWS cloud infrastructure and storage

Key Technical Challenge: Real-Time Collision Detection at Scale

The Problem

The recommendation engine needed to:

• Test all combinations = 60+ frame models × 1 to 5 sizes
• Simulate positioning on nose, ears, temples, cheeks, forehead for each
• Calculate optimal frame angle/tilt per unique facial geometry
• Calculate optical measurements after positioning (lens height, back vertex distance, etc)
• Rank results by fit quality for ready-to-wear recommendations

Standard collision detection libraries were too slow.

My Solution: Simplified Geometry with Cached Pre-Computation

I built a pragmatic collision system optimized for speed:

Geometric Simplification:

• Reduced nose and frame collision zones to essential triangular layers
• Focused on critical contact points (bridge, temples, ears)
• Cut unnecessary geometry from calculations

Fast Ray-Casting:

• Line-triangle intersection tests against simplified collision meshes
• Sequential testing with early exit for obviously bad fits
• Common sizes tested first to optimize average case

Cached Pre-Computation:

• All frame geometries loaded once and reused
• Pre-calculated collision inputs for every model
• No redundant file parsing or mesh processing

Results:

• Hit <2 second processing time for complete catalog analysis
• Maintained 0.5-1mm accuracy for optical measurements

I shipped fast and practical instead of chasing theoretical perfection.

Evolution: MVP to Production Platform

The Heru workflow

2019: Initial MVP

Built first working prototype as solo engineer:

• Basic iOS app with facial scanning
• Manual frame adjustment interface
• Limited catalog (few models)
• Core features: scan → select frame → adjust position/size/color → customize engraving
• Focus: Made-to-measure automation only

2020: COVID-19 Pivot to Comprehensive Recommendations

Lockdown turned Heru from a bespoke-only tool into a dual-purpose recommendation platform:

• Automated recommendation engine analyzing entire ready-to-wear catalog, ranking by fit quality
• Ready-to-wear prioritization helping customers find perfectly fitting frames from existing stock
• Made-to-measure automation (kept) generating adjustment specs for custom orders
• E-commerce integration enabling direct purchase of both ready-to-wear and bespoke frames
• Public beta development validated with over 2,000 scans across use cases

2021: Public Launch

• July beta launch with invite-only testing across 32 frame styles
• September public release integrating full ready-to-wear catalog
• Store deployment rolling out across UK locations for both ready-to-wear consultations and bespoke orders

2022: ML Enhancement & Refinement

• Machine learning landmarks improving facial feature detection accuracy
• Refined recommendation algorithms based on real customer fit data
• Production integration seamless export to manufacturing workflow for bespoke orders
• Global expansion supporting international store openings

Digital-to-Physical Manufacturing

For made-to-measure orders, Heru connects digital precision to artisanal craftsmanship:

1. Customer scan captured via iOS app
2. Recommendation engine selects optimal frame + size (or customer chooses from ready-to-wear options)
3. Automated adjustments calculated (temple width, bridge height, tilt) for bespoke orders
4. Production files exported with technical drawings and specs
5. Frames CNC-cut from acetate
6. Traditional finishing

This hybrid workflow kept Cubitts’ artisanal quality while making both ready-to-wear selection and made-to-measure accessible without in-person consultation.

Patent:

• EP4275092A2 / US20240061278A1: Frame Adjustment System

Working on something similar? Contact me or connect on LinkedIn .