Table of Contents
System Architecture
The face recognition door lock with ESP32-CAM and OpenCV combines three components: the ESP32-CAM streams MJPEG video over local WiFi, a Raspberry Pi running Python OpenCV identifies authorised faces, and an Arduino controls the door lock relay. This separation keeps the AI computation on the Pi for maximum accuracy while using the ESP32-CAM for low-cost image capture (Rs 350-600 vs Rs 5,000+ for a standalone IP camera).
Pipeline: PIR detects approach -> ESP32-CAM starts streaming -> Pi grabs frames -> OpenCV LBPH recogniser matches face -> if confident, sends serial command to Arduino -> solenoid lock opens for 5 seconds.
Hardware
- ESP32-CAM module (OV2640, Rs 350-600) + FTDI programmer (Rs 150-300)
- Raspberry Pi 4 or 5 (4GB)
- Arduino Nano (door lock relay control)
- 12V solenoid lock + 5V relay module
- 850nm IR illuminator (Rs 300-600) for night operation
- HC-SR501 PIR sensor for wake-on-approach
- 5V/3A supply for Pi + ESP32-CAM
ESP32-CAM Firmware
Flash the ESP32-CAM with MJPEG streaming firmware using Arduino IDE with ESP32 board package:
#include "esp_camera.h"
#include <WiFi.h>
#include "esp_http_server.h"
const char* ssid = "YourWiFi";
const char* password = "YourPassword";
// AI-Thinker ESP32-CAM pin assignments
#define PWDN_GPIO_NUM 32
#define RESET_GPIO_NUM -1
#define XCLK_GPIO_NUM 0
#define SIOD_GPIO_NUM 26
#define SIOC_GPIO_NUM 27
#define Y9_GPIO_NUM 35
#define Y8_GPIO_NUM 34
#define Y7_GPIO_NUM 39
#define Y6_GPIO_NUM 36
// (complete Y-pin mapping in AI-Thinker datasheet)
void setup() {
// Configure camera
camera_config_t config;
config.pixel_format = PIXFORMAT_JPEG;
config.frame_size = FRAMESIZE_VGA; // 640x480
config.jpeg_quality = 12;
config.fb_count = 2;
esp_camera_init(&config);
// Connect WiFi
WiFi.begin(ssid, password);
while(WiFi.status() != WL_CONNECTED) delay(500);
Serial.println(WiFi.localIP());
// Start HTTP stream server on /stream
}OpenCV Recognition on Raspberry Pi
import cv2, numpy as np, urllib.request, serial, time
# Load models
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
recognizer = cv2.face.LBPHFaceRecognizer_create()
recognizer.read('/home/pi/face_model.yml')
label_names = {0: 'Ravi', 1: 'Priya', 2: 'Guest'}
# Serial to Arduino for lock control
arduino = serial.Serial('/dev/ttyUSB0', 9600)
stream = urllib.request.urlopen('http://192.168.1.150/stream')
data = b''
while True:
data += stream.read(1024)
a = data.find(b'xffxd8') # JPEG start marker
b = data.find(b'xffxd9') # JPEG end marker
if a != -1 and b != -1:
jpg = data[a:b+2]
data = data[b+2:]
frame = cv2.imdecode(np.frombuffer(jpg, np.uint8), 1)
if frame is None: continue
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.3, 5)
for (x,y,w,h) in faces:
face_roi = cv2.resize(gray[y:y+h, x:x+w], (200,200))
label, confidence = recognizer.predict(face_roi)
if confidence < 65: # LBPH: lower = better match
name = label_names.get(label, 'Unknown')
arduino.write(b'UNLOCKn')
print(f'Access granted: {name} ({confidence:.0f})')
else:
print(f'Access denied: confidence {confidence:.0f}')Indian Lighting Tips
- Harsh backlight: South-facing doors get intense noon sun. Add a canopy above the camera or use HDR mode in ESP32-CAM settings.
- Night operation: 850nm IR illuminator (invisible to humans) provides excellent face illumination for OV2640 without IR-cut filter.
- Power cuts: 10,000 mAh power bank on ESP32-CAM + Pi supply gives 2-4 hours of backup during typical Indian power outages.
- Training data: Enrol 30+ training images per person in the actual door lighting conditions for best accuracy.
Frequently Asked Questions
How accurate is OpenCV LBPH face recognition?
LBPH achieves 90-95% accuracy with 30+ training images per person under good lighting. For better accuracy, use the Python face_recognition library (dlib-based, 99%+) or DeepFace. The tradeoff is computational cost – LBPH runs at 10+ FPS on Pi 5; face_recognition runs at 3-5 FPS.
Does it work with glasses or masks?
Enrol training images both with and without glasses for reliable recognition in both states. For masks (post-pandemic), train specifically with face-mask photos or use eye-region-only recognition. DeepFace with ResNet handles partial occlusion far better than LBPH.
What is the door response time?
Pi 5 processes a 640×480 frame in 100-200ms. Adding stream latency and serial communication, typical door unlock response is 1-2 seconds from face appearing in frame – acceptable for most applications. Reduce resolution to 320×240 for sub-second response if needed.
Is face recognition legal for home access control in India?
Face recognition for private property access control is currently legal in India with consent of the recognised persons. Get written consent from family members and employees. The DPDP Act (Digital Personal Data Protection) may require data handling documentation for non-personal household use applications.
How many faces can the system recognise?
LBPH reliably handles 50-100 enrolled faces with good accuracy. For larger deployments, use face_recognition library which maintains accuracy better at scale. Store all face data locally on the Pi – never upload biometric data to cloud services without explicit consent and data protection measures.
Add comment