360-Degree Camera Stitching Project with OpenCV and Pi

Creating a 360-degree camera using OpenCV image stitching with Raspberry Pi is an ambitious computer vision project that combines multiple cameras into a seamless panoramic view. Whether you want to build a simple 180-degree panorama for home monitoring or a full equirectangular 360-degree image for virtual tours, OpenCV’s Stitcher API and custom homography techniques provide the tools. This tutorial covers camera array setup, calibration, feature matching, and generating stitched 360-degree outputs.

Table of Contents

• How Image Stitching Works
• Camera Array Setup
• Camera Calibration
• OpenCV Stitcher API
• Custom Homography Stitching
• Real-Time Panorama
• 360-Degree Output Formats
• FAQ

How Image Stitching Works

Image stitching creates a panoramic image from overlapping photos through these steps:

1. Feature detection: SIFT, ORB, or AKAZE find distinctive keypoints in each image
2. Feature matching: Match corresponding keypoints between overlapping images
3. Homography estimation: RANSAC computes the 3×3 transformation matrix between image pairs
4. Warping: Images are projected onto a common surface (cylindrical or spherical)
5. Blending: Multi-band blending smooths seams at image boundaries

For 360-degree capture, a minimum of 4-6 cameras with 25-30% overlap is required. With Raspberry Pi CSI having one port (or two on Pi 5), USB cameras or a camera multiplexer are needed for multi-camera setups.

Camera Array Setup

Camera Calibration

Calibrate each camera to find its intrinsic matrix (focal length, principal point, distortion coefficients). Print a chessboard calibration pattern and capture 20+ images from different angles:

import cv2
import numpy as np
import glob

CHESSBOARD = (9, 6)  # Number of inner corners
CRITERIA = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)

objp = np.zeros((CHESSBOARD[0]*CHESSBOARD[1], 3), np.float32)
objp[:, :2] = np.mgrid[0:CHESSBOARD[0], 0:CHESSBOARD[1]].T.reshape(-1, 2)

objpoints = []  # 3D points in world space
imgpoints = []  # 2D points in image space

images = glob.glob('calibration/*.jpg')
for fname in images:
    img = cv2.imread(fname)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    ret, corners = cv2.findChessboardCorners(gray, CHESSBOARD, None)
    if ret:
        objpoints.append(objp)
        corners2 = cv2.cornerSubPix(gray, corners, (11,11), (-1,-1), CRITERIA)
        imgpoints.append(corners2)

ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None)

# Save calibration
np.save('camera_matrix.npy', mtx)
np.save('dist_coeffs.npy', dist)
print(f'Calibration RMS error: {ret:.3f} (should be < 1.0)')

OpenCV Stitcher API

import cv2
import numpy as np

stitcher = cv2.Stitcher.create(cv2.Stitcher_PANORAMA)

# Load images (already distortion-corrected)
images = [cv2.imread(f'frame_{i}.jpg') for i in range(4)]

status, panorama = stitcher.stitch(images)

if status == cv2.Stitcher_OK:
    print('Stitching successful!')
    cv2.imwrite('panorama.jpg', panorama)
    cv2.imshow('Panorama', panorama)
    cv2.waitKey(0)
else:
    print(f'Stitching failed: {status}')
    # status codes: 0=OK, 1=ERR_NEED_MORE_IMGS, 2=ERR_HOMOGRAPHY_EST_FAIL, 3=ERR_CAMERA_PARAMS_ADJUST_FAIL

Custom Homography Stitching

def stitch_two_images(img1, img2):
    # Convert to grayscale for feature detection
    g1 = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
    g2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
    # ORB feature detection (free, no patent issues)
    orb = cv2.ORB_create(nfeatures=2000)
    kp1, des1 = orb.detectAndCompute(g1, None)
    kp2, des2 = orb.detectAndCompute(g2, None)
    # Match features
    bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
    matches = bf.match(des1, des2)
    matches = sorted(matches, key=lambda x: x.distance)[:200]
    # Extract matched point coordinates
    pts1 = np.float32([kp1[m.queryIdx].pt for m in matches])
    pts2 = np.float32([kp2[m.trainIdx].pt for m in matches])
    # Find homography with RANSAC
    H, mask = cv2.findHomography(pts2, pts1, cv2.RANSAC, 5.0)
    # Warp and blend
    h1, w1 = img1.shape[:2]
    h2, w2 = img2.shape[:2]
    panorama_width = w1 + w2
    warped = cv2.warpPerspective(img2, H, (panorama_width, h1))
    result = warped.copy()
    result[0:h1, 0:w1] = img1  # Place img1 on left
    return result

Real-Time Panorama

For real-time stitching, pre-compute the homography from calibration (cameras are in fixed positions). Then at runtime, only apply the warp transform to each frame:

# Pre-calibration: compute and save H once
# H = compute_homography_from_calibration()
# np.save('H_cam0_to_cam1.npy', H)

# Runtime: load H and apply directly (very fast)
H = np.load('H_cam0_to_cam1.npy')

while True:
    frame1 = cam0.capture_array()
    frame2 = cam1.capture_array()
    warped = cv2.warpPerspective(frame2, H, (frame1.shape[1]*2, frame1.shape[0]))
    warped[0:frame1.shape[0], 0:frame1.shape[1]] = frame1
    cv2.imshow('360 View', warped)

360-Degree Output Formats

Equirectangular projection: Standard format for VR headsets and Google Street View. Use OpenCV’s cylindrical or spherical warp. Convert with cv2.remap() using precomputed lookup tables.

Viewing stitched 360 on Android/iOS: Open the equirectangular JPEG in Google Photos (enable 360 photo mode by adding XMP metadata), or use VR player apps. India-specific tip: Works with JioVR and Samsung VR viewer for cardboard headsets popular in India colleges.

FAQ

How much overlap do images need for successful stitching?

Minimum 25-30% overlap between adjacent cameras. More overlap improves matching accuracy but reduces the unique area covered. For three 120-degree FOV cameras in a 360-degree rig, 15 degrees of overlap on each side provides 60+ degrees of unique coverage per camera.

Why does OpenCV Stitcher fail sometimes?

Common causes: insufficient overlap, low texture scenes (plain walls, sky), very different exposures between cameras, or moving subjects between captures. Fix by adding artificial markers to low-texture areas, synchronise exposure settings, and capture all cameras simultaneously.

Can I do 360-degree stitching in real time on Raspberry Pi?

With pre-computed homography matrices, 2-camera stitching at 640×480 runs at 10-15 FPS on Pi 4. Full 4-camera 360-degree at 1280×720 requires Pi 5 and achieves 5-8 FPS. For real-time VR applications, use dedicated 360-degree cameras instead.

Shop Camera & Vision Modules