Building Expo Panoramic Stitcher

expo-panoramic-stitcher is an Expo native module (SDK 56+, verified on SDK 57 / RN 0.86) that stitches a set of photos into a 360°/wide panorama, using OpenCV 4.13 on both iOS and Android. Written in Swift + Kotlin against the Expo Modules API, with one small C++ shim per platform. It's MIT, open source, and published to npm as @notchip/expo-panoramic-stitcher. Six releases in, it's the stitching engine behind FieldDojo's Walkthrough feature.

Why I built it

Back in 2024 I shipped a 360° photosphere capture feature for a real-estate app — six phone photos in, a navigable panorama out. The image-processing half meant wiring OpenCV into React Native by hand: a vendored opencv2.framework on iOS, an OpenCV-android-sdk with jniLibs and CMake/JNI on Android, and a simulator arch hack so Apple Silicon would build at all. It worked, but the setup was the kind of thing you do once and dread touching again.

This module is that work done properly and pulled out as something reusable. Same OpenCV stitching, none of the manual SDK plumbing — and the same API surface on both platforms instead of two implementations that quietly drift apart.

What it does

It's on npm, so install is a normal dependency add:

npx expo install @notchip/expo-panoramic-stitcher
npx expo prebuild --clean

Four entry points in the core, one contract on both platforms:

import {
  stitchImagePaths,        // file paths in, JPEG file out — lowest memory
  stitchBase64,            // base64 in, base64 out — same payload both platforms
  stitchIncrementalBase64, // build a panorama one frame at a time
  stitchSweep,             // yaw-tagged photos in, wrap-closed 360° out
  isStitchingAvailable,
} from '@notchip/expo-panoramic-stitcher';
 
const res = await stitchImagePaths(photoPaths, {
  warpMode: 'spherical',
  outputWidth: 4096,
});
// res.path -> absolute path to the JPEG (prefix file:// for <Image>)
// res.usedIndices -> which inputs made it into the composite

warpMode covers spherical (360°), cylindrical, and plane (flat document/scan stitching). autoResize forces a clean equirectangular 2:1 output, and there are knobs for blend strength, match confidence, pano confidence, output width, and JPEG quality — all with sensible defaults so the zero-config call just works. Every result reports usedIndices and usedCount, so a partial panorama (OpenCV silently dropping the photos it couldn't match) is something you can detect and tell the user about instead of shipping a half-room.

Guided capture

Stitching is only half of a panorama feature. The other half is getting the user to take photos OpenCV can actually match, so the package ships a second entry, @notchip/expo-panoramic-stitcher/capture:

import { stitchSweep } from '@notchip/expo-panoramic-stitcher';
import { GuidedSweepCapture } from '@notchip/expo-panoramic-stitcher/capture';
 
<GuidedSweepCapture
  onComplete={async (photos) => {
    const res = await stitchSweep(photos); // photos: { uri, yawDeg }[]
  }}
/>

GuidedSweepCapture is an iOS-Panorama-style full-screen camera: it integrates the gyro to track yaw, projects it onto gravity so tilting the phone doesn't count as turning, and auto-shoots every 15° once the phone has settled. Turn too fast, tilt too far, or background the app and it tells you. Under it is useGuidedSweep, the same state machine with no UI, for anyone who wants their own HUD.

stitchSweep then uses the yaw tags to do what a plain cv::Stitcher can't: it closes the loop on a full 360° by re-appending the first two frames, salvages partial arcs into separate strips when the chain breaks, and reports the yaw ranges no photo covered so the app can say "re-sweep near 200°". It defaults to cylindrical warping (long spherical chains have a habit of diverging in bundle adjustment) and falls back from spherical to cylindrical exactly once if asked for spherical and it fails.

Killing the OpenCV setup tax

The whole point was that nobody installing this module should ever download OpenCV by hand. It took three tries to get there, and the answers ended up different on each platform.

Android — the official SDK, statically linked. My first version pulled org.opencv:opencv from Maven Central and called it from plain Kotlin. It compiled, and it couldn't stitch: the Maven AAR's libopencv_java4.so doesn't include the stitching module at all, and OpenCV ships no Java bindings for it anyway. So a Gradle task now downloads the official opencv-4.13.0-android-sdk.zip once (about 300 MB, SHA-256 verified) into the Gradle user-home cache, and CMake statically links libopencv_stitching.a and its dependency closure into one small JNI shim. Nothing is checked into the repo, there's no jniLibs folder, and every project on the machine shares the one cached download.

iOS — a prebuilt XCFramework, vendored automatically. My first version declared an SPM dependency on the yeatse/opencv-spm package in the podspec. That also compiled, and then every cv:: symbol came up undefined at the final app link under CocoaPods static frameworks, which is the Expo default and mandatory for anything using Firebase. An SPM product attaches to the pod target and never reaches the app's link line. A vendored framework does. So an npm postinstall script fetches the same prebuilt opencv2.xcframework (about 190 MB, SHA-256 verified) into the package's ios/ directory and the podspec vendors it:

s.vendored_frameworks = 'opencv2.xcframework'

The XCFramework ships device and arm64-simulator slices, which is the detail that kills the old EXCLUDED_ARCHS simulator hack. Offline builds point an environment variable at a pre-downloaded zip.

One shim per platform

OpenCV is a C++ library with no Swift or Kotlin API for stitching. So each platform keeps exactly one thin C++ file: PanoramaStitcherShim.mm on iOS and panorama_stitcher_jni.cpp on Android, each around 180 lines. They wrap cv::Stitcher, do file-IO with cv::imread/cv::imwrite, and hand back a plain result. No UIKit, no second bridge layer. The two shims contain the same stitch core and get edited together. Swift reaches its shim through a normal ObjC header; the .mm extension makes it compile as ObjC++ without any interop flag. (An earlier release set SWIFT_OBJC_INTEROP_MODE = objcxx in the podspec. That broke import ExpoModulesCore against SDK 57's precompiled binaries, and it turned out to be unnecessary.)

Worth saying: switching to Nitro Modules wouldn't remove either shim. OpenCV is still C++ underneath; something has to cross that boundary.

Symmetric by design

The trick that keeps the two platforms honest: native code only ever works on image file paths. stitchImagePaths is the core; stitchBase64 and the incremental variant just decode base64 to a temp file, run the same path-based stitch, and re-encode — base64 handling lives in Swift/Kotlin, the C++/OpenCV surface stays tiny. stitchSweep is plain TypeScript over stitchImagePaths. Both platforms return the exact same result shapes, so the JS layer never branches on Platform.OS.

JS / TS  (index.ts — defaults, validation, stitchSweep orchestration)
   │  requireNativeModule('ExpoPanoramicStitcher')
   ├── iOS:     Swift module  → PanoramaStitcherShim.mm   → cv::Stitcher   (vendored opencv2.xcframework)
   └── Android: Kotlin module → panorama_stitcher_jni.cpp → cv::Stitcher   (OpenCV Android SDK, static libs)

Honest caveats

• OpenCV stitching needs roughly 30–40% overlap between adjacent shots, or it returns a non-OK status — which the module surfaces as a rejected promise with a distinct message per status (ERR_NEED_MORE_IMGS, ERR_HOMOGRAPHY_EST_FAIL, ERR_CAMERA_PARAMS_ADJUST_FAIL), identical on both platforms, rather than a silent black image.
• Requirements: iOS 16.4+, Android minSdk 24, and the iOS postinstall download only runs on macOS.
• Inputs need to be JPEG or PNG. HEIC straight off an iPhone camera isn't supported; convert first.
• Web is a deliberate stub — isStitchingAvailable() returns false there.

Where it's at

Six releases since June, and every one of them came out of a real shoot. Partial-panorama detection exists because a room came back missing three walls. panoConfidence is exposed because the default silently threw away weakly-matched frames. The guided capture UI, wrap closure and arc salvage exist because hand-held 24-shot sweeps are what people actually do with this. It's the stitching engine inside FieldDojo's Walkthrough feature, and it's on GitHub at notchip/expo-panoramic-stitcher — MIT, with the full install and options reference in the README. If you've fought OpenCV into React Native before, this is the setup I wish I'd had, and bug reports from real shoots are still exactly what I'm after.