Calibration Primer

Table of Contents

• The General Idea
• Terminology
• The Camera
• The Target
• Types of calibrations
• Analysis
• Some Advice

This document discusses the "key concepts" and "terminology" that is commonly thrown around. We'll define some of these here so the documentation you find elsewhere in this repository makes sense.

The General Idea

Calibrating a camera in your workspace typically happens in two steps:

1. Calibrate the "intrinsics" (i.e., the camera sensor and lens)
2. Armed with the intrinsics, calibrate the "extrinsics" (i.e., the pose of the camera in your workcell).

In general, calibration routines compute the transform between the camera and the robot wrist or base, respectively. They do so by analyzing images of calibration target: a patterned set of features with known dimensions and spacing. This target is either statically fixed in your workcell if the camera moves (e.g., via attachment to a robot), or it is attached to a moving object (e.g., robot wrist) if the camera is statically mounted. The calibration routines start with a guess about where the camera and calibration target are. They then try to refine that guess through optimization into a more accurate estimation of the true position of the camera and target.

Terminology

• Extrinsic Parameters: "the extrinsic parameters define the position of the camera center and the camera's heading in world coordinates" [ref]. An extrinsic calibration thus tries to find WHERE your camera is relative to some frame of reference, usually the base of a robot or the wrist of a robot.
• Intrinsic Parameters: When talking about cameras, these parameters define how points in 3D space are projected into a camera image. They encompass internal properties of the camera sensor and lens such as focal length, image sensor format, and principal point. An intrinsic calibration tries to solve these
• Rectified Image: Real world cameras and their lenses are NOT perfectly described by the pinhole model of projection. The deviations from that model, called distortions, are estimated as part of intrinsic calibration and are used in software to produce an "undistorted" image called the rectified image. Most of the calibrations in this package assume they are operating on such a rectified image. The ROS image_proc package will usually do this for you.

The Camera

For 2D images, we use the OpenCV model. See OpenCV's discussion at the top of their calib3d module. In brief:

• +Z looks "down the barrel" of the camera lens
• +Y looks down the vertical axis
• +X looks from left to right along the horizontal axis.

When talking about pixel space, the origin of your image is the top left corner. The +Y axis runs down the height, and the +X axis runs right along the width. Most of the calibrations in this repository assume they are working on an undistored or rectified image.

The Target

The core calibrations don't make assumptions about the target; instead you just provide correspondences between sensor features (e.g., in 2D) to target features (e.g., in 3D) given a measurement from the sensor (e.g,. a 2D image). The Target class provides the definition of the calibration target interface. The TargetFinder class provides the definition of the interface for finding such calibration targets.

A number of OpenCV-based target finders are provided. Each target type has its advantages and disadvantages; see the documentation for more details.

Types of calibrations

This repository provides several calibration optimizations for the following types of problems:

• Camera Intrinsic Calibration
• Extrinsic Hand-eye Calibration
• Kinematic Calibration
• Static Multi-camera Extrinsic Calibration
• PnP Calibration
• Multi-camera PnP Calibration

See the corresponding documentation to learn more about how each optimization works.

Analysis

Just because a calibration converges does not mean it is accurate. Just because a calibration converged to a low final cost does not mean it's a good calibration. If you take 3000 images from the exact same position, you'll get good convergence, a very low score, and really crappy calibration.

This repository includes a number of tools in the analysis module for helping determine whether or not a calibration is good. See the documentation in this section for more details on the available analysis tools.

Some Advice

• Take lots of samples in lots of different positions. It's not uncommon to require tens of images from all over your workspace to get a good calibration.