Extrinsic Hand Eye Calibration GUI Application

This application performs extrinsic hand-eye calibration based on 2D camera images and measured camera/target poses. The application requires a file that specifies the configuration of the calibration problem and the observation data to be used in the calibration. The outputs of this calibration are:

• optimized estimate of the transform from the camera mount frame to the camera frame
• optimized estimate of the transform from the target mount frame to the target frame

Running the Application

Run the application by executing the compiled executable from the install directory of the colcon workspace in which it was built:

source <workspace>/install/setup.bash
./<workspace>/install/industrial_calibration/bin/industrial_calibration_extrnisic_hand_eye_calibration_app

Alternatively, use ROS2 to start the application:

source <workspace>/install/setup.bash
ros2 run industrial_calibration industrial_calibration_extrnisic_hand_eye_calibration_app

Observation Definition

The observation data for this application consists of:

• Pose measurements, saved to a YAML file
  • from the camera mount frame to the camera frame, for applications where the camera is moving, or
  • from the target mount frame to the target frame, for applications where the camera is static
• 2D image files in OpenCV-compatible formats (e.g., .png, .jpg, etc.)

The observation data and YAML file can be generated by the calibration data collector node in industrial_calibration_ros. Alternatively, users can collect calibration data manually and generate the observation YAML file by hand.

Pose files should be specified as a position vector in meters and quaternion, in the following format:

x:  0.0  # (m)
y:  0.0  # (m)
z:  0.0  # (m)
qx: 0.0
qy: 0.0
qz: 0.0
qw: 1.0

The observations should be collated into a YAML file (e.g., cal_data.yaml) that lists each corresponding pair of image and pose files. The image and pose files should be specified by their local directory, relative to the location of this file.

data:
  - image: <relative>/<path>/<to>/<image>
    pose:  <relative>/<path>/<to>/<pose>/<file>
  - image: ...
    pose:  ...
  - image: ...
    pose:  ...

Configuration Definition

Additional configuration information about the system also needs to be provided in order to perform the calibration. The configuration can be generated and saved to YAML file from the GUI using the icons and menu options. The format of the YAML configuration file is as follows:

camera_intrinsics

Camera intrinsic parameters for a pinhole model camera, without distortion

camera_mount_to_camera_guess

Initial guess for the transform from the camera mount frame to the calibrated camera frame

target_mount_to_target_guess

Initial guess for the transform from the target mount frame to the calibrated target frame

target_finder

Plugin configuration for target finder used to detect the calibration target in the image observations.

Here is an example configuration for a ModifiedCircleGridTargetFinder:

target_finder:
  type: ModifiedCircleGridTargetFinder
  rows: 10
  cols: 10
  spacing: 0.0254 # (meters between dot centers)
  circle_detector_params:
    minThreshold: 20
    maxThreshold: 220
    nThresholds: 20
 
    minRepeatability: 3
    circleInclusionRadius: 5
    maxRadiusDiff: 5
 
    maxAverageEllipseError: 0.02
 
    filterByColor: true
    circleColor: 0
 
    filterByArea: true
    minArea: 25.0
    maxArea: 5000.0
 
    filterByCircularity: false
    minCircularity: 0.8
    maxCircularity: 99999999.0
 
    filterByInertia: false
    minInertiaRatio: 0.1
    maxInertiaRatio: 99999999.0
 
    filterByConvexity: true
    minConvexity: 0.95
    maxConvexity: 99999999.0

Here is an example configuration for a CharucoGridBoardTargetFinder:

target_finder:
  type: CharucoGridTargetFinder
  rows: 7
  cols: 5
  chessboard_dim: 0.036195
  aruco_marker_dim: 0.018256
  dictionary: 10  # DICT_6X6_250

homography_threshold

In general, we want to ensure that the calibration target features observed in an image matches closely to the known geometry of the target. For planar calibration targets and 2D images, we can do this by computing a homography transform between the known target features and the observed target features. This homography transform projects the known target features onto the observed target features, allowing us to measure the average distance between corresponding features (in pixels). If the average homography error is above a specified threshold value, we want to exclude that observation from the calibration because the observed target match close enough to our known target geometry.

static_camera

Flag indicating whether the camera is static in the calibration

Example Configuration File

Here is an example of a calibration configuration YAML file:

# Camera intrinsics
intrinsics:
  fx: 1352.02747
  fy: 1356.14287
  cx: 789.67065
  cy: 627.2995
 
# Pose guesses
camera_mount_to_camera_guess:
  x:  0.0
  y:  0.0
  z:  0.0
  qx: 0.0
  qy: 0.0
  qz: 0.0
  qw: 1.0
target_mount_to_target_guess:
  x:  0.0
  y:  0.0
  z:  0.0
  qx: 0.0
  qy: 0.0
  qz: 0.0
  qw: 1.0
 
# Target finder
target_finder:
  type: <target finder plugin name>
  <target_finder_params>: ...
 
# Other parameters
homography_threshold: 2.0
static_camera: true