EgoExo4D: Different calibration parameters in aria01.vrs and aria01_noimagestream.vrs

1

Dear all,

thanks for your amazing work and effort on creating this dataset! I have a questions regarding the calibration.

What is the difference between ‘aria01.vrs’ and ‘aria01_noimagestream.vrs’ calibration files? They have different principal point and focal length, but to my understanding the noimagestream-file should be the same but without image data.

I only get the correct undistorted aria rgb images with aria01.vrs, but would like to avoid downloading all vrs files with images if that is not necessary.

Best,
lea

Here’s an example for take cmu_bike01_2:

# aria01.vrs
CameraCalibration(label: camera-rgb, model name: Fisheye624, principal point: [705.015, 704.749], focal length: [610.22, 610.22], projection params: [610.22, 705.015, 704.749, 0.390335, -0.362834, -0.239278, 1.72969, -2.09687, 0.749958, -0.000561914, -0.000926931, 0.00201643, -0.000420764, 0.000238061, -0.000507036], image size (w,h): [1408, 1408], T_Device_Camera:(translation:[-0.00438125, -0.0123959, -0.00536101], quaternion(x,y,z,w):[0.3293, 0.0417021, 0.0362213, 0.942608]), serialNumber:0450577b730305354401100000000000)

# aria01_noimagestream.vrs:
CameraCalibration(label: camera-rgb, model name: Fisheye624, principal point: [1442.53, 1442], focal length: [1220.44, 1220.44], projection params: [1220.44, 1442.53, 1442, 0.390335, -0.362834, -0.239278, 1.72969, -2.09687, 0.749958, -0.000561914, -0.000926931, 0.00201643, -0.000420764, 0.000238061, -0.000507036], image size (w,h): [2880, 2880], T_Device_Camera:(translation:[-0.00438125, -0.0123959, -0.00536101], quaternion(x,y,z,w):[0.3293, 0.0417021, 0.0362213, 0.942608]), serialNumber:0450577b730305354401100000000000)

# for reference: online_calib[0]['CameraCalibrations'][2]
{'Calibrated': True, 'Projection': {'Params': [1220.439291944814, 1442.529221859291, 1441.997151419051, 0.3903350032156671, -0.3628338118352796, -0.2392777738885203, 1.72968866010095, -2.096873915251233, 0.7499578205635813, -0.0005619143992431268, -0.0009269308782712309, 0.002016426267951603, -0.0004207638630777981, 0.0002380609461884488, -0.0005070359350339245], 'Description': 'see FisheyeRadTanThinPrism.h', 'Name': 'FisheyeRadTanThinPrism'}, 'T_Device_Camera': {'Translation': [-0.004381252520929506, -0.01239586212971901, -0.005361006094486243], 'UnitQuaternion': [0.9426082662605363, [0.3293001730031777, 0.04170212552646007, 0.03622133575613418]]}, 'SerialNumber': '0450577b730305354401100000000000', 'Label': 'camera-rgb'}
2

Hi all,

pushing this up since I am still facing the same problem even with v2 vrs files. It’s not clear why the calibration parameters in aria01.vrs and aria01_noimagestreams.vrs are not the same? My understanding is that the only difference is the image steam data being included or not included.

Yet, e.g. for cmu_bike18_6 the vrs file contains principal point: [713.877, 710.952], focal length: [611.282, 611.282] for aria01 and principal point: [1460.25, 1454.4], focal length: [1222.56, 1222.56] for aria01_noimagestreams.

Here’s how I get the calibration parameters:

import os 
from projectaria_tools.core import data_provider

for take_name in ["cmu_bike01_2", "cmu_soccer12_2", "cmu_bike18_6"]:
    aria_dir = os.path.join(release_dir, f"takes/{take_name}")
    for vrs_version in [f"aria01.vrs", f"aria01_noimagestreams.vrs"]:
        aria_path = os.path.join(aria_dir, vrs_version)
        provider = data_provider.create_vrs_data_provider(aria_path)
        calib = provider.get_device_calibration().get_camera_calib("camera-rgb")
        print(take_name, calib)