Question about the time synchronization

Hi, Ego-Exo4D team.

I want to get the corresponding information(like trajectory and gaze) about the frame_aligned_videos.

As I understand it, 1 capture folder corresponds to multiple takes. Therefore, I need to find the information about takes in trajectory.csv(or gaze.csv) in the capture folder. And I see that each take in takes.json has timesync_start_idx and timesync_end_idx. So I need to use these two indexes to find the timestamp of the corresponding period in timesync.csv, and then use the timestamp to go to trajectory.csv to get the trajectory information.

  1. Is this the right way to do it?

  2. If this is the right line of thought, I’m asking if the “timesync_start_idx” and “timesync_end_idx” represent the number of rows, or the number of rows in “timesync.csv” minus 1? (I’m not sure what the indexes mean here, due to the existence of the csv file culture table header)

Hey @zgchen333, thanks for the question!

That’s right, a single capture encapsulates 1 or more takes. The timesync start/end idx are 0-indexed pointers to rows in the capture’s timesync.csv.

For example, given the take metadata - you can load the trajectory for that section similar to this:

import json
import pandas as pd
from projectaria_tools.core import mps

def get_aria_start_end_sec(timesync_path, start_idx, end_idx):
    timesync = pd.read_csv(timesync_path)
    aria_rgb_col = next(
        col
        for col in timesync.columns
        if "aria" in col.lower() and "214-1" in col and "capture_timestamp_ns" in col
    )
    return (
        timesync.iloc[start_idx][aria_rgb_col] * 1e-9,
        timesync.iloc[end_idx][aria_rgb_col] * 1e-9,
    )

with open("path/to/metadata/takes.jsonl") as json_file: 
        take_metadata = json.load(json_file)
start_idx, end_idx = take_metadata[0]['timesync_start_idx'], take_metadata[0]['timesync_end_idx']
start_sec, end_sec = get_aria_start_end_sec("path/to/capture/timesync.csv", start_idx, end_idx)

mps_trajectory = mps.read_closed_loop_trajectory(closed_loop_trajectory_filepath)
for pose in mps_trajectory:
        if start_sec and pose.tracking_timestamp.total_seconds() < start_sec:
            continue
        if end_sec and pose.tracking_timestamp.total_seconds() >= end_sec:
            break
        # Get take-aligned poses here
        tracking_timestamp_ns = int(pose.tracking_timestamp.total_seconds()*1e9)
        T_world_device = pose.transform_world_device

We’re working on getting pre-trimmed versions of MPS out as well - stay tuned!

Okay. I got it. Thank you for your help.

Thanks a lot. Could you please tell me where are the intrinsics of the ego views? I can only get the pose from the trajectory.csv

They are in the VRS files.

thanks! However, are the VRS files released? I downloaded the dataset and nowhere to find them.