# Question about the time synchronization

**URL:** <https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443>\
**Category:** Q&A\
**Created:** [January 10, 2024, 8:13am UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443 "2024-01-10T08:13:28Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![zgchen333](https://avatars.discourse-cdn.com/v4/letter/z/e68b1a/32.png) [@zgchen333](https://discuss.ego4d-data.org/u/zgchen333)\
**Post date:** [January 10, 2024, 8:13am UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/1 "2024-01-10T08:13:28Z")

</div>

Hi, Ego-Exo4D team.

I want to get the corresponding information(like trajectory and gaze) about the frame\_aligned\_videos.

As I understand it, 1 `capture` folder corresponds to multiple takes. Therefore, I need to find the information about takes in `trajectory.csv`(or `gaze.csv`) in the `capture` folder. And I see that each take in `takes.json` has `timesync_start_idx` and `timesync_end_idx`. So I need to use these two indexes to find the timestamp of the corresponding period in `timesync.csv`, and then use the timestamp to go to `trajectory.csv` to get the trajectory information.

1. Is this the right way to do it?

2. If this is the right line of thought, I’m asking if the “timesync\_start\_idx” and “timesync\_end\_idx” represent the number of rows, or the number of rows in “timesync.csv” minus 1? (I’m not sure what the indexes mean here, due to the existence of the csv file culture table header)

---

<div class="post-metadata">

**Author:** ![dkukreja](https://avatars.discourse-cdn.com/v4/letter/d/71c47a/32.png) [@dkukreja](https://discuss.ego4d-data.org/u/dkukreja)\
**Post date:** [January 11, 2024, 10:44pm UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/2 "2024-01-11T22:44:17Z")

</div>

Hey @zgchen333, thanks for the question!

That’s right, a single capture encapsulates 1 or more takes. The timesync start/end idx are 0-indexed pointers to rows in the capture’s timesync.csv.

For example, given the take metadata - you can load the trajectory for that section similar to this:

```auto
import json
import pandas as pd
from projectaria_tools.core import mps

def get_aria_start_end_sec(timesync_path, start_idx, end_idx):
    timesync = pd.read_csv(timesync_path)
    aria_rgb_col = next(
        col
        for col in timesync.columns
        if "aria" in col.lower() and "214-1" in col and "capture_timestamp_ns" in col
    )
    return (
        timesync.iloc[start_idx][aria_rgb_col] * 1e-9,
        timesync.iloc[end_idx][aria_rgb_col] * 1e-9,
    )

with open("path/to/metadata/takes.jsonl") as json_file: 
        take_metadata = json.load(json_file)
start_idx, end_idx = take_metadata[0]['timesync_start_idx'], take_metadata[0]['timesync_end_idx']
start_sec, end_sec = get_aria_start_end_sec("path/to/capture/timesync.csv", start_idx, end_idx)

mps_trajectory = mps.read_closed_loop_trajectory(closed_loop_trajectory_filepath)
for pose in mps_trajectory:
        if start_sec and pose.tracking_timestamp.total_seconds() < start_sec:
            continue
        if end_sec and pose.tracking_timestamp.total_seconds() >= end_sec:
            break
        # Get take-aligned poses here
        tracking_timestamp_ns = int(pose.tracking_timestamp.total_seconds()*1e9)
        T_world_device = pose.transform_world_device

```

We’re working on getting pre-trimmed versions of MPS out as well - stay tuned!

---

<div class="post-metadata">

**Author:** ![zgchen333](https://avatars.discourse-cdn.com/v4/letter/z/e68b1a/32.png) [@zgchen333](https://discuss.ego4d-data.org/u/zgchen333)\
**Post date:** [January 12, 2024, 7:11am UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/3 "2024-01-12T07:11:31Z")

</div>

Okay. I got it. Thank you for your help.

---

<div class="post-metadata">

**Author:** ![benzmao](https://avatars.discourse-cdn.com/v4/letter/b/eada6e/32.png) [@benzmao](https://discuss.ego4d-data.org/u/benzmao)\
**Post date:** [February 22, 2024, 12:20pm UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/4 "2024-02-22T12:20:35Z")

</div>

Thanks a lot. Could you please tell me where are the intrinsics of the ego views? I can only get the pose from the `trajectory.csv`

---

<div class="post-metadata">

**Author:** ![miguelmartin](https://avatars.discourse-cdn.com/v4/letter/m/f19dbf/32.png) [@miguelmartin](https://discuss.ego4d-data.org/u/miguelmartin)\
**Post date:** [February 28, 2024, 12:55am UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/5 "2024-02-28T00:55:38Z")

</div>

They are in the VRS files.

---

<div class="post-metadata">

**Author:** ![benzmao](https://avatars.discourse-cdn.com/v4/letter/b/eada6e/32.png) [@benzmao](https://discuss.ego4d-data.org/u/benzmao)\
**Post date:** [February 28, 2024, 1:31am UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/6 "2024-02-28T01:31:43Z")

</div>

thanks! However, are the VRS files released? I downloaded the dataset and nowhere to find them.

---

<div class="post-metadata">

**Author:** ![zgchen333](https://avatars.discourse-cdn.com/v4/letter/z/e68b1a/32.png) [@zgchen333](https://discuss.ego4d-data.org/u/zgchen333)\
**Post date:** [March 6, 2024, 8:35am UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/7 "2024-03-06T08:35:03Z")

</div>

@dkukreja @miguelmartin Hey, Thank you for your great work.  
When I want to download the ‘take\_eye\_gaze’ by

```auto
egoexo -o ../data --parts take_eye_gaze

```

And the output is

```auto
ERROR: could not get manifests for all parts (nothing to download)

```

It is worth noting that I can download takes like"egoexo -o …/data --uid 853961be-a9e1-4b0c-81f6-d3e17c190b08 ".  
Therefore, I want to ask if this error resulted from your ongoing updating work.

---

<div class="post-metadata">

**Author:** ![miguelmartin](https://avatars.discourse-cdn.com/v4/letter/m/f19dbf/32.png) [@miguelmartin](https://discuss.ego4d-data.org/u/miguelmartin)\
**Post date:** [March 9, 2024, 3:09am UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/9 "2024-03-09T03:09:27Z")

</div>

> thanks! However, are the VRS files released? I downloaded the dataset and nowhere to find them.

Please use `--parts take_vrs`

---

<div class="post-metadata">

**Author:** ![miguelmartin](https://avatars.discourse-cdn.com/v4/letter/m/f19dbf/32.png) [@miguelmartin](https://discuss.ego4d-data.org/u/miguelmartin)\
**Post date:** [March 9, 2024, 9:59pm UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/10 "2024-03-09T21:59:48Z")

</div>

Thank you for the flag, I have resolved this issue, you can now download with `--parts take_eye_gaze`

You can alternatively use `--parts capture_eye_gaze` and then with the timesync.csv and start/end idx in the `takes.json` file you can correspond to the correct eye gaze region.

---

<div class="post-metadata">

**Author:** ![zgchen333](https://avatars.discourse-cdn.com/v4/letter/z/e68b1a/32.png) [@zgchen333](https://discuss.ego4d-data.org/u/zgchen333)\
**Post date:** [March 17, 2024, 2:08pm UTC](https://discuss.ego4d-data.org/t/question-about-the-time-synchronization/443/12 "2024-03-17T14:08:17Z")

</div>

Hello, Ego-Exo4D team.

I’ve observed transformation parameters and quaternions between the camera and world coordinates within the visualizer. Besides, The ‘gopro\_calibs.csv’ file contains parameters denoted as (tx\_world\_cam, ty\_world\_cam, tz\_world\_cam, qx\_world\_cam, qy\_world\_cam, qz\_world\_cam, qw\_world\_cam).

 ![微信图片_20240317211615](https://canada1.discourse-cdn.com/flex007/uploads/ego4d_data/original/1X/23c53940191751fb26b57b2307d3ac5a678fd31e.jpeg)

 ![image](https://canada1.discourse-cdn.com/flex007/uploads/ego4d_data/original/1X/3bd64a8303d72fa87197f2a39894e439c24d49d9.png)

The uid for this example is 853961be-a9e1-4b0c-81f6-d3e17c190b08, which was utilized for verification purposes.

In my analysis, I anticipated that the parameters in the ‘gopro\_calibs.csv’ file would correspond to the 3D transform observed in the visualizer, assuming they both refer to the same world coordinate system. However, in this example, they differ. Does this indicate that the world coordinate systems are distinct?

Additionally, could you provide sample code demonstrating how to obtain the transformation matrices between GoPro cameras and the Aria device?