The error above was from preprocess step of the training set.
Also for testing (unannotated challenge videos) I was curious whether we are expected to provide predictions for all 30fps frames from each video?
(So basically for preprocessing step of the challenge test set we have to extract all frames inside the video?)
@aram I don’t see this error before. Which version of av are you using? I am using av==10.0.0. Also, have you checked the file path?
Yes, you are absolutely correct! For the test, we are extracting 30FPS frames. But you only need to report tracking results every 6 frames. Please look at tools/eval_datasets/eval_ego4d_lt_tracking.py line 113 (for some reason, I cannot post link). So, we start from the frame number of the visual crop, and track forward and backward every 6 frames.
One reason for this was because of the frame number conversion issue we had (some annotation does not necessarily start from frame #0, but maybe frame #1). Another reason is different trackers might perform differently in different FPSs, so you could explore running on 5FPS video or 30FPS video.
Thank you for your kind explanation!
I still have the error but it seems to progress after skipping the video. I have the same av==10.0.0.
return _ForkingPickler.loads(buf.getbuffer())
Based on the line probably i’m missing a video? Not sure how that would have happened but I’ll update here if I figure out. Thanks!
@haotang Question on test set preprocess step. I meet the error “KeyError: exported_clip_uid” inside extract_clip_ids function. I found that the annotation actually doesn’t include the key “exported_clip_uid”. Should I just skip videos that do not have a ‘exported_clip_uid’ annotation? (Or should i use clip_uid instead?)
There are 5 clip_uid videos with this problem
8ef5815b-5067-46f3-a638-b62182e5356b
49636e60-5de5-4828-bfb0-85273d669fa7
0f628098-807e-48a5-90ec-3f5da7070b85
b021635b-8a08-4d18-84af-fb1ce3104df5
b4dc13ab-e30d-47e1-86ee-9c5b4ec8985c
2c091c01-f038-4fa7-b106-c640f24b4b64
@aram Thanks for letting us know this issue! We have updated the challenge test file to resolve this issue. Could you please download the updated test file?
@haotang Thank you so much for making the fix! Is it sufficient to download only the annotations?
ego4d --output_directory ./ --datasets egotracks --benchmark EM --version v2
Yes, downloading the annotations is sufficient
@haotang I also noticed that there are 4 videos that are missing in the training set. (exists in egotracks annotation but not downloaded) when you download the clips with
ego4d --output_directory ./ --datasets egotracks clips --benchmark EM --version v2
Below are the video file names.
[‘59daca91-5433-48a4-92fc-422b406b551f’, ‘db211359-c259-4515-9d6c-be521711b6d0’, ‘87b52dc5-3ac3-47e7-9648-1b719049732f’, ‘b7fc5f98-e5d5-405d-8561-68cbefa75106’]
Is it possible to use ego4d cli to download only the 4 videos that are missing?
@aram It is taking longer for us to investigate why these clips were not released to public. At the same time, please ignore these four clips in the training set. Do you experience any missing clips for the test set?