TTM submission failed

Ego4D Challenges
1

Hello,

Thank you for holding this challenge again this year! I just try my
first submission on the TTM challenge and I got the following error

image
image864×135 6.79 KB

It seems that my value for the “label” field is illegal. However, I can’t
tell from the submission guideline what a acceptable file should
look like. Here’s what I submitted (posting my exact answer may
violate rules so I censored the video id)

{
    "version": "1.0",
    "cahllenge": "ego4d_talking_to_me",
    "results": [
        {
            "video_id": "xxxxxxxxxxxxxxxxxxxx",  // folder name in test data (a str)
            "frame_id": 0,                       // file name of a image in test data (a int)
            "label": 0,                          // 1 for TTM; 0 for not TTM (a int)
            "score": 0.3                         // a floating point number between 0 and 1
        },
        ...
        ]
}

I’ve checked the total number of prediction is the same as the total
number images in the test data, and there is no repeated frame_id
for the same video_id. More detailed submission guideline or a
sample submission file may be helpful. Thank you!

2

challenge is mispelled there.

For the ValueError, that looks to be an error with the evaluation script. (It’s comparing a list to a value!) We’ll check with the challenge POC there and will come back shortly with an update.

3

The actual error is here:

raise ValueError(“Predictions CSV must contain only SPEAKING_AUDIBLE label.”)

Please confirm on your label value submissions.

4

Hello, please fill all the “label” field with 1 in your submission, and the “score” field should indicate the probability for predicting the frame as label=1.

5

Yeah, I submitted successfully yesterday by doing this (I found from the inference code in their repo). Thank you so much!