FHO State Change Object Detection Test Annotations Released

Ego4D Challenges
1

We have released the long overdue SCOD test annotations, fho_scod_test_unannotated.json.

Please download via the CLI for challenge submission, e.g. ego4d --output_directory="~/ego4d_data" --datasets annotations -y

See the challenge page for further instructions on submissions: https://eval.ai/web/challenges/challenge-page/1632/overview

And please post any issues either here or via github: Issues · EGO4D/hands-and-objects · GitHub

Thanks for your patience here if you were looking to submit, and we look forward to your entry!

1 Like
2

Hello, I have downloaded the latest annotations for SCOD.
But I find fho_scod_test_unannotated.json is about 480MB.
this file contains about 2000000 frames.
Is this file consistent with to test set in the paper?

3

Yes. To prevent information leakage, additional annotations have been included in the test set. Only the correct annotations from the test set used for the paper will be used to evaluate the final challenge submission.

4

Hello, we have submitted the results for evaluation, but there are some errors.

Traceback (most recent call last):
  File "/code/scripts/workers/submission_worker.py", line 491, in run_submission
    submission_metadata=submission_serializer.data,
  File "/tmp/tmp731rinei/compute/challenge_data/challenge_1632/main.py", line 91, in evaluate
    pred_coco = get_coco_detections(predictions, framesubpath2id)
  File "/tmp/tmp731rinei/compute/challenge_data/challenge_1632/main.py", line 29, in get_coco_detections
    x = box["x"]
TypeError: string indices must be integers

Also, we have a question: can each image of the prediction result only contain one box prediction?
Could you give me some help?
@ego4d-info

5

Hi, every PNR frame could have multiple box predictions. The bbox under "pnr_frame should be a list of bounding box parameters (x, y, width, height, score).

We also update the challenge page, please ensure your submission follows the format there:

6

Hi, we have submitted a new results file. But it has been evaluated for ten hours and has not been evaluated yet.
@qichenfu

7

Hello, we see that your submission was successfully run. Please see the below for more info on next steps

8

Hi, we have submitted a new results file. But it has been evaluating for over 12 hours and it still shows its running. @awestbury @qichenfu