Object State Change Classification challenge submission format

Ego4D Challenges
1

Hi, in the example provided in the submission guidelines the boolean values are capitalized, although this is not a valid json boolean type. Could you please clarify this? should I submit with capitalized boolean values although it is not valid json?

Thanks in advance,
Elad.

2

Hello Elad,

Thank you for reaching out to us!
Yes, we encourage you to submit with lowercase boolean (true, false).

All the best for your submissions!

Regards,
Siddhant Bansal

3

Thanks!
My submission to this task achieves really low accuracy (almost random), with more than 20% gap from the validation accuracy. I would really appreciate it you could help me with that. Few more details:

  1. I also submitted to the temporal localization task and achieved reasonable score, so it looks like the data I use is ok.
  2. I use the “fho_oscc-pnr_test_unannotated.json” file for the testing (which is the same as in the localization task).
  3. My submission file contains 28431 entries. Example:
{"unique_id": "88abcdef-6ghi-6789-4321-mnbvcxasfe", "state_change": false}

Thanks in advance,
Elad

4

Hello, we have submitted the results for the test set of OSCC.
But the system seems not to respond to the evaluation results.

5

Hello @takfate thank you for bringing this to our notice. We will have a look and get back to you.

6

Hello, Excuse me, has the problem been solved?

7

Hello @takfate,

Thanks to the EvalAI team, we have fixed the issue and your submission has been evaluated! Please go ahead and check it. Also, feel free to ping us if there is any concern.

We appreciate your patience!

8

@Sid
Hello, Excuse me, It seems that the same problem is occurring with the NLQ task.