# NLQ Annotation Zero Temporal Windows

**URL:** <https://discuss.ego4d-data.org/t/nlq-annotation-zero-temporal-windows/36>\
**Category:** Ego4D Challenges\
**Tags:** natural-language-queries\
**Created:** [May 19, 2022, 2:14am UTC](https://discuss.ego4d-data.org/t/nlq-annotation-zero-temporal-windows/36 "2022-05-19T02:14:05Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ego4d-info](https://avatars.discourse-cdn.com/v4/letter/e/7993a0/32.png) [@ego4d-info](https://discuss.ego4d-data.org/u/ego4d-info)\
**Post date:** [May 19, 2022, 2:14am UTC](https://discuss.ego4d-data.org/t/nlq-annotation-zero-temporal-windows/36/1 "2022-05-19T02:14:05Z")

</div>

We are aware that a significant number of NLQ annotations have a near-zero query window. (Nearly 10% of samples.) No updates to the annotations will be made prior to the challenge close, though we will update the annotations in the near future. For the test metric, results on such instances should be zero since it’s based on IoU given the 0 sized windows. Therefore, the impact should be to under-report the performance of models uniformly across all participants.

Additional details will be posted here and on the related issue ([A significantly large number of queries of the NLQ task have a response window of 0 seconds. · Issue #14 · EGO4D/episodic-memory · GitHub](https://github.com/EGO4D/episodic-memory/issues/14)) when we do update the annotations.

We apologize for the noise and confusion here.

---

<div class="post-metadata">

**Author:** ![takfate](https://avatars.discourse-cdn.com/v4/letter/t/85f322/32.png) [@takfate](https://discuss.ego4d-data.org/u/takfate)\
**Post date:** [August 8, 2022, 9:17am UTC](https://discuss.ego4d-data.org/t/nlq-annotation-zero-temporal-windows/36/2 "2022-08-08T09:17:33Z")

</div>

Excuse me. Has the NLQ updated the annotations?

---

<div class="post-metadata">

**Author:** ![awestbury](https://avatars.discourse-cdn.com/v4/letter/a/ba8739/32.png) [@awestbury](https://discuss.ego4d-data.org/u/awestbury)\
**Post date:** [August 8, 2022, 10:57pm UTC](https://discuss.ego4d-data.org/t/nlq-annotation-zero-temporal-windows/36/3 "2022-08-08T22:57:33Z")

</div>

Hi, no updates to the annotations will be made prior to the challenge close

---

<div class="post-metadata">

**Author:** ![takfate](https://avatars.discourse-cdn.com/v4/letter/t/85f322/32.png) [@takfate](https://discuss.ego4d-data.org/u/takfate)\
**Post date:** [August 9, 2022, 4:19am UTC](https://discuss.ego4d-data.org/t/nlq-annotation-zero-temporal-windows/36/4 "2022-08-09T04:19:20Z")

</div>

Hello, we have submitted the results for the test set of NLQ.  
But the system seems not to respond to the evaluation results.

---

<div class="post-metadata">

**Author:** ![awestbury](https://avatars.discourse-cdn.com/v4/letter/a/ba8739/32.png) [@awestbury](https://discuss.ego4d-data.org/u/awestbury)\
**Post date:** [August 9, 2022, 10:05pm UTC](https://discuss.ego4d-data.org/t/nlq-annotation-zero-temporal-windows/36/5 "2022-08-09T22:05:15Z")

</div>

Okay @skottur any ideas what might be going on?

---

<div class="post-metadata">

**Author:** ![awestbury](https://avatars.discourse-cdn.com/v4/letter/a/ba8739/32.png) [@awestbury](https://discuss.ego4d-data.org/u/awestbury)\
**Post date:** [August 15, 2022, 10:53pm UTC](https://discuss.ego4d-data.org/t/nlq-annotation-zero-temporal-windows/36/6 "2022-08-15T22:53:19Z")

</div>

Hello, we understand these problems have been addressed, but please let us know if otherwise.