KDD Cup 2020 AutoGraph Challenge Forum

Go back to competition Back to thread list Post in this thread

> Evaluation error

I wrote a simple program, which can be evaluated successfully in local run_local_test.py, but when I package and submit model.py, the score is all -1. It seems that online code can't read the prediction file when I check the log. Can anyone help me?

This is the content of log.
```
2020-03-31 03:05:55,091 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_2
2020-03-31 03:05:55,091 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_2 does not exist. Default values will be used.
2020-03-31 03:05:55,092 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_3
2020-03-31 03:05:55,092 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_3 does not exist. Default values will be used.
2020-03-31 03:05:55,092 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_4
2020-03-31 03:05:55,092 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_4 does not exist. Default values will be used.
2020-03-31 03:05:55,092 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_5
2020-03-31 03:05:55,092 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_5 does not exist. Default values will be used.
2020-03-31 03:05:55,093 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_6
2020-03-31 03:05:55,093 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_6 does not exist. Default values will be used.
2020-03-31 03:05:55,093 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_7
2020-03-31 03:05:55,093 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_7 does not exist. Default values will be used.
2020-03-31 03:05:55,093 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_8
2020-03-31 03:05:55,093 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_8 does not exist. Default values will be used.
2020-03-31 03:05:55,093 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_9
2020-03-31 03:05:55,093 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_9 does not exist. Default values will be used.
2020-03-31 03:05:55,094 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_10
2020-03-31 03:05:55,094 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_10 does not exist. Default values will be used.
2020-03-31 03:05:55,094 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_11
2020-03-31 03:05:55,094 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_11 does not exist. Default values will be used.
2020-03-31 03:05:55,094 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_12
2020-03-31 03:05:55,094 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_12 does not exist. Default values will be used.
2020-03-31 03:05:55,094 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_13
2020-03-31 03:05:55,094 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_13 does not exist. Default values will be used.
2020-03-31 03:05:55,095 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_14
2020-03-31 03:05:55,095 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_14 does not exist. Default values will be used.
2020-03-31 03:05:55,095 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_15
2020-03-31 03:05:55,095 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_15 does not exist. Default values will be used.
2020-03-31 03:05:55,095 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_16
2020-03-31 03:05:55,095 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_16 does not exist. Default values will be used.
2020-03-31 03:05:55,095 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_17
2020-03-31 03:05:55,095 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_17 does not exist. Default values will be used.
2020-03-31 03:05:55,096 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_18
2020-03-31 03:05:55,096 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_18 does not exist. Default values will be used.
2020-03-31 03:05:55,096 INFO evaluate.py: Checking /tmp/codalab/tmpSkeq_b/run/input/res_19
2020-03-31 03:05:55,096 WARNING evaluate.py: /tmp/codalab/tmpSkeq_b/run/input/res_19 does not exist. Default values will be used.
2020-03-31 03:05:55,096 INFO evaluate.py: [+] Results validation done.
2020-03-31 03:05:55,103 INFO evaluate.py: [+] Score reading done.
2020-03-31 03:05:55,103 INFO evaluate.py: Score list: [-1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0]
2020-03-31 03:05:55,103 INFO evaluate.py: [+] Score writing done.
2020-03-31 03:05:55,103 INFO evaluate.py: [+] Learning curve writing done.
2020-03-31 03:05:55,103 INFO evaluate.py: [+] Parent scoring program finished!
```

Posted by: LogicJake @ March 31, 2020, 4:03 a.m.

Hi,

Thanks for the feedback! We checked your submission, and the problem is that this submission runs out of time. You can find this logging information by:

click "My Submissions" - "Dataset 1" - “+"button at the end of one submission - "Error log".

Best regards,
KDD Cup 2020 AutoGraph Organizing Team

Posted by: automlai @ March 31, 2020, 7:04 a.m.

Thank you for your reponsonding.

Posted by: LogicJake @ April 2, 2020, 6:32 a.m.
Post in this thread