Results

What each verdict means, how to read per-test results, and what you see about tests after judging.

The judge compiles your submission and runs it on all of the problem’s tests. You see the result in the editor’s Last result tab, in the course’s Submissions, on the problem page and on My work.

How judging works

Compile. C++ code is compiled; Python code is checked for syntax errors. If this fails, the verdict is “Compilation error” and no tests run.
Tests. Your program runs on each test in order: the sample tests first, then the hidden ones. Usually every test runs, even after one fails.
Verdict. If every test passes, it is “Accepted”. Otherwise the submission’s verdict is the verdict of the first failing test.
Points. Usually each passed test earns one point, so “7 of 10” means 7 of 10 tests passed.

“Accepted” means the problem is solved. The problem’s status and points follow your best submission.

How output is compared

Usually your program’s output is compared with the expected output line by line:

trailing spaces and empty lines don’t matter;
every other line must match exactly;
an extra line is wrong.

Some problems compare only the words (line breaks don’t matter) or numbers within a tolerance.

What the verdicts mean

Verdict	What it means	What to check
Accepted	Every test passed	–
Wrong answer	The program finished, but its output is wrong, or the output file was not created	The algorithm, edge cases, the output format, the file name
Compilation error	The code could not be compiled. Python syntax errors are reported here too	The compiler messages. Doesn’t use an attempt
Runtime error	The program crashed or ended with an exit code other than 0	Array bounds, division by zero, `return 0;`, unhandled Python exceptions
Time limit exceeded	The program used more CPU time than allowed, or waited too long (e.g. for input that doesn’t exist)	The algorithm’s speed; whether the program reads the keyboard when the input is in a file
Memory limit exceeded	The program used more memory than allowed	Array sizes, recursion depth
Output limit exceeded	The program wrote more than 8 MB	Whether output happens in an endless loop
Check failed	The problem’s checker could not check the submission (e.g. it failed or isn’t supported yet)	The attempt doesn’t count. Tell your teacher
System error	Judging failed on our side: “Check failed on our side — this attempt doesn’t count.”	Nothing – the attempt doesn’t count

“Presentation error”, “Idleness limit exceeded” and “Security violation” appear only on old submissions moved from the previous system.

The verdict’s icon repeats its meaning: a check mark for “Accepted”, a cross for a wrong answer, a clock for a time or memory limit, a triangle for a runtime error.

The list of tests

Below the verdict you see the list of tests. Each row has:

the result icon and the test number, e.g. “Test 3”;
“Sample” or “Hidden”;
the verdict of a failing test, e.g. “Wrong answer”;
the run time, e.g. “12 ms”.

“Not run” means the test was not run, e.g. when the problem stops judging at the first failing test. Under Submissions, consecutive passed tests without data are merged into one row, e.g. “Tests 3–6”.

A row that has data can be expanded. It may contain:

Box	What’s in it
Input	The test’s input (in a file problem, the input file’s name)
Expected output	What the program should have printed
Your output	What your program printed (in a file problem, the output file’s contents)
Error output	What the program wrote to the error stream (`cerr`, `stderr`)
Checker message	The checker’s explanation, e.g. which line differs (in English)

Your program’s output is kept for the sample tests and every failing test. Very long data is shortened.

How much you see about tests

Once your submission is judged, you see every one of its tests, hidden ones included: the verdict, time and memory, the input, the expected output, your program’s output (when it was kept), the error output and the checker message. While it is being judged, the editor shows how many tests have run, e.g. “3 of 10”.

Before your submission is judged, hidden tests stay on the server: the problem page and the editor show only the sample tests from the statement. Hidden tests’ data is sent only with your own judged submissions – other students never see your submissions, and you never see theirs.

Some old submissions moved from the previous system have no per-test results. The editor’s Last result tab then says “No test results were recorded for this submission.”

Compiler messages

You always see compiler messages, because they are about your own code:

in the editor – underlined in the code and listed in the Compiler tab (see Code editor);
in a submission’s result – under “Compiler messages” when the submission got “Compilation error”. Some old submissions have no messages kept.