Tests

Open the tests page from the problem’s Tests card → Manage tests (for non-editors, View tests). The address is /library/problems/<problem>/tests. Excel tasks have no tests – their rules grade them.

Sample and hidden tests

	Sample tests	Hidden tests
Who sees the data	Students, in the statement.	The problem’s editors, on this page. Students see them only in the results of their own judged submissions.
Where the text is changed	In the statement editor.	On this page.
When they run	On Run samples and when any submission is judged.	When any submission is judged.
Default points	0	1

A submission is judged on every test in order: sample tests first, then hidden ones. Its score is the sum of the points of the tests it passed; the maximum is the sum of all tests’ points. The overall verdict is the verdict of the first failed test (or Accepted when every test passed).

The program’s output is compared with the expected output line by line: spaces and tabs at the ends of lines and empty lines are ignored; everything else must match exactly. Choosing or uploading a custom checker isn’t possible yet.

The test list

The top of the page shows the problem’s limits (for example Problem limits: 1 s and 256 MB. A test without its own limit uses these.) and its state: whether you see the published version or a draft with unpublished changes. Open the problem to publish leads to the problem page.

Each test’s row shows its number, name (Sample N or Test N), the sizes of its input and expected output, its points and its own time and memory limits (– means the problem’s limit).

Clicking a test’s name shows the first kilobyte of each file. Show full loads a file up to 256 KB and Download saves it. Only editors see hidden test data; others see sizes and points only.

Adding a hidden test

1. Click Add test.
2. For Input and Expected output, choose the source: Paste the text or File (Choose file…).
3. Leave Convert Windows line endings (CRLF) to LF and remove a byte-order mark ticked, so solutions that read lines see the same text on Linux.
4. Set the Points (0–1000) and, if needed, the test’s own Time limit (0.1–20 s) and Memory limit (16–4096 MB). An empty field uses the problem’s limit.
5. Click Add test.

The test is added after the hidden tests.

Editing tests

• Hidden test: the row’s ⋯ menu → Edit…. Files up to 64 KB can be edited as text; for larger ones choose Keep (the current file stays) or upload a new file.
• Sample test: ⋯ → Points and limits…. Its text is part of the statement and is changed in the statement editor (Edit the statement).
• Order: Move up / Move down in a hidden test’s menu. The order of sample tests is set in the statement.
• Downloads: Download input and Download expected output in the menu.

Several tests at once

1. Tick hidden tests, or Select all.
2. Choose:
   • Points and limits… – for each field choose Leave unchanged, Use the problem’s limit or Set to, then click Apply. Fields you leave unchanged keep each test’s own value.
   • Delete… – the tests are removed from the draft and the rest are renumbered.

Set the points and limits of all sample tests together with Points and limits… in the sample tests section.

Uploading a ZIP

Screenshot to comeThe “Upload tests as a ZIP” preview: the test pairs found, their sizes and notesTeacher account, the tests page of their own problem, a ZIP with 1.in/1.out pairs chosen, “Check ZIP” clickedstaff/library/tests-zip-preview

1. Click Upload ZIP and Choose ZIP….
2. Choose what happens to the current tests:
   • Add after hidden tests;
   • Replace hidden tests – the draft’s hidden tests are removed (the dialog says how many). Sample tests stay.
3. Set line-ending handling and the Points and limits of imported tests (by default 1 point and the problem’s limits).
4. Click Check ZIP. You see how many tests were found and by which names, the total size, each test’s files and notes (CRLF converted or CRLF kept), files without a partner and skipped system files.
5. If everything looks right, click Import N tests; otherwise Back.

Pairs are found by file name within the same folder, for example:

• 1.in and 1.out (also .inp, .input and .ans, .a, .ok, .sol, .res, .output, with or without .txt);
• 01 and 01.a;
• name.in and name.ans;
• input1.txt and output1.txt (also in_01, out_01).

Tests are sorted in natural order (2 before 10). Files without a partner, or whose name fits more than one pair, are not imported and are listed. __MACOSX folders, hidden files, Thumbs.db and desktop.ini are skipped. When the archive is a single folder, its name is left out of test names.

The import is all or nothing: if any file can’t be imported, it is marked and nothing is saved. Fix the files and upload the ZIP again.

Limits

What	Limit
One input or output file	8 MB
One upload (a test’s two files, or a ZIP)	9 MB
A ZIP unpacked	64 MB
Files in a ZIP	1000
Tests in one version	500

Files must be UTF-8 text without binary data. ZIPs that can’t be read – encrypted, ZIP64 or split – are refused: pack them again with standard settings. Upload bigger tests as a ZIP, which is compressed.

How changes reach students

Every test change goes into the problem’s draft. The published version and its submissions stay exactly as they are until you publish (see Drafts and publishing). The tests page shows which version is being prepared and whether students still get the earlier one.

You can check test changes before publishing by testing the unpublished version. Those test runs are deleted when the draft’s tests change again.

If another editor changed the tests or a new version was published meanwhile, the app says The tests changed in the meantime… and reloads the list – try again.