Skip to content

Allow more thorough checks of MultiPV lines - #166

Merged
vondele merged 1 commit into
vondele:masterfrom
robertnurnberg:multipv
Jul 25, 2026
Merged

Allow more thorough checks of MultiPV lines#166
vondele merged 1 commit into
vondele:masterfrom
robertnurnberg:multipv

Conversation

@robertnurnberg

Copy link
Copy Markdown
Contributor

This PR introduces the following two functional changes to the matecheck script, together with the associated new files matetrack_multipv.epd and matedtrack_multipv.epd.

  • Remove option --checkMultiPVs and now always check PVs of sidelines for completeness and correctness
  • Add option --multipvFile to allow providing (conjectured optimal) bm values for child positions, so that too short and wrongly signed mate scores can also be caught in MultiPV sidelines

Sample output for the example run python matecheck.py --epdFile matetrack.epd matedtrack.epd --nodes 1000 --bmMin 4 --bmMax 4 --multiPV 4 --multipvFile matetrack_multipv.epd matedtrack_multipv.epd --engine ./sf12:

multipv2: Found mate #-2 (better than #-3) for move d1c3 for FEN "N2k4/N6p/4pP2/3p4/B1p2p2/Kp2R2r/6pq/3nB1b1 b - -"  with bm #-4.

The format of the two new files matetrack_multipv.epd and matedtrack_multipv.epd is explained in their headers:

#####################################################################
# All the unique positions 1 ply from those in matetrack.epd.
# Comment format: "FEN bm #X;" |legalmoves| |moves| |unique|
#   where legalmoves = FEN's legal moves ...
#         moves      = those not leading to (check|stale)mate ...
#         unique     = those leading to yet unseen positions.
#####################################################################

Clearly, these two files will evolve over time, as we find more and faster mates for the positions they store. But for standard invocations of the matecheck script, their current state should already suffice.

@robertnurnberg

Copy link
Copy Markdown
Contributor Author

Example run for SF12:

> python matecheck.py --epdFile matetrack.epd matedtrack.epd --nodes 1000 --bmMin 4 --bmMax 4 --multiPV 4 --multipvFile matetrack_multipv.epd matedtrack_multipv.epd --engine ./sf12
Loaded 253 FENs with 253 bm values, with |bm| (min avg max): 4 4 4.
Loaded 231451 possible children FENs with 132655 bm values for MultiPV checks.

Matetrack started for ./sf12 on matetrack.epd matedtrack.epd with --bmMin 4 --bmMax 4 --nodes 1000 --multiPV 4 ...
100%|███████████████████████████████████████████| 51/51 [00:01<00:00, 39.58it/s]

multipv4: Found mate #-1 with PV status "short" for FEN "1bqQ4/1r6/2P1r2B/1P2P3/1p5p/3PpP2/4P1pp/R3K1kb b Q -"  with bm #-4.
PV: b4b3
Found mate #-4 with PV status "short" for FEN "5k2/4pP2/4p3/4p3/4P3/5Rp1/6p1/6Kb b - -"  with bm #-4.
PV: f8g7 f7f8q g7h7 f3a3 h7g6 a3g3 g6h5
multipv4: Found mate #-1 with PV status "short" for FEN "7R/2P5/P7/8/3Bp3/K3R1p1/1P1p2p1/5krb b - -"  with bm #-4.
PV: d2d1r
multipv3: Found mate #-1 with PV status "short" for FEN "8/4B3/8/6R1/7k/8/5rPK/6N1 b - -"  with bm #-4.
PV: f2f1
multipv3: Found mate #-1 with PV status "short" for FEN "4B2n/1r2Rr2/p1P2Pp1/2p1N3/3P1P1b/1pR2p2/1P2kPp1/2K2bB1 b - -"  with bm #-4.
PV: c5c4
multipv3: Found mate #-1 with PV status "short" for FEN "8/3N2R1/p6p/2p1p1Bk/2b1K1p1/3p1prr/2n5/R5n1 b - -"  with bm #-4.
PV: d3d2
multipv3: Found mate #-1 with PV status "short" for FEN "3R4/K1k5/p2R4/r7/5N2/3p4/r6n/1N3B2 b - -"  with bm #-4.
PV: d3d2
Found mate #3 (better) for FEN "2b5/2N2p2/6p1/4N3/1B1k1p1r/Kp6/n7/4Q3 w - -"  with bm #4.
Found mate #3 with PV status "short" for FEN "2b5/2N2p2/6p1/4N3/1B1k1p1r/Kp6/n7/4Q3 w - -"  with bm #4.
PV: e1d2 d4e4 d2d3 e4e5
multipv2: Found mate #5 with PV status "short" for FEN "2b5/2N2p2/6p1/4N3/1B1k1p1r/Kp6/n7/4Q3 w - -"  with bm #4.
PV: e1d1 d4e3 d1e1 e3d4
multipv4: Found mate #-2 with PV status "short" for FEN "5K2/5Nn1/6k1/8/8/4Q3/8/8 b - -"  with bm #-4.
PV: g7e8 e3g5 g6h7
multipv2: Found mate #-1 with PV status "short" for FEN "8/8/n7/8/7p/2pQ1KpN/2rpn1P1/2qb1kbR b - -"  with bm #-4.
PV: a6b4
multipv4: Found mate #-1 with PV status "short" for FEN "2R2r1k/p4Ppr/5p1p/7Q/7b/8/R5Kn/6n1 b - -"  with bm #-4.
PV: f6f5
multipv2: Found mate #-2 with PV status "short" for FEN "k7/P7/1Kp5/2p4q/2p5/2P1R3/7R/8 b - -"  with bm #-4.
PV: h5d1
multipv2: Found mate #-1 with PV status "short" for FEN "8/QBn5/R1p1rP2/bnPp4/3p1P2/3PkpP1/5N2/3R1K2 b - -"  with bm #-4.
PV: c7a6
multipv3: Found mate #-1 with PV status "short" for FEN "1NB3b1/1KP1bp1r/1R3p2/1P1pk3/3p2Rn/3P1pNP/1P4r1/5nq1 b - -"  with bm #-4.
PV: f3f2
multipv3: Found mate #-1 with PV status "short" for FEN "Kn6/BP1p4/k1rN3b/rb1Np3/p4Pq1/8/8/1R6 b - -"  with bm #-4.
PV: a4a3
multipv2: Found mate #-4 with PV status "short" for FEN "1RbK1B2/p2pNp1p/2k1p1pP/3p4/4R3/r1p2n2/n1B5/8 b - -"  with bm #-4.
PV: c6d6 e7c8 d6c6 f8a3 a2b4 c8a7 c6c5
Found mate #-5 with PV status "short" for FEN "1RbK1B2/p2pNp1p/2k1p1pP/3p4/4R3/r1p2n2/n1B5/8 b - -"  with bm #-4.
PV: c6c5 e7c8 d7d6 f8d6 c5c6 d6a3 a2b4 c8a7 c6c5
multipv2: Found mate #-1 with PV status "short" for FEN "8/8/p7/P7/n7/K7/B2B4/k7 b - -"  with bm #-4.
PV: a4c3
Found mate #-3 (better) for FEN "8/2p1pN1p/n1P4P/P1kB2K1/1pPp1Np1/1n6/5P2/B7 b - -"  with bm #-4.
Found mate #-3 with PV status "wrong" for FEN "8/2p1pN1p/n1P4P/P1kB2K1/1pPp1Np1/1n6/5P2/B7 b - -"  with bm #-4.
PV: b3a5 f4d3 c5b6 a1d4 a6c5 d4c5
multipv2: Found mate #-1 with PV status "short" for FEN "8/2p1pN1p/n1P4P/P1kB2K1/1pPp1Np1/1n6/5P2/B7 b - -"  with bm #-4.
PV: d4d3
multipv3: Found mate #-1 with PV status "short" for FEN "8/4p3/4Bp2/7p/Pp2k1qR/1PB1PNP1/2P1K1N1/8 b - -"  with bm #-4.
PV: g4h4
multipv2: Found mate #-1 with PV status "short" for FEN "B1n1rb2/2Q1B3/8/1p5p/5pp1/2p1k3/n3bN2/4K3 b - -"  with bm #-4.
PV: c3c2
multipv3: Found mate #-1 with PV status "short" for FEN "1n2B3/q5p1/2pQ3b/n3p3/2P1k2p/p2bp2N/4R1P1/R3K3 b Q -"  with bm #-4.
PV: a3a2
multipv2: Found mate #-2 with PV status "short" for FEN "8/1B3N2/8/K1k5/R6B/8/4r3/8 b - -"  with bm #-4.
PV: e2e1
multipv4: Found mate #-2 with PV status "short" for FEN "R6B/2K1p3/2p1k2N/2R1p1pp/8/5P2/4r1bn/8 b - -"  with bm #-4.
PV: g5g4 c5c6 e6d5
multipv2: Found mate #-2 with PV status "short" for FEN "7k/6pP/8/1p6/1P6/8/8/KQ6 b - -"  with bm #-4.
PV: g7g5 b1g6 g5g4
multipv2: Found mate #-2 with PV status "short" for FEN "B6B/8/2p5/Q2pN3/1N3q2/K2Rp3/PP2pbpn/4kn1R b - -"  with bm #-4.
PV: f2g3 b4c2 e1f2
Found mate #-4 with PV status "short" for FEN "rk1bn1b1/n1p4p/NpP2PPQ/pP5B/8/7q/P4r2/1K1RR3 b - -"  with bm #-4.
PV: b8c8 d1d8 c8d8 e1e8 d8e8 g6h7 h3h5
Found mate #-4 with PV status "short" for FEN "K2nk1Br/1P2p1NP/p1n1RpNp/4pr2/qP3P2/3pP2P/2b3pR/1Q4B1 b k -"  with bm #-4.
PV: e8d7 g6f8 d7c7 g7e8 c7b6
multipv3: Found mate #-1 with PV status "short" for FEN "8/4n2b/2p5/R1ppN3/R2P3r/2K3p1/6P1/1k6 b - -"  with bm #-4.
PV: c5c4
multipv3: Found mate #-1 with PV status "short" for FEN "5N2/4p3/8/8/3kN3/1B3K2/4P3/8 b - -"  with bm #-4.
PV: e7e5
multipv3: Found mate #-2 with PV status "short" for FEN "7B/5B2/6Pk/3N4/6K1/1N6/8/b7 b - -"  with bm #-4.
PV: a1f6 d5e3 f6h8
multipv4: Found mate #-3 with PV status "short" for FEN "5K1k/3P3b/6p1/8/7R/5n2/8/8 b - -"  with bm #-4.
PV: f3h2 h4h2 g6g5 d7d8q g5g4
multipv4: Found mate #-4 with PV status "short" for FEN "b7/3pK1p1/3P4/8/2p2P2/2p1Pk1P/2B2Pp1/2N3B1 b - -"  with bm #-4.
PV: a8c6 c2d1 f3e4 f2f3 e4f5
multipv3: Found mate #-1 with PV status "short" for FEN "8/4K3/8/4kp2/1N1pppbB/8/3PR2R/8 b - -"  with bm #-4.
PV: d4d3
multipv2: Found mate #-2 with PV status "short" for FEN "8/1N4pR/r3Pp2/4ppkp/5n1N/b2p2K1/3B4/8 b - -"  with bm #-4.
PV: a6e6
multipv3: Found mate #-2 with PV status "short" for FEN "3R2Kn/Q4Np1/3p1b2/1N1Bp3/Pkr1p3/5n2/P6R/8 b - -"  with bm #-4.
PV: f3d4 h2b2 d4b3
multipv4: Found mate #-1 with PV status "short" for FEN "8/8/4nrp1/4pr1b/5P1p/3Pp2Q/4P1pp/R3K1kq b Q -"  with bm #-4.
PV: e5e4
multipv2: Found mate #-1 with PV status "short" for FEN "1R6/2p1K3/8/5p1p/B2B4/P1N2p2/1P3P2/r1k2N1r b - -"  with bm #-4.
PV: h5h4
Found mate #-5 with PV status "short" for FEN "8/8/3p2p1/3Pk1Pp/2K4P/4pPP1/4P3/4B3 b - -"  with bm #-4.
PV: e5f5 e1a5 f5e5 a5b6 e5f5 b6e3 f5e5
multipv3: Found mate #-2 with PV status "short" for FEN "k4n2/2Q5/2K5/8/1q6/8/6B1/8 b - -"  with bm #-4.
PV: f8e6
multipv2: Found mate #-1 with PV status "short" for FEN "K3B3/8/k3Nppp/1R1ppp2/3p2r1/b1p4Q/5N2/r1q5 b - -"  with bm #-4.
PV: c3c2
multipv4: Found mate #-2 with PV status "short" for FEN "1R1K4/2ppp3/2k5/1N3n1b/1B1p1N2/1B6/1P1p4/q5n1 b - -"  with bm #-4.
PV: a1a2 b3d5 a2d5
multipv3: Found mate #-1 with PV status "short" for FEN "3B1nq1/8/2Npp1p1/K3Pp2/3p1k1b/2pp4/8/5NQB b - -"  with bm #-4.
PV: c3c2
multipv2: Found mate #-2 with PV status "short" for FEN "8/K1p1R3/2B5/1p6/1P6/N1Pk4/7r/1N6 b - -"  with bm #-4.
PV: h2h4
multipv2: Found mate #-2 with PV status "short" for FEN "8/2p1p3/4Ppp1/2p3P1/2KpP1Np/8/6p1/2Q3bk b - -"  with bm #-4.
PV: h4h3 g4f2 h1h2
multipv2: Found mate #-2 with PV status "short" for FEN "6b1/6K1/6N1/6k1/5RR1/6B1/5nrn/6r1 b - -"  with bm #-4.
PV: h2g4 g3h4 g5h5
multipv3: Found mate #-1 with PV status "short" for FEN "2r5/5p2/3N4/2p4B/p7/K1Bkp2p/2NP4/6R1 b - -"  with bm #-4.
PV: e3e2
multipv2: Found mate #-1 with PV status "short" for FEN "8/1p3p1p/kB4p1/PN3N2/6B1/1K2p2Q/4pr2/3n1q2 b - -"  with bm #-4.
PV: g6g5
multipv3: Found mate #-1 with PV status "short" for FEN "R7/2p5/2K5/2p4p/2k4P/2p5/2B5/2b5 b - -"  with bm #-4.
PV: c1b2
multipv3: Found mate #-3 with PV status "short" for FEN "1b4N1/7p/1p1pB2R/6k1/4p1p1/1Rp1pbP1/5P1P/K3B1nq b - -"  with bm #-4.
PV: c3c2
Found mate #-6 with PV status "short" for FEN "kn6/3p4/nPpP1p2/R1N2B2/p7/5K2/P7/8 b - -"  with bm #-4.
PV: a4a3 f5d3 f6f5 d3a6 b8a6 a5a6 a8b8 a6a3 f5f4 f3f4
multipv3: Found mate #-2 with PV status "short" for FEN "2brk3/3p4/3p1Q1p/8/4NK2/b1p1R3/1Pq2P1n/r7 b - -"  with bm #-4.
PV: a1d1
multipv3: Found mate #-2 with PV status "short" for FEN "r6k/3p1Ppr/4Bp1p/8/8/Q5Kn/1p4n1/2b1R3 b - -"  with bm #-4.
PV: g7g6
multipv2: Found mate #-1 with PV status "short" for FEN "8/8/7p/7P/1p1N1pK1/1P1PbP2/pp2N3/rk5R b - -"  with bm #-4.
PV: e3g1
multipv3: Found mate #-2 with PV status "short" for FEN "N2k4/N6p/4pP2/3p4/B1p2p2/Kp2R2r/6pq/3nB1b1 b - -"  with bm #-4.
PV: f4e3
multipv2: Found mate #-2 (better than #-3) for move d1c3 for FEN "N2k4/N6p/4pP2/3p4/B1p2p2/Kp2R2r/6pq/3nB1b1 b - -"  with bm #-4.
Found mate #-4 with PV status "short" for FEN "8/4p3/p3K2p/Pp2R1pP/pP1k2P1/P3p3/4P3/1N6 b - -"  with bm #-4.
PV: d4c4 e5e3 c4d4 e3f3 d4e4 b1d2 e4d4
multipv2: Found mate #-1 with PV status "short" for FEN "8/B1RN3K/3p1p1P/3k1P2/3q4/3P1Np1/6P1/8 b - -"  with bm #-4.
PV: d4a1
multipv4: Found mate #-1 with PV status "short" for FEN "8/K2N2N1/pp1p1P1p/4P1b1/2P1pk2/1nQ2PR1/2r4P/3q1BB1 b - -"  with bm #-4.
PV: e4e3
multipv2: Found mate #-1 with PV status "short" for FEN "7K/6R1/1p6/kP1P2Pp/Pp1N3B/1P6/1q6/b4R2 b - -"  with bm #-4.
PV: b2b1
Found mate #-3 (better) for FEN "4b3/4R2n/3R3K/2p2kpN/2p1p3/r3p2P/6PB/bq3Nr1 b - -"  with bm #-4.
Found mate #-3 with PV status "wrong" for FEN "4b3/4R2n/3R3K/2p2kpN/2p1p3/r3p2P/6PB/bq3Nr1 b - -"  with bm #-4.
PV: e8h5 f1g3 f5f4 e7e4 b1e4 g3h5
multipv2: Found mate #-2 with PV status "short" for FEN "4b3/4R2n/3R3K/2p2kpN/2p1p3/r3p2P/6PB/bq3Nr1 b - -"  with bm #-4.
PV: e3e2
multipv2: Found mate #-1 with PV status "short" for FEN "8/8/2B1N3/3rp2K/4k3/7Q/2r3Pn/1b1N4 b - -"  with bm #-4.
PV: h2f1
multipv2: Found mate #-6 with PV status "short" for FEN "8/6pK/8/R7/4P1k1/4P1P1/5BP1/8 b - -"  with bm #-4.
PV: g7g5 h7g7
multipv2: Found mate #-1 with PV status "short" for FEN "8/1Kp1pN1r/2P5/Bp6/8/kBN5/1pP1R3/1r3qb1 b - -"  with bm #-4.
PV: b5b4
multipv2: Found mate #-1 with PV status "short" for FEN "1N3k2/1R1p1ppK/p1bN1PpB/4n3/8/8/q6b/8 b - -"  with bm #-4.
PV: a6a5
multipv3: Found mate #-3 with PV status "short" for FEN "6Q1/4p3/N5B1/1p3K2/2k2pR1/1bb2nr1/3pp2B/5R2 b - -"  with bm #-4.
PV: c4d3
multipv3: Found mate #-3 (better than #-4) for move c4d3 for FEN "6Q1/4p3/N5B1/1p3K2/2k2pR1/1bb2nr1/3pp2B/5R2 b - -"  with bm #-4.
multipv2: Found mate #-1 with PV status "short" for FEN "r3b3/8/1B2pK2/p7/7k/5B1N/8/6N1 b - -"  with bm #-4.
PV: a5a4
multipv3: Found mate #-1 with PV status "short" for FEN "8/5p2/2N1k2R/1K6/4N1P1/8/8/8 b - -"  with bm #-4.
PV: e6d5
multipv3: Found mate #-3 with PV status "short" for FEN "1R6/4p3/3P2B1/8/3P4/KQP5/ppp1p3/qkrbN3 b - -"  with bm #-4.
PV: e7d6
multipv3: Found mate #-3 (better than #-4) for move e7d6 for FEN "1R6/4p3/3P2B1/8/3P4/KQP5/ppp1p3/qkrbN3 b - -"  with bm #-4.
multipv4: Found mate #-1 with PV status "short" for FEN "8/3p2r1/2n3B1/6B1/3P2PP/p1N1N3/6K1/4k3 b - -"  with bm #-4.
PV: a3a2
multipv2: Found mate #-1 with PV status "short" for FEN "8/8/3p2Q1/1K1kp3/3P4/8/3N4/8 b - -"  with bm #-4.
PV: e5e4
Found mate #-5 with PV status "short" for FEN "5N2/8/3p4/3Pp3/4p1P1/4kp2/4pN1P/B3K3 b - -"  with bm #-4.
PV: e3f4 f8e6 f4e3 e6g5 e3f4 g5e4 f4e3 a1b2 e3f4

Using ./sf12 on matetrack.epd matedtrack.epd with --bmMin 4 --bmMax 4 --nodes 1000 --multiPV 4
Engine ID:     Stockfish 12
Total FENs:    253
Found mates:   18
Best mates:    10

Parsing the engine's full UCI output, the following issues were detected:
Better mates:                10   (from 3 FENs)
MultiPV Better mates:        3   (from 3 FENs)
Bad PVs:                     30   (from 11 FENs)
MultiPV Bad PVs:             298   (from 63 FENs)

@robertnurnberg

Copy link
Copy Markdown
Contributor Author

I will keep track of improvements for the new files in robertnurnberg/matetools#127, in case others would like to contribute. Once enough improvements have accumulated, I would open a PR on the matetrack repo.

@vondele
vondele merged commit 2c6ef8c into vondele:master Jul 25, 2026
@robertnurnberg
robertnurnberg deleted the multipv branch July 25, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants