fix: do not treat blank lines with trailing whitespace as line continuations

A blank line that contains trailing whitespace (e.g. "    \n") produces an
NL token whose `.line` is not exactly "\n" or "\r\n".  The guard in
`is_newline_continuation` only excluded the exact-blank forms, so such a
line was misclassified as a continuation of the preceding line.  This
caused `_do_update_token_indices` to rewrite the token's start row back to
the previous token's end row, yielding a start position that precedes the
previous token's end.  `tokenize.untokenize` then raised
`ValueError: start (r,c) precedes previous end (r,c)` and docformatter
crashed (regression from the stdlib tokenize migration in #325, v1.7.8).

Classify any whitespace-only line as blank by testing `token.line.strip()`
instead of comparing the raw line against {"\n", "\r\n"}.  Genuine
continuations (whose stripped line is non-empty) are unaffected.

Fixes #355.

Author: apoorvdarshan

src/docformatter/classify.py

@@ -439,8 +439,8 @@ def is_newline_continuation(
     """
     if (
         token.type in (tokenize.NEWLINE, tokenize.NL)
+        and token.line.strip()
         and token.line.strip() in prev_token.line.strip()
-        and token.line not in {"\n", "\r\n"}
     ):
         return True
 

tests/_data/string_files/do_format_code.toml

@@ -1244,3 +1244,10 @@ expected='''"""A."""
 
 pass
 '''
+
+# A blank line with trailing whitespace must not crash the tokenizer round-trip.
+# The "\n" escapes are used so the four trailing spaces on the blank line are
+# explicit and cannot be stripped by editors or formatters.
+[issue_355]
+source="def foo():\n    \"\"\"Summary.\"\"\"\n    x = 1\n    # next line has 4 spaces of trailing whitespace\n    \n    return x\n"
+expected="def foo():\n    \"\"\"Summary.\"\"\"\n    x = 1\n    # next line has 4 spaces of trailing whitespace\n    \n    return x\n"

tests/formatter/test_do_format_code.py

@@ -141,6 +141,7 @@
         ("do_not_break_f_string_double_quotes", NO_ARGS),
         ("do_not_break_f_string_single_quotes", NO_ARGS),
         ("issue_331_black_module_docstring", ["--black", ""]),
+        ("issue_355", NO_ARGS),
     ],
 )
 def test_do_format_code(test_key, test_args, args):