Take line breaks into account for redundant periods

[engboris]

I had a fix for redundant periods: #521

However, it doesn't take line breaks into account, as in:

       IDENTIFICATION DIVISION.
       PROGRAM-ID. REDUNDANT-DOTS.

       ENVIRONMENT DIVISION.

       DATA DIVISION.
       WORKING-STORAGE SECTION.
       01  WS-MESSAGE        PIC X(30) VALUE "Hello, COBOL World!".
       01  WS-COUNTER        PIC 9     VALUE 1.

       .

I tried a new fix and the check syntax commands succeeds but I'm not sure about it as I'm not sure to understand exactly how tokens are manipulated. Moreover, the tests I have promoted look suspicious.

[nberth]

In this particular case may the period after 1 just be considered an interleaving token? I'm not sure adding even more state into the parser record is the way to go here. One approach I think might be cleaner is to allow lonesome periods when: (i) a full data description item is expected in the data division, and (ii) a statement is expected in the procedure division.

[engboris]

Hmm I merged with master and got a lot of weird output on make test. I'll try something else.

[nberth]

This here seems to be an issue unfortunately.