SSSOM/TSV test cases

[nichtich]

I've started a SSSOM/TSV parser in JavaScript but the format is quite nasty in some details such as "Multi-valued slots with a single value" and escaped tabulator characters. It's also not clear to me whether values starting or ending with a " only are errors or unescaped values and what to do with columns of unknown name and rows of different number of columns. I've looked into source code of sssom-py and sssom-java but these seem to delegate the parsing to an existing CSV library and I would not be surprised if these libraries don't strictly follow SSSSOM/TSV specification for edge cases!

tl;dr: a set of test cases is required, in particular for edge cases

[matentzn]

It would be good if you could make specific issues when you notice underspecification somewhere, that would be appreciated, we would want to clarify everything there, including edge cases (cc @gouttegd).

We keep a number of test cases here which we run (validate, convert) with every change to the schema using CI: https://github.com/mapping-commons/sssom/tree/master/examples/schema

We can add additional test cases of course but lets do that independently for every single issue like "quotes in TSV".

[nichtich]

I'd prefer one common repository with examples of:

• valid SSSOM/TSV files and their expected translation to the data model
• invalid SSSOM/TSV files (and optional error message/location)

And every SSSOM implementation to run against these examples.

Some edge cases I am not sure about:

• is the Mappings block mandatory or optional? Does an empty set of mappings require the column line?
• only two column names are required: predicate_id and mapping_justification, right?
• are additional column names in the mappings block allowed (=ignored) or forbidden?
• is TSV value "abc and error or does it translate to literal "abc?
• is TSV """ and error? Why?

[gouttegd]

There’s a bunch of test cases in the repository for the SSSOM-Java implementation. They can certainly be turned into a generic, implementation-independent test suite.

As for your ”edge cases”:

is the Mappings block mandatory or optional?

Mandatory. An empty file is not a valid SSSOM/TSV file. At a minimum, the header line (containing the column names) must be present.

From the spec:

The mappings block MUST follow immediately the metadata block within a SSSOM/TSV file. It starts with a header line containing the column names.

I don’t see how this could be interpreted as allowing the mapping block to be completely absent.

So a file representing a completely empty set (no metadata, no mapping) would be:

subject_id   predicate_id   object_id   mapping_justification

only two column names are required: predicate_id and mapping_justification, right?

No. subject_id, predicate_id, object_id, and mapping_justification. Those are the four mandatory slots.

(Leaving aside the special case of “literal” mappings.)

are additional column names in the mappings block allowed (=ignored) or forbidden?

They are allowed. It’s left to each implementation to decide whether they are ignored or taken into accounts as non-standard slots. For now SSSOM-Py ignores them; SSSOM-Java can either ignore them or take them into account, it’s user-configurable.

is TSV value "abc and error or does it translate to literal "abc?

That is very clearly an error as per the quoting rules. If the intention was to store a literal "abc, then it should have been written as """abc".

is TSV """ and error? Why?

Again, very clearly an error, because it violates the third rule: when a value is enclosed in double-quotes, any double-quote within the value must be escaped by being preceded by another double-quote. So if the intention was to store a literal double-quote ("), then it should have been written as """".

[gouttegd]

I've looked into source code of sssom-py and sssom-java but these seem to delegate the parsing to an existing CSV library and I would not be surprised if these libraries don't strictly follow SSSSOM/TSV specification for edge cases!

The TSV rules of the SSSOM specification don’t come out of nowhere. They have been chosen because they correspond to behaviours that are very common among CSV/TSV parsers – in particular, they correspond to the default behaviours of both Pandas’ parsers (used under the hood by SSSOM-Py) and Jackson’s parsers (used under the hood by SSSOM-Java).

In fact I would greatly encourage you to also rely on a well-designed, well-tested CSV/TSV parser library (don’t know what is available in the Javascript world, but I would hope there are such libraries) instead of trying to come up with your own parser. CSV/TSV is a much trickier format than most people initially assume, and writing a good parser for it is not trivial.

[nichtich]

Thanks for feedback and clarification. I also agree to rely on parser libraries for standards, but the SSSOM/TSV specification does not refer to a TSV specification, or am I missing something? Several TSV dialects exist so either refer to an existing document or define your own TSV. SSSOM has chosen the second option, so any existing TSV parser is at risk of not exactely conforming to SSSOM but to another TSV dialect.

[gouttegd]

but the SSSOM/TSV specification does not refer to a TSV specification, or am I missing something?

How could it refer to something that does not exist? I am not aware of anything that could be considered a “TSV specification”, whether official or not.

The CSV format has its own RFC (RFC 4180), but TSV does not. The only thing that remotely looks like a specification for TSV is the IANA entry for the text/tab-separated-values media type, which starts by explicitly stating that the format “does not really need any specification“ (which, I fully admit, is highly debatable, but that’s what we have).

Several TSV dialects exist so either refer to an existing document or define your own TSV. SSSOM has chosen the second option

We certainly did not “define our own dialect“. TSV dialects differ mostly, if not exclusively, in how they deal with the representation of “special characters” (including quotes, tabs, newlines, etc.) within values.

Options include:

• forbid special characters entirely (that’s the option chosen in the aforementioned IANA description; by the way, that’s why we did not refer to that description, because we did not want to forbid special characters in SSSOM/TSV files);
• C-style escape sequences (e.g. \n, \t, etc.);
• CSV-style quoting as described in the aforementioned RFC 4180.

We chose the third option, and explicitly referred to the RFC 4180.

As far as I know most TSV parsers have explicit support for those three options (with Java’s Jackson library for example, it’s one line of code to configure the parser to accept either C-style escape sequences or CSV-style quoting).

[nichtich]

Sorry for being pedantic. I misread "adapted from RFC 4180" as "inspired by RFC 4180" when it's more "conforming to RFC 4180 with comma/COMMA replaced by 'tabulator/HTAB". This still does not allow Unicode but an update to RFC 4180 is on is way.

[gouttegd]

valid SSSOM/TSV files and their expected translation to the data model

This raises the question of how should we represent, in that test suite, the “expected translation to the data model”.

The obvious answer would be “in JSON”, but then we run into the problem that the JSON serialisation of SSSOM is currently not specified at all.

[nichtich]

sssom-js now includes several test cases in directory valid and invalid, respectively. The sample files are mainly parsed in tsvreader.js.

[gouttegd]

They all look good, except the “invalid” array-value.sssom.tsv:

# see_also: ["//example.org/"]

This is valid. The see_also slot is multi-valued, and multi-valued slots can be represented using either the “flow sequence” syntax (the one shown above) or using the “block sequence” syntax:

# see_also:
#   - //example.org

Both forms are allowed in YAML and are not in the list of YAML features that SSSOM explicitly disallows, so they are both valid in SSSOM. The canonical SSSOM/TSV format mandates the use of the block syntax, but that is a recommendation for writers only.

[nichtich]

Good catch! The YAML is valid but the value //example.org is not. The current implementation checks for URI but it seems like IRI should be checked instead. The value is no valid IRI neither.