Modification to Acrobat PDF 1.4 - Portable Document Format 1.4

[NoeHuet]

Affected format(s)

• https://www.nationalarchives.gov.uk/PRONOM/fmt/18
• Maybe other PDF formats

Details of change

Some old PDFs from https://cairn.info have 0a at the beginning and 0d at the end of the file. A maximum offset must be defined to take this case into account.

Credit

Archives nationales de France

[thorsted]

This is a valid question. How flexible should signatures be to catch relatively well formed PDF's which may not be exactly as the specification mandates, when most software ignores minor indiscretions? I have quite a few PDF's which have null padding at the end of the file, they open fine and pass all simple checks, but technically are breaking the rules by not having the file end with EOF.

Is PRONOM the place to decide or is this for other tools to be more format specific?

[anjackson]

For what it's worth, when processing content from the wild web (at the UK Web Archive) I stacked format identification, using Apache Tika in the first instance, then running DROID/Siegfried and using that to refine the result.

I think PRONOM is stricter by design, because it's primary use case is in contexts where you'd want off-spec PDFs to get flagged for manual review (I presented about this at iPRES last year, but the papers aren't available yet. - I should probably republish them on my blog anyway). Tools like file or Apache Tika etc. are better suited to more permissive and automated workflows. (n.b. I believe SPAR at BNF uses file for this reason)

[NoeHuet]

Thanks a lot for both answers! @thorsted, we also have those strange PDFs with null padding at the end of the file, and identification from Siegfried/Droid doesn't work on them. I agree that a lack of identification is an effective way to flag files for manual review. @anjackson Thanks for sharing Apache Tika, which I hadn't heard of before. It seems really promising for dealing with large collection.

The problem we currently have is that our archival information system cannot handle multi-identification. When Siegfried proposes several formats, it always takes the first one, and this information is then unchangeable. Therefore, we would gain precision if our file signatures allowed fuzzier identification from the outset. This may be a local problem arising from the way our tool works and could definitely be resolved by adapting the signatures locally. However, it's interesting to hear other perspectives.

[ross-spencer]

it's primary use case is in contexts where you'd want off-spec PDFs to get flagged for manual review

I'm not sure about this assertion. May want / it depends might be important qualifiers. For one, a lot of signatures aren't written with the aid of specifications and a lot of software isn't written precisely to specification. So I don't think PRONOM is explicitly performing a first pass validation triage unless it is on accident. It's a clue, but probably not one to rely on.

It could be beneficial in a future model to be able to draw out signatures written precisely to spec though...

I think too, if there is an assertion to be made about this, seeing those encoded in a formal PRONOM modus operandi might also be beneficial for reasoning about this stuff (and pointing to!).

This is a valid question. How flexible should signatures be to catch relatively well formed PDF's
Is PRONOM the place to decide or is this for other tools to be more format specific?

I don't have an opinion here, but my recent work on incorporating fuzziness into JSON identification for PRONOM goes quite far and adds the generous {0-4095} into the beginning of all its signatures.

NB. We should circle back to conversations about this sometime for formats other than JSON, like XML.

I could imagine this equally being a configuration setting in a tool, however. In fact my first instinct was to check roy to see what it offered here. It has a max bof setting, but that only limits how far into a file it scans (if I read correctly and my quick trial worked correctly). In droid or siegfried I could imagine an extend bof setting to include excess up to _n-bytes_ at the beginning of the file to begin to implement fuzziness. This sort of setting in a tool like roy is then exposed in the metadata for the specific details about how a signature file was built.

and could definitely be resolved by adapting the signatures locally.

You definitely could, I guess you'd exclude fmt/18 and use a locally defined PDF signature? or simply include a locally defined extension on-top of the more precisefmt/18?