Skip to content

Feature/edan verifier utility#531

Draft
EMaslowskiQ wants to merge 19 commits into
developfrom
feature/EDAN-verifier-utility
Draft

Feature/edan verifier utility#531
EMaslowskiQ wants to merge 19 commits into
developfrom
feature/EDAN-verifier-utility

Conversation

@EMaslowskiQ

Copy link
Copy Markdown
Collaborator

Creation of EDAN verifier utility.

  • Accessed through endpoint: /verifiers/edan
  • Uses Workflows for managing execution
  • Uses WorkflowReports for storing results
  • Merged changes made to 'develop' and tested

* WIP snapshot of EDAN verifier
* compares subject units and identifiers outputting results to CSV
* currently runs as a test case in EdanCollection.test.ts
* WIP commit capturing refactor of EDAN verifier
* lacks cleanup and testing
*  WIP commit for EDAN Verifier utility
* cleaned out legacy code prior to refactor
* limiting returned EDAN records to 1, otherwise throw error
* fix issues with ARK ids not being outputted if also set to preferred
* added basic determination of DPO vs. EDAN subject sources
* support for name comparisons
* support for updating Packrat units from EDAN
* WIP commit for EDAN Verifier utility
* support for name comparisons
* support for updating Packrat units from EDAN
* improved messaging for automatic/manual fixing
* WIP commit for EDAN verifier
* added EDAN MDM column to ouput
* added ability to run a specific subject ID for debugging
* WIP commit for EDAN verifier
* moved EDAN verifier into own class derived from VerifierBase
* cleaned up how EDAN units are collected
* opting for comparing abbreviations when using fetchFromNamedSearch and receiving multiple hits
* WIP commit for EDAN verifier
* removed fallback for checking multiple EDAN units and opting for comparing abbreviations
* verifier now accepts EdanVerifierConfig object with settings
* moved all temporary debug statements to standard LOG
* EDAN Verifier updated to support running from any endpoint (/verifier/edan)
* support query params for basic configuration
* returns a CSV file if given param of: returnFile=true
* using WorkflowEngine for running the EDAN verifier (WorkflowVerifier)
* full support of query params for configuring verifier (accepts: objectId, limit, returnFile, detailedLogs)
* HTTP response includes hyperlink for downloading the report
* added 'Name' property to WorkflowReport to assist in filename creation
* cleaned up Packrat schema. comment prevented rebuilding the DB with initdb
* created utility routine that gathers all SystemObject details (via GraphQL) and returns as NDJSON. Not accessible.
* tested against 'develop' branch
* added cleaner/styled output for response
* updated model graph (mwb) to reflect changes in DB
* fixed eslint error with resolving VerifierBase

@jahjedtieson jahjedtieson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your first PR! What a great milestone!

  • The changes to server/tests/jest.config.js must not be committed. Doing so will disable regression tests from being run, by default, by other users, and by GitHub CI.
  • We don't add server/db/sql/models/Packrat.mwb.bak to our source tree.
  • server/utils/verifiers/VerifierBase.ts's replacePackratUnit() is doing the wrong thing. Instead of updating Unit records, it needs to update Subject.idUnit (i.e point the subject at the right unit).
  • I've asked a number of inline questions/provided feedback.

Comment thread server/db/api/WorkflowReport.ts
Comment thread server/db/api/WorkflowReport.ts Outdated
Comment thread server/http/index.ts Outdated
Comment thread server/http/routes/download.ts Outdated
Comment thread server/http/routes/verifiers.ts Outdated
}

// if identifier exists in our database (value & type) then store it
const identifiers: DBAPI.Identifier[] | null = await DBAPI.Identifier.fetchFromIdentifierValue(content);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused by the intention of this code. fetchFromIdentifierValue is going to get all identifiers that match the specified value ... this may include matches for identifiers with the same value, but applied to other system objects, and/or of random identifier types. In other words, identifiers are not guaranteed to have a unique value....

Comment thread server/utils/verifiers/VerifierBase.ts Outdated
Comment thread server/utils/verifiers/EdanVerifier.ts Outdated
Comment thread server/utils/verifiers/EdanVerifier.ts
Comment thread server/http/routes/dataQueries.ts
* formatting and cleanup for submission

@jahjedtieson jahjedtieson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove server/db/sql/models/Packrat.mwb.bak from this PR.

@EMaslowskiQ EMaslowskiQ marked this pull request as draft February 1, 2023 01:54
* changed LOG type top eSYS for better grouping
* clearer definition of VerifierBase types and their use
* comments describing future improvements
@EMaslowskiQ EMaslowskiQ force-pushed the feature/EDAN-verifier-utility branch from 756198b to b645f4b Compare May 11, 2026 14:05
@EMaslowskiQ EMaslowskiQ force-pushed the develop branch 2 times, most recently from 1b4dc49 to 0b420d9 Compare May 18, 2026 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants