Get rid of Trie based DATE fields, which are deprecated

[ndushay]

The goal of this ticket is to get rid of some deprecated Solr Trie field types for Argo.

Solr says Trie fields are deprecated, and that Point fields are faster. These field types are marked deprecated in argo-xx schema.xml

Fields to be updated

dor_indexing_app

18:      'released_to_searchworks_dttsi' => searchworks_release_date,
19:      'released_to_earthworks_dttsi' => earthworks_release_date

app/indexers/data_indexer.rb
18:      solr_doc['modified_latest_dttsi'] = modified_latest
19:      solr_doc['created_at_dttsi'] = created_at

app/indexers/embargo_metadata_indexer.rb
17:        solr_doc['embargo_release_dtsim'] = [embargo_release_date.utc.iso8601]

app/models/workflow_solr_document.rb
63:    data["wf_#{wf_name}_#{process_name}_dttsi"] = time.utc.iso8601
75:    # This is going to get the date fields, e.g. `wf_assemblyWF_jp2-create_dttsi'

app/services/workflow_fields.rb
50:      # create the published_dttsi and published_day fields and the like
52:        solr_doc["#{milestone}_dttsim"] ||= []
53:        solr_doc["#{milestone}_dttsim"] << date unless solr_doc["#{milestone}_dttsim"].include?(date)
55:      # fields for OAI havester to sort on: _dttsi is trie date +stored +indexed (single valued, i.e. sortable)
57:      solr_doc["#{milestone}_earliest_dttsi"] = dates.first
58:      solr_doc["#{milestone}_latest_dttsi"] = dates.last

argo

31:      solr_parameters['facet.query'] << '-rights_primary_ssi:"dark" AND published_dttsim:*'

app/search_builders/argo/date_field_queries.rb
15:      blacklight_params['f'].select { |k, _v| k =~ /.+_dt/ }.each do |key, values|

app/models/report.rb
92:      field: :registered_earliest_dttsi, label: 'Registered',
93:      proc: ->(doc) { DatePresenter.render(doc[:registered_earliest_dttsi]) },
139:      field: :accessioned_earliest_dttsi, label: 'Accession. Datetime',
140:      proc: ->(doc) { DatePresenter.render(doc[:accessioned_earliest_dttsi]) },
144:      field: :published_earliest_dttsi, label: 'Pub. Date',
145:      proc: ->(doc) { DatePresenter.render(doc[:published_earliest_dttsi]) },

app/models/solr_document.rb
11:  FIELD_EMBARGO_RELEASE_DATE = :embargo_release_dtsim
15:  FIELD_CREATED_DATE = :created_at_dttsi
16:  FIELD_REGISTERED_DATE = :registered_dttsim
17:  FIELD_LAST_ACCESSIONED_DATE = :accessioned_latest_dttsi
18:  FIELD_EARLIEST_ACCESSIONED_DATE = :accessioned_earliest_dttsi
19:  FIELD_LAST_PUBLISHED_DATE = :published_latest_dttsi
20:  FIELD_LAST_MODIFIED_DATE = :modified_latest_dttsi
21:  FIELD_LAST_OPENED_DATE = :opened_latest_dttsi
23:  FIELD_RELEASED_TO_EARTHWORKS = :released_to_earthworks_dttsi
24:  FIELD_RELEASED_TO_SEARCHWORKS = :released_to_searchworks_dttsi

app/presenters/profile_presenter.rb
90:    facet_query_field['-rights_primary_ssi:"dark" AND published_dttsim:*']

solrconfig

126:      <str name="facet.field">released_to_earthworks_dtsi</str>
127:      <str name="facet.field">released_to_searchworks_dtsi</str>

argo_qa/solrconfig.xml
126:      <str name="facet.field">released_to_earthworks_dtsi</str>
127:      <str name="facet.field">released_to_searchworks_dtsi</str>

argo_stage/solrconfig.xml
126:      <str name="facet.field">released_to_earthworks_dtsi</str>
127:      <str name="facet.field">released_to_searchworks_dtsi</str>

Steps

• dor_indexing_app populates new fields of type DatePoint (in addition to existing)
  • PR for this
  • deployed to qa
  • deployed to stage
  • deployed to prod
• argo solrconfig.xml file PR to use new fields in search results for stage and qa
  • solr config changes deployed to qa
  • solr config changes deployed to stage
• argo PR (on HOLD) for replacing usage of old field names
  • argo changes deployed to qa
  • argo changes deployed to stage
• tested and vetted on qa and/or stage
• argo solr prod new fields have begun populating (starting: )
• argo solr prod new fields have finished populating
• roll out new fields in argo prod
  • add prod to argo PR (take out of HOLD) for replacing usage of old field names
    • argo changes deployed to prod
  • argo solrconfig.xml file PR to use new fields in search results
    • solr config changes deployed to prod
    • argo repo gets solr config changes
    • dor-services-app repo gets solr config changes
• remove old fields
  • argo
  • dor_indexing_app
  • indexes
    • qa
    • stage
    • prod
• sul-solr-config removes field types
• argo Solr config files updated in the Solr cluster
  • qa
  • stage
  • prod
• argo repo solr config changes
• dor-services-app repo solr config changes