Use of `CustomerMgr.searchProfiles()` limits the result set to 1000 customers

[clstopher]

The historical customer export and delta customer export both use the CustomerMgr.searchProfiles() function to retrieve customer profile data:

bloomreach-salesforce-commercecloud-b2c-integration/cartridges/int_bloomreach_engagement/cartridge/scripts/jobSteps/customerInfoFeed.js

Line 105 in ec43840

customerProfilesItr = CustomerMgr.searchProfiles(query, 'customerNo DESC');

bloomreach-salesforce-commercecloud-b2c-integration/cartridges/int_bloomreach_engagement/cartridge/scripts/jobSteps/customerInfoFeed.js

Line 155 in ec43840

customerProfilesItr = CustomerMgr.searchProfiles(query, 'customerNo DESC', new Date(lastCustomerExport));

But this function is documented to only retrieve up to 1000 customer profiles:

https://salesforcecommercecloud.github.io/b2c-dev-doc/docs/current/scriptapi/html/index.html?target=class_dw_customer_CustomerMgr.html#dw_customer_CustomerMgr_searchProfiles_Map_String_DetailAnchor

• the search will return only the first 1000 hits from the search result

This will certainly cause a problem for the historical data load, and also may cause a problem for delta loads when customer activity volume is high.

See #5 regarding orders - We haven't tried running this job in PROD yet, but we are preemptively modifying this job to use CustomerMgr.processProfiles() along with the purchase jobs.