Full and Incremental Extracts from Symphony

This page is to document the process for Full and Incremental extracts processed from Symphony and how they are loaded to BlackCat Server.

What is included in a Symphony extract for BlackCat?

Everything from Sirsi,

• Excluding shadowed records, review records but not temporary records. (Most of the temporary records are shadowed.)

• Excluding item types:
  CARRELKEY, EBOOKREADR, EQUIP14DAY, EQUIP24FEE, EQUIP24HR, EQUIP3DAY, EQUIP4HR, EQUIP5DAY, EQUIP7DAY, ILL, LAPTOP, PALCI

• Excluding Junk tags: Generated a custom junk tag list for Blackcat (junktag_blackcat on Symphony):

  001, 029, 090, 590, 593, 594, 595, 596, 597, 598, 599, 799, 900, 901, 920, 921, 926, 927, 930, 948, 949, 960, 961, 977, 980, 990, 994, 998, 999
  

• Bound-with extract:
  selects all the call numbers which are a bound-with child with their parent's holdings information, title and catkey of that bound-with child. The parent's title and catkey is appended and exported as a 591 field and the parent's holding information as a 949 to the child's extracted record.

• Holdings extract:
  The 949 field is used for current holdings. During the export of a record, any legacy data (any 949 in the bib) is removed.

Full Extracts

1. Full extract can start anytime after adutext finishes it's daily run of clearing out the /sirsi/Unicorn/Database/Edit directory.

2. On the Symphony side, shut off the sirsi cron job that sends records over to Blackcat.

   #5,35 * * * * /sirsi/Psuscripts/Scripts/move_edited_records_to_blackcat >> /sirsi/Unicorn/PSULinkages/Blackcat/cron_log 2> /sirsi/Unicorn/PSULinkages/Blackcat/cron_error_log
   

3. On blackcat QA server

   $ sudo -i
   $ cd /data/symphony_data/full_extracts
   $ mkdir <extract date>
   $ chown ansible_deploy_bot:sirsi <extract date>
   $ chmod 775 <extract date>
   

4. Extract all the Symphony MARC data and secure copy it to /data/symphony_data/full_extracts/<extract date> directory on the Blackcat server. The full extract creates the following files:

   blackcat_<date>_<#>.mrc
   blackcat_<date>_bound_<#>.mrc
   blackcat_<date>_max_exceed.mrc
   blackcat_<date>_holdings_exceed.mrc
   

5. Run the full index on the Blackcat server.

6. On the Symphony side, turn on the Sirsi cron job that sends records over to Blackcat.

   #5,35 * * * * /sirsi/Psuscripts/Scripts/move_edited_records_to_blackcat >> /sirsi/Unicorn/PSULinkages/Blackcat/cron_log 2> /sirsi/Unicorn/PSULinkages/Blackcat/cron_error_log
   

Incrementals

The incrementals are the add/updates and delete extracts from Symphony. The process is similar to full extract.

TBD

• What happens if an item changes libraries? For instance, if a BOOKFLOAT item is checked into a different library, the item library changes (ie, the library changes). Since any change to the library (item library in Workflows) forces a re-index of the catalog record by putting the catalog key into /sirsi/Unicorn/Database/Edit, the incremental extracts will pick up records for re-indexing.

Translation Maps

TBD

---

Note: These processes will be elaborated as the process is fully tested and implemented.