De-dedups bookplate metadata further upstream

[shelleydoljack]

Fixes #1390

De-dups the bookplate metadata, starting with the bookplate_funds_polines task, which will return this data structure:

{
      "5513c3d7-7c6b-45ea-a875-09798b368873": {
        "bookplate_metadata": [
          {"fund_name": "...", "druid": "...", "image_filename": "...", "title": "..."},
          {"fund_name": "...", "druid": "...", "image_filename": "...", "title": "..."},
        ]
      },
      ...
 }

By turning the return of this task into a dict, with the po-line ID as key, we can also avoid extra calls to orders-storage/po-lines when getting the po line data. And since it could be the case where different po lines reference the same instance ID, we end up de-duping the bookplate metadata again in the instances_from_po_lines task (the test_instances_from_po_lines tests this scenario).

[jgreben]

I'd like to understand the tests better and the need for the explicit sorting in the tests.

[jgreben]

I am confused here. Should the instances_dict already be sorted and deduped because it calls the instances_from_po_lines function? If so why is it being forcibly sorted here?

[shelleydoljack]

You're right, we shouldn't need to sort the outcome of the function (that does call a sort to dedup). Let me take out the sorting; it seems unnecessary.

[jgreben]

Is this being sorted here only in order to assert the comparison for each row, or is this somehow related to the dedup function?

[shelleydoljack]

This sorting was unnecessary and wrong since the _dedup_bookplates function is sorting the bookplate dictionaries by key, value pairs when transforming to a tuple. Here, the mock_bookplate_metadata was sorting only the values. I removed the sorting altogether for the tests.

[shelleydoljack]

I found a small-ish issue with the trigger_digital_bookplate_979_task as I tested running the DAG with the LINDER fund in dev. The log for the dag run shows INFO - Total incoming instances 18 but this is the number of mapped tasks from the previous task. I fixed it in this commit.

I also removed the 979s that were in the records https://folio-test.stanford.edu/inventory/view/ca2c1623-bbe5-40fb-b724-e22aacf8a007 and https://folio-test.stanford.edu/inventory/view/66d4d0a9-f4bc-58de-a4b9-02b0f439ee65 before running the digital_bookplate_instances dag with the LINDER fund config. The digital_bookplate_979 dag runs are still running, but there are currently only 1 979 in each of those records nows.