Download Provenance Texts Dataset: CSV
Text Analysis Challenge: Detect Looted Art.
Help Automate Analysis, Flagging and Ranking of Museum Art Provenance Texts by the Probability of a Hidden History
The Question: How to sift through the millions of objects in museums to identify top priorities for intensive research by humans?
The Goal: Automatically Classify and Rank 60,000+ art provenance texts by probability that further research will turn up a deliberately concealed history of looting, forced sale, theft or forgery.
The Challenge: Analyse texts quickly for Red Flags, quantify, detect patterns, classify, rank, and learn. Whatever it takes to produce a reliable list of top suspects
For this challenge several datasets will be provided.
1) DATASET: 60,000+ art provenance texts for analysis
2) DATASET: 1000 Red Flag Names
(example)
3) DATESET: Red Flag Words or Phrases
What digital methods can we use - and put into practice during the Hackathon - to "diagnose" these texts and prioritize them for "treatment" (ie, additional provenance research)?
- IDENTIFY Red Flag Names and Words in each Text?
- COUNT Red Flag Names and Words in each Text?
- CHARACTERIZE each Text (number of words? sentiment? completeness v gaps? other features to be identified that may be useful)?
- ANALYZE for patterns, links and networks?
- CALCULATE probability that the provenance conceals a Nazi-era history that will prove problematic if investigated in detail
- RANK according to urgency for further in-depth provenance research
(Voyant-Tools Whitelist containing names:
keywords-14ca7131716f24c62c6529fcc143bbd2 )
What might a successful result look like?
- A list of 50 provenances from the DATASET ranked most likely to conceal looted art
- A color-coded evaluation of each provenance (RED, ORANGE, GREEN) by likelihood of concealing looted art
- Instructions how to analyze the provenances with the tools, functions or code to use (for example, how to use Voyant-Tools to count all the Red Flag Names and inject the result back into the spreadsheet)
- Ideas for going further....
Triage: "assignment of degrees of urgency to wounds or illnesses to decide the order of treatment of a large number of patients or casualties."
Link to Glamhack2020 project for participants
https://hack.glam.opendata.ch/project/7
Help us to test and improve and test and improve the code!
Link to Code on Githubhttps://github.com/parisdata/GLAMhack2020
Issues to resolve:
- While the word list counts and general name extraction seem to work pretty well, reconciliation with the list of 1000 Red Flag Names still needs work. To test: A tighter tolerance combined with a more complete listing of alias might help.
- The extraction of transaction years after 1900 gives interesting though incomplete indicators. The decision to exclude dates in parentheses (as they are sometimes biographical dates and not transaction dates) needs to be reviewed and refined.
(note: The texts are from multiple sources applying multiple formats and deliberately entered exactly as is)
Results RAW (with uncorrected errors for analysis)