DATASET: Text Analysis Challenge - Detect Looted Art - GLAMhack2020

Download  Provenance Texts Dataset: CSV 

Text Analysis Challenge: Detect Looted Art.

Help Automate Analysis, Flagging and Ranking of Museum Art Provenance Texts by the Probability of a Hidden History

The Question: How to sift through the millions of objects in museums to identify top priorities for intensive research by humans? 

The Goal: Automatically Classify and Rank 60,000+ art provenance texts by probability that further research will turn up a deliberately concealed history of looting, forced sale, theft or forgery. 

The Challenge: Analyse texts quickly for Red Flags, quantify, detect patterns, classify, rank, and learn. Whatever it takes to produce a reliable list of top suspects

For this challenge several datasets will be provided.

1) DATASET: 60,000+ art provenance texts for analysis

(example)

https://www.harvardartmuseums.org/collections/object/296887	[Pierre Matisse Gallery, New York, New York], by 1932, to M. Gutmann, 1936. Maurice Wertheim, by 1937, bequest, to Fogg Art Museum, 1951.;NOTE: Provenance derived from "Degas to Matisse: The Maurice Wertheim Collection," John O'Brian, Harry N. Abrams, New York, 1988.	1951.76
https://www.harvardartmuseums.org/collections/object/229045	Private Collector, Paris, Said to have been bought directly from artist, 1918.;Maurice Wertheim, Purchased from the Valentine Gallery, 1937, Bequest to Fogg Art Museum, 1951.	1951.52
https://www.harvardartmuseums.org/collections/object/229044	Paul Guillaume, Paris, France, 1925, 1935. By 1930, per caption of photo, Paul Guillaume dining room dated c. 1930, Georgel 2006;Maurice Wertheim, 1937, Bequest to Fogg Art Museum, 1951.	1951.51
https://www.harvardartmuseums.org/collections/object/229043	Unidentified owner, Paris, sold, [through Hôtel Drouot, Paris, June 16, 1906, no 30.]. Dr. Alfred Wolff, Munich, (1912). Sir Michael Sadler, England, (1912). [De Hauke & Co., New York], sold, to A. Conger Goodyear, New York, (1929-1937) sold, [through Wildenstein && Co., New York];to Maurice Wertheim, New York (1937-1951) bequest, to Fogg Art Museum, 1951.	1951.49

2) DATASET: 1000 Red Flag Names

(example)

Bignou, Etienne	Bignou	Etienne	Etienne Bignou
Billiet, Director	Billiet	Director	Director Billiet
Binder, Dr. Moritz Julius	Binder	Dr. Moritz Julius	Dr. Moritz Julius Binder
Bing Collection	Bing Collection		Bing Collection
Birtschansky, Zacharie	Birtschansky	Zacharie	Zacharie Birtschansky
Bisson, E.	Bisson	E.	E. Bisson
Blanc, Pierre	Blanc	Pierre	Pierre Blanc
Bleye, Willi	Bleye	Willi	Willi Bleye
Bloch, Dr. Vitale	Bloch	Dr. Vitale	Dr. Vitale Bloch
Bloch-Bauer Collection	Bloch-Bauer Collection		Bloch-Bauer Collection
Blot	Blot		Blot
Bode, Dr.	Bode	Dr.	Dr. Bode
Bodenschatz, General Karl	Bodenschatz	General Karl	General Karl Bodenschatz
Boedecker, Alfred	Boedecker	Alfred	Alfred Boedecker
Boehler, Julius, Jr.	Boehler	Julius	Julius Boehler
Boehler, Julius, Sr.	Boehler	Julius	Julius Boehler
Boehm, Dr. Franz	Boehm	Dr. Franz	Dr. Franz Boehm
Boehmer, Bernhard	Boehmer	Bernhard	Bernhard Boehmer

3) DATESET:  Red Flag Words or Phrases

(example)

flaguncertainty	flaganonymity	flagpuncutation	flagmove	flagreliability
likely	private collector	?	transfer	telephone
probably	anonymous	[	removed	to at least
possibly	art market			until at least
maybe	unidentified			by 19
?	unknown			before 19
	property of a European collector			according to
	private collection
	property of a lady
	anon.

You're the doctor and the texts are your patients! Who's in good health and who's sick? How sick? With what disease? What kind of tests and measurements can we perform on the texts to help us to reach a diagnosis? What kind of markers for should we look for? What kind of patterns? 

What digital methods can we use - and put into practice during the Hackathon - to "diagnose" these texts and prioritize them for "treatment" (ie, additional provenance research)?

• IDENTIFY Red Flag Names and Words in each Text?
• COUNT Red Flag Names and Words in each Text?
• CHARACTERIZE each Text (number of words? sentiment? completeness v gaps? other features to be identified that may be useful)?
• ANALYZE for patterns, links and networks?
• CALCULATE probability that the provenance conceals a Nazi-era history that will prove problematic if investigated in detail
• RANK according to urgency for further in-depth provenance research 

Download  Provenance Texts Dataset: CSV 

Download ALIU INDEX of Red Flag Names: CSV

(Voyant-Tools Whitelist containing names: 

keywords-14ca7131716f24c62c6529fcc143bbd2  )

Download Words Flags to be counted in provenances: CSV 

What might a successful result look like?

• A list of 50 provenances from the DATASET ranked most likely to conceal looted art
• A color-coded evaluation of each provenance (RED, ORANGE, GREEN) by likelihood of concealing looted art
• Instructions how to analyze the provenances with the tools, functions or code to use  (for example, how to use Voyant-Tools to count all the Red Flag Names and inject the result back into the spreadsheet)
• Ideas for going further....

Triage: "assignment of degrees of urgency to wounds or illnesses to decide the order of treatment of a large number of patients or casualties."

Link to Glamhack2020 project for participants
https://hack.glam.opendata.ch/project/7

Help us to test and improve and test and improve the code!

Link to Code on Github 

https://github.com/parisdata/GLAMhack2020

Issues to resolve:
- While the word list counts and general name extraction seem to work pretty well, reconciliation with the list of 1000 Red Flag Names still needs work. To test: A tighter tolerance combined with a more complete listing of alias might help.
- The extraction of transaction years after 1900 gives interesting though incomplete indicators. The decision to exclude dates in parentheses (as they are sometimes biographical dates and not transaction dates) needs to be reviewed and refined. 
(note: The texts are from multiple sources applying multiple formats and deliberately entered exactly as is)

Results RAW (with uncorrected errors for analysis)

Results CSV RAW

DATASET: Art Collectors who died or were detained in Nazi Concentration Camps, Ghettos or Jails

In this post, we publish a tiny subset of names that should immediately set off alarms when they appear in the provenance of an artwork:

Art collectors, dealers, curators, historians, and other art market professionals who were detained or died in Nazi custody, whether in concentration camps, ghettos, or jails.

This dataset is drawn from Wikidata Queries (see below), and so is necessarily limited to those very few names that have been properly documented in Wikidata, with occupation, date and place of death.

DATASET: Art Collectors who were detained or who died in Nazi Concentration Camps, Ghettos or Jails referenced in Wikidata

Version: 1

Licence: CC0

Date published: May 27, 2020

Publisher: Open Art Data

Source of information: Wikidata Query

File formats:

• CSV

DOWNLOAD CSV

• Google Sheet

VIEW GOOGLE SHEET

• RDF Turtle

(to be published)

Description:

In this first version, published on May 27, 2020, the Dataset contains only 47 names. (see below) with the following information:

Fields in the DATASET

Wikidata url

Wikidata Qcode

Name

pic

Detention P2632

PlaceDetention

DeathDate

PlaceDied

Died P20

Birth P19

PlaceBorn

BirthDate

VIAF_ID

Library_of_Congress_authority_ID

GND_ID

ULAN_ID

FirstName

Lastname

To appreciate how extremely incomplete this dataset is, one can compare it to the 351 names of Jewish collectors on the German Lostart site or to the 371 names (for France alone) of victims of spoliation listed in France's ERR database, or the over three million names in the central database of Yad Vashem.

It is obvious, then, that this tiny dataset makes no claim to completeness. Its purpose is rather to provide an angle of attack for linking up information about murdered Jewish collectors to other information, including provenance information in artworks. 

Each name has a Wikidata identifier, which serves as a linked data hub to other identifiers, authority files and references such as Viaf, Library of Congress, DNB, Dictionary of Art Historians, Lostart, Yad Vashem, Wikipedia, DBpedia, Ulan, etc.

At present, very few Jewish collectors who perished in the Holocaust are properly referenced and linked. 

But every journey begins with a first step. 

The DATASET presented  here is such a first step: to link Jewish art collectors and art professionals who were detained or died in Nazi camps or ghettos into the world of linked data via Wikidata, and thus, hopefully, to help anchor them in the digital Knowledge Graph.

A further observation:
In describing datasets it is common to inventory what is present. In describing datasets related to the Holocaust, however, it is also important to inventory what is absent. This is to avoid the frequent error of imagining that what we see is all there is to know, and to have an idea of the scope of the work that remains to be done.

Extract of Dataset:

List of Forty-Seven Names of Art Collectors, Dealers, Curators, Historians who Died or were Detained in Nazi Camps or Ghettos, or Jails

Wikidata Qcode	Name	BirthDate	VIAF_ID	Library of Congress authority ID	GND_ID	ULAN_ID
Q2546745	Walter Westfeld	1889-03-04	81690025		137508662
Q1039376	Carl Laszlo	1923-07-16	2511905	n83226831	118569996
Q1515593	Gertrud Kantorowicz	1876-10-09	18030172	n86032621	119368153
Q55676461	Jerzy Langman	1903-07-31	61280166	no2005063216	127273298
Q15995945	Wilhelm Mautner	1889-11-28	181186070		1013999878
Q1913457	Max Silberberg	1878-02-27	50130833		123274826
Q98887	Henri Hinrichsen	1868-02-05	54912104	no96023216	116897899
Q54501351	Max Frankenburger	1860-08-27	5294492	no98080821	116716649
Q19236821	Walter Cohen	1880-02-18	40183001	nr95021800	119198592
Q14777672	Ludwig Pollak	1868-09-14	39488480	nr89003403	118838172
Q89993	Robert Eisler	1882-01-01	763850	nr97031281	116435526
Q86225	Arthur Mahler	1871-08-01	2827185		1110216106
Q11779916	Mieczysław Paszkiewicz	1925-02-10	26463	n81056769
Q66832	Franz Roh	1890-02-21	34587588	n50048042	118602152	500048070
Q28192906	Q28192906	1863-05-03	54896846		116308257
Q762753	August Liebmann Mayer	1885-10-27	102317582	nr88011775	117542563	500322318
Q6280128	Josef Rosensaft	1911-01-15	20488355	n86056702
Q19258964	Alfred Werner	1911-03-31	56619941	n79148949	130178594
Q122226	Jan Krugier	1928-05-12	45150837	n85212724	121187519
Q62070061	Olga Bloch	1900-08-30
Q3085081	François Lang	1908-02-27	27877412	nr93019463
Q5503811	Friedrich Gutmann	1886-11-15	3375432		123271770
Q15433563	Lisa Hamburg	1890-09-10	306223608		1107446635
Q1597216	Heinrich Feurstein	1877-04-11	57367931	no2016109679	116484640
Q1598924	Heinrich Stahl	1868-04-13	68530888	no97004669	136897487
Q63110939	Wilhelm Kurtz	1897-08-20	316495431		1072437112
Q3174304	Jean Riboud	1919-11-15	18117722	n83185011	122914171
Q6202160	Jindřich Waldes	1876-07-02	55020467	n00009745	122092759
Q94971510	Sigmund Fein	1880-07-09
Q2038252	Otto Bernheimer	1877-07-14	64750637	no2004014234	116146672
Q62579	Joachim Ernst, Duke of Anhalt	1901-01-11
Q11779942	Mieczysław Porębski	1921-03-31	118048919	n85827144	119251752
Q468466	Karolina Lanckorońska	1898-08-11	76324812	n80001031	122554140
Q669893	Georges Mandel	1885-06-05	49264696	n91053655	119078872
Q87857	Fritz Grünbaum	1880-04-07	74077040	n86138300	119368129
Q47500000	Anton Mayer	1879-04-22	80994143	nr2005025243	117542393
Q9354188	Tadeusz Dobrowolski	1899-08-17	117652610	n80145073
Q47512272	Ludwik Rajewski	1900-12-16	93474687	n84219822
Q7085852	Ole Henrik Moe	1920-01-11	311036799	n82212059	13623674X
Q19753989	Karl Freund	1882-01-01	44770206	n90699618	140496785
Q3426274	René Gimpel	1881-01-01	91549548	n86036259	1044616733
Q9342555	Stanisław Małkowski	1889-08-22	102102730
Q60040474	Adam Abel	1886-01-01	68427564		116553375
Q55006961	Heinrich Rieger	1868-12-25	297121379	no2008119220	131888331
Q42887282	Lucien Graux	1878-04-04	9881824	n86022614
Q9138146	Abe Gutnajer	1888-01-01	303116224
Q88803939	Leo Grünstein	1876-07-18	69694062	no2008136862	116893001

About the Wikidata Queries and data preparation

Two queries were used:

1) for art people detained in Nazi camps or Ghettos
and 

2) for art people who died in Nazi camps or Ghettos

Art People

{ ?item wdt:P106 wd:Q1792450.} UNION { ?item wdt:P31 wd:Q1007870. } UNION { ?item wdt:P106 wd:Q173950.} UNION { ?item wdt:P921 wd:Q328376.} UNION { ?item wdt:P106 wd:Q10732476.} UNION { ?item wdt:P106 wd:Q446966.} UNION { ?item wdt:P106 wd:Q22132694.} UNION { ?item wdt:P106 wd:Q674426.}

(Please note that artists are not included in this selection.)

Died in Camp

{ ?placedied wdt:P31 wd:Q328468.} UNION { ?placedied wdt:P31 wd:Q152081. } UNION { ?placedied wdt:P31 wd:Q153813.} UNION { ?placedied wdt:P31 wd:Q153813.} UNION { ?placedied wdt:P31 wd:Q2583015.}}

Detained in Camp

{ ?place_detention wdt:P31 wd:Q328468.} UNION { ?place_detention wdt:P31 wd:Q152081. } UNION { ?place_detention wdt:P31 wd:Q153813.} UNION { ?place_detention wdt:P31 wd:Q153813.} UNION { ?place_detention wdt:P31 wd:Q2583015.}

Fields Added Manually

Last Name, First Name