Sources quoted by authorities to clamp down on the art market rarely stand up to scrutiny
How does false data come to influence policy and even law making on such a widespread basis when it comes to cultural property?
One reason is confirmation bias: if the results of your research match what you hope to find, you are less likely to check their validity – a point made by statistics guru Dr Tim Harford when discussing claims made about antiquities and crime.
Another can be the authority of the source. This is very common in the cultural heritage sphere.
This two-part article analyses two studies from what should be an impeccable single source, showing how false data can spread from one official report to another to gain traction, and ultimately become an unchallenged authority among those who should know better.
They also demonstrate that many apparently learned pieces of research published by acknowledged authorities simply can’t be trusted, because it is clear that these professionals are not checking their sources adequately.
Both reports were published by the United Nations Office of Drugs and Crime (UNODC).
One was titled PRACTICAL ASSISTANCE TOOL to assist in the implementation of the International Guidelines for Crime Prevention and Criminal Justice Responses with Respect to Trafficking in Cultural Property and Other Related Offences. It was published in 2016.
The two relevant claims it included were as follows:
• The Museums Association has estimated that profits from the illicit antiquities trade range for $225 million and $3 billion per year.
AND
• The Organized Crime Group of the United Kingdom Metropolitan Police and INTERPOL has calculated that profits from the illicit antiquities trade amounted to between $300 million and $6 billion per year.
Footnotes indicated the source for each of these statements.
For the first it was “See Neil Brodie, Jenny Doole and Peter Watson, Stealing History: The Illicit Trade in Cultural Material (Cambridge, McDonald Institute for Archaeological Research, 2000); and Simon Mackenzie, “Trafficking antiquities” in International Crime and Justice, Mangai Nataajan, ed. (Cambridge, Cambridge University Press, 2011).”
For the second it was “United Kingdom, House of Commons, Culture, Media and Sport Select Committee, Cultural Property: Return and Illicit Trade, seventh report, vols. 1, 2 and 3 (London, 2000).”
These were very precise references, if rather out of date for a 2016 report by the UNODC.
The problem is that whoever researched the UNODC report failed to check where its quoted sources got their data from. If they had, they would have found the following:
– The Brodie, Doole and Watson report from 2000 did not refer to the Museums Association $225 million and $3 billion per year claim at all. Instead, on page 23 in the introduction to section 1.9, The Financial value of the illicit trade, it stated: “Geraldine Norman has estimated that the illicit trade in antiquities, world-wide, may be as much as $2 billion a year.” The footnote for this statement identified the source as journalist Geraldine Norman’s November 24, 1990, Independent article Great sale of the century. However, apart from the fact that the article was actually titled Great sale of the centuries, it included no such claim or figure.
The Simon Mackenzie chapter on Trafficking Antiquities is not open source data, but is available on subscription to CUP.
– In fact, the Museums Association did give estimated figures as part of its evidence to the UK House of Commons, Culture, Media and Sport Select Committee, Cultural Property: Return and Illicit Trade, seventh report, vols. 1, 2 and 3 (London, 2000) – the same source as the second claim quoted by the UNODC. In the Seventh Report, Chapter II The problem of illicit trade, The nature and scale of illicit trade, paragraph 9 reads: “The scale of the illicit trade taken is said to be very considerable. According to the Museums Association, as an underground, secretive activity, it is impossible to attach a firm financial value to the illicit trade in cultural material. Estimates of its worldwide extent vary from £150 million up to £2 billion per year.” The Museums Association gave as its source the Brodie, Doole and Watson 2000 report, quoted above, which in turn gave the Geraldine Norman article as the source, when, in fact, it provided no such figures.
So, the Museums Association’s actual claim was that “it is impossible to attach a firm financial value to the illicit trade in cultural material”, but that estimates worldwide [by others] varied greatly between £150 million and £2 billion.
This was rather different from the UNODC claim based on this source: “The Museums Association has estimated that profits from the illicit antiquities trade range for $225 million and $3 billion per year.”
To summarise, then, the £150 million to £2 billion claim ultimately came from nowhere. Its claimed primary source, the Geraldine Norman article from 1990, quoted no such figures. The secondary source which mistakenly quoted them was the Brodie, Doole & Watson report from ten years later in 2000, leading to the tertiary source of the Museums Association. In turn, this was quoted by the UNODC in 2016 – 26 years after the Norman article which gave no figures anyway. The UNODC report then became a new ‘primary’ source, with the figures quoted as UNODC estimates, which they weren’t at all.
Recent Comments