Gwen C. Katz
@gwenckatz
Mon Apr 12 16:54:58 +0000 2021

Colorization APIs are becoming widespread; AI-colorized historical photos are circulated without caveat. But is AI colorization providing an accurate image of the past? To find out, I digitally desaturated these color photos by Sergey Prokudin-Gorsky, taken between 1909 and 1915. https://t.co/YzJO2b0aza

I then colorized the photos using the DeepAI Image Colorization API: https://t.co/5XFGTEieIA

Here's the AI's attempt to color these photos. https://t.co/SVr4dKsnZL

And here are the original color photos. https://t.co/SN3wcOclSt

AI colorization strips away the vibrant colors from history and replaces them with a world of dull tans, muddy browns, and slate grays. (Notice how it removes the painted railing.) It reinforces our impression that the past was drab and lifeless--in contradiction of reality. https://t.co/ulTRs11XFr

We can't let AIs that propagate our own biases define our image of the past. Colorization should be left to human experts who can use context to pick accurate colors. Look to primary sources, such as paintings, to see the real colors of the past. Because it wasn't all mud. https://t.co/CiTp3TAZCE

The photos are:
"Group of Jewish children with a teacher. Samarkand"
"Palace. Another facade. Bukhara"
"Peasant Girls"
"Clothing Display"

Prokudin-Gorsky created them by taking three separate exposures with red, green, and blue filters. The prints were composited digitally.

If you'd like to see more of Prokudin-Gorsky's photos of Russia, Central Asia, and the Caucasus in the final years before industrialization, the entire collection of almost 2500 photos is available digitally from the Library of Congress:

https://t.co/S78jUuGdkV

Some more on why AIs make these mistakes, and how the problem is more intractable than just "Developers told the AIs to make everything brown."

https://t.co/1kVbt6oW6T

One of the wonderful volunteers who digitized Prokudin-Gorsky's photos!

https://t.co/TpMwA5FW4i

Another bot has a go. It does better on the peasants but is still utterly defeated by the palace.

This one seems to have a lot of difficulty detecting the edges of garments, so the colors are brighter but very patchy.

https://t.co/STzHaasbQO

And speaking of Russia, if you like Russian history, I wrote a novel about it that you can read!

https://t.co/7a5Ofc0E6Z

Since you all are interested, here are a few more sets of photos (original/BW/colorized).

"Haying, near rest time." https://t.co/25F7M7egsl

"Group of workers harvesting tea. Greek women." https://t.co/ZCyjUM1FEs

"Four people seated on a carpet, in front of a backdrop of textiles."

Notice that the AI successfully divines that red and green are present, but inverts them! https://t.co/KFrgaAvOL7

Final note: The API was trained on desaturated color photos. This is what it should be best at. If it fails at these images, it will fail far worse at natural grayscale photos. And if desaturated photos are inherently nonrepresentative, the API is fundamentally flawed. https://t.co/ysl9Y9gZt1

Tue Apr 13 18:29:54 +0000 2021