Using Civil War Photo Sleuth

Using Civil War Photo Sleuth harnesses facial recognition technology and crowdsourcing to identify people in Civil War images. Do you have unidentified Civil War photos? Read on!

Creating the Software

Virginia Tech computer science professor Kurt Luther created Civil War Photo Sleuth. With the help of colleagues and graduate students, the site launched in August 2019.

Results over time using Civil War Photo Sleuth (courtesy civilwarphotosleuth.com)

Identifying people in historical photographs is important for researching family history, correcting the historical record, and increasing understanding of the war. But it is also a complex and challenging task, according to Luther and Virginia Tech’s Crowd Intelligence Lab:

The Civil War [was] the first widely photographed conflict. Many thousands of these portraits survive, but only ten to twenty percent are identified. We created Photo Sleuth, a web-based platform that combines crowdsourced human expertise and automated face recognition to support Civil War portrait identification. …One month after its public launch, [the site demonstrated] that it helped users successfully identify unknown portraits and provided a sustainable model for volunteer contribution.

While facial recognition features get the most attention, Luther believes credit is also due to more than 4,000 users who have added photos and helped identify the subjects of portraits. More than 85 percent of proposed identifications were “probably or definitely correct,” according to an early analysis by Luther and his students.

Using Civil War Photo Sleuth

How It Works

Time magazine writes:

…[T]he facial recognition technology for Civil War Photo Sleuth uses a set of 27 facial landmarks, such as the corners of the mouth or the tip of the nose, to analyze a given photo. Faces have different proportions, so the software calculates various distances between facial landmarks in the uploaded portrait — such as the distance between left pupil and right pupil — and then looks for photos with similar distances between different facial features. (Sometimes discoloration or holes in such old images can obscure these facial landmarks and prevent identifications.)

The site then pulls up previously uploaded images that match the new one, with the hope that one of the matches will help identify the man in the picture.

The more photos are uploaded, the more likely a match will be found, and the Civil War Photo Sleuth team adds Library of Congress photos and other collections to enhance the probability of matches. Since launching in August 2018, nearly 30,000 photos have been added to the site by nearly 14,000 registered users, including employees at the Library of Congress and the National Archives. A little over 3,300 identifications have been made, including photos in the collections of the New York Public Library, the Massachusetts Historical Society and the Henry Ford museum in Michigan. The site also won Microsoft’s $25,000 Cloud AI Research Challenge and a National Science Foundation grant.

For more Sassy Jane Genealogy posts on Civil War research, click here. All images in this post courtesy of civilwarphotosleuth.com.