Tips & tricks to batch edit EXIF metadata of photos

I have thousands of photos in a perfect, organized state with correct EXIF metadata. However, this was not the case couple of years ago.

Not all of my photos had proper EXIF data. Some had EXIF metadata but were incorrectly named, some didn't have EXIF metadata, but I knew the time they were taken and most of them had filenames with random numbers based on the camera
shutter count.

Because I didn't like the state of all my photos, I've decided to fix them. During this time, I've gathered a set of small, short code pieces that allows me to change anything on a set of photos.

Before we dive in, please be cautious with the commands, some of them are disastrous. Always do a backup before you start working manipulating the data. One final reminder, all the commands are only used on macOS, it might be different based on what OS you use.

Now, let's dive in:

Install tools

exiftool is a command line tool that allows you to change and manipulate the EXIF metadata of images. On macOS you can easily install it with:

$ brew install exiftool

detox is a utility designed to clean up filenames. It replaces difficult to work with characters, such as spaces, with standard equivalents. It will also clean up filenames with UTF-8 or Latin-1 (or CP-1252) characters in them (i.e: foo bar - quz.avi -> foo_bar-quz.avi). This will come handy for scripting because you don't have to escape white spaces in your files. To install:

$ brew install detox

Make a backup of your photos folder

Instead of using cp, it's better to use a tool that is faster and copes better with file permissions. To make a backup of your photo folder run:

rsync -vaE --progress photos photos_backup

These flags mean:

• v increases verbosity.
• a applies archive settings to mirror the source files exactly, including
  symbolic links and permissions.
• E copies extended attributes and resource forks (OS X only).
• progress provides a countdown and transfer statistics during the copy

Remove all white spaces in filenames

Assuming all your photos are in one place, to remove all white spaces recursively, run the following:

detox -r -v photos/

Find images without an EXIF metadata

The first mistake when editing batch EXIF metadata is assuming that all images have EXIF metadata. That's not always the case. To find all images (in JPEG) without any date run the following, recursively:

$ cd photos
$ exiftool -p '$filename' -r -if '(not $datetimeoriginal) and $filetype eq "JPEG"' . > nodates.txt
  157 directories scanned
23507 files failed condition
 1641 image files read

verify:

$ cat nodates.txt | wc -l
  2409

Set EXIF date based on file modify date

Suppose a photo does have an incorrect date or no EXIF metadata. But you know that the modified date of the file is correct because that's is the date the photo was created. To update the EXIF date based on this date, call:

$ exiftool -v "-FileModifyDate>AllDates" *

Change EXIF date by shifting timezone

Sometimes I forgot to change the time of my camera when I travel and I shoot in my local timezone. When this happens, I have to change all my photos and update it with the correct timezone. The following examples shifts the time by -11 hours (suppose local time is 10:00 PM, Photos will be adjusted to 11AM (previous day)). Change it according to your own needs:

$ exiftool "-DateTimeOriginal-=0:0:0 11:0:0" .

Set the file modification date based on EXIF date

With the previous action, the EXIF date might be correct, but now the modified date is set to the date the tool was invoked. To fix the modified date of the file that was destroyed with the previous action, run:

$ exiftool -v "-DateTimeOriginal>FileModifyDate" *

Set EXIF date based on the file name

If you have a file with the name foo_20180418_103000.jpeg (which is in the format YYYYmmdd_HHMMSS), but the EXIF date is incorrect, you can update the EXIF date from the filename with:

$ exiftool "-AllDates<filename" *

The format needs to be similar to the format above

Set file name based on the EXIF date

If you have a correct EXIF date, you can easily rename all files based on the EXIF date of the file:

$ exiftool '-FileName<DateTimeOriginal' -d %Y%m%d_%H%M%S%%-c.%%e .

Set file name based on the file modified date

If you have a correct modified date, you can easily rename all files based on the modified date of the file:

$ exiftool '-FileName<FileModifyDate' -d %Y%m%d_%H%M%S%%-c.%%e .

Rebuild and fix corrupted EXIF metadata

Sometimes exiftool refuses to operate on a photo because of the EXIF metadata being corrupted. To fix all photos recursively, run:

$ exiftool -all= -tagsfromfile @ -all:all -unsafe -icc_profile -F .

Filter out non JPEG files

Your photos/ folder might contain various kind of data, such as videos, gif, png images, etc.. Let's filter out all non JPEG files.

The file let us find the mime type:

$ file --mime-type ./2013/05/20130513_0212.png

gives us this:

./2013/05/20130513_0212.png: image/png

let's pipe it to awk to get the filename:

$ file --mime-type ./2013/05/20130513_0212.png | awk '{print $1}'

gives us:

./2013/05/20130513_0212.png:

to strip last char (:) we use substr

$ file --mime-type ./2013/05/20130513_0212.png | awk '{print substr($1, 1, length($1)-1)}'

this gives us:

./2013/05/20130513_0212.png

To pick up files that are not of mime type image/jpeg, we can use $NF builtin variable, which prints the last field in a line and compare it to image/jpeg. If it's not equal to it, we're going to continue parsing the file:

$ file --mime-type ./2013/05/20130513_0212.png |  awk '{if ($NF != "image/jpeg") print substr($1, 1, length($1)-1)}'

which gives us:

./2013/05/20130513_0212.png

Now to list all files with inside the current directory you should run:

find . -type f -exec file --mime-type {}  \;

combining this with the awk directive above and direct the result to a file called nonjpeg.txt, we get:

$ find . -type f -exec file --mime-type {}  \; | awk '{if ($NF != "image/jpeg") print substr($1, 1, length($1)-1)}' > nonjpeg.txt

As a bonus, this is how you can iterate over the files in nonjpeg.txt and move it to a new folder called others:

$ mkdir others

$ while read p; do
  mv "$p" others/
done <nonjpeg.txt

Recursive action: * vs .

Some of the example above uses * or . for selecting all files recursively. The difference is that * is a bash expansion. If you have hundreds of files, this won't work as it fails after a threshold. Using . is better, because in that case exiftool automatically selects the files.

Moving without overwriting

You might want to move files to a folder after fixing the issues. It's possible to have a duplicate filename due to how your EXIF metadata was created.

To avoid overwriting with mv, use the -n flag. This will not move if a field with the same name exists. As a bonus, I also use -v so it shows me the progress (this is not very useful actually as it's moving very fast, but I see that mv is working when I use it in a script, iterating over thousands of files)

Find files newer than given date

You can use find to get a list of files that are newer than the given data. For example, the command below will give you a list of all files that are newer than Jun 18, 2017:

$ find . -type f -newermt "jun 18, 2017" > newfiles.txt

I've found these little snippets very useful for a folder with thousands of
unorganized photos with corrupted and missing EXIF metadata. These are meant to be combined with each other depending on the files you have. Use one that fits your workflow.