Search:  
Gentoo Wiki

TIP_Organizing_digital_photos

Contents

Intro

From one obsessive compulsive pack rat to another, here's a little trick / convention that I use to organize / manage my digital photos:

First I just copy them to the hard drive from my camera or download them from my e-mail. Then put them in folders. Then on occasion I run a script that moves all of the photos into a permanent chronologically ordered directory but leaves symlinks behind for me to muck around with. On top of that I use a web-based program to share them with family and friends.

So for this project we'll use mainly three tools - picdir.sh, Gallery2, and RenRot

picdir.sh

This script will by default sift through all files in your home directory, move any JPEG / MP4 / AVI / 3GP files into a special directory which is ordered chronologically while leaving symlinks in place of the originals and also deletes any duplicates (but again leaves symlinks).

The script supports images and videos from my mobile phone (also older Nokia phones which store the date and time in comments). AVI video files from digital cameras can be sorted according to the video creation time (I didn't find that AVI files have the creation timestamp included in the header). The script also asks user to input a date and time for images which don't have timestamps and optionally to create EXIF header and save the information there.

Code: Get what you need
 emerge jhead mpeg4ip exiftool
 mkdir ${HOME}/bin
 nano -w ${HOME}/bin/picdir.sh # insert script below
 sudo ln -s ${HOME}/bin/picdir.sh /usr/local/bin/

 picdir.sh /media/HP733_CARD/ /home/USER/Pictures/tmp/
File: picdir.sh
#!/bin/bash

# Picdir v0.9 rc2

# Copyleft 2005 Gentoo-Wiki.com - This software is OpenSource
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

# Guaranteed to be my personal best!
# ...not guaranteed to work...


# What will this do?
# Search for timestamps in all JPEG, 3GP, MP4 and AVI files within a directory
# Use the timestamp to relocate and rename the file
# Using checksums, determine and remove duplicate files
# Uniquely name files which have the same timestamp but different data

# needs BASH FIND GREP CUT FILE DATE JHEAD MP4DUMP MD5SUM MV SED WC EXIFTOOL

TIMEZONE="0"
OPERATION="mv"

if [ ! -n "${1}" ]; then
        echo "USAGE: ${0} [OPTIONS] SOURCE [DESTINATION]"
        echo "Try ${0} -h for help"
        exit
fi

while [ -n "${1}" ]; do
case $1 in
    -h | --help)
        echo "Organizes pictures and videos with EXIF/MP4/3GP/AVI timestamps"
        echo ""
        echo "USAGE: ${0} [OPTIONS] /my/pictures/unsorted/ [/my/pictures/sorted/]"
        echo ""
        echo "Optional arguments:"
        echo "   -m, --move    (default) moves files to destination"
        echo "   -L, --no-ln   just move, don't leave symlinks"
        echo "   -c, --copy    copies files to destination instead of moving (implies -L)"
        echo "   -h, --help    shows this help"
        echo "   -e, --exif    creates EXIF date for images which don't have it"
        echo "   -q, --quiet   be quiet and don't bother the user"
        echo ""
        echo "If no DESTINATION directory is given, files will be moved/copied"
        echo "${HOME}/.Pictures/"
        echo "(Note that it's a hidden directory)"
        echo ""
        echo "EXAMPLE: ${0} -c ~/Desktop/pics/unsorted/ ~/Pictures/chrono/"
        echo ""
        exit 1;;
    -c | --copy)
        COPYOPTION=${1}
        OPERATION="cp -p"
        LINKOPTION="no"
        shift 1;;
    -m | --move)
        shift 1;;
    -L | --no-ln)
        LINKOPTION="no"
        shift 1;;
    -e | --exif)
        EXIFOPTION=${1}
        shift 1;;
    -q | --quiet)
        QUIETOPTION=${1}
        shift 1;;
    -*)
        echo "Error: no such option $1. -h for help"
        exit 1;;
     *)
        break;;
esac
done

if [ ! -n "${2}" ]; then
        mkdir -p "${HOME}/.Pictures/"
        MOVETO="${HOME}/.Pictures/"
else
        # Add trailing "/" character if it is missing
        LASTCHAR=$(echo -n "${2}" | cut -b $(echo -n "${2}" | wc -c))
        if [ "$LASTCHAR" != "/" ]; then
                MOVETO="${2}/"
        else
                MOVETO="${2}"
        fi
fi
LOGDIR="${MOVETO}"

# Call self recursively
if [ ! "${3}" = "RECUR" ]; then
        echo "Finding and organizing all pictures with EXIF/MP4/3GP/AVI timestamps..."
        find "${1}" -type f -exec "${0}" ${COPYOPTION} ${EXIFOPTION} ${QUIETOPTION} {} "${2}" "RECUR" \;
        echo "Done!"
else
        FILE="${1}"
        IS_JPEG=$(file "${FILE}" | grep 'JPEG')
        if [ -n "${IS_JPEG}" ]; then
                SUFFIX=jpg
                TIMESTAMP=`jhead "${FILE}" 2>/dev/null | grep 'Date/Time' | cut -d':' -f2-6`

                if [ ! -n "${TIMESTAMP}" ]; then
                        # Older Nokia phones (3650 6600 etc.) have the timestamp in the comment - dig it out.
                        COMMENTDATE=$(jhead "${FILE}" 2>/dev/null | grep '^Comment' | grep '[0-9]\{2\}/[0-9]\{2\}/[0-9]\{4\}' | sed 's/Comment      : //g')
                        COMMENTTIME=$(jhead "${FILE}" 2>/dev/null | grep '^Comment' | grep '[0-9]\{2\}:[0-9]\{2\}:[0-9]\{2\}' | sed 's/Comment      : //g')
                        if [ -n "${COMMENTDATE}" -a -n "${COMMENTTIME}" ]; then
                                COMMENTYEAR=`echo ${COMMENTDATE} | cut -d'/' -f3`
                                COMMENTMONTH=`echo ${COMMENTDATE} | cut -d'/' -f2`
                                COMMENTDAY=`echo ${COMMENTDATE} | cut -d'/' -f1`
                                TIMESTAMP="$COMMENTYEAR:$COMMENTMONTH:$COMMENTDAY $COMMENTTIME"
                        else
                                # Get initial timestamp from file modification time and ask if it's ok
                                TIMESTAMP=$(ls -l --full-time "${FILE}" | cut -d ' ' -f6-7 | cut -d '.' -f1 | sed 's/-/:/g')
                                while [ ! -n "$VALID" ]; do
                                        if [ ! -n "${QUIETOPTION}" ]; then
                                                echo -n "Enter new date and time for ${FILE} or press ENTER to accept [$TIMESTAMP]:"
                                                read READDATE READTIME
                                                if [ ! -n $READDATE -o -n $READTIME ]; then
                                                        TIMESTAMP="$READDATE $READTIME"
                                                fi
                                        fi

                                        VALID=$(echo $TIMESTAMP | grep '[1-2][0-9][0-9][0-9]:[0-1][0-9]:[0-3][0-9] [0-2][0-9]:[0-5][0-9]:[0-5][0-9]')
                                done
                        fi
                fi
        fi

        # MP4/3GP files have timestamp but it is stored as seconds from 1904-01-01 00:00:00.
        IS_MP4=$(file "${FILE}" | grep 'MPEG v4')
        if [ -n "${IS_MP4}" ]; then
                SUFFIX=mp4
                IS_3GP=$(file "${FILE}" | grep '3GPP')
                if [ -n "${IS_3GP}" ]; then
                        SUFFIX=3gp
                fi
                TIMESTAMP=$(date -d "1970-01-01 00:00:00 $(($(date -d "1904-01-01 00:00:00" +%s) + $(mp4dump "${FILE}" | grep creationTime | cut -d '=' -f2 | cut -d ' ' -f2 | head -n1))) sec + $TIMEZONE hours" "+ %Y:%m:%d %H:%M:%S")
                if [ ! -n "${TIMESTAMP}" ]; then
                        # '${FILE}' has no creation timestamp!"
                        exit
                fi
        fi

        # AVI doesn't have timestamp so trust the creation time.
        IS_AVI=$(file "${FILE}" | grep 'AVI')
        if [ -n "${IS_AVI}" ]; then
                SUFFIX=avi
                TIMESTAMP=$(ls -l --full-time "${FILE}" | cut -d ' ' -f6-7 | cut -d '.' -f1 | sed 's/-/:/g')
        fi

        if [ -n "${TIMESTAMP}" ]; then
                # Deciding path
                DATE=`echo ${TIMESTAMP} | cut -d' ' -f1`
                YEAR=`echo ${DATE} | cut -d':' -f1`
                MONTH=`echo ${DATE} | cut -d':' -f2`
                DAY=`echo ${DATE} | cut -d':' -f3`
                MOVETO="${MOVETO}${YEAR}/${MONTH}/" #Directory Structure
                mkdir -p ${MOVETO}
                # Deciding filename
                TIME=`echo ${TIMESTAMP} | cut -d' ' -f2`
                HOUR=`echo ${TIME} | cut -d':' -f1`
                MINUTE=`echo ${TIME} | cut -d':' -f2`
                SECOND=`echo ${TIME} | cut -d':' -f3`
                NEWFILE="${YEAR}${MONTH}${DAY}${HOUR}${MINUTE}${SECOND}.$SUFFIX"
                # Complete path
                ABSPATH="${MOVETO}${NEWFILE}"
                # Complete path

                if [ -f "${ABSPATH}" ]; then
                # 1) File exists with that name"

                        if [ ! "${ABSPATH}" = "${FILE}" ]; then
                        # 2) The file isn't itself ... yeah ... that makes sense"

                                SUM0=`/usr/bin/md5sum "${FILE}" | cut -d' ' -f1`
                                SUM1=`/usr/bin/md5sum "${ABSPATH}" | cut -d' ' -f1`
                                if [ ! "${SUM0}" = "${SUM1}" ]; then
                                # 3) It isn't a duplicate of the same picture"

                                        i=0
                                        ABSPATH_1="d${i}-${ABSPATH}"
                                        ABSPATH_1="${MOVETO}d${i}-${YEAR}${MONTH}${DAY}${HOUR}${MINUTE}${SECOND}.${SUFFIX}"
                                        until [ ! -f ${ABSPATH_1} ]; do
                                                (( i++ ))
                                                ABSPATH_1="${MOVETO}d${i}-${YEAR}${MONTH}${DAY}${HOUR}${MINUTE}${SECOND}.${SUFFIX}"
                                        done

                                        # Giving it a unique name 'd#-FILE'"

                                        if [ ! -n "${QUIETOPTION}" ]; then
                                                echo "Appending 'd${i}-' to ${ABSPATH}: file exists."
                                        fi

                                        ABSPATH="${ABSPATH_1}"

                                # 3) non-duplicate of same name was renamed"
                                else
                                        if [ ! -n "${QUIETOPTION}" ]; then
                                                echo "Checksum match: Duplicate file overwritten"
                                        fi
                                fi
                        fi
                        # 2) if the file is itself, pass-along, mv/cp will handle it"
                fi
                # 1) All that settled, should be safe to move/copy the file
                if [ ! "${FILE}" = "${ABSPATH}" ]; then
                        if [ ! -n "${QUIETOPTION}" ]; then
                                if [ "$OPERATION" == "mv" ]; then
                                        echo "Moving ${FILE} to ${ABSPATH}"
                                else
                                        echo "Copying ${FILE} to ${ABSPATH}"
                                fi
                                if [ ! -n "$LINKOPTION" ]; then
                                        echo "Creating symbolic link"
                                fi
                        fi

                        $OPERATION -f "${FILE}" ${ABSPATH}
                        if [ ! -n "${LINKOPTION}" ]; then
                                ln -s ${ABSPATH} "${FILE}"
                        fi
                        echo "${ABSPATH}:::::${FILE}" > ${LOGDIR}catalog.log

                        # Create Date/Time EXIF tag for image
                        if [ -n "${EXIFOPTION}" ]; then
                                exiftool -overwrite_original "-dateTimeOriginal=$TIMESTAMP" "${ABSPATH}" >/dev/null
                        fi
                fi
        fi
        #  0) Not supported file."
fi

gphoto2 might have some tools that could further enhance this? Anyone with know-how, please report.

Gallery2

  1. Download Gallery2
  2. Follow the instructions
  3. Create an upload directory in the g2data folder
  4. As admin, specify that folder as the upload folder
  5. Link that folder to your ${HOME}/pictures/tmp
  6. Use the local server upload page when uploading pictures
  7. Delete the pictures from ${HOME}/pictures/tmp
  8. Link the g2data/${USER} folder to your ${HOME}/pictures/public
  9. Keep your non-public pictures in ${HOME}/pictures/private

Convention

I upload all of my photos via gallery2 and then organize in albums in chronological order. These are my 'permalinks', if you will. Most of my pictures I don't actually rename from 200506261265.jpg to meaningfull names, like Lake_Placid.jpg - I use the 'Title' feature in gallery2, but for the sake of illistration, I'll pretend that I do.

username/life/chrono/2005/06/Lake_Placid.jpg
username/life/chrono/2005/06/John_Bytheway.jpg
username/life/chrono/2005/06/Painting.jpg

Additionally, I have a misc and tech album for things that most people wouldn't be interested in, but are important to me.

username/tech/wallpaper/gentoo_1024x768.png
username/tech/screenshots/gaim_fullscreen.png
username/misc/road_signs/dead-end_natures-way.jpg
username/misc/bumper_stickers/MEN_we_re_just_better.jpg

But that's not really great for keeping track of recent events, or currrent friends, etc - just for permalinks. So as temporary albums (people, places, things, events - basically) that I leave up for a few months and then delete, I use the 'create link' functionality of gallery2. So these would actually be symbolic links:

username/life/recent/bbq_at_johns/MEN_we_re_just_better.jpg
username/life/recent/bbq_at_johns/John_Bytheway.jpg
username/life/recent/vacation_vt/dead-end_natures-way.jpg
username/life/friends/john_bytheway/John_Bytheway.jpg

Just thought I'd share that. Hope you find it useful.

RenRot

RenRot is a program to rename and lossless rotate (for now only JPEG format) files according to their EXIF tags values.

RenRot is intended to work with files of --extension extension containing EXIF date and can do two things with them - rename and rotate. It runs in batch mode in current or set with --work-directory directory as well as selective mode for separate files given as arguments in command line.

Template ideology was implemented. It includes flexibility in file name construction. The template can contain different data, from direct name to EXIF data (the date, id or shooting details such as WhiteBalance, ISO, e.t.c.). For further information, please, see applied manual.

RenRot rotates file and its thumbnail, according to EXIF tag Orientation. If the tag is absent or miss set, than the script allows to rotate the file as well as it's thumbnail "by hands".

Furthermore, the script put a comment to: - Comment tag from comment file; - UserComment tag from configuration variable or command line option.

Personal data could be specified via XMP tags defined in configuration file.

In addition RenRot can aggregate all files in directories according the given date/time pattern template, set with --aggr-template.

Code: rename each file according to the given template
 renrot --name-template="01.%c.%Y%m%d%H%M%S.%E%F%W%I" --extension JPG
Code: rename each file according to the given template and aggregate according the date
 renrot --name-template="%y%m%d%H%M%S.%i" --aggr-mode="template" --aggr-template="%Y%m%d" *.JPG
Code: aggregate files by yymmddHHMM
 renrot --aggr-mode="template" --aggr-template="%y%m%d%H%M" --extension jpg
Code: rotate each file and their thumbnail by 90CW in specified directory
 renrot --rotate-angle 90 --work-directory="/tmp/images" --extension jpg
Code: rotate thumbnails, included to EXIF, for each file by 270CW (same as 90CCW)
 renrot --rotate-thumb 270 --extension jpg
Code: rotate given files by Orientation tag (no real rotation will be done)
 renrot --no-rename --mtime --rotate-angle=90 --only-orientation *.JPEG
Code: fix file mtime according to its EXIF tags or current time stamp, when tags are invalid
 renrot --no-rotate --no-rename --mtime --extension jpeg
Code: leave mtime untouched for couple of files
 renrot --no-mtime *.jpg

See Also

CoolAJ86's Review of Gallery2 In version 0.16 and later the previous behaviour still present with default NameTemplate.

Retrieved from "http://www.gentoo-wiki.info/TIP_Organizing_digital_photos"

Last modified: Sun, 07 Sep 2008 03:55:00 +0000 Hits: 9,200