↓ Skip to main content
hofiverse
  1. Projects/

Image Privacy and Optimization Pipeline with Folder Watching

Table of Contents
Status Stable
Cost $0
Time Spent ~2 hours
Outcome

Automated EXIF stripping and WebP conversion, 80% average size reduction across 47 images

Stack
Bash ImageMagick ExifTool systemd

Every photo I take on my iPhone 13 Pro Max contains a hidden report about me. GPS coordinates, camera serial number, exact timestamp down to the millisecond, and even the angle I was holding the phone. Drop that image on a website and all of that travels with it. I needed a single tool that would handle both the privacy problem and the performance problem at once.

The Problem
#

Modern devices are extremely chatty when it comes to image metadata. A single HEIC photo from my iPhone contained all of the following before I started scrubbing:

  • GPS latitude, longitude, and altitude
  • Camera make, model, and serial number
  • Software version
  • Timestamp down to the millisecond including timezone offset
  • Subject area, acceleration vector, and Maker Notes

None of that should be public. Beyond privacy, the files themselves were too large. An 11 MB PNG screenshot or a full-resolution HEIC has no place on a personal website.

The Goal
#

A Bash script that handles the full pipeline in one shot:

  • Accept JPG, PNG, and HEIC as input
  • Convert everything to WebP at quality 82
  • Auto-orient and resize to 1920px max dimension
  • Rename files to lowercase, hyphenated, web-safe filenames
  • Strip all metadata with ExifTool
  • Delete the original only after confirming a successful conversion
  • Print a summary showing space reclaimed, compression ratio, and average processing speed

The Build
#

The Investigation
#

The core issue was that two separate problems needed solving together. Privacy stripping alone does not help if the files are still 11 MB. Resizing alone does not help if GPS coordinates are still embedded. The tool needed to handle both in a single pass so there was no way to accidentally skip one step.

I also needed it to be safe. The original file should never be deleted until a successful conversion is confirmed. Losing originals because a conversion silently failed would be worse than the original problem.

Finding the Right Approach
#

Three tools cover everything needed:

ImageMagick converts between formats, handles resizing, and corrects rotation from EXIF orientation data.

ExifTool strips every metadata field from the output file after conversion.

inotify-tools provides inotifywait for the folder watcher, which reacts the moment a new file appears.

Install all three:

sudo apt install -y imagemagick libimage-exiftool-perl inotify-tools

The folder structure the script expects:

~/metadata_scrub/
├── needs_scrub/    # drop images here
├── scrubbed/       # processed output lands here
└── logs/           # timestamped log files

Create it:

mkdir -p ~/metadata_scrub/{needs_scrub,scrubbed,logs}

The Fix
#

scrub.sh handles the full processing pipeline. Save it as ~/metadata_scrub/scrub.sh.

#!/bin/bash

start_time=$(date +%s.%N)

BASE_DIR="$HOME/metadata_scrub"
INPUT_DIR="$BASE_DIR/needs_scrub"
OUTPUT_DIR="$BASE_DIR/scrubbed"
LOG_DIR="$BASE_DIR/logs"

mkdir -p "$LOG_DIR"
LOG_FILE="$LOG_DIR/scrub_$(date +%Y%m%d_%H%M%S).log"

GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m'

echo -e "${BLUE}Image Processing Pipeline Initialized...${NC}" | tee -a "$LOG_FILE"

cd "$INPUT_DIR" || { echo "Error: needs_scrub folder not found!"; exit 1; }

shopt -s nocaseglob

count=0
count_jpg=0
count_png=0
count_heic=0
total_orig_raw=0
total_new_raw=0

for img in *.{jpg,jpeg,png,heic}; do
    [ -e "$img" ] || continue

    orig_size_raw=$(stat -c%s "$img")
    orig_size_h=$(du -h "$img" | cut -f1)
    ((total_orig_raw += orig_size_raw))

    ext="${img##*.}"
    case "${ext,,}" in
        jpg|jpeg) ((count_jpg++)) ;;
        png)      ((count_png++)) ;;
        heic)     ((count_heic++)) ;;
    esac

    raw_name="${img%.*}"
    clean_name="${raw_name,,}"
    clean_name="${clean_name// /-}"
    clean_name="${clean_name//[^a-z0-9_-]/}"
    final_output="$clean_name.webp"

    if convert "$img" -auto-orient -resize 1920x\> -quality 82 "$OUTPUT_DIR/$final_output" 2>> "$LOG_FILE"; then
        exiftool -overwrite_original -all= "$OUTPUT_DIR/$final_output" > /dev/null 2>&1

        new_size_raw=$(stat -c%s "$OUTPUT_DIR/$final_output")
        new_size_h=$(du -h "$OUTPUT_DIR/$final_output" | cut -f1)
        ((total_new_raw += new_size_raw))

        rm "$img"

        echo -e "  [${GREEN}OK${NC}] $img ($orig_size_h) -> $final_output ($new_size_h)" | tee -a "$LOG_FILE"
        ((count++))
    else
        echo -e "  [${RED}FAIL${NC}] Could not convert $img. Original preserved." | tee -a "$LOG_FILE"
    fi
done

shopt -u nocaseglob

end_time=$(date +%s.%N)
runtime=$(echo "$end_time - $start_time" | bc)

if [ $count -gt 0 ]; then
    total_saved_raw=$((total_orig_raw - total_new_raw))
    total_mb=$(echo "scale=2; $total_saved_raw / 1048576" | bc)
    avg_speed=$(echo "scale=2; $runtime / $count" | bc)
    compression_ratio=$(echo "scale=1; ($total_saved_raw / $total_orig_raw) * 100" | bc)

    echo -e "\n${YELLOW}--- IMAGE PROCESSING SUMMARY ---${NC}" | tee -a "$LOG_FILE"
    echo -e "Total Images:          $count" | tee -a "$LOG_FILE"
    [ $count_jpg -gt 0 ]  && echo -e "  - JPEGs:             $count_jpg" | tee -a "$LOG_FILE"
    [ $count_png -gt 0 ]  && echo -e "  - PNGs:              $count_png" | tee -a "$LOG_FILE"
    [ $count_heic -gt 0 ] && echo -e "  - HEICs:             $count_heic" | tee -a "$LOG_FILE"
    echo -e "Space Reclaimed:       ${GREEN}${total_mb} MB${NC} (${compression_ratio}% smaller)" | tee -a "$LOG_FILE"
    echo -e "Average Speed:         ${YELLOW}${avg_speed}s / image${NC}" | tee -a "$LOG_FILE"
    echo -e "Total Execution Time:  ${runtime:0:4}s" | tee -a "$LOG_FILE"
    echo -e "Log file:              $LOG_FILE" | tee -a "$LOG_FILE"
else
    echo -e "\n${RED}No images were successfully processed.${NC}" | tee -a "$LOG_FILE"
fi

Mark it executable and run it:

chmod +x ~/metadata_scrub/scrub.sh
~/metadata_scrub/scrub.sh

watcher.sh calls scrub.sh automatically when a new image lands in needs_scrub/. Save it as ~/metadata_scrub/watcher.sh.

#!/bin/bash

WATCH_DIR="$HOME/metadata_scrub/needs_scrub"
SCRUB_SCRIPT="$HOME/metadata_scrub/scrub.sh"

echo "Watching $WATCH_DIR for new images..."

inotifywait -m -e close_write --format '%f' "$WATCH_DIR" | while read -r filename; do
    case "${filename,,}" in
        *.jpg|*.jpeg|*.png|*.heic)
            echo "Detected: $filename. Starting scrub..."
            sleep 2
            bash "$SCRUB_SCRIPT"
            ;;
    esac
done

Register the watcher as a systemd user service so it starts automatically on login:

mkdir -p ~/.config/systemd/user
nano ~/.config/systemd/user/image-scrub-watcher.service
[Unit]
Description=Image Privacy and Optimization Folder Watcher
After=default.target

[Service]
Type=simple
ExecStart=%h/metadata_scrub/watcher.sh
Restart=on-failure

[Install]
WantedBy=default.target
systemctl --user daemon-reload
systemctl --user enable image-scrub-watcher.service
systemctl --user start image-scrub-watcher.service
systemctl --user status image-scrub-watcher.service

Results
#

The first production run processed 47 images:

--- IMAGE PROCESSING SUMMARY ---
Total Images:          47
  - JPEGs:             29
  - PNGs:              1
  - HEICs:             17
Space Reclaimed:       66.74 MB (80.0% smaller)
Average Speed:         .79s / image
Total Execution Time:  37.4s

The before and after on a single iPhone photo tells the full story. Before: 1.4 MB HEIC with over 100 metadata fields including GPS coordinates, acceleration vectors, camera serial number, and device identifiers. After: 432 kB WebP with 16 fields, all derived from the file container itself with no sensitive data.

The standout result was an 11 MB PNG screenshot that came out at 144 kB, a 99% reduction for a single file.

What I Would Do Differently
#

The sleep 2 in watcher.sh is a hack. It works but a proper solution would poll the file until its size stabilizes rather than assuming 2 seconds is always enough for a file to finish copying. On a slow network mount or a very large file this could still trigger processing before the file is fully written.

I would also add a flag to keep the originals in an archive folder rather than deleting them outright, at least for the first few runs until confidence in the script is high.

What Is Next
#

The current setup handles images dropped into the folder manually or via a file manager. The logical next step is integrating this with the site’s deploy pipeline so any image added to the Hugo static folder gets automatically processed before deployment. That would remove the manual step of dropping files into needs_scrub/ entirely.

← Fixing a Whirlpool Dishwasher Start Button for $4
Disclaimer: Content on this site is provided as-is with no guarantees of accuracy, completeness, or fitness for any particular purpose. Always test in a safe, non-production environment before applying anything documented here to systems you rely on. I am not responsible for data loss, damage, or security issues resulting from following these guides. Software and services change over time and steps that worked when this was written may not work in the future.