File management script for Android and Linux

Hi all, :wave:

I donΒ΄t know whether the following would be of any interest to any of you but I thought IΒ΄d post it here anyway. :blush:

Often enough thereΒ΄s a situation in which I find myself obliged to scan physical documents (i.e. documents on paper) in order to produce a digital equivalent of them. In most cases I need to have them in digital format for being able to quickly and easily access them e.g. when dealing my income tax return.

Anyhow, due the setup of my computer peripherals itΒ΄s pretty inconvenient for me to take all those paper documents from one room into another (where my printer-scanner resides) and then manually scan all of those documents.

Now I found a way of accessing and dealing with them just where they are. Hassle-free process. :wink:

  1. I take a photograph of all the pages of the document(s) I need with my Android smartphone

  2. As those images take up too much space (data-wise) I cannot easily send them via e-mail to my 2nd e-mail address as attachments.
    So I came up with a script which I run in termux on the smartphone which processes them before I can send them.

  3. In the end I have very small PDF documents instead of heavy-weight pictures, which I often enough can send as one single file (packed and encrypted).

  4. When accessing my Linux Lite PC (mostly the next day) the attachment can be downloaded and I have all documents readily available.

And hereΒ΄s what the script does:

  • Step 1: Convert jpg to jpeg # That step might not be necessary but I noticed that jpeg in some context worked better for me in the past. Feel free to skip this step if youΒ΄d like

  • Step 2: Convert to monochrome # No need for having coloured output for documents

  • Step 3: Resize to 50% # Good enough for me and itΒ΄s economical on data consumption

  • Step 4: Convert to PDF # ThatΒ΄s what I need in the end

  • Step 5: Cleanup # Deleting temporary jpeg files on the smartphone

  • Step 6: Create the 7z archive of PDFs # with encryption so that it can safely be sent as attachment via e-mail

  • Step 7: Cleanup # Deleting the original PDFs on the smartphone

  • Step 8: Move the original JPG files back to the original folder # So they are not lost. Delete them manually if you need space back

As a result we have produced a compressed and packed 7zip file which holds the processed data (end result: PDFs) which can be safely sent as e-mail attachment. :wink:

Note:

When unpacking them on the PC I realized that in most cases the the PDFs were somehow rotated by 90Β°. But it wasnΒ΄t always the case.
So you might need to run the following command on Linux after unpacking:

for f in *.pdf; pdftk $f cat 1-endeast output (string replace '.pdf' '_90.pdf' $f); end # fish syntax

Now hereΒ΄s the script:

#!/bin/bash

for file in *.jpg; do
    if [[ -f "$file" ]]; then
        base_name="${file%.*}"  

        echo "Processing $file..."

        # Step 1: Convert jpg to jpeg
        echo "Step 1: Converting to JPEG..."
        magick "$file" "${base_name}_step1.jpeg" || { echo "Error in Step 1"; continue; }

        # Step 2: Convert to monochrome
        echo "Step 2: Converting to grayscale..."
        magick "${base_name}_step1.jpeg" -colorspace Gray "${base_name}_step2.jpeg" || { echo "Error in Step 2"; continue; }

        # Step 3: Resize to 50%
        echo "Step 3: Resizing..."
        magick "${base_name}_step2.jpeg" -resize 50% "${base_name}_step3.jpeg" || { echo "Error in Step 3"; continue; }

        # Step 4: Convert to PDF
        echo "Step 4: Converting to PDF..."
        magick "${base_name}_step3.jpeg" "${base_name}.pdf" || { echo "Error in Step 4"; continue; }

        # Cleanup
        echo "Cleaning up temporary files..."
        rm -f "${base_name}_step1.jpeg" "${base_name}_step2.jpeg" "${base_name}_step3.jpeg"

        echo "Finished processing $file."
    fi
done

# Create the 7z archive of PDFs
echo "Creating encrypted 7z archive..."
7z a -p'ABC' -mhe=on -t7z DocumentPhoto.7z *.pdf  # see Note below

# Check if the command was successful
if [[ $? -eq 0 ]]; then
    echo "7z archive created successfully."
    # Delete the original PDFs
    rm -f *.pdf
    echo "Original PDFs deleted to free up space."
else
    echo "Error creating 7z archive."
fi

# Move the original JPG files back to the original folder
echo "Moving original JPG files back to their original folder..."
mv *.jpg /data/data/com.termux/files/home/storage/dcim/Camera/

# Check if the move was successful
if [[ $? -eq 0 ]]; then
    echo "JPG files moved successfully."
else
    echo "Error moving JPG files."
fi

Note:

The line

7z a -p'ABC' -mhe=on -t7z DocumentPhoto.7z *.pdf

needs to be altered as to your preference. It holds your personal password. So just replace ABC (which is just an example) with your PW.

In my experience data consumption could be reduced significantly, although it varies a bit. In most cases the file size (original photo taken by the smartphone vs the produced PDF) could be brought down to a 4th or 5th.

Last time I used the script I had to process 20 photos which would have taken up around 80 MB (much too big for e-mail attachments) …
… and they could be reduced to less that 20 MB. I could easily send them using just one single e-mail (attachment). :smiling_face:

HereΒ΄s an example of original photo vs PDF:


(original)

and:


(PDF)

And here are even pictures involved. For text-based documents itΒ΄s more than good enough.:wink:

O.K., thatΒ΄s it.

Many greetings from Rosika :slightly_smiling_face:

P.S.:

see also β€œAddendum” in post #7.

7 Likes

Hi Rosika,
You use imagemagick.
I might try that. I have a small booklet which I scanned. I would prefer it as a pdf.
The scan quality is poor… it may not do the OCR well?
We shall see.
Regards
Neville

3 Likes

Hi Neville, :waving_hand:

thanks for your reply. :heart:

Yes, I forgot to mention imagemagick has to be installed as a prerequisite in termux. We need that for processing the files.

Thanks for mentioning it.

Cheers from Rosika :slightly_smiling_face:

3 Likes

That’s marvellous!
:penguin:

I achieved a very similar goal in a different way.
What we use is the simple-scan.
https://packages.debian.org/bookworm/simple-scan

That can handle the scanners via SANE, which connects to networked scanners (a Lide 60 connected to an Rpi, and shared via airsane, and a Samsung MFP).
On the Debian boxes sane-airscan has to be installed to reach those networked scanners. Even Android devices can use them via Mopria scan :wink:

Simple scan has the option to scan multiple pages, I set up 10 seconds delay between pages.
So I start the multi-page scanning, take the document, walk to the choosed scanner, and feed it manually until I finish.
Sometimes I’m clumsy, and despite the 10 seconds delay, I can’t correctly change the page, that causes a messy scan of that page, but that will be repeated. After the last page is correctly scanned, I walk back to the computer, stop the scanning, and before acutally saving the document, delete the messed pages. Usually the first one, the last one, and one in the middle, where I was clumsy.
That’s kind of comfortable, for me, and my family just adapted to it :slight_smile:
I never had problems with filesize of pdf’s saved from Simple-scan.

Your solution @Rosika using a smartphone and involving image-magick, termux, 7z, is ingenious! Very impressive hack! (In this context β€œhacking” means using a device for a purpose in a way that it was originally not designed for…)
That’s marvellous!

3 Likes

I do the same with ``xsane` . It does not have a 10 sec time window… you press a button for each page when ready.
I was most interested in using imagemagick to clean it up.
I would also like to try OCR…that would make a very small pdf document.

https://www.baeldung.com/linux/ocr-tools

3 Likes

I use a shell script and TermUX to get my photos off my phone…

╭─x@titan ~/tmp/rip  
β•°β”€βž€  bat ~/bin/today-photos.bash 
───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       β”‚ File: /home/x/bin/today-photos.bash
───────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   β”‚ #!/usr/bin/env bash
   2   β”‚ # Grab today's photos off a samdung piece of crap
   3   β”‚ # 
   4   β”‚ # source folder on the samdung crud :
   5   β”‚ SRC="/storage/8FC6-9FA1/DCIM/Camera/"
   6   β”‚ DSTRING=$(date '+%Y%m%d')
   7   β”‚ FCKDSTRING="$SRC$DSTRING""_*.jpg"
   8   β”‚ DSTRING="$DSTRING""_??????.jpg"
   9   β”‚ YY=$(date '+%Y')
  10   β”‚ echo $DSTRING
  11   β”‚ echo $YY
  12   β”‚ FSTRING=$SRC$DSTRING
  13   β”‚ # e.g. 20230705_161756.jpg
  14   β”‚ # counting on "homedung" having an entry in ~/.ssh/config or ~/.ssh/home (might be a symlink)
  15   β”‚ HO=homedung
  16   β”‚ # set -vx
  17   β”‚ DEST=$HOME/ResilioSync/bigshit/Photos/$YY/Camera
  18   β”‚ if [ ! -d $DEST ] ; then
  19   β”‚     echo "$DEST doesn't exist... yet ..."
  20   β”‚     echo "exiting..."
  21   β”‚ fi
  22   β”‚ echo "------------------"
  23   β”‚ echo "$SRC"$DSTRING" to $DEST"
  24   β”‚ echo "or even : "
  25   β”‚ echo -e "\trsync -av $HO:\"$FSTRING\" $DEST/"
  26   β”‚ echo "------------------"
  27   β”‚ echo "or even (would be more betterer): "
  28   β”‚ echo -e "\trsync -av $HO:\"$FCKDSTRING\" $DEST/"
  29   β”‚ echo "------------------"
  30   β”‚ echo "\tor just to update : "
  31   β”‚ echo -e "\t""\t""rsync -av --ignore-existing $HO:\"$SRC\" $DEST/"
  32   β”‚ ## URL="$(xclip -o)"
  33   β”‚ ## STRING=$(echo "rsync -av --ignore-existing $HO:\"$SRC\" $DEST/" |xclip -o)
───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

It doesn’t actually copy anything - but the end output is tells me the command I need to run :
rsync -av --ignore-existing homedung:"/storage/8FC6-9FA1/DCIM/Camera/" /home/x/ResilioSync/bigshit/Photos/2025/Camera/

β€œhomedung” is my SSH nickname for my Samsung Galaxy in ssh (~/.ssh/config) and rsync uses the SSHD in TermUX… Was trying to get double-tab on the last output - but it doesn’t seem to work…

Some time ago - I was using imagemagick to deliberately pixelate bitmaps…

───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       β”‚ File: /home/x/bin/piggzel8.bash
───────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   β”‚ #!/usr/bin/env  bash
   2   β”‚ <<'COMMENTATO'
   3   β”‚   pixelate a image into "pixel art"...  
   4   β”‚   will do either 5, 7 or 10 % reducto then 1000% grobow - will expect two arguments - but NO MORE ARGUING! we're gunna assume P
       β”‚ NG in and PNG out (but input shouldn't matter, output will always be PNG - actually input other than PNG might break it)... 
   5   β”‚   Like DIS mofo : 
   6   β”‚            convert -scale 5% -scale 1000% thalidomide-robot-face.png thalidomide-robot-face-out.png
   7   β”‚   Version 3 - added 4th option to do 20% if too much pixelization
   8   β”‚ COMMENTATO
   9   β”‚ # Global Variables : 
  10   β”‚ PROG=$(basename "$0")
  11   β”‚ IMAGIO="$1"
  12   β”‚ SIZIO=$2
  13   β”‚ TRIMD=$(basename "$IMAGIO" .png)
  14   β”‚ BING="/usr/bin/convert"
  15   β”‚ # set -vx
  16   β”‚ if [ "$#" -ne 2 ] ; then
  17   β”‚         echo "Need TWO arguments:"
  18   β”‚     echo "     e.g.  $PROG \"imagefile.png\" (1,2,3 or 4)"
  19   β”‚     echo ""
  20   β”‚     echo "     e.g.     $PROG factunt.png 1"
  21   β”‚     echo "                 will downsize \"factunt\" to  5% before grobowing up to 1000%"
  22   β”‚     echo "     e.g.     $PROG factunt.png 2"
  23   β”‚     echo "                 will downsize \"factunt\" to  7% before grobowing up to 1000%"
  24   β”‚     echo "     e.g.     $PROG factunt.png 3"
  25   β”‚     echo "                 will downsize \"factunt\" to 10% before grobowing up to 1000%"
  26   β”‚     echo "     e.g.     $PROG factunt.png 4"
  27   β”‚     echo "                 will downsize \"factunt\" to 20% before grobowing up to 1000%"
  28   β”‚         exit 1
  29   β”‚ fi
  30   β”‚ if [ ! -f "$IMAGIO" ] ; then
  31   β”‚     echo "$IMAGIO doesn't exist - be more specific ya moron!"
  32   β”‚     exit 1
  33   β”‚ fi
  34   β”‚ 
  35   β”‚ if [ $SIZIO -lt 1 ]  || [ $SIZIO -gt 4 ] ; then 
  36   β”‚     echo "1 = 5%, 2 = 7%, 3 = 10%, 4 = 20% - THERE ARE NO OTHER >>ACCEPTABLE<< VALUES "
  37   β”‚     exit 1
  39   β”‚ 
  40   β”‚ case $SIZIO in
  41   β”‚     1)
  42   β”‚         REDUCTO="5"
  43   β”‚         GROWBO="1000"
  44   β”‚     ;;
  45   β”‚     2)
  46   β”‚         # because sometimes 5% is too much, but 10% is not enough
  47   β”‚         REDUCTO="7"
  48   β”‚         GROWBO="1000"
  49   β”‚     ;;
  50   β”‚     3)
  51   β”‚         REDUCTO="10"
  52   β”‚         GROWBO="1000"
  53   β”‚     ;;
  54   β”‚     4)
  55   β”‚         REDUCTO="20"
  56   β”‚         GROWBO="1000"
  57   β”‚     ;;
  58   β”‚     # anything else situation probably redundant - but not really...
  59   β”‚     *)
  60   β”‚         echo "not a number between 1 and 3 moron!"
  61   β”‚         exit 1
  62   β”‚     ;;
  63   β”‚ esac    
  64   β”‚ $BING -scale $REDUCTO% -scale "$GROWBO"% "$IMAGIO" "$TRIMD"-out-"$SIZIO".png
  65   β”‚ echo "iffenya rundis again - will overrite previous output..."
───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Start with original :

Then reduce to 5% then grow 1000% in the same line :

Imagemagick actually does a decent job of anitaliasing

4 Likes

@all:

Hi again, :waving_hand:

sorry that I couldnΒ΄t reply earlier.

Thanks to all of you for your great posts. :heart:

@kovacslt:

Thank you for your praise. :blush:

Your emplyoment of simple-scan is great. Actually I used to use it in the past as well, particularly when I was on Lubuntu. Linux Lite even came with simple-scan installed.

Splendid solution. :+1:

Well, since I have the smartphone at my disposal now, I was thinking: why not use it to its full potential? This way i wonΒ΄t have to carry around my documents. :wink:

But I like your way of doing scans quite a lot, especially in view of the fact you could get your family to adapt to it as well.

Thanks again, LΓ‘szlΓ³.

@nevj :

Thanks for mentioning xsane (and for providing the baeldung link).
ThereΒ΄s a quite extensive article on ubuntuusersΒ΄ wiki:

(in German though, but available in English e.g via firefoxΒ΄s built-in tranlator)

XSane is a very comprehensive and powerful scan program with a GTK+ interface, which acts as a so-called β€œfront end” for SANE. It is therefore suitable to use XSane under GNOME, Xfce and Unity.

It seems to be an impressive tool.

@daniel.m.tripp :

As always you came up with a great shell script, Dan :+1: .
Highly impressive.

In fact, I believe it was you who mentioned termux in some other context quite some time ago.
I had never heard of it before, so I got interested in it. Accordingly I installed it on my smartphone and IΒ΄m delighted to have it. :wink:

@all:

Addendum:

In fact I forgot to mention a certain step in my first post.
In the paragraph β€œAnd hereΒ΄s what the script does” under Step 8 it says:

Some of you might ask β€œmove back from where?”
ThatΒ΄s the part I forgot to mention. Sorry. Here it is as an addendum:

When I take photographs on my smartphone they reside in the pictures folder.
Termux sees its path as:
/data/data/com.termux/files/home/storage/dcim/Camera/ .

So I created a working directory:
/data/data/com.termux/files/home/storage/dcim/Camera/work/
in which the script resides.

Suppose I had taken a bunch of photos on April 30 this year their name would be e.g. 20250430_123814.jpg etc.

Starting from within the directory
/data/data/com.termux/files/home/storage/dcim/Camera/
a move command will take care of transferring all the relevant photos to the work directory:
mv 20250430*.jpg work/
Now the script can be run and step 8 hopefully makes sense now. :blush:

Many greetings from Rosika :slightly_smiling_face:

4 Likes

I’m glad you built an open source method for doing this.

Currently, I use Google Stacks app on Android to do this.
Can you comment on how your method is different/better.

Also, have you thought to release this in a .debian or .rpm package?

2 Likes

I’m curious which version of Android you’re running on your phone - my Galaxy S9+ is EOL (end of life) and stuck with Android 10…

There are problems with accessing files from third party apps - this is something google deliberately broke with Android 9 and later (I think they might have fixed it - or replaced an API with Android 11 or 12?).

Anyway - on my Android 10 - I can only READ things on the phone storage or SD-Card… I can’t write. When I want to β€œwrite” I mostly use adb-sync… The only location I can write to with TermUX is those β€œabstracted” TermUX folders (e.g. /data/data/com.termux/). Did you reconfigure your Android to save photos in that com.termux path? I never thought of that? Hmmm - my /data/data/com.termux/ is located on the internal phone storage (64 GB - and I only have 15 free)… my DCIM (photos) folder is on the 512 GB SD card - which I can’t write to from TermUX (the same SD card has my portable music collection subset).

Anyway - this S9+ seems to do everything I need - but I may be in the market for a new phone in the next 12-18 months (β€œnew” - as in replacement - I’ll probably buy 2nd hand). Nearly all the handset manufacturers have removed expandable storage (i.e. no SD-Card slot) - so you need to buy knowing what your longer term storage might be… I’d probably have to spend a tad more and get something with 512 GB of storage… I think Motorola still have SD-Card slots - but - I really like DeX so will probably stick with Samsung (even though Android 16 has some sort of β€œdesktop mode”).

3 Likes

Hi again, :waving_hand:

@John_Hansen :

Welcome to our community, John. :slightly_smiling_face:

Well, actually no, as donΒ΄t know β€œGoogle Stacks app”.
I presume itΒ΄s from the Goggle Play Store. I tend to avoid stuff from that source and install the few apps I need from the f-droid store.

I donΒ΄t release my humble scripts, far from it. :blush:

All I wanted to achieve is solving a personal problem for me…
… which meant that I wouldnΒ΄t have to carry physical documents over quite some distance and scan it with my printer/scanner.

The script helps me to process the photos which are locally available on the smartphone and create monochrome PDFs and pack them in an encrypted file.
This can be sent to my 2nd e-mail address.
EverythingΒ΄s done within the smartphone.
ThatΒ΄s all. :wink:

@daniel.m.tripp :

I believe itΒ΄s Android 14.

Hmm, sorry to hear that.
However my script writes to this abstracted termux folder as well:

/data/data/com.termux/files/home/storage/dcim/Camera/work.

When I need to send the newly created 7zip file (holding the PDFs) this work directory is available for the fennec browser to use for attachments.

I didnΒ΄t have to reconfigure anything.
When taking the photos theyΒ΄re saved in AndroidΒ΄s default Gallery.
Termux sees the path as
/data/data/com.termux/files/home/storage/dcim/Camera/ .

With the move command I transfer the photos to
/data/data/com.termux/files/home/storage/dcim/Camera/work, where the script resides.
So no reconfiguration necessary. I donΒ΄t know whether it would have been doable in the first place…

Hope it helps clarifying things.

Many greetings from Rosika :slightly_smiling_face:

2 Likes