Problems with bulk renaming command

Hi all, :wave:

I suddenly ran into a problem with bulk-renaming.
It used to work well with Lubuntu but now - on Linux Lite 6.2 - it doesn´t seem to work any more. :slightly_frowning_face:

With bash I used to apply the following command with any given directory:

for f in *\ *; do mv "$f" "${f// /_}"; done

That took care of eliminating any whitespaces of the contents of the directory an replacing them with the underscore character.

But now I get an error:

mv: cannot stat '* *': No such file or directory

I even tried replacing the double quotes with single quotes for this part:

“${f// /_}”

but it didn´t seem to do anything aside from getting me into a loop. I had to abort with “CTRL+C”.

Hmm… :thinking:
I was looking around the net and found on cannot stat error in Linux – meaning and fixes – LinuxPip :

This error means that stat cannot read information about the file because it was denied by disallowing “execute” permission.

So I changed the rights of the contents of the respective folder Videos:

chmod -R 755 Videos/

and issued the command once more. But I still get the same error message.

Can anybody help me solve the problem :question:

Thanks a lot in advance.

Many greetings
Rosika :slightly_smiling_face:

If you want to replace whitespaces with underscore in the name of the files and directories, I prefer using the rename command with the find command.

The rename command needs to be installed because it’s not a standard Linux command.

For directories, use:

find . -name "* *" -type d | rename 's/ /_/g'

For files, use:

find . -name "* *" -type f | rename 's/ /_/g'

I hope that helps.

@abhishek :

Hi Abhishek. :wave:

thank you so much for your help. :heart:

I was astonished to see LL alrerady has rename installed on it by default. So I could immediately try the commands you kindly provided and indeed they work well. :+1:

In the respective directory I have files and a folder, so I could use both commands with great success.
Well, that´s really great.

Thanks a lot.
Many greetings from Rosika :slight_smile:

P.S.:

@all:

Just to learn something about the background of commands…

I´d still like to know what the difference of those two commands might be:

for f in *\ *; do mv "$f" "${f// /_}"; done

and

for f in *\ *; do mv "$f" '${f// /_}'; done

Well, the first one gave me the error message I posted…

…but the seconde one (in which I replaced the double quotes with single quotes for the second part of the command) got me into a loop but seemingly didn´t change anything.

Any ideas regarding the difference of those two commands :question:

2 Likes

Hi Neville, :wave:

thanks a lot for providing the link to the explanation.

Single quotes won’t interpolate anything, but double quotes will.

I see.

$ echo "$(echo "upg")"
upg
$ echo '$(echo "upg")'
$(echo "upg")

Yes, that sums it up quite nicely.

So the '${f// /_}' part in my command wouldn´t make any sense as it

preserves the literal value of each character within the quotes.

and doesn´t interpolate the values…

O.K., I guess no harm done as the command was still restricted to the folder I started it from, right?
And the contents (or rather the names of them) weren´t changed at all.

Thanks a lot, Neville.

Many greetings
Rosika :slightly_smiling_face:

Hi Rosika,
Should that not be $(f// /_)
and
does the *\ * not need quotes?
and
what will the script do if it matches no files?

Not sure my scripting is rusty
Regards
Neville

1 Like

I like @abhishek 's answer - I swear by that rename command… Beware there are two versions.

The one Abishek’s suggesting, and the one I use are from perl, CPAN… sometimes called “prename”… This is the “default” rename command in Debian and Ubuntu…

On Red Hat based distros (e.g. Fedora) it’s a completely different command that doesn’t do regex the same way that CPAN rename does (see further below).

One thing I always have difficulty with is trying to use regex to remove brackets in filenames… it almost always stumps me…

When I buy music from Bandcamp - it comes in a ZIP file - and the extracted files have “Artist - Album” in front - which is too much information, given I stored them in “Artist/Album” folders :

So :

╭─x@titan ~/MPZ/LordDying  
╰─➤  mkdir 2019-MysteriumTremendum-FLAC  
╭─x@titan ~/MPZ/LordDying  
╰─➤  cd 2019-MysteriumTremendum-FLAC 
╭─x@titan ~/MPZ/LordDying/2019-MysteriumTremendum-FLAC  
╰─➤  unzip ../Lord\ Dying\ -\ Mysterium\ Tremendum.zip 
Archive:  ../Lord Dying - Mysterium Tremendum.zip
 extracting: Lord Dying - Mysterium Tremendum - 01 Envy The End.flac  
 extracting: Lord Dying - Mysterium Tremendum - 02 Tearing at the Fabric of Consciousness.flac  
 extracting: Lord Dying - Mysterium Tremendum - 03 Nearing the End of the Curling Worm.flac  
 extracting: Lord Dying - Mysterium Tremendum - 04 The End of Experience.flac  
 extracting: Lord Dying - Mysterium Tremendum - 05 Exploring Inward (An Unwelcome Passenger).flac  
 extracting: Lord Dying - Mysterium Tremendum - 06 Severed Forever.flac  
 extracting: Lord Dying - Mysterium Tremendum - 07 Even the Darkness Went Away.flac  
 extracting: Lord Dying - Mysterium Tremendum - 08 Freed From the Pressures of Time.flac  
 extracting: Lord Dying - Mysterium Tremendum - 09 Lacerated Psyche.flac  
 extracting: Lord Dying - Mysterium Tremendum - 10 Split From a World Within, Devoid of Dreams Death, The Final Loneliness.flac  
 extracting: Lord Dying - Mysterium Tremendum - 11 Saying Goodbye to Physical Form.flac  
 extracting: cover.jpg               
╭─x@titan ~/MPZ/LordDying/2019-MysteriumTremendum-FLAC  
╰─➤  rename 's/Lord\ Dying\ -\ Mysterium\ Tremendum\ -\ //' *
╭─x@titan ~/MPZ/LordDying/2019-MysteriumTremendum-FLAC  
╰─➤  ls -al                                                                                                                                                                     2 ↵
total 1127272
drwxrwxr-x 2 x family        14 Feb  4  2023  .
drwxrwxr-x 7 x family         9 Feb  4 15:25  ..
-rw-rw-r-- 1 x family 128839185 Feb  4 14:27 '01 Envy The End.flac'
-rw-rw-r-- 1 x family  46938047 Feb  4 14:27 '02 Tearing at the Fabric of Consciousness.flac'
-rw-rw-r-- 1 x family 138962189 Feb  4 14:27 '03 Nearing the End of the Curling Worm.flac'
-rw-rw-r-- 1 x family 101695486 Feb  4 14:27 '04 The End of Experience.flac'
-rw-rw-r-- 1 x family  49599424 Feb  4 14:27 '05 Exploring Inward (An Unwelcome Passenger).flac'
-rw-rw-r-- 1 x family 167051233 Feb  4 14:27 '06 Severed Forever.flac'
-rw-rw-r-- 1 x family  54701546 Feb  4 14:27 '07 Even the Darkness Went Away.flac'
-rw-rw-r-- 1 x family  96814403 Feb  4 14:27 '08 Freed From the Pressures of Time.flac'
-rw-rw-r-- 1 x family 103724538 Feb  4 14:27 '09 Lacerated Psyche.flac'
-rw-rw-r-- 1 x family 154305181 Feb  4 14:27 '10 Split From a World Within, Devoid of Dreams Death, The Final Loneliness.flac'
-rw-rw-r-- 1 x family 107771830 Feb  4 14:27 '11 Saying Goodbye to Physical Form.flac'
-rw-rw-r-- 1 x family   3083985 Feb  4 14:27  cover.jpg
╭─x@titan ~/MPZ/LordDying/2019-MysteriumTremendum-FLAC  
╰─➤  rename 's/\,//g' *
╭─x@titan ~/MPZ/LordDying/2019-MysteriumTremendum-FLAC  
╰─➤  ls -al
total 1127272
drwxrwxr-x 2 x family        14 Feb  4  2023  .
drwxrwxr-x 7 x family         9 Feb  4 15:25  ..
-rw-rw-r-- 1 x family 128839185 Feb  4 14:27 '01 Envy The End.flac'
-rw-rw-r-- 1 x family  46938047 Feb  4 14:27 '02 Tearing at the Fabric of Consciousness.flac'
-rw-rw-r-- 1 x family 138962189 Feb  4 14:27 '03 Nearing the End of the Curling Worm.flac'
-rw-rw-r-- 1 x family 101695486 Feb  4 14:27 '04 The End of Experience.flac'
-rw-rw-r-- 1 x family  49599424 Feb  4 14:27 '05 Exploring Inward (An Unwelcome Passenger).flac'
-rw-rw-r-- 1 x family 167051233 Feb  4 14:27 '06 Severed Forever.flac'
-rw-rw-r-- 1 x family  54701546 Feb  4 14:27 '07 Even the Darkness Went Away.flac'
-rw-rw-r-- 1 x family  96814403 Feb  4 14:27 '08 Freed From the Pressures of Time.flac'
-rw-rw-r-- 1 x family 103724538 Feb  4 14:27 '09 Lacerated Psyche.flac'
-rw-rw-r-- 1 x family 154305181 Feb  4 14:27 '10 Split From a World Within Devoid of Dreams Death The Final Loneliness.flac'
-rw-rw-r-- 1 x family 107771830 Feb  4 14:27 '11 Saying Goodbye to Physical Form.flac'
-rw-rw-r-- 1 x family   3083985 Feb  4 14:27  cover.jpg╭─x@titan ~/MPZ/LordDying/2019-MysteriumTremendum-FLAC  
╰─➤  rename 's/\,//g' *
╭─x@titan ~/MPZ/LordDying/2019-MysteriumTremendum-FLAC  
╰─➤  ls -al
total 1127272
drwxrwxr-x 2 x family        14 Feb  4  2023  .
drwxrwxr-x 7 x family         9 Feb  4 15:25  ..
-rw-rw-r-- 1 x family 128839185 Feb  4 14:27 '01 Envy The End.flac'
-rw-rw-r-- 1 x family  46938047 Feb  4 14:27 '02 Tearing at the Fabric of Consciousness.flac'
-rw-rw-r-- 1 x family 138962189 Feb  4 14:27 '03 Nearing the End of the Curling Worm.flac'
-rw-rw-r-- 1 x family 101695486 Feb  4 14:27 '04 The End of Experience.flac'
-rw-rw-r-- 1 x family  49599424 Feb  4 14:27 '05 Exploring Inward (An Unwelcome Passenger).flac'
-rw-rw-r-- 1 x family 167051233 Feb  4 14:27 '06 Severed Forever.flac'
-rw-rw-r-- 1 x family  54701546 Feb  4 14:27 '07 Even the Darkness Went Away.flac'
-rw-rw-r-- 1 x family  96814403 Feb  4 14:27 '08 Freed From the Pressures of Time.flac'
-rw-rw-r-- 1 x family 103724538 Feb  4 14:27 '09 Lacerated Psyche.flac'
-rw-rw-r-- 1 x family 154305181 Feb  4 14:27 '10 Split From a World Within Devoid of Dreams Death The Final Loneliness.flac'
-rw-rw-r-- 1 x family 107771830 Feb  4 14:27 '11 Saying Goodbye to Physical Form.flac'
-rw-rw-r-- 1 x family   3083985 Feb  4 14:27  cover.jpg

I also use perl rename / prename in my Android Sync script (to sync music from my sub-collection, to my phone)… Because Samsung FAT32 SD-Card barfs on dodgy characters like words with umlauts on vowels - or question marks in filenames (WHO EVEN DOES THAT??? They should be shot!).

Right now all it does is find “?” or “|” (also another deadly sin - WHO EVEN does that??? They should be shot) :

find $SRC -iname \*\?\* -exec rename 's/\?/-Q/g' {} \;
find $SRC -iname \*\|\* -exec rename 's/\|/-/' {} \;

Spaces in filenames used to really bother me too - but I let it slide these days… But it is a PITA when you have to escape those spaces if you’re working with them in the shell… What I really need is a script that will do proper camel case - e.g.
11 Saying Goodbye to Physical Form.flac
to something like
11-SayingGoodbyeToPhysicalForm.flac
(i.e. capitalise any lowercase characters that start a word)
This bit of regex would insert a “-” at the first space in a string :
rename 's/\ /-/' *
then this
rename 's/\ //g' * (the g for “global”) would remove ALL subsequent instances of space… But that’s a bit extreme… I kinda hate regex too… I really don’t care that much about space character in filenames anymore - there’s probably some regex for :

  • capitalise any lowercase letter that starts a word

But it would probably look hideously complex and unfathomable!


On my Red Hat 8 system :

╭─x@fuglpige ~  
╰─➤  /usr/bin/rename
rename: not enough arguments
Try 'rename --help' for more information.
╭─x@fuglpige ~  
╰─➤  /usr/bin/rename --help                                                                                                                                                     1 ↵

Usage:
 rename [options] <expression> <replacement> <file>...

Rename files.

Options:
 -v, --verbose       explain what is being done
 -s, --symlink       act on the target of symlinks
 -n, --no-act        do not make any changes
 -o, --no-overwrite  don't overwrite existing files

 -h, --help          display this help
 -V, --version       display version

For more details see rename(1)
2 Likes

Hi again, :wave:

@nevj :

Well, Neville, you surely know much more about the correct syntax than I do.
All I can say is this command:

for f in *\ *; do mv "$f" "${f// /_}"; done

(bash syntax) used to work with my Lubuntu setup.
I did a research on the topic quite a while ago and found a discussion here:

I got the respective command from there. :wink:

I guess the discussion is quite thorough.

@daniel.m.tripp :

I totally agree. :+1:
In the meantime I used the rename command already twice.

Also: thanks for the detailed account of your experience with rendering filenames.
Quite impressive.

Thanks a lot to all of you and many greetings
Rosika :slightly_smiling_face:

1 Like

Here’s how I do it:

rename ‘s/ //g’ *

where
the ‘g’ means remove all instances in the file name, not just the first occurrence
the ‘*’ means do this to all files in the folder. I don’t know if it will recurse into folders.

As a test you can use the -v and -n switches (be sure to use both)
where
-v means verbose mode, i.e., show what will (or has) happen(ed)
-n means ‘no action’, i.e., don’t actually do it

Note: there is no easy way to undo the changes made so it’s wise to be very careful

2 Likes

@don.karon :

Hi Don, :wave:

thanks for the explanation.

I just looked up the syntax for brushing up on my ancient sed skills :blush:

(Bash Guide for Beginners by Machtelt Garrels)

So the s stands for the command “regular find and replace”, i.e. “substitute”…
…acording to my notes.
Yes, now it all comes back to me.

Thanks also for hinting towards the “-v” and “-n” options. I´ll try them out.

Many greetings from Rosika :slightly_smiling_face:

It’s the same thing in vi / vim… sed, perl, vi…

In vi (I refuse to call it vim) it’s :

:%s/<Search String>/<Replace With/[g] and a “g” at the end to do all occurences…

Geez - I still remember the days of Solaris where “sed” couldn’t edit a file (i.e. unlike in modern times “sed -i blah”) and you had to output to another file, then mash them around etc… perl however I think was the first to have “i” argument… been so long since I used perl to edit text files with regex (sed style - I don’t know who did it first - probably sed) I can’t remember the syntax, as these days I can just use sed directly - here’s one sed I use often - to remove entries in from my known_hosts file :

sed -i '55d' ~/.ssh/known_hosts

will delete line 55 from my known hosts file…

1 Like

Yes sed is older than perl. Sed is part of the original AT&T Unix.

I use a small C program called nbs , which filters all blanks and special characters from its input.
I use it in a script as follows

#! /bin/sh
for i in * ; do
mv “$i” “nbs $i”
done

The source code of nbs is available in an old post called “Best one-liner”

I originally wrote nbs just to show how things used to be done when there was no nice rename command. I ended up adding it to my toolkit.

2 Likes

Hi all, :wave:

thanks for your new posts.

@daniel.m.tripp :

I see. Thanks for the info.
I´ve never come into contact with vi so far. I´ve always used nano for editing config files.
But it´s good to know that some sytnax seems to be kind of universal… :blush:

@nevj :

Thanks Neville.
I guess you´re referring to: Best one-liner? - #12 by nevj .

Many greetings to all.
Rosika :slightly_smiling_face:

In any discussion on bulk-renaming, I’d be remiss if I didn’t bring up the very excellent mmv family of tools. — “mmv” for “mass move” or “multiple move”, there are also obvious and analogous mcp and mln forms.

For most types of bulk “name-mangling”, mmv is my go-to tool. Though, in truth, it’s actually 100% useless for solving this particular problem. (mmv uses wildcard-style patterns, which means the one thing it can’t handle is regexp-style filtering of arbitrary numbers of a specific character (or characters) found in arbitrary locations throughout the source string.)

But for anything that involves pattern-based matching, deconstruction, and reassembly, mmv is the bomb.

Basically, it extends the shell’s familiar wildcard matching: ? (match-1), * (match 0-∞), [...] (match any of a set of characters) by adding a reference syntax that lets you build output templates based on your input patterns. (It’s a simple 1-based numeric index, every #n in the output pattern refers to whatever was matched by the nth wildcard in the source pattern.)

It can also operate recursively, and adds an additional wildcard ; that can only be either the very first pattern character, or immediately follow a slash. A ; will expand to a directory path of any arbitrary length — basically, mmv ';*.c' would match the same as zsh’s **/*.c, but also (like the other wildcards) stores what it matches for later reference.

The syntax opens up all sort of neat tricks. (With patterns that are always single-quoted, for the exact reasons discussed above. You don’t ever want the shell touching these.) You can do things like:

# Rename all *.mp4 to *.m4a, in the same directory
mmv '*.mp4' '#1.m4a'
# do it recursively...
mmv ';*.mp4' '#1#2.m4a'  # #1 is now what ; matched, incl. /
# do it recursively across the *entire* filesystem
mmv '/;*.mp4' '/#1#2.m4a'
# Discover music files in a hierarchy where tracks are
# organized into directories by Artist:
#   Pharrell/Probably Something Autotuned.mp4
# but some of those contain album subdirectories:
#   Adele/Eleventy/Probably Something Weepy.mp3
# ...and collect it all into the current directory as:
#   Artist - Title.xtn
mcp '*/;*.m??' './#1 - #3.m#4#5'
# (where...)
#  #1 - is the * that matched the Artist directory name
#  #2 - is the ; match, either nothing or the Album/ subdir,
#         so we discard it either way
#  #3 - is the * after the ;, which matches from the start
#       of the filename up to the ".m" of the extension
#  #4#5 - are ??, the other two characters of the extension

Finally…

# Rename all TV series episode files that use the format:
#   Series.sNNeMM.Episode.Title.Format.Scene.Noise.mkv
# into the alternative format:
#   series.NxMM.episode.title.blah.blah.blah.mkv
# and lowercase any capitalized names along the way
mmv '*[Ss]0[0-9][eE][0-9][0-9]*.mkv' '#l1#3x#5#6#l7.mkv'
mmv '*[sS][1-9][0-9][eE][0-9][0-9]*.mkv' '#l1#3#4x#6#7#l8.mkv'

Let’s try that last pair:

$ ls *.mkv
some.sequel.s01e01.itll.never.last.1080p.HiMOM.mkv
Some.Show.S01E22.The.Last.Episode.720p.J011Y.R0GR.mkv
the.after.show.s14e01.all.new.cast.576i.DVDrip.mkv
The.After.Show.S14E02.Part.Deux.4K.240Hz.Humblebrag.mkv

$ mmv -v '*[Ss]0[0-9][eE][0-9][0-9]*.mkv' '#l1#3x#5#6#l7.mkv'
Some.Show.S01E22.The.Last.Episode.720p.J011Y.R0GR.mkv -> some.show.1x22.the.last.episode.720p.j011y.r0gr.mkv : done
some.sequel.s01e01.itll.never.last.1080p.HiMOM.mkv -> some.sequel.1x01.itll.never.last.1080p.himom.mkv : done

$ mmv -v '*[sS][1-9][0-9][eE][0-9][0-9]*.mkv' '#l1#3#4x#6#7#l8.mkv'
The.After.Show.S14E02.Part.Deux.4K.240Hz.Humblebrag.mkv -> the.after.show.14x02.part.deux.4k.240hz.humblebrag.mkv : done
the.after.show.s14e01.all.new.cast.576i.DVDrip.mkv -> the.after.show.14x01.all.new.cast.576i.dvdrip.mkv : done

$ ls *.mkv
some.sequel.1x01.itll.never.last.1080p.himom.mkv
some.show.1x22.the.last.episode.720p.j011y.r0gr.mkv
the.after.show.14x01.all.new.cast.576i.dvdrip.mkv
the.after.show.14x02.part.deux.4k.240hz.humblebrag.mkv
2 Likes

@FeRDNYC

Thanks, Frank, for introducing us to mmv.

It seems to be a very mighty and effective tool.
But I have to admit in order to fully understand it one has to read through your examples more than once. :wink:

At first sight it all seems complicated.

I didn´t know this command before, so I took a look at ubuntuusers (a pretty good source for German speakers; can be viewed in English with “TranslateLocally for Firefox” add-on).
… and it´s covered there, too.

Cannot believe I didn´t come across it before… :blush:

Thanks again, Frank. I´ll take a closer look at it.

Many greetings from Rosika :slightly_smiling_face:

1 Like

Oh, believe me, I had to write them more than once. And in fact, I got the second command of the last pair wrong the first time. (When I added the extra match for the first digit of the season number, which was previously hardcoded, I confused myself into thinking it became a #2 I’d no longer have to skip — in reality, it was a new #3, and all of the following indexes increased by 1.)

Basically, I agree…

Not just “seems”, it is! But it’s that nice kind of complicated where all of the complexity is built out of just a few simple rules. After all, there are only four wildcards, and three of them everyone is probably already familiar with. The reference syntax is even easier, since there’s only one form to worry about. (Well, except for the lowercase/uppercase flags I snuck in there — while a reference of #n inserts exactly what was matched by the nth wildcard, using #ln will lowercase the matched string, and #un will uppercase it.)

; makes things a bit more confusing, and honestly I rarely use that one myself. I prefer to rename files only in one directory at a time, if I need more than that I’ll cd and repeat the command until I’ve got them all.

But the nice thing about the complexity of mmv is, because it’s all built from a few simple pieces, it’s actually a lot easier to write than it is to read, explain, or demonstrate. When writing patterns, we tend to do them one or two pieces at a time (at least, I do), building them up in layers as you go. Which makes it pretty quick to assemble some incredibly complex commands, and being surprised that you actually still understand the whole thing perfectly once you get there.

…At that point, if it’s something I expect I might be reusing in the future, I’ll usually save it to a script. I don’t want to have to figure that crap out twice! :wink:

For example, I have one script I was using for years (actually more like decades), to normalize the names of downloaded radio broadcasts. There were a few different “styles” of naming that the names would be in (like the S##E## vs. #x## naming in my example above). Plus, people’s styles would evolve slightly through the years, and then I’d add more matches to account for the new formats.

That’s why, today, that name-correcting script is 67 lines long. There are 21 different mmv commands, to account for the various possible ways the filenames could be formatted. And another 27 lines are just comments explaining those commands, because without that even I’d never be able to make sense of it.

1 Like

When I said that mmv can’t do string filtering the way regular expressions can, that’s not strictly true. It can do it, it’s just really clunky.

If you want an mmv command that removes a space character from a filename, that’s simple. It’s just

mmv '* *' '#1#2'

…But that’ll only remove a single space character from a filename. If there’s more than one, the rest will be left.

…which is why, you just repeat it:

mmv '* *' '#1#2'

And if you use mmv -v, you’ll see what replacements it does, so you can just keep repeating the command over and over again until it stops matching anything. At that point, all of the spaces in your filenames are gone.

It’s not perfect, but it gets you there. So, the reason there are 21 mmv commands in that script I mentioned is, some of them are literally that simple. In fact, here are the last three lines of that script, verbatim:

mmv "*   *" "#1 #2"	# Removes tripled(!?) space
mmv "*  *" "#1 #2"	# Removes doubled space
mmv "*- -*" "#1-#2"	# Removes doubled dash

They rarely match anything, but when they do it’s free cleanup I don’t have to do by hand.

1 Like

@FeRDNYC :

Wow, Frank, you´ve offered a real tutorial here. :+1:

Thanks so much.
I´ve just read through the mmv page on ubuntuusers as it is in German.
Yet it seems that wouldn´t help much as it still appears to be complicated… :slightly_frowning_face: .

It´s especially the usage of the hash-sign in the target syntax which seems to elude my grasp…

I´ll have to read it through a few more times, I think.
Hopefully I´ll then be able to better understand your tutorial too. :blush:

Thanks again for your help.

Many greetings
Rosika :slightly_smiling_face:

How is it that if we have seemingly progressed past the the complexity and confusion of mmv etc in the last decades etc why not use a easy to use gui eg KRename that have find and replace options without having to remember all this bragging rights user crap ?