Attack of the clones - cloning

1

OK - I normally just use DD to clone things (e.g. an RPi raspbian image or SD card, onto another medium)…

But it drives me insane sometimes knowing that just because I’m copying a 256 GB partition - it shouldn’t worry about all them empty zero’d bits - but it copies them anyway… i.e. why can’t it be smarter and not attempt to copy empty data? Seems a very common question out there in the wild… and I’ve never seen an answer (other than doing things like pipe it through gzip or bzip or something).

So - here’s the thing :
I’m trying to clone a 256 USB 3 thumb (Samsung brand - it’s genuine) drive with Raspbian on it to a 2 TB USB 3 thumb drive (that’s got a Samsung logo silk screen onto it - but I strongly suspect it’s a fake).

Note : I’m doing all these things on my Ryzen 7 desktop machine running Ubuntu 20.04… mostly via a powered USB 3 hub…

“dd” was taking WAY too long so I canned it (and the inefficiency of “dd’ing” empty bits annoyed the crap out of me).

Installed clonezilla - but it continually barfs with ugly red writing I can barely read (red text on black terminal) - something about the partitions being mounted… so I unmount them and try again, then it complains in the text UI that there’s nothing to clone from or to - so how to do I use it to clone stuff that’s mounted… tried it as “me” and started from “sudo” : same results…

So - I tried “ddrescue” (apt get install gddrescue)… and seriously, despite someone saying it doesn’t clone empty data, it must be - because I let it run and finish and it must have taken EIGHT hours!
And the copying was dodgy anyway - because if I try to boot the Pi from it I get a kernel panic… and I’m guessing it’s default behaviour is to assume it’s copying from a possible bad source and doing lotsa CRC and stuff on the data it reads before writing - i.e. probably too much overhead…

I may just go back to using simple “dd”… if it keeps happening then I’m probably thinking this is because it’s a VERY shonky Chinese knock off of a Samsung thumb drive with shoddy parts and components… buyer beware and all that…

Because last time I did this - cloning from an SD-card to an external USB C “3” Samsung SSD - I used DD - and it took ages (but not EIGHT HOURS), but it worked in the end…

Hmmm - I just remembered Raspbian desktop installs some GUI app to clone your SD card… might try that… but I expect it will brown out as there’s not usually enough power in the Pi (Pi4B 8 GB) to do I/O on TWO USB 3 devices… might have to use a powered USB 3 hub to accomplish this… nothing’s ever easy… :smiley: and I’d rather do it on the Ryzen 'cause it’s got way more grunt than the Pi…

2

Because the whole point of cloning is based on the fundamental premise, that “everything is left as it was”. This way, you can be absolutely sure (well, assuming the hardware is not broken) that everything copy you created is absolutely precisely the same as the source.

If you’d want to only have the “occupied” space, then you’d need a solution, that isn’t based on the same fundamental premise, that cloning is.

I’m not a storage media expert. As far as I know, you can clone a half-empty drive with a normal non-copy-on-write file-system on it, for only the half that is actually occupied and it should work either way. You just need to stop cloning shortly after the point, the occupied space ends. This method may work on hard drives.

Did you make sure the 2TB are legit and it works as it is supposed to?

I think the empty data thing in this case is less about saving time and more about saving space. The program still has to understand which parts of the filesystem are actually occupied and which aren’t. So my guess would be, it still needs to go through everything. If the thumb drive has low performance, because it’s simply bad or just has too much stuff on it, relative to its max size, then it may take a long while. Especially, I/O’ing it all through USB 3 stuff does make it slower. You would need something like USB 3.2 (the good version of it) to really see a much smaller performance impact. Just plain old USB 3 isn’t holy in terms of performance. It’s just better than USB 2 and this one is truly slow and limited, not to mentioned USB 1.1.

When I read this sentence, I actually remembered what the best thing about ddrescue was, which is also the same reason I recommended and would recommend it again. It also is related to saving time:

If DD encounters a read/write error during cloning, it will entirely abort the operation and you’ll have to start over again, using a different tool anyway, or configuring dd in a very complicated, error-prone, way.

If it is the case, check if it’s legit. Though – even if it’s legit, I would rather get a really good stick for stuff like this. If you are copying so much data, perhaps even important data, then I wouldn’t recommend anyone to use some Xing Zhang Mai Arse USB stick as the destination for such a cloning operation.

That said, I actually have a couple of Samsung USB 3 thumb drives at home, which packaging and marketing is in Chinese only. They are legit though. So, one just needs to really watch out and be especially careful with bigger drives. The ones I bought are only 32GB, each.

Yes, and no. It’s not the best thing but my Raspberry Pi setup does stuff like this every single day and “it works”. It is slow. It has its issues. But it works – good enough for me.

I would generally recommend to only use powered USB (3) hubs. Implicitly powered ones are way overrated and people use them way too often, just because they are more convenient. I’d say in 50% of the cases, where people use an implicitly powered one, they should actually use one that is explicitly powered by a dedicated power supply or at least with a dedicated second USB 3 cable, to get at least some extra juice into the device, from a second USB 3 slot.

I can’t tell you how many times I’ve already seen people complaining that their 2.5" hard drive does not work as expected on the Raspberry Pi 3 B or similar models. Just use a damn explicitly powered enclosure or an explicitly powered USB hub!

People never see electricity with their own eyes, that’s why they disregard the importance of electrical power and the quality of components that deliver power, like e.g. power supply units in a computer. Such components are the most important of the entire device.

3

how dumb am I?

The penny dropped the other day… well yesterday… Apparently that GUI tool in Raspbian Pixel desktop is just a wrapper that uses “dd” to copy the /boot partition - but - it uses “cp -x” to copy the 2nd “root” partition to another medium…

So - I’m cloning this way :
dd for the /boot partition - it’s only 0.5 G and a mere blip… and then I can use rsync or cp to “clone” the root disk (e.g. use fdisk to create a 2nd partition if it doesn’t already have one, then mkfs to format), then rsync the mounted src to the mounted dest and Annette Funicello’s your aunty or something like that :

image
image300×400 45.7 KB

4

Confirmed : these alleged Samsung 2 TB USB 3 thumb drives are shonky knock offs…

Ya win some, ya lose some I guess - buyer beware…

5

For future readers, the explanation, why that is bad:

Knock-off “hard drives” (that are essentially just USB thumb drives in a box) or just plain knock-off thumb drives, which lie about their size, are tricking the user in believing the drive is 2TB, when in reality it probably only has 8GB or 16GB, by constantly overwriting the first stuff that was written, when more than the actual size of the drive is written to itself.

So, if you write 16GB to this drive with an actual size of 16GB all should be fine, however, if you continue to write more than 16GB, then the following data just overwrites the previous data. This results (obviously) in the drive never carrying more than 16GB of data.

1 Like
6

How hard would it have been for me to look on Samsung’s product page to discover the the highest capacity USB 3 thumb drives they have in this range are only 256GB? Too hard apparently…

It’s amazing how blase these shonky fly-by-nighters are… So - mostly people bypass them or “do the research” (i.e. a bit more thoroughly than the average flat earther or Q@n0n dispshit “doing research on youtube”) - but every so often one of them slips through the cracks (like me) and there’s enough of us idiots out there they can actually make some money out of spoonfed mooncalves…

This was on e-bay - and it would be probably nigh on impossible for them to police every possibly shonky item on every tiny little e-bay store out there in the wild wild west…

What amazes me yet further - is google and facebook allowing advertisers who are quite obviously scammers - especially facebook… Day after f–king day of ads in my face for things like “gaming” laptops for $29, because the factory closed and they’re runout models - my arse! They’re so obviously just taking the money and not delivering anything at all - worse yet maybe also harvesting credit card numbers - I personally believe that Facebook should be culpable for this behaviour - the onus on them is to vet advertisers… I report at least 2 or 3 of them a day, but it doesn’t seem to be a deterrent… Google is still also a bit shonky, but not as bad as Facebook IMHO…

Anyway - long story short - I’m going to fill up one of these 2TB drives and see what happens… So far it’s holding at least 200 GB - and - it’s not blazingly fast (not even just “ordinary” fast like the real deal Samsung it’s faking) - but - its not piggishly sluggish either… The big clue is when you list it and it displays manufacturer info the manufacturer is “Vendor Inc” :smiley:

Also - note - I think I paid like $18.69 USD each for them… yeah still too much for a shonky product… and that low price point should have been a big clue they weren’t genuine… WHAT WAS I THINKING? :smiley: probably a bad hair day (like most WFH days - I don’t “groom” my hair, I shave my scalp if I have to go into the office - which is rare just lately - we’re in another CV19 3 day lockdown here in West Oz [THANKS GLADYS!] - and - I’ve only had my first AZ vax so far)…

1 Like
7

If I were you, I would blame the fact how easy it is to buy something online nowadays. It starts with “Buy now” buttons, that instantly buy you the thing, and goes through all that “return if you have a problem” stuff, that all manipulates the consumer into buying the product as quickly and as thoughtlessly as possible.

1 Like
8

I don’t usually impulse buy - but - I do often plonk shit in my shopping cart, and leave it there for days and days while I think about it (and maybe do more research).

I guess it’s kinda like the old department store “lay buy” system, where you take your purchase to the lay buy counter, slap down a deposit and keep paying installments till you’ve paid the whole thing and the lay buy counter cashier gives you your item (which in my case might be payday from my newspaper “round”)… Me and my little brother used to do that with toys - but - we usually also swapped price tags with something much cheaper, then lay buyed it - heck man back in 1974 $5 was a shitload of money (in 1974 the AUD was worth more than the USD!) - you could buy 5 packets of cigarettes and two ice cold bottles of coke (unfortunately they’d already stopped putting cocaine in it 30-40 years earlier), and a small white paper bag full of mixed lollies (candy) with the change!

I’ve been eyeing off Graphic Tablet displays, from the likes of Wacom, Huiion and some mob called XP-PEN, over the last 6 months… So I put an XP-PEN 13.3" Pro in my shopping cart sometime last week - finally paid for it last night… They’re much better priced than their competitors - AND - they openly support Linux and have Linux driver downloads for all their products! $415 AUD for the tablet and an extension cable - the nearest competitor from Wacom is closer to $1000… and they supposedly ship to ANZ from within ANZ (Australia and New Zealand - AKA Oceania).

9

I do that, too. Sometimes it helps to see if the item is forgettable. If it is, there is a big chance you wouldn’t need to buy it, in the first place.

I bought a Wacom Pro graphic tablet thing (not display) quite some time ago, but in the end I used it much more rarely, than I expect, before buying that thing.

Though, I’m sure you would make more use of your device, as you tend to sketch something from time to time.
I bought it, because I wanted to stop using paper for my programming notes, as they get lost easier and are hard to sort. If you have notes on your computer, it’s much easier to manage and much easier to keep sorted, even for years.

10

My last Wacom purchase was in 2004 - a USB 8" (not a display - I don’t know if they had Cintiq back then) - and - I’ve still got it - and it still works - and - it just “works out of the box” in every Ubuntu I plug it into… I like 'em… but $400 VS $1000 pricetag difference for a similar product is too hard to ignore… I know it won’t be as good as a Wacom but I’m not a professional - and I’ve got shit eyesight and I’m red green colour blind, so all those bells and whistles from Wacom may be wasted on me…

That USB Wacom replaced a serial port 6" Wacom which I loaned to an “aquaintance” and I never got it back off them when they became a non-aquaintance…

Anyway - after drawing on an actual display, e.g. Samdung [sic] Note tablets and phones, and iPad with iPencil, and ThinkPad X series “pen enabled hybrids” - I can’t go back to a non-display version of a graphics tablet…

1 Like
11

Handy tool for verifying your storage is what you bought and paid for…

“f3probe”

On Debian / Ubuntu :

sudo apt install f3probe

To check some media inserted as /dev/sde (verify using sudo fdisk -l or sudo lsblk) - note this is destructive - do this BEFORE you put any valuable memories on there, if you want to verify your ebay purchase is not shonky :

sudo f3probe --destructive --time-ops /dev/sde

And it will report back (e.g. like those knock off Samsungs, with identical alloy metal body and Samsung logo silkscreened on them and marked [and marketed] as 2 TB) what the real size is - and show you a command to reduce the partition to the “real” data area : like :

sudo f3fix --last-sec=4897535 /dev/sde

This “downgraded” a 2 TB drive to a 2 GB drive (it shitcans the 2 TB partition, creates a 2 GB partition - but the drive still shows up as a 2 TB) - yeah… that’s like 1000 x ripped off :smiley: … Still can’t believe I fell for it… Anyway - I’ve contacted the seller and reported them to ebay for fraud (I’m not normally a snitch - but this behaviour just aint cricket).

I just don’t see the point - well maybe I guess… if they got 2 GB drives for free, and it cost them 50c to manufacture each one, and they find enough suckers like me to pay more than $10 each ???
– edit –
case in point, someone went to a reasonable amount of effort to make these shonky things look like the real thing :

20210707_005814
20210707_0058142072×1932 531 KB

The top one is the dodgy one from e-bay, the bottom one is 256 GB I used to boot one of my Pi4B’s (which I bought a bricks’n’mortar store in my town).
It would only have take a little bit more effort to make them at least 128 or 256 GB…

1 Like
12
1 Like