Using 'sort --unique' I end up with a tiny handful of duplicate lines

Shell Scripting & Linux Command Line
21

@FBClark, I think I found it!
The forum motor plays havoc with my copy-paste :crazy_face:
It automagiclly replaced my simple apostrophs to something opening, then closing apostroph:
tr -d ‘\015’
There should be only the simple (opening?) apostrophs.

1 Like
22

If you can isolate a pair of your apparent duplicates into their own little file, and then pipe it into “od -xc” to dump it in hex and ascii, you may see hidden differences:

[kurt@T3400 ~]$ cat dupes
X       Y
X       Y

[kurt@T3400 ~]$ cat dupes | od -xc
0000000    0958    0a59    2058    2020    2020    2020    0a59    000a
          X  \t   Y  \n   X                               Y  \n  \n
0000017
[kurt@T3400 ~]$
23

Thanks Kurt, I’ll put this in my notes. I got my problem solved by installing and using dos2unix.

1 Like
24

Hi @FBClark

I edited the links in your original thread because it was leading to an adult website. You were trying to blacklist it I think. I replaced it with example.com so that other members don’t click on it accidently, specially on a work computer.

I hope you understand.