Re: Grep and mv

From: WCB (wbarwell_at_Mungggedd.mylinuxisp.com)
Date: 07/15/05 

Date: Fri, 15 Jul 2005 14:54:43 -0500

Chris F.A. Johnson wrote:

>>> If it does contain spaces my script will work. AS possible problem
>>> is that leading spaces are converted to hyphens which mv then
>>> interprets as options.
>>>
>>> for i in xx*
>>> do
>>> f=$(grep -m 1 "HCO*" "$i")
>>> while :
>>> do
>>> case $f in
>>> \ *) f=${f# } ;; ## remove leading space
>>> *\ ) f=${f% } ;; ## remove trailing space
>>> *\ *) f=${f// /-} ;; ## convert spaces to hyphens
>>> *--*) f=${f//--/-} ;; ## convert multiple hyphens to a single
>>> hyphen
>>> *) break ;; ## nothing needs doing; exit loop
>>> esac
>>> done
>>> mv "$i" "$f"
>>> done
>>>
>>> You repeatedly leave out the quotes in the snippets you have
>>> posted. Use the above script EXACTLY as it is; copy it (either cut
>>> and paste it, or save the message and edit it), do not retype it.
>>>
>>> If it doesn't work, tell us EXACTLY what happens. Use set -x and
>>> redirect stderr to a file.
>>
>> Cut and pasted. changed mod and owner.
>> Loaded fresh test files in directory
>>
>> I ran this script.
>> still it has trailing spaces.
>
> What is "it"?

File names.

>
>> In the terminal, they show up as ?
>
> In what context do they (whatever "they" are) "show up"?

If I type ls in the terminal, I see a ? at end of each file name.
This is how a terminal shows a blank at end of a file name.
If I go to KDE and peek at it with Kwrite is shows a space
by a square. If I click on a file using properties, I can see
the cursor is not at the end of the name, but a space beyond.
If I erase that space. I no longe see the box in Kwrite nor
nor "?" when I type ls in terminal.

The file is a file, I can open and read it despite the trailing
space. So it is a space, not some control character or the other
nor a quirk in how its displayed.

Its a very real space.

Each and every file name has a jolly little space appended
at the end this way.

 
>> and are seen boxes in properties.
>
> What are?

the spaces.
>
>> I can easily edited it out from properties.
>
> Properties of what?

A given file. Properties gives me name (with obvious space
size of file, permissions, the usual We are interested in what
is at the end of these file names. Properties in KDE allows
me to not only confirm, it is a real live space, but to remove
that space.
No more "box" in Kwrite, no more "?" at end of that file as
now viewed in at a terminal.

>> listing in the terminal looks like
>>
>> HCO-BULLETIN-OF-10-MARCH-1965?
>> HCO-BULLETIN-OF-29-MARCH-1965?
>> HCO-BULLETIN-OF-2-APRIL-AD15?
>> HCO-BULLETIN-OF-4-APRIL-AD15?
>> HCO-BULLETIN-OF-5-APRIL-1965?
>> HCO-BULLETIN-OF-5-MARCH-1965?
>> HCO-BULLETIN-OF-7-APRIL-AD15?
>>
>> Each file is a file, so its renaming a real file.
>
> What files did you have to start with?

xx03, xx04, xx05 and so on.

These files have been checked and run through dos2unix
just in case.
No control characters, not DOS not Mac, not Word,
simple ascii. At the end of the name to be extracted,
no spaces. cat -v shows a ^M at end of the string extracted.
"^M", not " ^M". So our mystery space is not coming from
there.

cat -v xx03 show no other characters beyond ascii than ^M.
All other test files are likewise clean.

My original file was extracted from a CD. It was one
big text file I broke down into sections and then into
individual files using csplit which had no
problems with doing so. The Nano editor displays them
with no artifacts or problems, nor does Kate nor Kwrite
have problems. Cat -v shows no problems.

So I am very definitely sure its not a buggered file
nor word processor control characters that are at any
way an issue.

Now all I have to do is find how to make 4000 xxNN
files have somewhat meaningful names.

The files I am using here are clean and readable.
 My silly little grep script extracts the names as
expected. No spaces or artifacts there showing
up, so its again, not a problem with the files.

> What files did you end up with?

HCO-BULLETIN-OF-10-MARCH-1965?
HCO-BULLETIN-OF-29-MARCH-1965?
HCO-BULLETIN-OF-2-APRIL-AD15?
HCO-BULLETIN-OF-4-APRIL-AD15?
HCO-BULLETIN-OF-5-APRIL-1965?
HCO-BULLETIN-OF-5-MARCH-1965?
HCO-BULLETIN-OF-7-APRIL-AD15?

They are files, I can cat them and read them.
  
> Is that what you wanted? If not, what is different from what you
> wanted? Please post the script you used, directly from the file.

This script almost does the trick except for the spaces.
Since cat -V shows the string extracted ends as an example,

                  HCO-BULLETIN-OF-4-APRIL-AD15^M

The space is not coming from within the file with this string.
So there should not be a trailing space to cut.
So something seems to be ADDING a space.
Since line 8 in your script cuts trailing spaces
it seems a logical deduction that the space is added
after line 8, somewhere. I have no idea where that artifact
is coming from. I tried moving the cut trailing spaces
to last after dealing with hyphens.
The resulting files still have a trailing space. So its obviously an
artifact coming from the break statment on.

The other thing that weirds me out big time is, your
mv "$i" "$f" seems to work.
When I extract $f and $i, my echo "$f" >> F
tests show repeatedly that I am extracting real
data that mv should then use.

Neither mv $i $f nor mv "$i" "$f" work, I just get
different error messages.

My script does one file and then gives a bunch of error
messages.

Just before that step, if I do echo $i >> I
and echo $f >> F I show that I am getting
the xxNN files and extracted name files OK
to that point.

It does not matter if I use mv "$i" "$f" or mv $i $f
 
****************
#1/bin/bash

# mover4

for i in xx*
 do
  
grep -m 1 HCO* $i > x
sed 's/^ *//' x > y
sed 's/ /-/g' y > z

# cat x >> X
# cat y >> Y
# cat z >> Z
# OK to here so far

f=$(cat z)
# echo $i >I
# echo $f >F
# mv $i $f
mv "$i" "$f"
done

*****************************

files to start

xx03 xx10 xx100 xx101 xx102 xx103 xx104

If echo "$f" >> I
if echo "$i" >> F
  
I

xx104
xx03
xx10
xx100
xx101
xx102
xx103
xx104

F

HCO-BULLETIN-OF-27-DECEMBER-1967
HCO-BULLETIN-OF-5-MARCH-1965
HCO-BULLETIN-OF-18-APRIL-AD15
HCO-BULLETIN-OF-11-OCTOBER-1967
HCO-BULLETIN-OF-9-NOVEMBER-1967
HCO-POLICY-LETTER-OF-22-NOVEMBER-1967
HCO-BULLETIN-OF-28-NOVEMBER-1967
HCO-BULLETIN-OF-27-DECEMBER-1967

OK, this works!

************************************
 
Now mv "$i" "$f"
Exactly as used in your script.

results?

ls ..

HCO-BULLETIN-OF-5-MARCH-1965?* xx10* xx101* xx103* y
x xx100 xx102* xx104*
z

It does not work like yours. And adds a space
as a final insult.
Which space is NOT at end of echo "$f" >> F names
This is bash 2.05 B patch level 0. As supplied by
Mandrake 10.1.

The error messages I get when running this script are:
mv: cannot move `xx10' to 1`': No such file or directory
This for all remaining xxNN files.

Why your mv works, and not mine is a mystery to me.
But this is where the mystery space is coming from also,
obviously since the one file that did work with my script
has one also.

It sure look like a mv bug to me.

Next step, googling for info on possible broken mv. 

-- 
When I shake my killfile I can hear them buzzing.

Relevant Pages

  • CMUCL "-script" hack now available
    ... CMUCL won't start up correctly.] ... $ cat script_header.cmucl ... If you want a script which just sets up an environment ... Welcome to the customized test REPL ...
    (comp.lang.lisp)
  • Re: Batch Cisco Configuration Script
    ... configuration of many routers but now I cannot find it unfortunately. ... The way the script would work is as follows. ... I can't use Kiwi Cat ... open telnet session. ...
    (comp.dcom.sys.cisco)
  • Re: script reports file not found
    ... any and all feedback on this script appreciated. ... when it's kicking out the cat of the file in question. ... primitive linux script to run some ruby stuff. ... why all the cat commands? ...
    (Fedora)
  • Re: My auth.log analyze script
    ... users on this group I decided to write my own auth.log analyzing script. ... rm $TMPFILE1 ... You don't need cat: ... # Sort out Unique IPs ...
    (alt.os.linux)
  • Re: Plain text file on web server, questions please...
    ... strictly for online viewing that would add the HTML tags to the online ... a utility that would do this or can it be done with a shell script ... thing, this way that if anything changed in the FAQ, it would also be ... cat head.html joke.txt tail.html> joke_$.html ...
    (alt.os.linux)