2. Script de modificare a fisierelor in recursie pe foldere

Script de modificare a fisierelor in recursie pe foldere

Date Tags bash

Table of content:

Intro

La inceput voi spune ca acest lucru trivial poate fi facut in diferite moduri, fie prin comenzi direct in consola, fie intr-un manager de fisiere. Insa eu am scris acest script pentru a intari cunostintele in limbajele de sconsola. :)

Sarcina

Avem doua directorii “input” si “output”. In fiecare din ele se afla fisire , nu are importanta de ce tip si ce contin. Este necesar de a efectua urmatoarele operatii

  1. Crearea folderelor in “output” dupa cele din “input”

  2. Efectuarea unei operatii, de ex. modificarea unui cuvint cu sed, in fisierele din “Input” si transferarea lor in “Output” cu pastrarea denumirii si denumirii folderului.

  3. Daca folderul nu exista in “Output” , acesta trebuie creat

  4. Feetch-uri:

    4.1 Raportarea erorilor.

    4.2 Numarul fisierelor modificate.

    4.3 Timpul de efectuare a ciclului.

Scriptul propriu zis

Datele de intrare vor fi niste foldere de ex. pe ani. 

.
|-- input
|   |-- 1991
|   |   |-- XX-2-145-91.pdf
|   |   |-- XX-2-189-91.pdf
|   |   |-- XX-2-4479-91.pdf
|   |   |-- XX-2-4489-91.pdf
|   |   |-- XX-2-4579-91.pdf
|   |   |-- XX-2-A-2459-91.pdf
|   |   `-- XX-2-A2-459-91.pdf
|   |-- 1992
|   |   |-- XX-2-137-92.pdf
|   |   |-- XX-2-336-92.pdf
|   |   |-- XX-2-737-92.pdf
|   |   `-- XX-2-777-92.pdf
|   |-- 1993
|   |   |-- XX-2-1654-93.pdf
|   |   |-- XX-2-3424-93.pdf
|   |   |-- XX-2-3644-93.pdf
|   |   |-- XX-2-3654-93.pdf
|   |   `-- XX-2-656-93.pdf
|   |-- 1994
|   |   |-- XX-2-211-94.pdf
|   |   |-- XX-2-221-94.pdf
|   |   |-- XX-2-3451-94.pdf
|   |   |-- XX-2-4411-94.pdf
|   |   `-- XX-2-4451-94.pdf
|   |-- 1995
|   |   |-- XX-2-3221-95.pdf
|   |   |-- XX-2-3424-95.pdf
|   |   |-- XX-2-3521-95.pdf
|   |   |-- XX-2-3522-95.pdf
|   |   |-- XX-2-3524-95.pdf
|   |   `-- XX-2-3527-95.pdf
|   |-- 1996
|   |   |-- XX-1-110-97.pdf
|   |   |-- XX-1-123-97.pdf
|   |   |-- XX-1-143-97.pdf
|   |   `-- XX-1-587-97.pdf
|   |-- 1997
|   |   |-- XX-2-22-97.pdf
|   |   |-- XX-2-993-97.pdf
|   |   `-- XX-2-995-97.pdf
|   |-- 1998
|   |   |-- XX-2-143-98.pdf
|   |   `-- XX-2-465-98.pdf
|   `-- 1999
|       |-- XX-2-110-99.pdf
|       |-- XX-2-25-99.pdf
|       |-- XX-2-3389-99.pdf
|       `-- XX-2-5555-99.pdf
|-- output
`-- script.sh

Scriptul “script.sh” - Acest script va crea doar folderele

1
2
3
4
5
6
7
8
9
#!/bin/bash
i_dir="input/"
o_dir="output/"
i=0
for dir in `ls -d $i_dir*` ; do
        mkdir $o_dir/`echo "$dir" | awk -F"/" '{print $2}'`;
        i=$(expr $i + 1);
done;
echo "Au fost create "$i " foldere"

Dupa executie avem: 

root@xubuntu:/home/jeka/scripting# ./script.sh 
Au fost create 9 foldere 


.
|-- input
|   |-- 1991
|   |   |-- XX-2-145-91.pdf
|   |   |-- XX-2-189-91.pdf
|   |   |-- XX-2-4479-91.pdf
|   |   |-- XX-2-4489-91.pdf
|   |   |-- XX-2-4579-91.pdf
|   |   |-- XX-2-A-2459-91.pdf
|   |   `-- XX-2-A2-459-91.pdf
|   |-- 1992
|   |   |-- XX-2-137-92.pdf
|   |   |-- XX-2-336-92.pdf
|   |   |-- XX-2-737-92.pdf
|   |   `-- XX-2-777-92.pdf
|   |-- 1993
|   |   |-- XX-2-1654-93.pdf
|   |   |-- XX-2-3424-93.pdf
|   |   |-- XX-2-3644-93.pdf
|   |   |-- XX-2-3654-93.pdf
|   |   `-- XX-2-656-93.pdf
|   |-- 1994
|   |   |-- XX-2-211-94.pdf
|   |   |-- XX-2-221-94.pdf
|   |   |-- XX-2-3451-94.pdf
|   |   |-- XX-2-4411-94.pdf
|   |   `-- XX-2-4451-94.pdf
|   |-- 1995
|   |   |-- XX-2-3221-95.pdf
|   |   |-- XX-2-3424-95.pdf
|   |   |-- XX-2-3521-95.pdf
|   |   |-- XX-2-3522-95.pdf
|   |   |-- XX-2-3524-95.pdf
|   |   `-- XX-2-3527-95.pdf
|   |-- 1996
|   |   |-- XX-1-110-97.pdf
|   |   |-- XX-1-123-97.pdf
|   |   |-- XX-1-143-97.pdf
|   |   `-- XX-1-587-97.pdf
|   |-- 1997
|   |   |-- XX-2-22-97.pdf
|   |   |-- XX-2-993-97.pdf
|   |   `-- XX-2-995-97.pdf
|   |-- 1998
|   |   |-- XX-2-143-98.pdf
|   |   `-- XX-2-465-98.pdf
|   `-- 1999
|       |-- XX-2-110-99.pdf
|       |-- XX-2-25-99.pdf
|       |-- XX-2-3389-99.pdf
|       `-- XX-2-5555-99.pdf
|-- output
|   |-- 1991
|   |-- 1992
|   |-- 1993
|   |-- 1994
|   |-- 1995
|   |-- 1996
|   |-- 1997
|   |-- 1998
|   `-- 1999
`-- script.sh

Versiunea prefinalaa scriptului

Acest script va insera in fiecare fisier “Sandel scrie in bash” Acest script este prefinal deoarece acesta insereaza ceva date in fisierele din folderele sursa

Scriptul “script.sh”

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
#!/bin/bash
i_dir="input/"    #Folderul in care se afla folderele cu fisierele sursa
o_dir="output/"   #Folderul in care vor fi create folderele cu fisiere
d=0               #Contorul pentru directorii 
f=0               #Contorul pentru fisiere
for dir in `ls -d $i_dir*` ; do                                 #Ciclul pentru foldere
        mkdir $o_dir/`echo "$dir" | awk -F"/" '{print $2}'`;
        for file in `ls $dir/*.pdf`; do                         #Ciclul pentru fisiere
                echo "Sandel scrie in bash" > $file;
                f=$(expr $f + 1);                               #Incrementare contor penru fisiere
        done;
        i=$(expr $i + 1);                                       #Incrementare contor pentru foldere
done;
echo "Au fost create "$i " foldere."
echo "Au fost modificate "$f "fisiere."

Executam

root@xubuntu:/home/jeka/scripting# ./script.sh    
Au fost create 9  foldere.
Au fost modificate 40 fisiere.

Versiunea finala a scriptului

Acest script va insera in fiecare fisier “Sandel scrie in bash” Acest script este prefinal deoarece acesta insereaza ceva date in fisierele din folderele sursa

Scriptul “script.sh”

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
#!/bin/bash
i_dir="input/"    #Folderul in care se afla folderele cu fisierele sursa
o_dir="output/"   #Folderul in care vor fi create folderele cu fisiere
d=0               #Contorul pentru directorii
f=0               #Contorul pentru fisiere
for dir in `ls -d $i_dir*` ; do                                 #Ciclul pentru foldere
        mkdir $o_dir/`echo "$dir" | awk -F"/" '{print $2}'`;
        for file in `ls $dir/*.pdf`; do                         #Ciclul pentru fisiere
                cp $file $o_dir/`echo "$dir" | awk -F"/" '{print $2}'`/  #Copie fisierele
                f=$(expr $f + 1);                               #Incrementare contor penru fisiere
        done;
        i=$(expr $i + 1);                                       #Incrementare contor pentru foldere
done;
echo "Au fost create "$i " foldere."
echo "Au fost modificate "$f "fisiere."


.
|-- input
|   |-- 1991
|   |   |-- XX-2-145-91.pdf
|   |   |-- XX-2-189-91.pdf
|   |   |-- XX-2-4479-91.pdf
|   |   |-- XX-2-4489-91.pdf
|   |   |-- XX-2-4579-91.pdf
|   |   |-- XX-2-A-2459-91.pdf
|   |   `-- XX-2-A2-459-91.pdf
|   |-- 1992
|   |   |-- XX-2-137-92.pdf
|   |   |-- XX-2-336-92.pdf
|   |   |-- XX-2-737-92.pdf
|   |   `-- XX-2-777-92.pdf
|   |-- 1993
|   |   |-- XX-2-1654-93.pdf
|   |   |-- XX-2-3424-93.pdf
|   |   |-- XX-2-3644-93.pdf
|   |   |-- XX-2-3654-93.pdf
|   |   `-- XX-2-656-93.pdf
|   |-- 1994
|   |   |-- XX-2-211-94.pdf
|   |   |-- XX-2-221-94.pdf
|   |   |-- XX-2-3451-94.pdf
|   |   |-- XX-2-4411-94.pdf
|   |   `-- XX-2-4451-94.pdf
|   |-- 1995
|   |   |-- XX-2-3221-95.pdf
|   |   |-- XX-2-3424-95.pdf
|   |   |-- XX-2-3521-95.pdf
|   |   |-- XX-2-3522-95.pdf
|   |   |-- XX-2-3524-95.pdf
|   |   `-- XX-2-3527-95.pdf
|   |-- 1996
|   |   |-- XX-1-110-97.pdf
|   |   |-- XX-1-123-97.pdf
|   |   |-- XX-1-143-97.pdf
|   |   `-- XX-1-587-97.pdf
|   |-- 1997
|   |   |-- XX-2-22-97.pdf
|   |   |-- XX-2-993-97.pdf
|   |   `-- XX-2-995-97.pdf
|   |-- 1998
|   |   |-- XX-2-143-98.pdf
|   |   `-- XX-2-465-98.pdf
|   `-- 1999
|       |-- XX-2-110-99.pdf
|       |-- XX-2-25-99.pdf
|       |-- XX-2-3389-99.pdf
|       `-- XX-2-5555-99.pdf
|-- output
|   |-- 1991
|   |   |-- XX-2-145-91.pdf
|   |   |-- XX-2-189-91.pdf
|   |   |-- XX-2-4479-91.pdf
|   |   |-- XX-2-4489-91.pdf
|   |   |-- XX-2-4579-91.pdf
|   |   |-- XX-2-A-2459-91.pdf
|   |   `-- XX-2-A2-459-91.pdf
|   |-- 1992
|   |   |-- XX-2-137-92.pdf
|   |   |-- XX-2-336-92.pdf
|   |   |-- XX-2-737-92.pdf
|   |   `-- XX-2-777-92.pdf
|   |-- 1993
|   |   |-- XX-2-1654-93.pdf
|   |   |-- XX-2-3424-93.pdf
|   |   |-- XX-2-3644-93.pdf
|   |   |-- XX-2-3654-93.pdf
|   |   `-- XX-2-656-93.pdf
|   |-- 1994
|   |   |-- XX-2-211-94.pdf
|   |   |-- XX-2-221-94.pdf
|   |   |-- XX-2-3451-94.pdf
|   |   |-- XX-2-4411-94.pdf
|   |   `-- XX-2-4451-94.pdf
|   |-- 1995
|   |   |-- XX-2-3221-95.pdf
|   |   |-- XX-2-3424-95.pdf
|   |   |-- XX-2-3521-95.pdf
|   |   |-- XX-2-3522-95.pdf
|   |   |-- XX-2-3524-95.pdf
|   |   `-- XX-2-3527-95.pdf
|   |-- 1996
|   |   |-- XX-1-110-97.pdf
|   |   |-- XX-1-123-97.pdf
|   |   |-- XX-1-143-97.pdf
|   |   `-- XX-1-587-97.pdf
|   |-- 1997
|   |   |-- XX-2-22-97.pdf
|   |   |-- XX-2-993-97.pdf
|   |   `-- XX-2-995-97.pdf
|   |-- 1998
|   |   |-- XX-2-143-98.pdf
|   |   `-- XX-2-465-98.pdf
|   `-- 1999
|       |-- XX-2-110-99.pdf
|       |-- XX-2-25-99.pdf
|       |-- XX-2-3389-99.pdf
|       `-- XX-2-5555-99.pdf
`-- script.sh

Varianta optimizata

Am optimizat scriptul prin includerea unei comenzi ls care arata fisierele sau folderele ls -1

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
#!/bin/bash
i_dir="input/"    #Folderul in care se afla folderele cu fisierele sursa
o_dir="output/"   #Folderul in care vor fi create folderele cu fisiere
d=0               #Contorul pentru directorii
f=0               #Contorul pentru fisiere
start=`date +%s`
for dir in `ls $i_dir -1` ; do          #Ciclul pentru foldere
        mkdir $o_dir/$dir;
        for file in `ls $i_dir/$dir/*.pdf`; do  #Ciclul pentru fisiere
                cp $file $o_dir/$dir/       #Copie fisierele
                f=$(expr $f + 1);       #Incrementare contor penru fisiere
        done;
        i=$(expr $i + 1);           #Incrementare contor pentru foldere
done;
end=`date +%s`
runtime=$((end-start))
echo "Au fost create "$i " foldere."
echo "Au fost modificate "$f "fisiere."
echo "A durat" $runtime " sec."

Recomandari

Putem folosi acest ciclu if pentru a verifica daca este deja folderul

 #-Check dir
 if [ ! -d "$DIR" ];then
        echo "Error: Please create directory $DIR, with write permissions for user \"$(/bin/id -un)\""
        exit 1
 fi

Surse

  1. https://stackoverflow.com/questions/4651437/how-to-set-a-variable-to-the-output-of-a-command-in-bash
  2. https://unix.stackexchange.com/questions/52313/how-to-get-execution-time-of-a-script-effectively

Comments

comments powered by Disqus