Move files based on condition in python

Contents of /tmp/mydir/

$ ls /tmp/mydir/ | paste -

logWA241.dat
logWA249.dat
logWA258.dat
logWA259.dat

Required: Move the above files to directories under /tmp/mydir such that logWA241.dat should go to dir /tmp/mydir/1, similarly logWA258.dat to /tmp/mydir/8 (i.e. dir name with last digit before .dat extn)

The python script:

import os
DIR="/tmp/mydir"
for file in os.listdir(DIR):
   Absfile = os.path.join(DIR,file)
   if os.path.isfile(Absfile) and file.endswith(".dat"):
       Dname = Absfile.split(".")[:-1][-1][-1:]
       Dname = os.path.join(DIR,Dname)
       if not os.path.exists(Dname):
           os.mkdir(Dname)
           os.system('mv '+Absfile+' '+Dname)
       else:
           os.system('mv '+Absfile+' '+Dname)

The contents of /tmp/mydir/ after exection of the above script.

$ ls -R /tmp/mydir/

o/p:

/tmp/mydir/:
1  8  9

/tmp/mydir/1:
logWA241.dat

/tmp/mydir/8:
logWA258.dat

/tmp/mydir/9:
logWA249.dat  logWA259.dat

Related concepts and modules:
- Python os module

Remove duplicate based on field using python

Input file:

$ cat file.txt
DD:12
AA:11
EE:13
AA:11
BB:09
DD:13
AA:78

Required output: Keep only 1st occurrence of each unique first field. i.e. required output:

DD:12
AA:11
EE:13
BB:09

Python script:

d = {}

input = file('file.txt')
for line in input:
   ff = line.split(':',1)[0]
   if ff not in d:
      d[ff] = 1
      print line,

Awk alternative:

$ awk -F ":" '!x[$1]++' file.txt
DD:12
AA:11
EE:13
BB:09

Grouping related items using python dictionary

Thought of trying a awk post that I did someday back on my bash scripting blog.

Input file:

$ cat data.txt
Manager1|sw1
Manager3|sw5
Manager1|sw4
Manager2|sw9
Manager2|sw12
Manager1|sw2
Manager1|sw0

Required output: Group the similar engineers which are under common Manager. i.e. required output:

Manager3|sw5
Manager2|sw9,sw12
Manager1|sw1,sw4,sw2,sw0

The python program:

d={}

fp = open("grp.txt","w")
for line in open("data.txt"):
   line=line.strip().split("|")
   d.setdefault(line[0],[])
   d[line[0]].append(line[1])

print d
for i,j in d.iteritems():
   fp.write(i+"|"+','.join(j)+"\n")

Output file after executing above script:

$ cat grp.txt
Manager3|sw5
Manager2|sw9,sw12
Manager1|sw1,sw4,sw2,sw0

Related concepts:
setdefault(key[, default])
If key is in the dictionary, return its value. If not, insert key with a value of default and return default. default defaults to None.

Dictionary iteritems : Read here