Python merge line with line above

Input file:

$ cat file.txt
500:120:100:X
:100:120
200:900:125
120:120
:900
120:345
12:900:1234:34
:90

Required: Join the lines which startswith : with the previous line.
i.e. required output is:

500:120:100:X:100:120
200:900:125
120:120:900
120:345
12:900:1234:34:90

The python script to achieve this:

data=open("file.txt").read().split("\n")
for i,line in enumerate(data):
    if line.startswith(":"):
        data[i-1]= data[i-1]+line
        data.pop(i)
print '\n'.join(data),

Executing it:

$ python merge-lines.py
500:120:100:X:100:120
200:900:125
120:120:900
120:345
12:900:1234:34:90

The solution using UNIX Awk can be found here

More about python enumerate function can be found here. Mentioned below is a small example on python enumerate function

>>> for i, student in enumerate(['Alex', 'Ryan', 'Deb']):
...     print i, student
...
0 Alex
1 Ryan
2 Deb
>>>

Related Posts:
- Print section of file using line number - Python
- Print line next to pattern using Python
- Print line above pattern using Python

New learning:
Python list pop method:
list.pop([i])
Remove the item at the given position in the list, and return it. If no index is specified, a.pop() removes and returns the last item in the list.

Python list append example - divide by two

Input file:

$ cat file.txt
h1|u|1
h2|5|1|1
rec1|1239400800|Sat|fan1|AX|2|10035|-|2|50
rec2|1239400800|Sat|fan1|AX|2|-|-|2|17
rec5|1239400801|Sat|fan3|AY|5|10035|-|2|217
rec8|1239400804|Sat|fan5|AX|2|5|-|2|970

Required Output:
- Lines starting with "h1" or "h2", no action required, just print.
- Lines starting with "rec", divide the values starting from 6th field by 2.

Required output is:

h1|u|1
h2|5|1|1
rec1|1239400800|Sat|fan1|AX|1|5017|-|1|25
rec2|1239400800|Sat|fan1|AX|1|-|-|1|8
rec5|1239400801|Sat|fan3|AY|2|5017|-|1|108
rec8|1239400804|Sat|fan5|AX|1|2|-|1|485

The python script:

fp = open("file.txt", "rU")
lines = fp.readlines()
fp.close()

for line in lines:
       if line.startswith("h1"):
               print line,
       if line.startswith("h2"):
               print line,
       if line.startswith("rec"):
               f=line.split("|")
               r = f[5:]
               l = []
               for each in r:
                       try:
                               l.append(str(int(each)/2))
                       except ValueError:
                               l.append(each)

               t = "|".join(f[0:5]) + "|" + "|".join(l)
               print t.rstrip()

Python - Replace based on another file

$ cat main.txt
P|34|90
T|12
R|0|1291870414|ip1|890
R|1|1291870415|ip5|690
R|2|1291870415|ip1|899
R|3|1291870412|ip2|896
R|4|1291870418|ip3|999
R|5|1291870419|ip5|191

$ cat lookup.txt
ip7|172.17.4.8
ip1|172.17.4.3
ip5|172.17.4.9
ip4|172.17.4.2
ip3|172.17.4.1
ip2|172.17.4.6
ip6|172.17.4.7

Required Output:
Replace the 4th field (pipe delimited) of the 'R' lines of 'main.txt' with the corresponding lookup value from 'lookup.txt' i.e. 'ip1' to be replaced with '172.17.4.3', 'ip2' with '172.17.4.6' etc.

P|34|90
T|12
R|0|1291870414|172.17.4.3|890
R|1|1291870415|172.17.4.9|690
R|2|1291870415|172.17.4.3|899
R|3|1291870412|172.17.4.6|896
R|4|1291870418|172.17.4.1|999
R|5|1291870419|172.17.4.9|191

The python script:

import sys
d={}
for line in open("lookup.txt"):
    line=line.strip().split("|")
    d[line[0]]=line[-1]
for line in open(sys.argv[1]):
    if line.startswith('P'):
        print line,
    if line.startswith('T'):
        print line,
    if line.startswith('R'):
        line=line.strip().split("|")
        print '|'.join(line[0:3])+'|'+d[line[3]]+'|'+'|'.join(line[4:])

Executing it:

$ python replace-from-file.py main.txt
P|34|90
T|12
R|0|1291870414|172.17.4.3|890
R|1|1291870415|172.17.4.9|690
R|2|1291870415|172.17.4.3|899
R|3|1291870412|172.17.4.6|896
R|4|1291870418|172.17.4.1|999
R|5|1291870419|172.17.4.9|191

Related Posts:
- Lookup file operation using Python
- Lookup file in python using Dictionary
- Simple python file lookup function
- Find text string in file in Python

Simple python file lookup function for newbie

Config file 'ip-mapping.txt' is a file of the following format:

$ cat /home/testusr/work/ip-mapping.txt
#id:ip1,ip2,ip3
200:172.17.4.12,172.17.4.14,172.17.4.10
205:172.17.4.14,172.17.4.14,172.17.4.11
210:172.17.4.12,172.17.4.18,172.17.4.18
208:172.17.4.11,172.17.4.10,172.17.4.19

Required: Create a simple python function which will accept an 'id' and will return 'ip1' from the list of ips.

The python script:

import os,sys

config = '/home/testusr/work/ip-mapping.txt'
if not os.path.exists(config):
    print config+' file not present'
    sys.exit()

def getip(id):
   all = open(config).readlines()
   for line in all:
        if line.startswith('#'):
            continue
        f=line.split(":")
        if f[0]==id:
                return f[1].split(',')[0]

ip=getip('205')
print ip

Executing it:

$ python get-ip.py
172.17.4.14

I am sure there will be much better solutions to this problem, please comment, really appreciated.

The description about Exit function of 'sys' module (source) :

sys.exit([arg])
Exit from Python. This is implemented by raising the SystemExit exception, so cleanup actions
specified by finally clauses of try statements are honored, and it is possible to intercept the exit
attempt at an outer level. The optional argument arg can be an integer giving the exit status
(defaulting to zero), or another type of object. If it is an integer, zero is considered “successful
termination” and any nonzero value is considered “abnormal termination” by shells and the
like. Most systems require it to be in the range 0-127, and produce undefined results otherwise.

Some systems have a convention for assigning specific meanings to specific exit codes, but these
are generally underdeveloped; Unix programs generally use 2 for command line syntax errors
and 1 for all other kind of errors. If another type of object is passed, None is equivalent to
passing zero, and any other object is printed to sys.stderr and results in an exit code of 1. In
particular, sys.exit("some error message") is a quick way to exit a program when an error occurs.

Related posts on lookup on file using python:

• Lookup file in python using dictionary
• Find text string in file using python
• Lookup file operation in python
  

Python - count instances without a specific line

Input file:

$ cat data.txt
k:begin:0
i:0:66
i:1:76
t:1:143
k:end:0
k:begin:7
i:0:55
i:1:65
i:2:57
k:end:7
k:begin:2
i:0:10
i:1:0
t:1:10
k:end:7
k:begin:2
i:0:46
t:0:46
k:end:7
k:begin:9
i:0:66
i:1:56
i:2:46
i:3:26
k:end:7

Required: Count total number of instances (one instance being from a 'k:begin' to 'k:end' line) which do not have a 't' line associated.

The python program:

import sys
count=0
data = open(sys.argv[1]).readlines()
for i in range(len(data)):
       if data[i].startswith("k:end") and data[i-1].split(":")[0]!="t":
               count=count+1
print count

Executing it:

$ python count_no_t.py data.txt
2

Related post:

- Print last instance of a file using Python
- Print line next to pattern in Python
- Print line above pattern in Python

Split a file into sub files in python

Input file 'file.txt' is basically a log file containing running information of certain device interfaces in the following format:

$ cat file.txt
debug: on
max allowed connection: 3
tr#45
Starting: interface 78e23
Fan Status: On
Speed: -
sl no: 3431212-2323-90
vendor: aledaia
Stopping: interface 78e23
tr#90
newdebug received
Starting: interface 78e24
Fan Status: Off
Speed: 5670
sl no: 3431212-2323-90
vendor: aledaia
Stopping: interface 78e24
Starting: interface 68e73
Fan Status: On
Speed: 1200
sl no: 3431212-2323-90
vendor: aledaia
Stopping: interface 68e73
tr#99

Required:

Split the above file into sub-files such that
- Each sub file conatins information of an interface (basically information from 'Starting' and 'Stopping' of the interface)
- Sub-file name should be of the format: interface-name_someSLno.txt

The python script:

flag=0;c=0
for line in open("file.txt"):
    line=line.strip()
    if line.startswith("Stopping"):
        flag=0
        o.close()
    if line.startswith("Starting"):
        interface=line.split(" ")[2]
        flag=1;c=c+1
        o=open(interface+"_"+str(c)+".txt","w")
    if flag and not line.startswith("Starting"):
        print >>o, line

Output:

$ cat 78e23_1.txt
Fan Status: On
Speed: -
sl no: 3431212-2323-90
vendor: aledaia

$ cat 78e24_2.txt
Fan Status: Off
Speed: 5670
sl no: 3431212-2323-90
vendor: aledaia

$ cat 68e73_3.txt
Fan Status: On
Speed: 1200
sl no: 3431212-2323-90
vendor: aledaia

Print line next to pattern in python

Input file: 'file.txt' contains results of a set of students in the following format (i.e. for any student result precedes the student id)

$ cat file.txt
Result:Pass
id:502
Result:Fail
id:909
Result:Pass
id:503
Result:Pass
id:501
Result:Fail
id:802

Required: Print the Ids of the students who have passed the exam.

The python program:

fp = open("passedids.txt","w")
data = open("file.txt").readlines()
for i in range(len(data)):
        if data[i].startswith("Result:Pass"):
                fp.write(data[i+1].split(":")[1])

Executing it:

$ python printnext.py
$ cat passedids.txt
502
503
501

Another python alternative:

fp=open('file.txt','r')
previous_line = ""

for current_line in fp:
    if 'Result:Pass' in previous_line:
        print current_line.split(":")[1],
    previous_line = current_line
fp.close()

Executing it:

$ python printnext1.py
502
503
501

Related post:

- Print line above pattern in python

Print line above pattern in python

Input file: 'data.txt' contains results of a set of students in the following format.

$ cat data.txt
id:502
Result:Pass
id:909
Result:Fail
id:503
Result:Pass
id:501
Result:Pass
id:802
Result:Fail

Required:
Print the Ids of the students who have passed the exam.

The python program:

fp = open("passedids.txt","w")
data = open("data.txt").readlines()
for i in range(len(data)):
        if data[i].startswith("Result:Pass"):
                fp.write(data[i-1].split(":")[1])

Output:

$ cat passedids.txt
502
503
501

Python - delete lines between two pattern

Input file:

$ cat input.txt
test1
test2
test3
BEGIN
test4
test5
test6
END
test7
test8
test9
BEGIN
test10
test11
END
test12

Required:
From the above file delete the lines which are between a BEGIN-END block and print rest of the lines.

The python script deleteline.py:

flag = 1
linelist = open("input.txt").readlines()
for line in linelist:
    if line.startswith("BEGIN"):
        flag = 0
    if line.startswith("END"):
        flag = 1
    if flag and not line.startswith("END"):
       print line,

Executing it:

$ python deleteline.py
test1
test2
test3
test7
test8
test9
test12

Now if we need to print the lines which are between a BEGIN-END block.
Here is a modification of the above scirpt.

flag = 1
linelist = open("input.txt").readlines()
for line in linelist:
    if line.startswith("BEGIN"):
        flag = 0
    if line.startswith("END"):
        flag = 1
    if not flag and not line.startswith("BEGIN"):
       print line,

Executing it:

$ python printline.py
test4
test5
test6
test10
test11

Related post:

- Lookup file operation using python

Performing multiple split in python

This post is mainly for python newbies.
Input file:

$ cat data.txt
File Start
#Comment
Gid034:s9823,I1290,s9034,s1230
Gid309:s9034,I5678,s1293,s4590
Gid124:s2145,K9008,s2381,s0234
Gid213:s9012,N9034,s8913,s9063
#End

Required: Extract the 3 rd field (colored blue) from the above file. i.e. required output:

I1290
I5678
K9008
N9034

Here we would need to split the required lines twice (one for field separator : and then for comma) to extract the desired column.

Using bash cut command or awk, the solution would be:
$ grep ^Gid data.txt | cut -d":" -f2 | cut -d"," -f2
$ awk -F"[:,]" '/^Gid/{print $3}' data.txt

The python code:

for line in open("data.txt"):
    if line.startswith("Gid"):
        print line.split(":")[1].split(",")[1]