Remove Empty Line Printed From Hive Query Output Using Python

i am performing a hive query and storing the output in a tsv file in the local FS. I am running a for loop for the hive query and passing different parameters. If the hive query re

Solution 1:

Usually you would open the input file and write the non-empty lines to a second file:

withopen('file.tsv') as infile, open('filtered_file.tsv', 'w') as outfile:
    for line in infile:
        if line.strip():
            outfile.write(line)

If you want to filter the file inplace you can use FileInput with the inplace option:

import fileinput
for line in fileinput.FileInput("infile", inplace=1):
    if line.strip():
        print line

however, this uses an intermediate file and may not work in low disk space situations.

To filter the file inplace without allocating any additional disk space you could try something like this:

withopen('file.tsv', 'r+') as infile:
    read_pos = write_pos = 0
    line = infile.readline()
    while line:
        read_pos += len(line)
        if line.strip():
            infile.seek(write_pos)
            infile.write(line)
            write_pos += len(line)
        infile.seek(read_pos)
        line = infile.readline()
    # update file size to the new, possibly reduced, size
    infile.truncate(write_pos)