Python Watchdog Windows Wait Till Copy Finishes

I am using the Python watchdog module on a Windows 2012 server to monitor new files appearing on a shared drive. When watchdog notices the new file it kicks off a database restore

Solution 1:

In your on_modified event, just wait until the file is finished being copied, via watching the filesize.

Offering a Simpler Loop:

historicalSize = -1while (historicalSize != os.path.getsize(filename)):
  historicalSize = os.path.getsize(filename)
  time.sleep(1)
print"file copy has now finished"

Solution 2:

I would add a comment as this isn't an answer to your question but a different approach... but I don't have enough rep yet. You could try monitoring filesize, if it stops changing you can assume copy has finished:

copying = True
size2 = -1while copying:
    size = os.path.getsize('name of file being copied')
    if size == size2:
        breakelse:
        size2 = os.path.getsize('name of file being copied')
        time.sleep(2)

Solution 3:

I'm using following code to wait until file copied (for Windows only):

from ctypes import windll
import time

defis_file_copy_finished(file_path):
    finished = False

    GENERIC_WRITE         = 1 << 30
    FILE_SHARE_READ       = 0x00000001
    OPEN_EXISTING         = 3
    FILE_ATTRIBUTE_NORMAL = 0x80ifisinstance(file_path, str):
        file_path_unicode = file_path.decode('utf-8')
    else:
        file_path_unicode = file_path

    h_file = windll.Kernel32.CreateFileW(file_path_unicode, GENERIC_WRITE, FILE_SHARE_READ, None, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, None)

    if h_file != -1:
        windll.Kernel32.CloseHandle(h_file)
        finished = Trueprint'is_file_copy_finished: ' + str(finished)
    return finished

defwait_for_file_copy_finish(file_path):
    whilenot is_file_copy_finished(file_path):
        time.sleep(0.2)

wait_for_file_copy_finish(r'C:\testfile.txt')

The idea is to try open a file for write with share read mode. It will fail if someone else is writing to it.

Enjoy ;)

Solution 4:

I had a similar issue recently with watchdog. A rather simple but not very smart workaround was for me to check the change of file size in a while loop using a two-element list, one for 'past', one for 'now'. Once the the values are equal the copying is finished.

Edit: something like this.

past = 0
now = 1
value = [past, now]
while True:
    # change# test
    if value[0] == value[1]:
        break
    else:
        value = [value[1], value[0]]

Solution 5:

This works for me. Tested in windows as well with python3.7

while True:
        size_now = os.path.getsize(event.src_path)
        if size_now == size_past:
            log.debug("file has copied completely now size: %s", size_now)
            break
            # TODO: why sleep is not working here ?
        else:
            size_past = os.path.getsize(event.src_path)
            log.debug("file copying size: %s", size_past)