Ctrl-c Crashes Python After Importing Scipy.stats
Solution 1:
Here's a variation on your posted solution that may work. Maybe there's a better way to solve this problem -- or maybe even avoid it all together by setting an environment variable that tells the DLL to skip installing a handler. Hopefully this helps until you find a better way.
Both the time module (lines 868-876) and _multiprocessing module (lines 312-321) call SetConsoleCtrlHandler. In the case of the time module, its console control handler sets a Windows event, hInterruptEvent. For the main thread, time.sleep waits on this event via WaitForSingleObject(hInterruptEvent, ul_millis), where ul_millis is the number of milliseconds to sleep unless interrupted by Ctrl+C. Since the handler that you've installed returns True, the time module's handler never gets called to set hInterruptEvent, which means sleep cannot be interrupted.
I tried using imp.init_builtin('time') to reinitialize the time module, but apparently SetConsoleCtrlHandler ignores the 2nd call. It seems the handler has to be removed and then reinserted. Unfortunately, the time module doesn't export a function for that. So, as a kludge, just make sure you import the time module after you install your handler. Since importing scipy also imports time, you need to pre-load libifcoremd.dll using ctypes to get the handlers in the right order. Finally, add a call to thread.interrupt_main to make sure Python's SIGINT handler gets called.
For example:
import os
import imp
import ctypes
import thread
import win32api
# Load the DLL manually to ensure its handler gets# set before our handler.
basepath = imp.find_module('numpy')[1]
ctypes.CDLL(os.path.join(basepath, 'core', 'libmmd.dll'))
ctypes.CDLL(os.path.join(basepath, 'core', 'libifcoremd.dll'))
# Now set our handler for CTRL_C_EVENT. Other control event # types will chain to the next handler.defhandler(dwCtrlType, hook_sigint=thread.interrupt_main):
if dwCtrlType == 0: # CTRL_C_EVENT
hook_sigint()
return1# don't chain to the next handlerreturn0# chain to the next handler
win32api.SetConsoleCtrlHandler(handler, 1)
>>> import time
>>> from scipy import stats
>>> time.sleep(10)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyboardInterrupt
[1] interrupt_main calls PyErr_SetInterrupt. This trips Handlers[SIGINT] and calls Py_AddPendingCall to add checksignals_witharg. In turn this calls PyErr_CheckSignals. Since Handlers[SIGINT] is tripped, this calls Handlers[SIGINT].func. Finally, if func is signal.default_int_handler, you'll get a KeyboardInterrupt exception.
Solution 2:
Setting the environment variable FOR_DISABLE_CONSOLE_CTRL_HANDLER to 1seems to fix the issue, but only if it is set before loading offending packages.
import osos.environ['FOR_DISABLE_CONSOLE_CTRL_HANDLER'] = '1'
[...]
EDIT: While Ctrl+C doesn't crash python anymore, it also fails to stop the current calculation.
Solution 3:
I have been able to get a half-workaround by doing this:
from scipy import stats
import win32api
defdoSaneThing(sig, func=None):
returnTrue
win32api.SetConsoleCtrlHandler(doSaneThing, 1)
Returning true in the handler stops the chain of handlers so that the meddling Fortran handler is no longer called. However, this workaround is only partial, for two reasons:
- It does not actually raise a KeyboardInterrupt, meaning that I can't react to it in Python code. It just drops me back to the prompt.
- It doesn't fully interrupt things in the way that Ctrl-C normally does in Python. If in a fresh Python session I do a
time.sleep(3)and hit Ctrl-C, the sleep is immediately aborted and I get a KeyboardInterrupt. With the above workaround, the sleep is not aborted, and control returns to the prompt only after the sleep time is up.
Nonetheless, this is still better than crashing the whole session. To me this raises the question of why SciPy (and any other Python libraries that rely on these Intel libraries) don't do this themselves.
I'm leaving this answer unaccepted in the hope that someone can provide a real solution or workaround. By "real" I mean that pressing Ctrl-C during a long-running SciPy calculation should work just like it does when SciPy is not loaded. (Note that this doesn't mean it has to work immediately. Non-SciPy calculations like plain Python sum(xrange(100000000)) may not immediately abort on Ctrl-C, but at least when they do, they raise a KeyboardInterrupt.)
Solution 4:
Here's code to patch the dll to remove the call that installs the Ctrl-C handler:
import os
import os.path
import imp
import hashlib
basepath = imp.find_module('numpy')[1]
ifcoremd = os.path.join(basepath, 'core', 'libifcoremd.dll')
withopen(ifcoremd, 'rb') as dll:
contents = dll.read()
m = hashlib.md5()
m.update(contents)
patch = {'7cae928b035bbdb90e4bfa725da59188': (0x317FC, '\xeb\x0b'),
'0f86dcd44a1c2e217054c50262f727bf': (0x3fdd9, '\xeb\x10')}[m.hexdigest()]
if patch:
contents = bytearray(contents)
contents[patch[0]:patch[0] + len(patch[1])] = patch[1]
withopen(ifcoremd, 'wb') as dll:
dll.write(contents)
else:
print'Unknown dll version'EDIT: Here's how I added a patch for the x64. Run python.exe in the debugger, and set a breakpoint for SetConsoleCtrlHandler until you get to the call you want to patch out:
Microsoft (R) WindowsDebuggerVersion6.12.0002.633AMD64Copyright (c) MicrosoftCorporation. All rights reserved.
CommandLine: .\venv\Scripts\python.exe
...
0:000> .symfix0:000> bp kernel32!SetConsoleCtrlHandler0:000> g
Breakpoint0 hit
KERNEL32!SetConsoleCtrlHandler:
00007ffc`c25742f0 ff252af00400 jmp qword ptr [KERNEL32!_imp_SetConsoleCtrlHandler (00007ffc`c25c3320)] ds:00007ffc`c25c3320={KERNELBASE!SetConsoleCtrlHandler (00007ffc`bfa12e10)}
0:000> k 5Child-SPRetAddrCallSite00000000`007ef7a8 00000000`71415bb4 KERNEL32!SetConsoleCtrlHandler
*** ERROR: Symbol file could not be found. Defaulted to export symbols forC:\WINDOWS\SYSTEM32\python27.dll -
00000000`007ef7b0 00000000`7035779f MSVCR90!signal+0x17c00000000`007ef800 00000000`70237ea7 python27!PyOS_getsig+0x3f00000000`007ef830 00000000`703546cc python27!Py_Main+0x21ce700000000`007ef880 00000000`7021698c python27!Py_InitializeEx+0x40c0:000> g
Python2.7.11 (v2.7.11:6d1b6a68f775, Dec52015, 20:40:30) [MSC v.150064 bit (AMD64)] on win32
Type"help", "copyright", "credits" or "license"for more information.
>>> import numpy
...
Breakpoint0 hit
KERNEL32!SetConsoleCtrlHandler:
00007ffc`c25742f0 ff252af00400 jmp qword ptr [KERNEL32!_imp_SetConsoleCtrlHandler (00007ffc`c25c3320)] ds:00007ffc`c25c3320={KERNELBASE!SetConsoleCtrlHandler (00007ffc`bfa12e10)}
0:000> k 5Child-SPRetAddrCallSite00000000`007ec308 00000000`7023df6e KERNEL32!SetConsoleCtrlHandler00000000`007ec310 00000000`70337877 python27!PyTime_DoubleToTimet+0x10ee00000000`007ec350 00000000`7033766d python27!PyImport_IsScript+0x4f700000000`007ec380 00000000`70338bf2 python27!PyImport_IsScript+0x2ed00000000`007ec3b0 00000000`703385a9 python27!PyImport_ImportModuleLevel+0xc820:000> g
...
>>> import scipy.stats
...
Breakpoint0 hit
KERNEL32!SetConsoleCtrlHandler:
00007ffc`c25742f0 ff252af00400 jmp qword ptr [KERNEL32!_imp_SetConsoleCtrlHandler (00007ffc`c25c3320)] ds:00007ffc`c25c3320={KERNELBASE!SetConsoleCtrlHandler (00007ffc`bfa12e10)}
0:000> k 5
*** ERROR: Symbol file could not be found. Defaulted to export symbols forC:\Users\kevin\Documents\\venv\lib\site-packages\numpy\core\libifcoremd.dll -
Child-SPRetAddrCallSite00000000`007ed818 00007ffc`828309eb KERNEL32!SetConsoleCtrlHandler00000000`007ed820 00007ffc`828dfa44 libifcoremd!GETEXCEPTIONPTRSQQ+0xdb00000000`007ed880 00007ffc`828e59d7 libifcoremd!for_lt_ne+0xc27400000000`007ed8b0 00007ffc`828e5aff libifcoremd!for_lt_ne+0x1220700000000`007ed8e0 00007ffc`c292ddc7 libifcoremd!for_lt_ne+0x1232f0:000> ub 00007ffc`828309eb
libifcoremd!GETEXCEPTIONPTRSQQ+0xbb:
00007ffc`828309cb 00e8 add al,ch
00007ffc`828309cd df040b fild word ptr [rbx+rcx]
00007ffc`828309d0 0033 add byte ptr [rbx],dh
00007ffc`828309d2 c9 leave
00007ffc`828309d3 ff15bf390e00 call qword ptr [libifcoremd!for_lt_ne+0x40bc8 (00007ffc`82914398)]
00007ffc`828309d9 488d0d00efffff lea rcx,[libifcoremd!for_rtl_finish_+0x20 (00007ffc`8282f8e0)]
00007ffc`828309e0 ba01000000 mov edx,1
00007ffc`828309e5 ff158d390e00 call qword ptr [libifcoremd!for_lt_ne+0x40ba8 (00007ffc`82914378)]
We'll patch out the lea instruction with a relative jmp (which is 0xeb followed by the number of bytes to jump)
0:000> ? 00007ffc`828309eb - 00007ffc`828309d9
Evaluate expression: 18 = 00000000`00000012
0:000> f 00007ffc`828309d9 L2 eb 10
Filled 0x2 bytes
0:000> ub 00007ffc`828309eb
libifcoremd!GETEXCEPTIONPTRSQQ+0xbe:
00007ffc`828309ce 040b add al,0Bh
00007ffc`828309d0 0033 add byte ptr [rbx],dh
00007ffc`828309d2 c9 leave
00007ffc`828309d3 ff15bf390e00 call qword ptr [libifcoremd!for_lt_ne+0x40bc8 (00007ffc`82914398)]
00007ffc`828309d9 eb10 jmp libifcoremd!GETEXCEPTIONPTRSQQ+0xdb (00007ffc`828309eb)
00007ffc`828309db 0d00efffff or eax,0FFFFEF00h
00007ffc`828309e0 ba01000000 mov edx,100007ffc`828309e5 ff158d390e00 call qword ptr [libifcoremd!for_lt_ne+0x40ba8 (00007ffc`82914378)]
I don't know how the .dll file is mapped in this process, so I'll just search for 0d 00 ef ff ff in the file with a hex editor. It is a unique hit, so we can calculate the location in the .dll to patch.
0:000> db 00007ffc`828309d0
00007ffc`828309d0 0033 c9 ff 15 bf 390e-00 eb 100d 00 ef ff ff .3....9.........
00007ffc`828309e0 ba 01 00 00 00 ff 15 8d-39 0e 00 48 8d 0d 0e 9c ........9..H....
00007ffc`828309f0 0900 e8 092e 0a 0048-8d 0d 329f 0900 e8 fd .......H..2.....
00007ffc`82830a00 2d 0a 00 48 8d 0d ca ee-0e 00 e8 51 90 00 00 85 -..H.......Q....
00007ffc`82830a10 c0 0f 8588020000 e8-38 fa 0a 00 ff 154e 39 ........8.....N9
00007ffc`82830a20 0e 00 89 c1 e8 d7 2d 0a-00 48 8d 05 f8 be 11 00 ......-..H......
00007ffc`82830a30 4532 e4 c7 050b 4a 13-000000000041 bd 01 E2....J......A..
00007ffc`82830a40 00 00 00 48 89 05 06 4a-13 00 ff 15 30 39 0e 00 ...H...J....09..
0:000> ? 00007ffc`828309d9 - 00007ffc`828309d0
Evaluate expression: 9 = 00000000`000000090:000> ? 00007ffc`828309d9 - 00007ffc`828309d0 + 3FDD0
Evaluate expression: 261593 = 00000000`0003fdd9
0:000>
Ok, I've patched the dll at 0x3fdd9. Let's see what it looks like now:
Microsoft (R) Windows Debugger Version 6.12.0002.633 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
CommandLine: .\venv\Scripts\python.exe
...
0:000> bp libifcoremd!GETEXCEPTIONPTRSQQ+c9
Bp expression 'libifcoremd!GETEXCEPTIONPTRSQQ+c9' could not be resolved, adding deferred bp
0:000> g
Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 52015, 20:40:30) [MSC v.150064 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license"for more information.
>>> import scipy.stats
...
Breakpoint 0 hit
libifcoremd!GETEXCEPTIONPTRSQQ+0xc9:
00007ffc`845909d9 eb10 jmp libifcoremd!GETEXCEPTIONPTRSQQ+0xdb (00007ffc`845909eb)
0:000> u
libifcoremd!GETEXCEPTIONPTRSQQ+0xc9:
00007ffc`845909d9 eb10 jmp libifcoremd!GETEXCEPTIONPTRSQQ+0xdb (00007ffc`845909eb)
00007ffc`845909db 0d00efffff or eax,0FFFFEF00h
00007ffc`845909e0 ba01000000 mov edx,100007ffc`845909e5 ff158d390e00 call qword ptr [libifcoremd!for_lt_ne+0x40ba8 (00007ffc`84674378)]
00007ffc`845909eb 488d0d0e9c0900 lea rcx,[libifcoremd!GETHANDLEQQ (00007ffc`8462a600)]
00007ffc`845909f2 e8092e0a00 call libifcoremd!for_lt_ne+0x30 (00007ffc`84633800)
00007ffc`845909f7 488d0d329f0900 lea rcx,[libifcoremd!GETUNITQQ (00007ffc`8462a930)]
00007ffc`845909fe e8fd2d0a00 call libifcoremd!for_lt_ne+0x30 (00007ffc`84633800)
0:000>
So now were are jmping over pushing the arguments on the stack and the function call. So its Ctrl-C handler will not be installed.
Solution 5:
Workaround: patch SetControlCtrlHandler
import ctypes
SetConsoleCtrlHandler_body_new = b'\xC2\x08\x00'if ctypes.sizeof(ctypes.c_void_p) == 4elseb'\xC3'try: SetConsoleCtrlHandler_body = (lambda kernel32: (lambda pSetConsoleCtrlHandler:
kernel32.VirtualProtect(pSetConsoleCtrlHandler, ctypes.c_size_t(1), 0x40, ctypes.byref(ctypes.c_uint32(0)))
and (ctypes.c_char * 3).from_address(pSetConsoleCtrlHandler.value)
)(ctypes.cast(kernel32.SetConsoleCtrlHandler, ctypes.c_void_p)))(ctypes.windll.kernel32)
except: SetConsoleCtrlHandler_body = Noneif SetConsoleCtrlHandler_body:
SetConsoleCtrlHandler_body_old = SetConsoleCtrlHandler_body[0:len(SetConsoleCtrlHandler_body_new)]
SetConsoleCtrlHandler_body[0:len(SetConsoleCtrlHandler_body_new)] = SetConsoleCtrlHandler_body_new
try:
import scipy.stats
finally:
if SetConsoleCtrlHandler_body:
SetConsoleCtrlHandler_body[0:len(SetConsoleCtrlHandler_body_new)] = SetConsoleCtrlHandler_body_old
Post a Comment for "Ctrl-c Crashes Python After Importing Scipy.stats"