cosscript crash with long running scripts

[afedor]

I think this is very similar to issue #5 but I though I'd leave a simple example script that illistrates it for me. It really only appears in 10.9 for me. I've spent a lot of time trying to find the cause but most of the JavaScript stuff is black magic to me. It appears though that due to some garbage collection, javascript objects are disappearing before I'm done using them, but this really only manifests itself when you have a long running javascript script. I even went back and bisected the JavaScriptCore framework to find where the change might have started. I found something around Jun 2012 that when I reverted fixed the problem in most cases (I can give you the exact rev if you want), but that didn't really help with the real problem of finding how I could update CocoaScript to avoid the problem.

Anyway, here's the script. It always seems to crash for me around loop 26, with: 1 0x7fff8d1a0fbc JSC::JSValue::get(JSC::ExecState_, unsigned int, JSC::PropertySlot&) const 2 0x7fff8d1d06fa JSC::getByVal(JSC::ExecState_, JSC::JSValue, JSC::JSValue, JSC::ReturnAddressPtr) 3 0x7fff8d1d034e cti_op_get_by_val_generic 4 0x5971a9c02bd7 5 0x7fff8d1168b6 JSC::Interpreter::execute(JSC::ProgramExecutable_, JSC::ExecState_, JSC::JSObject_) 6 0x7fff8d1156b6 JSC::evaluate(JSC::ExecState_, JSC::SourceCode const&, JSC::JSValue, JSC::JSValue*) 7 0x7fff8d1153ad JSEvaluateScript 8 0x106c1a535 -[Mocha evalJSString:scriptPath:] 9 0x106c1a405 -[Mocha evalJSString:] 10 0x106c1a34a -[Mocha evalString:] 11 0x106c3d1fc -[COScript executeString:baseURL:] 12 0x106c3cf8a -[COScript executeString:] 13 0x106c3b1eb main 14 0x7fff969955fd start

==== script ==== var toClass = {}.toString; var itemDictPath = "/Applications/Xcode.app/Contents/Info.plist"; var itemDict = NSDictionary.dictionaryWithContentsOfFile_(itemDictPath); var itemList = itemDict["CFBundleDocumentTypes"];

function readDict(item) { var name = item["CFBundleTypeName"]; var role = item["CFBundleTypeRole"]; print("name class length " + name.length()); //NSLog("role class " + role.class() + " length " + role.length()); }

for (var j = 0; j < 100; j++) { print(" === LOOP " + j + " ===="); for (var i = 0; i < itemList.count(); i++) { readDict(itemList[i]); } }

[ccgus]

I got it to reproduce as well. Is this on 10.9.2? And I assume you're using the source straight from this repository?

[afedor]

Yes. I'm currently running 10.9.2 using the latest pull from the repository.

[samdeane]

I'd be interested in that rev, just in case it suggests anything.

One of my theories is that it's a problem relating to MOBox, which Mocha uses to associate a javascript object with a cocoa object.

These are stored in a map with weak keys but strong values. The key is the cocoa object, so if that goes away, then I think that the MOBox will get dropped from the map. The problem being that a non-retaining pointer to the box is also stored as the private data for the associated JS object. If that object is still alive, it may attempt to access the MOBox after it has died.

All of which should never happen, as MOBox sets up a retain cycle with the object it represents. If something breaks that cycles somehow though...

[samdeane]

With my hacked together Mocha 2.0 based version, I've managed to get this monster script to crash:

for (var n = 0; n < 10000; n++) { var test = [NSString stringWithString:"1"]; print(n + ": " + test); }

[samdeane]

Whereas this doesn't, even with 10 x the iterations:

for (var n = 0; n < 100000; n++) { var test = [NSString stringWithString:@"1"]; print(n + ": " + test); }

[afedor]

Well the actual revision where the break occurs is SVN r120897 of WebKit, but note that if I revert that particular revision it fixes the problem with the example script I gave above, but it does not fix it for other (more complicated) examples. Also note that it breaks both jstalk and Cocoascript, but I was just focusing on Cocoascript as it's being maintained. Here's the actual patch I made to try to revert it, in case you don't want to spend several hours or so checking out webkit, etc...

Index: runtime/JSObject.cpp
===================================================================
--- runtime/JSObject.cpp    (revision 151722)
+++ runtime/JSObject.cpp    (working copy)
@@ -57,12 +57,16 @@

 JSCell* getCallableObjectSlow(JSCell* cell)
 {
+#if 1
+    return getJSFunction(cell);
+#else
     Structure* structure = cell->structure();
     if (structure->typeInfo().type() == JSFunctionType)
         return cell;
     if (structure->classInfo()->isSubClassOf(&InternalFunction::s_info))
         return cell;
     return 0;
+#endif
 }

[samdeane]

My suspicion is that it's some sort of memory overwrite coming from Mocha, in which case the WebKit revision might be a coincidence. If the memory access pattern changes slightly, it could mask the crash again for any given case of it.

[afedor]

I currently believe this is more an issue with JavaScriptCore than with CS (or Mocha), so you can probably close this if you like. I've filed an Apple rdar://19378158 (http://openradar.appspot.com/radar?id=6174800997253120). I also feel this is similar or perhaps the same bug reported here (https://bugs.webkit.org/show_bug.cgi?id=131682), which is just raw JavaScriptCore functions.

[lukasondre]

Any updates on this issue? I'm using CocoaScript to write Sketch plugins and I keep getting cti_op_call_NotJSFunction crashes. Even just iterating over a simple array with several 1000s elements crashes it.

Interestingly, if I put in a delay of about a second after each few 100s, it doesn't crash! But that makes execution way too slow and unreliable.

Please let me know how I can help and what the progress with fixing this major issue is.

[afedor]

No response from the Apple or WebKit guys. I still believe that JSC is garbage collecting some object when it shouldn't be, but I could spend hours single-stepping through the JSC framework and not get anywhere. I'll keep trying though.

[ccgus]

I'm wondering if a similar crash could be reproduce with JavaScript for Automation- and that's another way to get a real radar filed. I've not been able to reproduce it in Script Editor though.

[afedor]

In case anyone is interested, I have a fix for this in my fork https://github.com/afedor/CocoaScript master branch. It uses the dreaded JSValueProtect/Unprotect, so it's a bit of a hack, but it works really well. I've been using it in production code with many scripts some running several hundred lines long.

[samdeane]

It's a nice pragmatic workaround...

[ccgus]

And thus it was merged. We'll see how this goes :)

[afedor]

FYI #24 is not related to this bug, although it does help a bit.

[samdeane]

It looks like we might have an alternative fix here: https://github.com/logancollins/Mocha/pull/23.

The Mocha code in the main branch of Cocoascript is a copied-in version of Mocha 1.0 (with a few modifications).

I have a branch which is re-worked to pull in Mocha 2.0 as a submodule. We should be able switch over to using that (for Sketch, at least), which means we can pull in that Mocha fix directly.

[ccgus]

Sounds good, where's the branch? I can merge that guy in (or if you want to do a pull request, that's super easy as well).

[afedor]

This is actually just a smarter implementation of my fix #24 (or logaincollins/Mocha#21), which, while it does help some things, it doesn't actually fix the crashes, either on the master or 2.0 branch of Mocha

[afedor]

I could provide a pull request that reverts my change and uses this one if you want. It's currently on the same branch as my previous pull

[ccgus]

Hello.

So, I've been trying to get Cocoa Script (mainline) to crash using some simple examples, and can't seem to do it anymore on 10.10.4. Anyone got an example that still does it? I'm also trying out the address sanitizer, to see if that finds anything.

[ccgus]

wooooooo:

=================================================================
==47764==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fff8a06c064 bp 0x7fff5d10f3f0 sp 0x7fff5d10f3d8 T0)
    #0 0x7fff8a06c063 in objc_retain (/usr/lib/libobjc.A.dylib+0x9063)
    #1 0x1051a902d in MOObject_finalize (/builds/acorn-cduspeibhaptzecrrqdrotwjeezx/Build/Products/Debug/CocoaScript.framework/Versions/A/CocoaScript+0x4802d)
    #2 0x7fff8c282226 in JSC::JSCallbackObjectData::finalize(JSC::Handle<JSC::Unknown>, void*) (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x145226)
    #3 0x7fff8c177217 in JSC::WeakBlock::sweep() (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x3a217)
    #4 0x7fff8c156617 in JSC::WeakSet::sweep() (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x19617)
    #5 0x7fff8c1565a8 in JSC::MarkedBlock::sweep(JSC::MarkedBlock::SweepMode) (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x195a8)
    #6 0x7fff8c564edd in JSC::MarkedAllocator::tryAllocateHelper(unsigned long) (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x427edd)
    #7 0x7fff8c15582b in JSC::MarkedAllocator::allocateSlowCase(unsigned long) (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x1882b)
    #8 0x7fff8c1a9040 in JSC::JSCallbackObject<JSC::JSDestructibleObject>::create(JSC::ExecState*, JSC::JSGlobalObject*, JSC::Structure*, OpaqueJSClass*, void*) (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x6c040)
    #9 0x7fff8c1a8e0f in JSObjectMake (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x6be0f)
    #10 0x1051b7e3e in -[Mocha boxedJSObjectForObject:] (/builds/acorn-cduspeibhaptzecrrqdrotwjeezx/Build/Products/Debug/CocoaScript.framework/Versions/A/CocoaScript+0x56e3e)
    #11 0x1051b72a4 in -[Mocha JSValueForObject:] (/builds/acorn-cduspeibhaptzecrrqdrotwjeezx/Build/Products/Debug/CocoaScript.framework/Versions/A/CocoaScript+0x562a4)
    #12 0x1051ad31a in MOBoxedObject_getProperty (/builds/acorn-cduspeibhaptzecrrqdrotwjeezx/Build/Products/Debug/CocoaScript.framework/Versions/A/CocoaScript+0x4c31a)
    #13 0x7fff8c4fa302 in JSC::JSCallbackObject<JSC::JSDestructibleObject>::callbackGetter(JSC::ExecState*, JSC::JSObject*, long long, JSC::PropertyName) (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x3bd302)
    #14 0x7fff8c3538bd in llint_slow_path_get_by_id (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x2168bd)
    #15 0x7fff8c558dea in llint_entry (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x41bdea)
    #16 0x7fff8c556490 in callToJavaScript (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x419490)
    #17 0x7fff8c4da5e2 in JSC::JITCode::execute(JSC::VM*, JSC::ProtoCallFrame*) (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x39d5e2)
    #18 0x7fff8c17dd7b in JSC::Interpreter::execute(JSC::ProgramExecutable*, JSC::ExecState*, JSC::JSObject*) (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x40d7b)
    #19 0x7fff8c17b943 in JSC::evaluate(JSC::ExecState*, JSC::SourceCode const&, JSC::JSValue, JSC::JSValue*) (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x3e943)
    #20 0x7fff8c17b669 in JSEvaluateScript (/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore+0x3e669)
    #21 0x1051baaa1 in -[Mocha evalJSString:scriptPath:] (/builds/acorn-cduspeibhaptzecrrqdrotwjeezx/Build/Products/Debug/CocoaScript.framework/Versions/A/CocoaScript+0x59aa1)
    #22 0x1051ba2f3 in -[Mocha evalString:atURL:] (/builds/acorn-cduspeibhaptzecrrqdrotwjeezx/Build/Products/Debug/CocoaScript.framework/Versions/A/CocoaScript+0x592f3)
    #23 0x1051d334d in -[COScript executeString:baseURL:] (/builds/acorn-cduspeibhaptzecrrqdrotwjeezx/Build/Products/Debug/CocoaScript.framework/Versions/A/CocoaScript+0x7234d)
    #24 0x1051d295e in -[COScript executeString:] (/builds/acorn-cduspeibhaptzecrrqdrotwjeezx/Build/Products/Debug/CocoaScript.framework/Versions/A/CocoaScript+0x7195e)
    #25 0x102b6300c in -[TSApplication scriptingDoJavaScript:] (/builds/acorn-cduspeibhaptzecrrqdrotwjeezx/Build/Products/Debug/Acorn.app/Contents/MacOS/Acorn+0x10007700c)```

[afedor]

I have some Unit Tests written for the Mocha framework that illustrate it. I could adapt them for CS

[ccgus]

I'd love to see one, danke.

[ccgus]

Some notes to myself here.

If I change a script from newDoc.dataOfType("public.png").writeToFile("/tmp/foo.png")

to:

var data = newDoc.dataOfType("public.png") data.writeToFile("/tmp/foo.png")

Then I don't get a crasher from an overrelease of an NSData object.

[samdeane]

Yay for address sanitizer - seems like just what we need here.

[ccgus]

Now if I can just get JSCore to build…

[afedor]

If you're thinking of build JSCore from WebKit, it's easiest to just checkout the package from svn:

svn co https://svn.webkit.org/repository/webkit/trunk

and build all of webkit (takes a long time)

./Tools/Scripts/build-webkit --debug

[ccgus]

I'll try that out this evening, thanks.

[ccgus]

Well… I couldn't get JSCore to build with the sanitizer. Boo.

[afedor]

Well if it helps, I don't think this is really a memory issue. It's really sort-of expected behavior. JSCore can garbage collect values at any time if it doesn't think they are being used internally or on the stack. What I can't figure out is that Mocha is supposed to get a Finalize callback when the objects are garbage collected so it can remove them from it's own reference, but that doesn't happen.