foxreplace
foxreplace copied to clipboard
Woundorf
A Large Substitution List Disables FireFox
If there are many regex rules FF freezes until FoxReplace completes. Not just
the window where the tab being operated on is, but every FF window. I have more
than 1Gig free RAM during the process. Is this a problem with JAVA or with FF
or with FoxReplace?
What version of the product are you using? On what operating system?
FireFox 20.0.1
FoxReplace 0.15.0
Windows 7
Original issue reported on code.google.com by [email protected] on 26 Sep 2013 at 4:30
It has nothing to do with Java because Java isn't used. Also, I don't think it
has anything to do with RAM unless the Firefox process is consuming 1 GB or
more. So, I suppose the freeze is due to a high CPU usage for a while.
Maybe there is some part of the code that can be optimized, but I can't do
anything without a specific reproducible case. Can you provide a substitution
list and a webpage where this is reproducible? Also, how much freeze time are
we talking about? Less than a second? A few seconds? More? And what CPU do you
have?
Original comment by [email protected] on 26 Sep 2013 at 8:33
Overall CPU's is about 12% and none of them is maxing out. Every second CPU is
idle (to be expected since each pair is not two physically discrete CPU's
AFAIK).
Freeze lasts minutes. But the substitution list is probably the longest one you
have seen :)
i7-2630QM @2GHz
Windows7 in 64 bit mode
The pages I am operating on are password protected so I cannot give access to
them easily but maybe you can edit the json subtitutions file so that it
applies to *any* random page (aferall actual successful substitutions are not
necessary to put it through its paces, just checking for matches should be
intensive enough). There are *many* subtitution lists so manually editing the
names of the pages they all apply to would be tedious so hopefully you can do a
global replace in a json editor.
Original comment by [email protected] on 26 Sep 2013 at 9:21
Attachments:
Hi, I haven't been able to reproduce a long freeze, even replacing the urls by
*, but I have some guesses.
First: I have observed that most substitution groups use HTML. Every time the
HTML is changed Firefox must render the whole page again. The HTML is changed
at most one time in a substitution group (the HTML is copied from innerHTML,
all substitutions applied, and the result is assigned back to innerHTML), but
there are a lot of groups sharing the same URL, so the page may be re-rendered
several times. This could be improved from my part by applying all the HTML
substitutions from all the matching gropus before assigning the result to
innerHTML.
Second: For each individual substitution the string is always replaced. I don't
know how the JavaScript engine handles this when the RegExp doesn't match and
the result is the same as the input, i.e. I don't know whether it creates a
deep copy or a shallow copy. I could try to first check if there is a match and
then apply the substitution only if it matches. I'd have to try if this is
faster or slower.
I guess that the combination of both factors (or just the first) and the large
number of substitution groups and substitutions in each group is the cause of
the problem, and maybe also the "complexity" of the affected webpages.
Original comment by [email protected] on 26 Sep 2013 at 5:05
Original comment from
https://code.google.com/p/foxreplace/issues/detail?id=130#c2
#2 [email protected]
The situation is worse than this.
Large pages (or long substitution lists) can make FoxReplace either
(1) complete but not with all substitutions executed
(2) hang indefinitely (while hogging resources) or
(3) crashing
To be fair, I am talking about a *huge* file and/or a *huge* substitution list.
You can test the limits via the files at http://jo.koan.net/foxreplace (file
names and explanation in the screenshot png should be enough but let me know if
it does not make sense).
Am using FireFox 26.0
Original comment by [email protected] on 26 Dec 2013 at 9:15
Marc although you already have two workarounds in mind to remedy the problem
may I suggest you apply them to a *paid* version? It really takes large/complex
pages/substitutions to encounter the problem and I would be happy to pay $20. I
am not using it professionally but imagine most people encountering this would
be professionals so they ought to contribute to your efforts.
Original comment by [email protected] on 29 Jan 2014 at 12:20
Thank you for your suggestion of a paid version, but it's not in my plans. I
wouldn't like to have a slower free version and the full-speed paid one,
neither from a philosophical point of view or from a practical one (having to
maintain both implementations). Also, the user base isn't big enough to make
any significant money out of it, considering only a small percentage of users
would need the paid version.
I haven't had much time lately to dedicate to FoxReplace development, but
seeing that you really need this, I will step up the priority on this issue so
it comes at the beginning of the queue when I resume.
Original comment by [email protected] on 29 Jan 2014 at 7:17
- Changed state: Accepted
- Added labels: Performance, Priority-High
- Removed labels: Priority-Medium
Thanks for escalating the priority. It may even be related to other issues
people have logged(?)
marc sez...
> Also, I don't think it has anything to do with RAM unless the Firefox process
is consuming 1 GB or more.
Actually it *is* using more than 1GB. Does that explain my problem? However (as
I said) I do have at least a gig RAM to spare.
It seems as my subs list grows incrementally, this problem grows exponentially.
FoxReplace is now useless for the large files it used to handle (albeit slowly)
previously. From what you say, a fix will not only stop the freezes but improve
performance overall. Sounds worth waiting for. It may even make a port to
Android machines (with feeble CPU) feasible(?)
Original comment by [email protected] on 11 Feb 2014 at 2:51
I have found that Firefox is sluggish when it uses 1 GB or more RAM, even when
I have 12 GB at home. And I mean sluggish for general use, not when using my
addon. I suppose it has something to do with Firefox's memory management.
Original comment by [email protected] on 12 Feb 2014 at 9:26
Hello, this happens to me with https://dl.dropbox.com/u/4967399/FoxReplace.json
- its a page load script, but it happens on auto page reloads too. Examples:
pretty much anything linked in https://news.google.pt/nwshp (ex:
http://expresso.sapo.pt) and http://pplware.sapo.pt (ex:
http://pplware.sapo.pt/informacao/engenheiros-informaticos-tem-desemprego-zero-e
m-portugal)
Sometimes it triggers a unresponsive script warning about foxreplace.js or some
other webpage script, but not always, and it's unrelated to memory usage (ff at
around 600MB will do), but it actually brings cpu usage down to next to zero,
even freezing task manager, with a dualcore + 3gb mem, and win7 sp1 x64.
Original comment by [email protected] on 6 Dec 2014 at 7:32
Forgot to add, I'm using FR 0.16.1 and FF x64 Nightly 37.0.
Original comment by [email protected] on 6 Dec 2014 at 7:34
Thank you for the examples, they will be helpful.
Original comment by [email protected] on 6 Dec 2014 at 8:56
I improved the performance slightly in 0.17.2. Now with multiprocess Firefox it shouldn't freeze the whole window but all tabs. When you receive the next release (1.0.0, currently in the review queue), it should also be faster because it's multiprocess compatible.
I leave it open though because probably there are more things that can be done.
