greenscript icon indicating copy to clipboard operation
greenscript copied to clipboard

Published 20 hours ago • verified publisher icon greenlaw110

running greenscript in a cluster on heroku

Open scottlet opened this issue 13 years ago • 14 comments

When greenscript runs in a cluster, the minified css/js often fail to load, giving 404s. Could Greenscript be creating two different CSS/JS keys, one for each node of the cluster, then trying to serve one from the other, mistakenly, or something similar?

Single instance: JS url: /public/gs/2453b7a5-2095-4d3c-979f-4cfb1963f15a.js

CSS urls: /public/gs/02344583-8dce-4cde-95ad-88b4fb9156cd.css
/public/gs/d679a5e3-3631-4ec1-af5d-de7c32c8f061.css

Clustered instance: JS url: /public/gs/2453b7a5-2095-4d3c-979f-4cfb1963f15a.js on server 1 /public/gs/763d23cb-af76-424f-ba0b-83da64cfc302.js on server 2 CSS urls: /public/gs/02344583-8dce-4cde-95ad-88b4fb9156cd.css on server 1 /public/gs/d679a5e3-3631-4ec1-af5d-de7c32c8f061.css on server 1 /public/gs/85f67f15-ae1b-4900-9f10-634a76707b96.css on server 2 /public/gs/22b8e334-8f5a-458b-bb16-60020fed3a82.css on server 2

Am I doing something wrong? Is there anything that needs to be set in the conf file?

This seems like a very similar issue to Issue #28, I'm wondering if he was running in a cluster and failed to mention it?

Regards,

Scott

scottlet avatar Jan 09 '12 20:01 scottlet

I see the problem. The keys are generated randomly with either java.io.File.createTempFile (for file cache) or UUID.randomUUID (for in-memory cache).

Will see how to address this issue. Basically I need an algorithm to generate a unique key based on a list of path/file names. At the moment the key is generated randomly as stated above, if there is an algorithm which could generate the same key for the same list of file names/paths, then the problem could be solved.

greenlaw110 avatar Jan 10 '12 05:01 greenlaw110

Hi Mr Green,

Have you been able to give this any more thought? Could you generate a hash from the paths that greenscript is loading or something similar?

scottlet avatar Jan 17 '12 06:01 scottlet

Hi generate hash not secure coz there might be hash collisions (which is seldom but does happen). I got one idea, but don't have time to work through it yet.

In your case if you are sure that your path series will not have any hash collision, you could hack into the code and use hash code as the key as a temporary workaround.

On Tue, Jan 17, 2012 at 5:37 PM, scottbert < [email protected]

wrote:

Hi Mr Green,

Have you been able to give this any more thought? Could you generate a hash from the paths that greenscript is loading or something similar?

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-3523849

greenlaw110 avatar Jan 17 '12 06:01 greenlaw110

Hi Green, I've had a look to see if I can resolve this but struggled to understand exactly what needs to change. Can you provide some pointers? cheers, Chris

tazmaniax avatar Jan 27 '12 00:01 tazmaniax

Here is a rough idea

  1. for a list of resources (js or css) lookup a cache for the generated code
  2. if the code cannot find in the cache, then generate a uuid for that list of resources, and put it into the cache (indexed by List<String>)
  3. the cache should be an interface and able to plug into greenscript core with configuration
  4. implement the cache using Play's cache mechanism in play-greenscript plugin
  5. implement Play's cache based on memcached, which could be accessed across the cluster.

On Fri, Jan 27, 2012 at 11:22 AM, tazmaniax < [email protected]

wrote:

Hi Green, I've had a look to see if I can resolve this but struggled to understand exactly what needs to change. Can you provide some pointers? cheers, Chris

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-3679732

greenlaw110 avatar Jan 27 '12 00:01 greenlaw110

And the uuid generation would have to be predictable because a cache entry could expire and all of the servers in the cluster would need to be able to generate the same cache key, yes?

On 27 Jan 2012, at 00:30, Green Luo [email protected] wrote:

Here is a rough idea

  1. for a list of resources (js or css) lookup a cache for the generated code
  2. if the code cannot find in the cache, then generate a uuid for that list of resources, and put it into the cache (indexed by List<String>)
  3. the cache should be an interface and able to plug into greenscript core with configuration
  4. implement the cache using Play's cache mechanism in play-greenscript plugin
  5. implement Play's cache based on memcached, which could be accessed across the cluster.

On Fri, Jan 27, 2012 at 11:22 AM, tazmaniax < [email protected]

wrote:

Hi Green, I've had a look to see if I can resolve this but struggled to understand exactly what needs to change. Can you provide some pointers? cheers, Chris

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-3679732

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-3679830

tazmaniax avatar Jan 27 '12 00:01 tazmaniax

If possible u could allow developer to persist the path list and uuid into a database in case cache outdated

On Fri, Jan 27, 2012 at 11:35 AM, tazmaniax < [email protected]

wrote:

And the uuid generation would have to be predictable because a cache entry could expire and all of the servers in the cluster would need to be able to generate the same cache key, yes?

On 27 Jan 2012, at 00:30, Green Luo [email protected] wrote:

Here is a rough idea

  1. for a list of resources (js or css) lookup a cache for the generated code
  2. if the code cannot find in the cache, then generate a uuid for that list of resources, and put it into the cache (indexed by List<String>)
  3. the cache should be an interface and able to plug into greenscript core with configuration
  4. implement the cache using Play's cache mechanism in play-greenscript plugin
  5. implement Play's cache based on memcached, which could be accessed across the cluster.

On Fri, Jan 27, 2012 at 11:22 AM, tazmaniax < [email protected]

wrote:

Hi Green, I've had a look to see if I can resolve this but struggled to understand exactly what needs to change. Can you provide some pointers? cheers, Chris

Reply to this email directly or view it on GitHub:

https://github.com/greenlaw110/greenscript/issues/30#issuecomment-3679732

Reply to this email directly or view it on GitHub:

https://github.com/greenlaw110/greenscript/issues/30#issuecomment-3679830

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-3679900

greenlaw110 avatar Jan 27 '12 06:01 greenlaw110

Is there some way that greenscript can hook into the Play precompiler so that the css & js get generated at precompile time? that would solve this problem on Heroku.

jamesward avatar Jan 30 '12 23:01 jamesward

Can you check the current version on master? That has integrated a contribution coming from https://github.com/tazmaniax/greenscript which is supposed to solved the Heroku problem. I didn't tested it because I don't have a app running on Heroku

On Tue, Jan 31, 2012 at 10:45 AM, James Ward < [email protected]

wrote:

Is there some way that greenscript can hook into the Play precompiler so that the css & js get generated at precompile time? that would solve this problem on Heroku.

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-3730535

greenlaw110 avatar Jan 31 '12 00:01 greenlaw110

I just tested this in Heroku and this is working great!

Thanks again for the help!

mgonto avatar Feb 16 '12 17:02 mgonto

After trying this more, it's still happening. However, it happens less often.

What do you need from me to reproduce this?

mgonto avatar Mar 06 '12 19:03 mgonto

The solution is use the same UUID for the same list of js file names. Meaning everytime your server start the same list of js file requests will generate the same output js file name. I am not sure how this still lead to 404 error coz I am not on Heroku. I will probably need you guys to find out what happened. Might due to for certain cause that one of the requests to your server does not load the same sequence of js file list, in that case I have solution at the moment.

On Wed, Mar 7, 2012 at 6:22 AM, Martin Gontovnikas < [email protected]

wrote:

After trying this more, it's still happening. However, it happens less often.

What do you need from me to reproduce this?

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-4351122

greenlaw110 avatar Mar 06 '12 19:03 greenlaw110

We're having this problem with the CSS, it's the same as with the JS, right?

I'll try to figure this out on Heroku.

Thanks

On Tue, Mar 6, 2012 at 4:37 PM, Green Luo [email protected] wrote:

The solution is use the same UUID for the same list of js file names. Meaning everytime your server start the same list of js file requests will generate the same output js file name. I am not sure how this still lead to 404 error coz I am not on Heroku. I will probably need you guys to find out what happened. Might due to for certain cause that one of the requests to your server does not load the same sequence of js file list, in that case I have solution at the moment.

On Wed, Mar 7, 2012 at 6:22 AM, Martin Gontovnikas < [email protected]

wrote:

After trying this more, it's still happening. However, it happens less often.

What do you need from me to reproduce this?

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-4351122

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-4351386

mgonto avatar Mar 06 '12 19:03 mgonto

Yes, css is the same as with js.

On Wed, Mar 7, 2012 at 6:58 AM, Martin Gontovnikas < [email protected]

wrote:

We're having this problem with the CSS, it's the same as with the JS, right?

I'll try to figure this out on Heroku.

Thanks

On Tue, Mar 6, 2012 at 4:37 PM, Green Luo [email protected] wrote:

The solution is use the same UUID for the same list of js file names. Meaning everytime your server start the same list of js file requests will generate the same output js file name. I am not sure how this still lead to 404 error coz I am not on Heroku. I will probably need you guys to find out what happened. Might due to for certain cause that one of the requests to your server does not load the same sequence of js file list, in that case I have solution at the moment.

On Wed, Mar 7, 2012 at 6:22 AM, Martin Gontovnikas < [email protected]

wrote:

After trying this more, it's still happening. However, it happens less often.

What do you need from me to reproduce this?

Reply to this email directly or view it on GitHub:

https://github.com/greenlaw110/greenscript/issues/30#issuecomment-4351122

Reply to this email directly or view it on GitHub:

https://github.com/greenlaw110/greenscript/issues/30#issuecomment-4351386

Reply to this email directly or view it on GitHub: https://github.com/greenlaw110/greenscript/issues/30#issuecomment-4351770

greenlaw110 avatar Mar 06 '12 20:03 greenlaw110