django-unicorn icon indicating copy to clipboard operation
django-unicorn copied to clipboard

Error 500 in production

Open xaitec opened this issue 2 years ago • 23 comments

Every thing runs fine in development but when I try to run on the production server I get this message. I will say that some calls work about 20% of the time. Not sure what else to do or what info to give as it works well in development.

For production I use a docker container with nginx as a proxy.

Can unicorn assets be served by whitenoise? Would I need to have nginx handle serving this url?

image

xaitec avatar May 18 '22 12:05 xaitec

It's curious that things work 20% of the time. You are showing the JS console error, but can show the Python error log on the server (or do you use something like Sentry, Datadog, etc.)? That will give you the Python traceback and might give us some more clues.

adamghill avatar May 18 '22 13:05 adamghill

This is the message that was returned

image

xaitec avatar May 20 '22 11:05 xaitec

Hmm, I've never seen that happen only in production before. Is employee_services.components.employee_disabilities.EmployeeDisabilitiesView the correct class? Do you have different settings for production than local dev? Maybe UNICORN app settings are different per environment?

adamghill avatar May 21 '22 19:05 adamghill

Yes it's the correct class. I have not looked into the unicorn app settings yet, I like to use default settings unless necessary.

On Sat, May 21, 2022, 15:01 Adam Hill @.***> wrote:

Hmm, I've never seen that happen only in production before. Is employee_services.components.employee_disabilities.EmployeeDisabilitiesView the correct class? Do you have different settings for production than local dev? Maybe UNICORN app settings https://www.django-unicorn.com/docs/settings/#apps are different per environment?

— Reply to this email directly, view it on GitHub https://github.com/adamghill/django-unicorn/issues/408#issuecomment-1133752643, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFFXNU5XQ5QWPPIWAZW5HLVLEXHPANCNFSM5WIHEVSA . You are receiving this because you authored the thread.Message ID: @.***>

xaitec avatar May 21 '22 21:05 xaitec

I would like to add that I do have django-htmx in my project, as i was planning to convert components to unicorn.

xaitec avatar May 24 '22 09:05 xaitec

I removed htmx and tried again. this was the resulting error message

Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py", line 47, in inner response = get_response(request) File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 181, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "", line 2, in message File "/usr/local/lib/python3.10/site-packages/django_unicorn/decorators.py", line 17, in timed return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django_unicorn/views/init.py", line 44, in wrapped_view return view_func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django/utils/decorators.py", line 130, in _wrapped_view response = view_func(request, *args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django/views/decorators/http.py", line 40, in inner return func(request, *args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django_unicorn/views/init.py", line 528, in message json_result = _handle_component_request(request, component_request) File "/usr/local/lib/python3.10/site-packages/django_unicorn/views/init.py", line 374, in _handle_component_request return _process_component_request(request, component_request) File "/usr/local/lib/python3.10/site-packages/django_unicorn/views/init.py", line 83, in _process_component_request component = UnicornView.create( File "", line 2, in create File "/usr/local/lib/python3.10/site-packages/django_unicorn/decorators.py", line 17, in timed return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django_unicorn/components/unicorn_view.py", line 864, in create raise ComponentClassLoadError(message, locations=locations) django_unicorn.errors.ComponentClassLoadError: The component class 'employee_services.components.employee_disabilities.EmployeeDisabilitiesView' could not be loaded.

xaitec avatar Jun 01 '22 18:06 xaitec

Yeah, I wouldn't expect htmx to interfere with Unicorn, but I'm glad you tried just in case. That error is raised at https://github.com/adamghill/django-unicorn/blob/main/django_unicorn/components/unicorn_view.py#L867 which should only happen if the EmployeeDisabilitiesView class could not be instantiated. I'm not sure how else to debug this problem that only happens 20% of time and only in production, but I'd love to help out however I can.

adamghill avatar Jun 03 '22 02:06 adamghill

Just a silly question but is https a requirement?

xaitec avatar Jun 29 '22 12:06 xaitec

https is not technically a requirement, but probably a good idea so that the JSON payloads are protected.

adamghill avatar Jun 29 '22 14:06 adamghill

I understand that. I recently tried again as I upgraded to Django 4. The application in now not public and only used on the LAN. The first action to the unicorn component works, but any other action gives the same browser error as before. I will investigate more on my down time and give the feedback.

On a side note I really like the project and wondered if there was anyway I can help (non financial for now)

xaitec avatar Jun 29 '22 15:06 xaitec

I'm sorry it's still not working. :/ I'll try to dig into this again when I have time.

Since this is just a side project of mine, any help is appreciated! PRs for features/bug fixes, documentation updates, adding better test coverage, etc. :)

adamghill avatar Jun 29 '22 15:06 adamghill

Does this having anything to do with Gunicorn at all?

When I start my application with something like gunicorn --workers=4 app.wsgi a lot of requests fail, but some go through.

If I start gunicorn with only 1 worker, then everything works perfectly.

Also, if I start it with 'gunicorn --workers=4 --worker-class=gevent app.wsgi' and use gevent, then again, everything works 100% of the time.

@adamghill Is this expected behaviour?

nickpolet avatar Nov 22 '22 11:11 nickpolet

Hmm, the number of workers shouldn't matter. My gunicorn.conf for django-unicorn.com.

One thing I didn't ask above: what are your cache settings in production? Unicorn uses cache to store component information and I'm wondering if you are using local memory the cache would fail for some of the workers, but would work for one of them perhaps? I always use redis in production, so maybe that's why I never saw this before?

adamghill avatar Nov 23 '22 17:11 adamghill

I was using in memory cache, 3 workers.

On Wed, Nov 23, 2022 at 1:01 PM Adam Hill @.***> wrote:

Hmm, the number of workers shouldn't matter. My gunicorn.conf https://github.com/adamghill/django-unicorn.com/blob/main/gunicorn.conf for django-unicorn.com.

One thing I didn't ask above: what are you cache settings in production? Unicorn uses cache to store component information and I'm wondering if you are using local memory the cache would fail for some of the workers, but would work for one of them perhaps? I always use redis in production, so maybe I never saw this before?

— Reply to this email directly, view it on GitHub https://github.com/adamghill/django-unicorn/issues/408#issuecomment-1325390745, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFFXNWBA5DHQXNVFJEJH7DWJZEVJANCNFSM5WIHEVSA . You are receiving this because you authored the thread.Message ID: @.***>

xaitec avatar Nov 23 '22 18:11 xaitec

If you are able to try it with a filesystem cache (if you only have one server running) or memcache/redis if you have more than one server that would be helpful. I can also come up with some fallback mechanisms if caching is not available.

adamghill avatar Nov 23 '22 21:11 adamghill

Ok, I'll check it out. I really think this can be the real issue

On Wed, 23 Nov 2022, 17:42 Adam Hill, @.***> wrote:

If you are able to try it with a filesystem cache (if you only have one server running) or memcache/redis if you have more than one server that would be helpful. I can also come up with some fallback mechanisms if caching is not available.

— Reply to this email directly, view it on GitHub https://github.com/adamghill/django-unicorn/issues/408#issuecomment-1325693597, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFFXNTP65RMDI5Q5WYG6Z3WJ2FU3ANCNFSM5WIHEVSA . You are receiving this because you authored the thread.Message ID: @.***>

xaitec avatar Nov 25 '22 23:11 xaitec

FWIW I had the same issue, and changing from a memory cache to a Memcached instance has seemed to fix it.

Thanks @adamghill for the pointer with this!

regoawt avatar Dec 03 '22 12:12 regoawt