Messages in queue not cleaned

[willpower232]

There are a number of messages in the queued_messages table which have the locked_by field filled in with a reference presumably to a no-longer-existing process. The messages date back some months which means the original message has been removed from the server.

This means that there are several queue pages in the UI which throw exceptions because they're trying to load messages which don't exist.

Is the issue that queued_messages are never cleaned up by the automatic process or that the automatic clean up process removes messages despite them being present in the queue?

[Manishjodhani]

@willpower232 we can try that by using smtp relays......but there is problem with authentication in relays Is there any solution for that????

[willpower232]

(Manish's comment is unrelated and resolved by https://github.com/atech/postal/issues/48#issuecomment-373334092)

[willpower232]

In case anyone else needs the fix, the workaround looks like this

Add the following at the end of the queue method here https://github.com/atech/postal/blob/master

       countbroken = 0
       for message in @messages
               begin
                       test = message.message
               rescue
                       message.destroy
                       countbroken += 1
               end
       end

       if countbroken > 0
               @messages = @server.queued_messages.order(:id => :desc).page(params[:page])
       end

[hadifarnoud]

can this help with #717 ? and can I run this via rails console bin/rails c?

[willpower232]

If you have deleted the messages from the main table and are now unable to open the queued messages page, then this will help you.

The process of catching the exception to delete a disassociated message might be able to be run from the console but I am not sure how.

[saporus]

In case anyone else needs the fix, the workaround looks like this

Add the following at the end of the queue method here https://github.com/atech/postal/blob/master

       countbroken = 0
       for message in @messages
               begin
                       test = message.message
               rescue
                       message.destroy
                       countbroken += 1
               end
       end

       if countbroken > 0
               @messages = @server.queued_messages.order(:id => :desc).page(params[:page])
       end

Hello @willpower232

I trust you are doing great.

What is the file location to apply this solution?

Thanks

[willpower232]

Terribly sorry, not sure what happened to the link.

I added that code between these two lines https://github.com/atech/postal/blob/master/app/controllers/servers_controller.rb#L70-L71

[saporus]

Alright.

Thank you!

[saschaarthur]

can confirm that the "queued messages" counter is increasing over month. locked at and locked by was set, thats why those messages were never cleaned up.

my fix for this is running regulary:

update queued_messages set locked_at=null, locked_by=null WHERE locked_at is not NULL AND locked_at <= NOW() - INTERVAL 6 WEEK

Somewhat dirty, but still solving the issue of stuck queued messages bringing the counter back to 0

[pjv]

@willpower232 are you still running this patch in Postal V2? Is it still needed?

[willpower232]

I'm not aware of this issue being fixed so am still running this patch.

[ndTEC]

@willpower232 Is this going to be fixed for V2? Is not feasible this approach with docker. In fact any fix that require us to bind files from containers and modify them is not ok.

[SHELA]

Hello, any updates? still error in v2

[leoborlot]

@willpower232

How the def will be at the end?

71  def queue
 72 #    @messages = @server.queued_messages.order(:id => :desc).page(params[:page])
 73 #    @messages_with_message = @messages.include_message
 74     countbroken = 0
 75        for message in @messages
 76                begin
 77                        test = message.message
 78                rescue
 79                        message.destroy
 80                        countbroken += 1
 81                end
 82        end
 83
 84        if countbroken > 0
 85                @messages = @server.queued_messages.order(:id => :desc).page(params[:page])
 86        end
 87
 88   end

[willpower232]

I added the code after @message = and before @messages_with_message = as those lines need to remain

def queue
	@messages = @server.queued_messages.order(:id => :desc).page(params[:page])

	countbroken = 0
	for message in @messages
		begin
			test = message.message
		rescue
			message.destroy
			countbroken += 1
		end
	end
	if countbroken > 0
		@messages = @server.queued_messages.order(:id => :desc).page(params[:page])
	end

	@messages_with_message = @messages.include_message
end

[adamcooke]

See #2869