PQsocket() can't get socket descriptor (PG::ConnectionBad) locker error

[k1r8r0wn]

Hey there, I'm periodically getting this:

Que 2.3.0 started worker process with:  Worker threads: 2 (priorities: any, any)  Buffer size: 8  Queues:    - default (poll interval: 5s)Que waiting for jobs...#<Thread:0x00007fcbdf582598 /usr/src/app/vendor/cache/ruby/3.2.0/gems/que-2.3.0/lib/que/locker.rb:179 run> terminated with exception (report_on_exception is true):/usr/src/app/vendor/cache/ruby/3.2.0/gems/que-2.3.0/lib/que/connection.rb:63:in `exec': PQsocket() can't get socket descriptor (PG::ConnectionBad)

As I see the locker file code doesn't have the re-start logic on errors (and also there is no way to put such logs to the default logger, cause it's another locker-process).

So the k8s pod always restarts...

Any thoughts or suggestions?

Much appreciated in advance!

P.S. Also I'm using the que-scheduler gem & have one job in schedule for test purposes & have Sequel as ORM.

[k1r8r0wn]

It seems to me there is a pattern - when the Queue::Scheduler::Scheduler Job by run_at starts polling, then restarts stop... Maybe there is an issue in que-scheduler?

[hlascelles]

Interesting logs! It's strange the way the timezone changes format. Were new boxes rolling in with a new Ruby or gems?

How complex is the schedule yaml? The scheduler is nothing more than an inline single thread algorithm in a job. It spawns no new connections, processes, threads or similar. It doesn't monkey patch anything. What timezone are you in?

If you're thinking it is a problem with the scheduler do raise an issue there. But on balance I'd think it is revealing a deploy issue, memory / cpu constraints, or maybe a que issue that could happen to any other job. But I could be wrong.

[k1r8r0wn]

It's custom logger, actually (I've forgotten to tell about). It's just an example that Queue::Scheduler::Scheduler Job works without failures.