Long stalls/stutters (1.4+)

[karlrado]

Hey @ProgrammingDinosaur,

Your recent work on the download/processing threads looks really good. I haven't seen a missing color in quite a while.

But sometimes there's a pretty long stutter or stall. It can last several seconds and the frame rate drops to near zero during this stall.

I've been looking at the code and have some questions:

First, I changed the log message here to info

autoortho4xplane/autoortho/getortho.py

Line 1459 in 57e09fc

log.debug(f"Processing {len(unprocessed_chunks)} chunks that never started downloading (fallback only)")

and see this message come up at about the time of the stalls. So I think this is the area causing them.

Then here:

autoortho4xplane/autoortho/getortho.py

Lines 1462 to 1464 in 57e09fc

	# Don't wait for download, go straight to fallback
	future = executor.submit(process_chunk, chunk)
	result = future.result(timeout=5) # Short timeout for fallback

we seem to process the chunks serially.

Question 1: Why is this done serially and could we change this to submit them (the unprocessed_chunks) all at once and then handle the resulting futures? I know that the fallback processing is relatively fast, but sometimes there are hundreds of chunks.

Question 2: Also, when the chunk is (re) submitted to process_chunk here, what is there to keep it from waiting on the download again? The comment says don't wait on the download and go straight to the fallback, but I don't see how that works. Maybe chunk.ready is already set/signaled?

Just wanted to get your thoughts on this. I could work on a fix myself if you think I'm asking the right questions. :-)

[ProgrammingDinosaur]

Hi @karlrado.

Sorry for the late response, I distanced myself this week from this code hehe. I am just seeing the new discussions and issues

Thanks for reporting your findings, as I perhaps coul have missed them during my own testing.

About your questions:

Q1: You are right, the code currently processes the chunks sequentially, while It can be done in parallel, must have missed it.

Q2: Nothing :(, just something more I missed

Thanks a lot for bringing up the issues. Will work today on a fix

[KeyGlock666]

Would love to see this ASAP.

[karlrado]

Seems resolved or no longer relevant.