When is BigQueryInsertJobOperator supposed to reattach to an existing big query job?

[mallman-experian]

When is BigQueryInsertJobOperator supposed to reattach to an existing big query job? Because from what I can tell there are absolutely no circumstances whatsoever that BigQueryInsertJobOperator will ever successfully reattach to an existing job id.

And I mean never, as in, under no circumstances and under no input configuration whatsoever.

I believe this is unintended behavior that was introduced in this commit:

d2eed7a

Note that there is literally no code path in this commit where re-attachment will succeed. It will always raise an exception, because _handle_job_error raises an exception if job.state != "DONE" and the last thing the reattachment strategy does is call that method with a job.state != "DONE".

Sorry I don't have an actual bug to report. This is just my observation from scrutinizing this source code.

Cheers.

[potiuk]

Sorry I don't have an actual bug to report. This is just my observation from scrutinizing this source code.

You can always propose a PR to fix it.

[mallman-experian]

I don't want to.

[potiuk]

Then you can create an issue describing it well, and we can assign a good first issue and let someeon fix it.

[mallman-experian]

The problem is, I don't have a bug report. I don't want to send someone on a wild goose chase in case the code actually works correctly.

I'm not actually using this code. I just saw this seeming contradiction and wanted to raise it as possible issue with the community. I hope this is helpful, though it falls short of a bug report.

Maybe someone will eventually report a bug related to this. I looked for related reports on google and came up empty. Looks like no one is using this code or I'm just wrong about this being a problem.

[potiuk]

Hopefully yes if there is an issue. It's one of thousands of operators we have and hopefully someone who will have a problem will post an issue or even better PR fixing it.