error when run create_memmaps.sh

[WhuRan]

I followd the instruction, download the datasets from https://duebenchmark.com/data, but when I run create_memmaps.sh as section 2.1(Process datasets into memmaps (binarization)), exception raised:
WARNING:benchmarker.data.reader.benchmark_dataset:Cannot locate directory data/DocVQA/png/hnnp0227_15
WARNING:benchmarker.data.reader.benchmark_dataset:Cannot locate directory data/DocVQA/png/hnnp0227_13
WARNING:benchmarker.data.reader.benchmark_dataset:Cannot locate directory data/DocVQA/png/mxlm0227_1
WARNING:benchmarker.data.reader.benchmark_dataset:Cannot locate directory data/DocVQA/png/rsvb0228_1
WARNING:benchmarker.data.reader.benchmark_dataset:Cannot locate directory data/DocVQA/png/hnnp0227_72
WARNING:benchmarker.data.reader.benchmark_dataset:Cannot locate directory data/DocVQA/png/xrcy0227_34
WARNING:benchmarker.data.reader.benchmark_dataset:Cannot locate directory data/DocVQA/png/xnhw0228_29

What the 'png' dir means? and how can I download the origin images?

[adda1221]

hi, I met this problem same to u, do u have solved it? thx for response!

[jshtok]

I have the same question. There is PDF data available on the website, but no png images.
Thanks!

[eshani-agrawal]

Hi, I am stuck with the same issue, did anyone resolve the issue of how to get png images?
Thanks

[jshtok]

I didn't find the png images, but notice that the PDFs have same names as the png files and there is a single page in each PDF. So I believe pngs can be created by a pdf2png converter

[eshani-agrawal]

Hi, Yeah that's true for DocVQA dataset, but not for other datasets like DeepForm. So I was wondering if there exists a standard way. Thanks, Eshani

…

On Tue, Oct 3, 2023, 08:11 Joseph Shtok ***@***.***> wrote: I didn't find the png issues, but notice that the PDFs have same names as the png files and there is a single page in each PDF. So I believe pngs can be created by a pdf2png converter — Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2WVGBJU4HSSRJJ63MW457DX5P6IVAVCNFSM6AAAAAAVWJ5SFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBUHAZTSMBYGE> . You are receiving this because you commented.Message ID: ***@***.***>

[jshtok]

Hi Eshani, There is a field 'page_number', or something like that, in the metadata. I used PDF converter and this field to fetch the relevant page, and visualized results for all the datasets. Two of them - PWC and Kleister - look wrong, in the sense that the page does not contain the information relevant to the question. Others are Ok. If you like, I can send you my version of the code for the dataset. Regards, Joseph On Tue, Oct 3, 2023 at 6:13 PM eshani-agrawal ***@***.***> wrote:

…

Hi, Yeah that's true for DocVQA dataset, but not for other datasets like DeepForm. So I was wondering if there exists a standard way. Thanks, Eshani On Tue, Oct 3, 2023, 08:11 Joseph Shtok ***@***.***> wrote: > I didn't find the png issues, but notice that the PDFs have same names as > the png files and there is a single page in each PDF. So I believe pngs can > be created by a pdf2png converter > > — > Reply to this email directly, view it on GitHub > < #6 (comment)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/A2WVGBJU4HSSRJJ63MW457DX5P6IVAVCNFSM6AAAAAAVWJ5SFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBUHAZTSMBYGE> > . > You are receiving this because you commented.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACOBU6RM3QYZ66NMRMYFK3DX5QTQJAVCNFSM6AAAAAAVWJ5SFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBVGE4DIOJQHA> . You are receiving this because you commented.Message ID: ***@***.***>