Skip to content

Commit abf316c

Browse files
dpgeorgeprojectgus
authored andcommitted
tarfile: Fix FileSection.skip to not rely on extended readinto args.
Commit 2ca1527 optimized `FileSection.skip()` for memory use. But that introduced a dependency on the MicroPython-extension to stream read methods for an additional argument specifying a maximum read size. This optimization meant that all file-like objects passed into TarFile must support the extended 2-argument `readinto` form. This is problematic for at least two use cases: 1. Nested tar files, because `FileSetion` itself doesn't support 2-argument `readinto`. 2. Using `mpremote mount` and reading a tar file from the remote mount, which also doesn't support 2-argument `readinto`. Instead of requiring all file-like objects to implement this extended form of `readinto`, this commit changes `FileSection.skip()` so that it doesn't use this form. A test is added for this case which fails without the fix here. Signed-off-by: Damien George <damien@micropython.org>
1 parent 1390722 commit abf316c

File tree

3 files changed

+15
-4
lines changed

3 files changed

+15
-4
lines changed

python-stdlib/tarfile/manifest.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
metadata(description="Read-only implementation of Python's tarfile.", version="0.4.1")
1+
metadata(description="Read-only implementation of Python's tarfile.", version="0.4.2")
22

33
# Originally written by Paul Sokolovsky.
44

python-stdlib/tarfile/tarfile/__init__.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,9 +55,12 @@ def skip(self):
5555
if sz:
5656
buf = bytearray(16)
5757
while sz:
58-
s = min(sz, 16)
59-
self.f.readinto(buf, s)
60-
sz -= s
58+
if sz >= 16:
59+
self.f.readinto(buf)
60+
sz -= 16
61+
else:
62+
self.f.read(sz)
63+
sz = 0
6164

6265

6366
class TarInfo:

python-stdlib/tarfile/test_tarfile.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,3 +35,11 @@ def test_iter(self):
3535
def test_contents(self):
3636
tf = tarfile.TarFile("test.tar")
3737
self.check_contents(test_tar_contents, tf)
38+
39+
def test_nested_tar(self):
40+
tf = tarfile.TarFile("test.tar")
41+
for file in tf:
42+
if file.name == "tar.tar":
43+
subf = tf.extractfile(file)
44+
subtf = tarfile.TarFile(fileobj=subf)
45+
self.check_contents(test_sub_tar_contents, subtf)

0 commit comments

Comments
 (0)