Skip to content

Optimize fprof for Lustre attribute retrieval #54

@adilger

Description

@adilger

The "lfs getstripe" and "lfs find" code is using an ioctl(LL_IOC_MDC_GETINFO) to fetch both the stat attributes and Lustre layout from the MDS in a single RPC. This also opens only the parent directory for the ioctl(), so that each file does not need to be opened (which avoids inode cache pollution and DLM lock/cancel overhead as well).

With the "Lazy Size-on-MDT" (LSOM) feature that is being added in 2.12 it would also be possible to fetch a reasonable estimate of the file size from the MDS as well, which would avoid the need for any OSS RPCs entirely, which can add significant overhead if the MDT(s) are SSDs and the OSTs are HDDs. For the purposes of fprof (histograms and average file sizes), the LSOM size is sufficient.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions