Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Support
    • Submit feedback
    • Contribute to GitLab
  • Sign in
D
darshan
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 72
    • Issues 72
    • List
    • Boards
    • Labels
    • Milestones
  • Merge Requests 5
    • Merge Requests 5
  • Analytics
    • Analytics
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
  • darshan
  • darshan
  • Issues
  • #187

Closed
Open
Opened May 20, 2016 by Glenn K. Lockwood@glock
  • Report abuse
  • New issue
Report abuse New issue

lustre module cannot obtain ost list efficiently

We are using Lustre's ioctl interface to get striping information about each file that is opened. There are a few options to get stripe information:

  1. LL_IOC_LOV_GETSTRIPE only needs an open file handle, but only returns stripe size and width
  2. IOC_MDC_GETFILESTRIPE needs an open handle on the parent directory as well as the file basename as a string. It returns stripe size, width, and exact layout.

Darshan already has the file handle for option 1, but has to issue an open(2) call to use option 2. Our tests show that having each MPI process issue this additional open introduces introduces overhead that scales with MPI process count.

I made some simple code that demonstrates the two ioctl calls' behaviors here:

https://github.com/glennklockwood/nersc-ssio/blob/master/llapi-perf/test-getstripe.c

And here's some example output of the above tool to illustrate:

$ ./test-getstripe /scratch1/scratchdirs/glock/random.bin
This is with ioctl LL_IOC_LOV_GETSTRIPE:
Stripe size: 1048576
Stripe count: 2
OST idx: 0
  os_id: 0 oi_seq: 0 f_seq: 0 f_oid: 0 f_ver: 0 l_ost_gen: 0 l_ost_idx: 0
  os_id: 0 oi_seq: 0 f_seq: 0 f_oid: 0 f_ver: 0 l_ost_gen: 0 l_ost_idx: 0

$ ./test-getstripe /scratch1/scratchdirs/glock/ random.bin
This is with IOC_MDC_GETFILESTRIPE:
Stripe size: 1048576
Stripe count: 2
OST idx: 54
  os_id: 76620879 oi_seq: 0 f_seq: 76620879 f_oid: 0 f_ver: 0 l_ost_gen: 0 l_ost_idx: 54
  os_id: 76656970 oi_seq: 0 f_seq: 76656970 f_oid: 0 f_ver: 0 l_ost_gen: 0 l_ost_idx: 22

We need to find a way to get the stripe layout from an open file handle alone so we don't have to issue any additional open(2)s.

Assignee
Assign to
3.1.0
Milestone
3.1.0
Assign milestone
Time tracking
None
Due date
None
0
Labels
None
Assign labels
  • View project labels
Reference: darshan/darshan#187