Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Support
    • Submit feedback
    • Contribute to GitLab
  • Sign in
D
darshan
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 74
    • Issues 74
    • List
    • Boards
    • Labels
    • Milestones
  • Merge Requests 6
    • Merge Requests 6
  • Analytics
    • Analytics
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
  • darshan
  • darshan
  • Issues
  • #232

Closed
Open
Opened Aug 08, 2017 by Glenn K. Lockwood@glock
  • Report abuse
  • New issue
Report abuse New issue

add support for DataWarp accounting API

Recent versions of Cray Linux include a DataWarp telemetry API that provides information analogous to what the Lustre module provides. Specifically, dw_get_stripe_configuration(3) returns

  • int stripe_size
  • int stripe_width
  • int starting_index

And the DataWarp servers to which starting_index refers can be divined from /proc/mounts or other kernel interfaces.

Furthermore, CLE 6.0UP04 now includes dw_get_accounting_data_json(3) which returns aggregate server-side telemetry. While not broken out into a per-file basis, the data types returned may be of interest at some point. An example output is:

{
  "realm": {
    "namespaces": [
      {
        "stage_bytes_written": 0,
        "namespace_id": 378,
        "stripe_width": 2,
        "file_size_limit": 0,
        "stage_bytes_read": 0,
        "stripe_size": 8388608,
        "files_create_threshold": 0,
        "substripe_width": 12,
        "substripe_size": 8388608,
        "files_created": 0,
        "num_data_created": 0,
        "bytes_read": 0,
        "max_offset_written": 0,
        "bytes_written": 0,
        "max_offset_read": 0
      }
    ],
    "fragments": [
      {
        "fs_capacity": 0,
        "files_created": 0,
        "capacity_used": 0,
        "capacity_max": 0,
        "max_window_write": 86400,
        "write_high_water": 0,
        "server_hostname": "c0-0c0s6n2",
        "write_moving_avg": 0,
        "write_limit": 1730066513920,
        "bytes_written": 0,
        "bytes_read": 0
      },
      {
        "fs_capacity": 172920012800,
        "files_created": 0,
        "capacity_used": 34369536,
        "capacity_max": 34369536,
        "max_window_write": 86400,
        "write_high_water": 0,
        "server_hostname": "c0-0c0s6n1",
        "write_moving_avg": 0,
        "write_limit": 1730066513920,
        "bytes_written": 0,
        "bytes_read": 0
      }
    ],
    "realm_id": 378,
    "namespace_count": 1,
    "server_count": 2
  }
}

These sorts of data may be best collected by some other tool (just as we decided client-side file system counters should be).

Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
1
Labels
enhancement
Assign labels
  • View project labels
Reference: darshan/darshan#232