Opened 4 years ago

Closed 3 years ago

#657 closed task (fixed)

Add NVMe support to the smartmontools

Reported by: Alex Samorukov Owned by: Christian Franke
Priority: major Milestone: Release 6.5
Component: all Version:
Keywords: nvme Cc:

Description (last modified by Alex Samorukov)

About

It would be great to add NVMe support to the smartctl. This task is not very trivial - because smart in NVMe is implemented differently then in SCSI/SAS or SATA. This ticket will be used to track changes in smartmontools and to collect important related links.

NVMe standard and vendor specific log pages

Hardware and emulation

  • QEMU fork which can emulates NVMe device with most of the features supported, including SMART log pages, namespaces, etc. Tested with Linux and FreeBSD, full list of the options could be found in the source code.

Linux NVMe support

FreeBSD NVMe support

Windows NVMe support

OS X NVMe support

  • Kind of official support added from 10.10.3, closes source, not documented, only apple-branded NVMe devices are supported. Implements NVMeSMARTClient library, however API is not documented.
  • MacVidCards NVMe driver - third party closed-source driver, coming with closed-source nvme-cli management utility, which is based on open-source one.

Solaris NVMe support

Change History (28)

comment:1 Changed 4 years ago by Alex Samorukov

https://github.com/linux-nvme/nvme-cli - open source tool to get NVME information on Linux (including SMART data).
https://github.com/xnox/nvme-user - another NVMe Linux utility
http://nvmexpress.org/specifications/ - specifications, including SCSI translation interface
http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-dc-p3608-spec.pdf - Intel specification with some Intel-only pages documented

Last edited 3 years ago by Alex Samorukov (previous) (diff)

comment:2 Changed 4 years ago by Christian Franke

Milestone: undecidedunscheduled

Adding NVMe support makes sense.

Logical structure of NVMe commands and returned SMART related information is different both to ATA and SCSI. A new (-d nvme) pass_through interface and related printing routines are required. Unlike SAT, the NVMe SCSI translation interface is not very useful as it does not (yet?) provide a NVMe pass-through command.

comment:3 Changed 4 years ago by Christian Franke

https://svn.openfabrics.org/svnrepo/nvmewin/ - OpenFabrics NVMe driver for Windows.
It provides a NVME_PASS_THROUGH I/O-control which is reportedly also supported by the NVMe Windows drivers from Intel and OCZ.

comment:4 Changed 4 years ago by Alex Samorukov

In the FreeBSD NVMe SMART data could be obtained using nvmecontrol utility.

comment:5 Changed 4 years ago by Alex Samorukov

Windows opensource NVMe tool set: http://naraeon.net/en/latest-naraeon-ssd-tools/

comment:6 Changed 4 years ago by Alex Samorukov

QEMU with nvme support: https://github.com/OpenChannelSSD/qemu-nvme/. It could be used for the development purpose.

Last edited 3 years ago by Alex Samorukov (previous) (diff)

comment:7 Changed 4 years ago by Christian Franke

IOCTL_STORAGE_PROTOCOL_COMMAND - NVMe pass-through I/O-control supported by the generic NVMe driver for Windows (Stornvme.sys). Requires Windows 10.

comment:8 Changed 3 years ago by Alex Samorukov

Recent (10.10.3+) OSX/Darwin also supports NVME and there is undocumented NVMeSMARTClient library. No userland client [yet] found. Also only apple-branded devices are supported by official driver.

Non-apple NVMe devices are supported by 3rd party driver, see http://www.macvidcards.com/nvme-faq.html. This driver has some management tool, but no API description found.

Last edited 3 years ago by Alex Samorukov (previous) (diff)

comment:9 Changed 3 years ago by Alex Samorukov

Description: modified (diff)

comment:10 Changed 3 years ago by Alex Samorukov

Description: modified (diff)

comment:11 Changed 3 years ago by Alex Samorukov

Description: modified (diff)

comment:12 Changed 3 years ago by Alex Samorukov

Description: modified (diff)

comment:13 Changed 3 years ago by Alex Samorukov

Description: modified (diff)

comment:14 Changed 3 years ago by Alex Samorukov

Description: modified (diff)

comment:15 Changed 3 years ago by Alex Samorukov

Description: modified (diff)

comment:16 Changed 3 years ago by Christian Franke

Owner: set to Christian Franke
Status: newaccepted

Work is in progress.

comment:17 Changed 3 years ago by Christian Franke

r4238, r4239 added basic NVMe support for smartctl on Linux:

smartctl [-d nvme[,NSID]] -i -H -A /dev/nvmeX[nY]

comment:18 Changed 3 years ago by Christian Franke

r4240 added NVMe Error Information log support:

smartctl -l error[,NUM] /dev/nvmeX[nY]

comment:19 Changed 3 years ago by Christian Franke

r4242 added hex dump support for NVMe logs:

smartctl -l nvmelog,PAGE,SIZE /dev/nvmeX[nY]

Useful for testing.

comment:20 Changed 3 years ago by Christian Franke

r4249 added basic NVMe support to smartd:

/dev/nvmeN [-d nvme[,NSID]] -H -l error -W DIFF[,INFO[,CRIT]]

comment:21 Changed 3 years ago by Alex Samorukov

r4257:4258 added initial FreeBSD support

Last edited 3 years ago by Alex Samorukov (previous) (diff)

comment:22 Changed 3 years ago by Christian Franke

r4262 added NVMe DEVICESCAN support on Linux. It needs to be explicitly enabled with -d nvme unless --with-nvme-devicescan is used during build.

With r4265, it is possible to add NVMe to existing device scanning:

DEVICESCAN -d ata -d scsi -d sat -d nvme -a

comment:23 Changed 3 years ago by Christian Franke

r4278:4281 added initial NVMe support for Windows.

Support for DEVICESCAN ... -d nvme or smartctl -d nvme --scan is included.

Devices are accessed via NVME_PASS_THROUGH I/O-control as specified by OpenFabrics NMVe driver. This should work (at least) with NVMe drivers from Intel, OCZ and Samsung. It does not work with the Microsoft NVMe driver.

Pseudo device names are similar to Linux:

smartctl [-d nvme[,NSID]] -x /dev/nvmeX[nY]

Alternatively /dev/nvmesX[nY] could be used to directly address NVMe device behind Windows device name \\.\ScsiX:.

comment:24 Changed 3 years ago by Christian Franke

Milestone: unscheduledRelease 6.5

comment:25 Changed 3 years ago by Alex Samorukov

I created initial version of the NVMe support page for the smartmontools wiki - https://www.smartmontools.org/wiki/NVMe_Support . Feel free to edit.

comment:26 Changed 3 years ago by Christian Franke

r4311 added smartctl -c which prints info about various capabilites (optional commands, firmware slots, power states, LBA sizes).

comment:27 Changed 3 years ago by Christian Franke

Sample smartctl -x output:

smartctl 6.5 2016-04-27 r4312 [x86_64-w64-mingw32-win10] (daily-20160427)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Samsung SSD 950 PRO 256GB
Serial Number:                      ...
Firmware Version:                   1B0QBXX7
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Controller ID:                      1
Number of Namespaces:               1
Namespace 1 Size/Capacity:          256,060,514,304 [256 GB]
Namespace 1 Utilization:            117,410,267,136 [117 GB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Thu Apr 28 19:32:07 2016 CEST
Firmware Updates (0x06):            3 Slots
Optional Admin Commands (0x0007):   Security Format Frmw_DL
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         32 Pages

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     6.50W       -        -    0  0  0  0        5       5
 1 +     5.80W       -        -    1  1  1  1       30      30
 2 +     3.60W       -        -    2  2  2  2      100     100
 3 -   0.0700W       -        -    3  3  3  3      500    5000
 4 -   0.0050W       -        -    4  4  4  4     2000   22000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:                   0x00
Temperature:                        40 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    1,769,281 [905 GB]
Data Units Written:                 1,384,224 [708 GB]
Host Read Commands:                 24,646,213
Host Write Commands:                19,105,374
Controller Busy Time:               38
Power Cycles:                       32
Power On Hours:                     129
Unsafe Shutdowns:                   6
Media and Data Integrity Errors:    0
Error Information Log Entries:      44

Error Information (NVMe Log 0x01, max 64 entries)
Num   ErrCount  SQId   CmdId  Status  PELoc          LBA  NSID    VS
  0         44     0  0x002a  0x4016  0x000            0   255     -
  1         43     0  0x0029  0x4016  0x000            0   255     -
  2         42     0  0x0049  0x4016  0x000            0   255     -
  3         41     0  0x0048  0x4016  0x000            0   255     -
  4         40     0  0x001f  0x4004  0x000            0     0     -
  5         39     0  0x001e  0x4004  0x000            0     0     -
  6         38     0  0x001f  0x4004  0x000            0     0     -
  7         37     0  0x001e  0x4004  0x000            0     0     -
  8         36     0  0x001f  0x4004  0x000            0     0     -
  9         35     0  0x001e  0x4004  0x000            0     0     -
 10         34     0  0x001d  0x4004  0x000            0     0     -
 11         33     0  0x001c  0x4004  0x000            0     0     -
 12         32     0  0x001d  0x4004  0x000            0     0     -
 13         31     0  0x001c  0x4004  0x000            0     0     -
 14         30     0  0x001d  0x4004  0x000            0     0     -
 15         29     0  0x001c  0x4004  0x000            0     0     -
... (28 entries not shown)

comment:28 Changed 3 years ago by Christian Franke

Resolution: fixed
Status: acceptedclosed

Basic NVMe support is now available for Linux, FreeBSD and Windows. See above comments for details.

For NVMe related bug reports and enhancement requests, please create new tickets (e.g. #687, #688, #689, #690, #691).

Note: See TracTickets for help on using tickets.