= Smartmontools NVMe support = [[TOC]] == About NVMe == VM Express, NVMe, or Non-Volatile Memory Host Controller Interface Specification (NVMHCI), is a logical device interface specification for accessing non-volatile storage media attached via PCI Express (PCIe) bus. Some links related to the standard and vendor-specific implementations provided below: - [http://nvmexpress.org/specifications/ Official specifications] - [http://www.flashmemorysummit.com/English/Collaterals/Proceedings/2013/20130812_PreConfD_Marks.pdf An NVM Express Tutorial], Kevin Marks (Dell) in ''[http://www.flashmemorysummit.com/ Flash Memory Summit] 2013''. - [http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-dc-p3608-spec.pdf Intel DC PC3608 specification] with Intel-specific log pages (CAh, C5h, DDh, C1h/C2h). == Smartmontools NVMe support == Smartmontools supports NVMe starting from version 6.5. Please note, that currently NVMe support is considered as experimental. Currently implemented features: - Basic information about controller name, firmware, capacity (`smartctl -i`) - Controller and namespace capabilities (`smartctl -c`) - SMART overall-health self-assessment test result and warnings (`smartctl -H`) - NVMe SMART attributes (`smartctl -A`) - NVMe error log (`smartctl -l error[,NUM]`) - Ability to fetch any nvme log (`smartctl -l nvmelog,N,SIZE`) - The `smartd` daemon tracks health (`-H`), error count (`-l error`) and temperature (`-W DIFF,INFO,CRIT`) Currently NVMe support is implemented on Linux, Windows and FreeBSD. Details about supported version/driver combinations are provided in the next section. == NVMe support in the different OS == ||=OS and driver=||=Supported by OS=||=Supported by Smartmontools=||=Native management tools=||=Notes=|| ||Linux 3.3 or later|| ✓ || ✓ ||[https://github.com/linux-nvme/nvme-cli nvme-cli], [https://github.com/xnox/nvme-user nvme-user] || || ||FreeBSD 9.2 or later|| ✓ || ✓ ||[https://www.freebsd.org/cgi/man.cgi?query=nvmecontrol nvmecontrol]|| || ||NetBSD|| (✓) || (✓) || [http://netbsd.gw.com/cgi-bin/man-cgi/man?nvmectl+8+NetBSD-current nvmectl] ||NVMe support will be available in upcoming [https://www.netbsd.org/changes/changes-8.0.html#nvme%284%29 NetBSD 8.0]. Smartmontools supports this since r4343. || ||OpenBSD 6.0 or later|| ✓ || - || ?? || Does the [http://man.openbsd.org/?query=nvme&apropos=1 OpenBSD NVMe] driver provide NVMe pass-through functionality? || ||OSX 10.10.3+ native NVMe support || ✓ || - || ||Only Apple-branded NVMe devices are supported, monitoring API is implemented (NVMeSMARTClient) but not documented. It is also possible to use this driver with non-Apple devices using [https://github.com/RehabMan/patch-nvme patch-nvme] project. Attempt to use private api is [https://pikeralpha.wordpress.com/2017/01/30/apple-nvme-smart-monitor-under-control/ here] (w/o source) and in the [https://binaryfruit.com/drivedx DriveDx] app (commercial, closed source)|| ||OSX with [http://www.macvidcards.com/nvme-driver.html MacVidCards NVMe driver] || ✓ || - || nvme-cli tool distributed with the driver ||IOCTL format is not documented, driver is closed source and seems to be abandoned || ||Oracle Solaris 11.2+|| ✓ || - ||[https://docs.oracle.com/cd/E36784_01/html/E36871/nvmeadm-1m.html nvmeadm] ||Interface is not documented || ||IllumOS [https://github.com/illumos/illumos-gate/commit/3c9168fa8e9c30d55b3aa2fde74bd7da46df53f5 NVMe driver]|| ✓ || - ||not yet implemented ||[https://www.illumos.org/issues/6235 no management interface/tool] yet implemented || ||Windows with [https://www.openfabrics.org/index.php/windows-nvm-express/nvm-express-subversion-repos.html OpenFabrics] compatible NVMe driver|| ✓ || ✓ || Various vendor specific tools || Devices are accessed via [https://svn.openfabrics.org/svnrepo/nvmewin/releases/IOCTL_Sample_App/ NVME_PASS_THROUGH] I/O-control. Successfully tested with NVMe drivers from Intel, OCZ/Toshiba and Samsung. Does not work with Intel RST drivers or Microsoft NVMe drivers. || ||Windows 7 to 2012.R2 with [https://support.microsoft.com/en-us/kb/2990941 Microsoft NVMe driver] || (✓) || - || ?? || Older Microsoft NVMe drivers [https://social.msdn.microsoft.com/Forums/en-US/bc138b94-da4b-4851-9778-cc63ab65077b/ do not provide a NVMe pass-through I/O-control]. || ||Windows 10 with Microsoft NVMe driver (stornvme.sys) || ✓ || (✓) || ?? || Access via [https://msdn.microsoft.com/en-us/library/windows/desktop/mt718131.aspx enhanced version of IOCTL_STORAGE_QUERY_PROPERTY] is supported since smartmontools r4348. || == Sample smartctl output == Sample `smartctl -x` output: {{{ smartctl 6.5 2016-04-27 r4312 [x86_64-w64-mingw32-win10] (daily-20160427) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Number: Samsung SSD 950 PRO 256GB Serial Number: ... Firmware Version: 1B0QBXX7 PCI Vendor/Subsystem ID: 0x144d IEEE OUI Identifier: 0x002538 Controller ID: 1 Number of Namespaces: 1 Namespace 1 Size/Capacity: 256,060,514,304 [256 GB] Namespace 1 Utilization: 117,410,267,136 [117 GB] Namespace 1 Formatted LBA Size: 512 Local Time is: Thu Apr 28 19:32:07 2016 CEST Firmware Updates (0x06): 3 Slots Optional Admin Commands (0x0007): Security Format Frmw_DL Optional NVM Commands (0x001f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Maximum Data Transfer Size: 32 Pages Supported Power States St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat 0 + 6.50W - - 0 0 0 0 5 5 1 + 5.80W - - 1 1 1 1 30 30 2 + 3.60W - - 2 2 2 2 100 100 3 - 0.0700W - - 3 3 3 3 500 5000 4 - 0.0050W - - 4 4 4 4 2000 22000 Supported LBA Sizes (NSID 0x1) Id Fmt Data Metadt Rel_Perf 0 + 512 0 0 === START OF SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff) Critical Warning: 0x00 Temperature: 40 Celsius Available Spare: 100% Available Spare Threshold: 10% Percentage Used: 0% Data Units Read: 1,769,281 [905 GB] Data Units Written: 1,384,224 [708 GB] Host Read Commands: 24,646,213 Host Write Commands: 19,105,374 Controller Busy Time: 38 Power Cycles: 32 Power On Hours: 129 Unsafe Shutdowns: 6 Media and Data Integrity Errors: 0 Error Information Log Entries: 44 Error Information (NVMe Log 0x01, max 64 entries) Num ErrCount SQId CmdId Status PELoc LBA NSID VS 0 44 0 0x002a 0x4016 0x000 0 255 - 1 43 0 0x0029 0x4016 0x000 0 255 - 2 42 0 0x0049 0x4016 0x000 0 255 - 3 41 0 0x0048 0x4016 0x000 0 255 - 4 40 0 0x001f 0x4004 0x000 0 0 - 5 39 0 0x001e 0x4004 0x000 0 0 - 6 38 0 0x001f 0x4004 0x000 0 0 - 7 37 0 0x001e 0x4004 0x000 0 0 - 8 36 0 0x001f 0x4004 0x000 0 0 - 9 35 0 0x001e 0x4004 0x000 0 0 - 10 34 0 0x001d 0x4004 0x000 0 0 - 11 33 0 0x001c 0x4004 0x000 0 0 - 12 32 0 0x001d 0x4004 0x000 0 0 - 13 31 0 0x001c 0x4004 0x000 0 0 - 14 30 0 0x001d 0x4004 0x000 0 0 - 15 29 0 0x001c 0x4004 0x000 0 0 - ... (28 entries not shown) }}} == Development notes Development and debugging could be performed using [https://github.com/OpenChannelSSD/qemu-nvme/ QEMU fork] which can emulates NVMe device with most of the features supported, including SMART log pages, namespaces, etc. Tested with Linux and FreeBSD, full list of the options could be found in the [https://github.com/OpenChannelSSD/qemu-nvme/blob/master/hw/block/nvme.c source code]. The NVMe emulation in the [http://wiki.qemu.org/Main_Page upstream version of QEMU] (2.7.0) is still more limited (single namespace, no log pages). [https://www.virtualbox.org/ VirtualBox] 5.1.0 or later also provides basic NVMe emulation. See list of the [/query?status=!closed&keywords=~nvme NVMe related tickets] if you want to improve smartmontools NVMe support.