TrueNas VM set up - All SSD's passthrough.. Trim not working?

[fahadshery]

Hi,

I passed through the SSD's using the official tutorial.
Here is the VM configs:



I am getting the following error when try to enable Trim:

Error: Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 355, in run
    await self.future
  File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 391, in __run_body
    rv = await self.method(*([self] + args))
  File "/usr/local/lib/python3.9/site-packages/middlewared/schema.py", line 981, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/pool.py", line 841, in do_update
    verrors.check()
  File "/usr/local/lib/python3.9/site-packages/middlewared/service_exception.py", line 62, in check
    raise self
middlewared.service_exception.ValidationErrors: [EINVAL] pool_update.topology: Disks have duplicate serial numbers: '' (da0, da1, da2).

what am I doing wrong???

 

[fahadshery]

is there a way for smartctl and Trim to work if doing a SSD passthrough?

 

[leesteken]

is there a way for smartctl and Trim to work if doing a SSD passthrough?

When using disk/partition passthrough, SMART is not passed through. Use SMART on the Proxmox host instead. There are several threads on this forum about that.

I am getting the following error when try to enable Trim:

Error: Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 355, in run
    await self.future
  File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 391, in __run_body
    rv = await self.method(*([self] + args))
  File "/usr/local/lib/python3.9/site-packages/middlewared/schema.py", line 981, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/pool.py", line 841, in do_update
    verrors.check()
  File "/usr/local/lib/python3.9/site-packages/middlewared/service_exception.py", line 62, in check
    raise self
middlewared.service_exception.ValidationErrors: [EINVAL] pool_update.topology: Disks have duplicate serial numbers: '' (da0, da1, da2).

It looks like your trim problem is caused by duplicate serial numbers. When using disk/partition passthrough, drive serial numbers are not passed through. The work-around it to add a unique serial number via a parameter (manually, via the VM configuration file). There are several threads on this forum about that. Also, see the manual: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_options

serial=<serial>
The drive’s reported serial number, url-encoded, up to 20 bytes long.

 

[fahadshery]

serial=<serial>
The drive’s reported serial number, url-encoded, up to 20 bytes long.

YES! This has solved the issue. qm set 1002 -scsi2 /dev/disk/by-id/ata-SSD_4TB_008,serial=myserial01

I can't see the Disks in the Proxmox host when they are passthrough to the VM. So need to lookup how to enable SMARTCTL for those.

thanks for your help

 

[leesteken]

I can't see the Disks in the Proxmox host when they are passthrough to the VM.

Weird; what do you mean with "I can't see the Disks in the Proxmox host"? What is the output of ls -l /dev/disk/by-id/ata-SSD_4TB_008 while the VM is running?

So need to lookup how to enable SMARTCTL for those.

What is the output of smartctl -a /dev/disk/by-id/ata-SSD_4TB_008 (while the VM is running)?

 

[fahadshery]

Weird; what do you mean with "I can't see the Disks in the Proxmox host"? What is the output of ls -l /dev/disk/by-id/ata-SSD_4TB_008 while the VM is running?

What is the output of smartctl -a /dev/disk/by-id/ata-SSD_4TB_008 (while the VM is running)?

I had to reboot the node and now I can see the disks. Here is the output:

smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.5.11-8-pve] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     SSD 4TB
Serial Number:    003348
LU WWN Device Id: 0 000000 000000000
Firmware Version: VA001CBN
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic
Device is:        Not in smartctl database 7.3/5319
ATA Version is:   ACS-3, ATA8-ACS T13/1699-D revision 6
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Thu Feb 29 08:08:52 2024 GMT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (    1) seconds.
Offline data collection
capabilities:                    (0x59) SMART execute Offline immediate.
                                        No Auto Offline data collection support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0002) Does not save SMART data before
                                        entering power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        ( 130) minutes.
Extended self-test routine
recommended polling time:        ( 134) minutes.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   100   100   050    Pre-fail  Always       -       0
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       11
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       3
161 Unknown_Attribute       0x0032   100   100   050    Old_age   Always       -       171
162 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       641
163 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       3000
164 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       1
166 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       171
167 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       0
168 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       0
169 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       100
171 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       0
172 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       0
174 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       2
175 Program_Fail_Count_Chip 0x0032   100   100   000    Old_age   Always       -       0
181 Program_Fail_Cnt_Total  0x0022   100   100   000    Old_age   Always       -       920
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
194 Temperature_Celsius     0x0022   100   100   000    Old_age   Always       -       40
195 Hardware_ECC_Recovered  0x003a   100   100   000    Old_age   Always       -       0
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
199 UDMA_CRC_Error_Count    0x0032   100   100   000    Old_age   Always       -       0
206 Unknown_SSD_Attribute   0x0032   100   100   000    Old_age   Always       -       0
207 Unknown_SSD_Attribute   0x0032   100   100   000    Old_age   Always       -       3
232 Available_Reservd_Space 0x0032   100   100   000    Old_age   Always       -       0
233 Media_Wearout_Indicator 0x0032   100   100   000    Old_age   Always       -       202
241 Total_LBAs_Written      0x0032   100   100   000    Old_age   Always       -       102
242 Total_LBAs_Read         0x0032   100   100   000    Old_age   Always       -       0
249 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       158
250 Read_Error_Retry_Rate   0x0032   100   100   000    Old_age   Always       -       102

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Aborted by host               90%        10         -

SMART Selective self-test log data structure revision number 0
Note: revision number not 1 implies that no selective self-test has ever been run
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

I ran TrueNas VM fine. But this morning it was giving a weird error:

plugin_dispatch_values: Low water mark reached. Dropping 100% of metrics

Had to reboot the node and it seems to be ok. but people seem to think it returns with the error. So will keep you posted