In this final article in the OneFS SmartQuotas series we focus on data reduction and storage efficiency reporting:
SmartQuotas reports both data reduction and efficiency as a ratio across the desired dataset as specified in the quota path field. These efficiency and data reduction ratios are for the full quota directory and its contents, including any overhead, and reflects the net efficiency of both compression and deduplication.
The ‘isi quota quotas view’ CLI command provides considerably more detailed storage capacity and efficiency metrics. These include the following:
| Metric | Description |
| AppLogical | The application data that can be written to the cluster, irrespective of where it’s stored from. |
| FSLogical | Removing sparse data (data that was always sparse, was zero block eliminated, or data that’s been moved to the cloud, etc) results in filesystem logical, which is the non-sparse data stored on the filesystem. |
| AppPhysical | Data reduction techniques, such as compression and dedupe, reduce filesystem logical to application physical, or pre-protected physical. This is the physical size application data on the filesystem disks, and does not include metadata, protection overhead, or data moved to the cloud. |
| FSPhysical | Application physical with data protection overhead added – including inode, mirroring and FEC blocks, etc. Filesystem physical is also referred to as protected physical. |
| Reduction | The data reduction ratio is the amount that’s been reduced from the filesystem logical down to the application physical. |
| Efficiency | Storage efficiency ratio is the filesystem logical divided by the filesystem physical. |
With OneFS, the relationship between the capacity, data reduction and storage efficiency elements is as follows:
SmartQuotas reports the capacity saving from in-line data reduction as a storage efficiency ratio across the desired data set, or quota domain, as specified in the quota path field. The efficiency ratio is for the full quota directory and its contents, including any overhead, and reflects the net efficiency of compression and deduplication. On a cluster with licensed and configured SmartQuotas, this efficiency ratio can be easily viewed from the WebUI by navigating to ‘File System > SmartQuotas > Quotas and Usage’. In OneFS 9.2 and later, in addition to the storage efficiency ratio, the data reduction ratio is also displayed.
Similarly, the same data can be accessed from the OneFS command line via is ‘isi quota quotas list’ CLI command. For example:
# isi quota quotas list Type AppliesTo Path Snap Hard Soft Adv Used Reduction Efficiency ------------------------------------------------------------------------------ directory DEFAULT /ifs No - - - 6.02T 2.54 : 1 1.77 : 1 ------------------------------------------------------------------------------ Total: 1
More detail, including both the physical (raw) and logical (effective) data capacities, is also available via the ‘isi quota quotas view <path> <type>’ CLI command. For example:
# isi quota quotas view /ifs directory Path: /ifs Type: directory Snapshots: No Enforced: No Container: No Linked: No Usage Files: 5759676 Physical(With Overhead): 6.93T FSPhysical(Deduplicated): 3.41T FSLogical(W/O Overhead): 6.02T AppLogical(ApparentSize): 6.01T ShadowLogical: - PhysicalData: 2.01T Protection: 781.34G Reduction(Logical/Data): 2.54 : 1 Efficiency(Logical/Physical): 1.77 : 1
To configure SmartQuotas for in-line data efficiency reporting, create a directory quota at the top-level file system directory of interest, for example /ifs. Creating and configuring a directory quota is a simple procedure and can be performed from the WebUI by navigate to ‘File System > SmartQuotas > Quotas and Usage’ and selecting ‘Create a Quota’. In the create pane, field, set the Quota type to ‘Directory quota’, add the preferred top-level path to report on, select ‘application logical size’ for Quota Accounting, and set the Quota Limits to ‘Track storage without specifying a storage limit’. Finally, select the ‘Create Quota’ button to confirm the configuration and activate the new directory quota.
The efficiency ratio is a single, current-in time efficiency metric that is calculated per quota directory and includes the sum of in-line compression, zero block removal, in-line dedupe and SmartDedupe. This is in contrast to a history of stats over time, as reported in the ‘isi statistics data-reduction’ CLI command output, described above. As such, the efficiency ratio for the entire quota directory will reflect what is actually there.
When using SyncIQ replication on a cluster pair that are also running SmartQuotas, the quotas are matched one-to-one across the replication set. Multiple quotas are supported within a source directory or domain structure, and the target directory is included in a quota domain.
During replication SyncIQ ignores quota limits. However, if a quota is over limit, quotas still prevent users from adding additional data. SyncIQ will never automatically delete an existing target quota. Instead, a SyncIQ will fail, as opposed to deleting an existing quota. This may occur during an initial sync where the target directory has an existing quota under it, or if a source directory is deleted that has a quota on it on the target. The quota still remains and requires administrative removal if desired
Application logical quotas, available in OneFS 8.2 and later, provide a quota accounting metric, which accounts for, reports and enforces on the actual space consumed and available for storage, independent of whether files are on-premises or cloud-tiered.
In addition to data-protection overhead, the option is provided on whether to include snapshot data when calculating a quota’s usage limits.
SmartQuotas will only report on snapshots created after the quota domain was created. This is because determining quota governance (including QuotaScan job) for existing snapshots is a very time and resource consuming operation. However, as snapshots age out, SmartQuotas will gradually accrue accounting information for the entire set of relevant snapshots.
Compressed and deduplicated files appear no differently than regular files to standard quota policies. However, for deduplicated files, if the quota is configured to include data-protection overhead, the additional space used by the shadow store will not be accounted for by the quota.