Storage QA, especially Network Attached Storage, (NAS) QA, poses many challenges for the OEM/ODM. It is not difficult by itself, but it needs a wide array of expertise like File System knowledge, Protocol functionality, understanding differences between various versions of the Protocol and interoperability between different protocol types & the underlying File System.
Moreover, it needs relevant knowledge of various tools & techniques that can effectively cover all the possible use cases.
All these need to be done before the product is released either to its Beta Users or for GA release of the software/firmware/hardware.
Types of Testing:
- Functionality Testing of the features offered.
- Protocol validation, a.k.a conformance (to RFC) testing.
- Interoperability testing across matrix of major OS for verification of functionality & conformance.
- Compatibility testing against legacy version of the protocol versions.
- Scalability testing.
- Data Path testing by doing data ingestion of different types, size, and combination.
- Control Path testing, a.k.a Configuration Testing
- Out-of-Box (OOB) Testing for getting end-user perspective.
- User Interface Testing of CLI, GUI, and API if applicable.
- Stress/Load testing for identifying stability across various load condition.
- Error injection testing for identifying recoverability aspect of the system.
- Limit or Boundary Analysis Testing to identify limits of the overall system.
- File System interop with the network filesystem protocol; ext3/4 (POSIX), NTFS. ZFS, Reiser, HPFS or any other proprietary File System against SMB 2.x/3.x and NFS 3/4.x
- File System and File System Protocol locking semantics testing.
- 3rd Party Authentication/Authorization testing such as NIS+/AD/LDAP/Kerberos, etc;
- Solution Testing a.k.a. Use Case Testing, such as use in simple file serving, host virtualization, Backup & DR, High Performance Computing (HPC), etc.
- Performance testing for publishing to Solution White Papers, Technical Journals, or online vendor neutral charting of performance numbers under typical workloads.
At Ryussi, we’ve addressed most of the NAS testing challenges by adopting the appropriate derivatives of the solutions discussed above & have successfully delivered quality testing as per the needs of our customers.
- Leverage the existing knowledge within the organization to support testing
- Use of existing Test Automation frameworks to perform functional regression, Stress, Error-Injection, Limit, and data path testing.
- Use of readymade open source tools to carry out the above testing using automation framework & also perform other complex testing such as file-locking, interop, scalability, and solution testing.
- Use of Tables, Graphs & Charts to create performance charting for publishing and also knowledge of other 3rd party vendor neutral testing techniques for generating performance numbers under typical workloads.
- Ability to identify weakly developed feature/function and focus on specific areas to find maximum issues early during the test cycle rather than later.
- In order to adapt to the changing requirements of the customers & quickly come-up with test strategy to address the change.