Some tips & pointers when load testing a non-windows file server using FSCT

Performance testing is the process of determining a system’s behaviour in terms of responsiveness and stability when the system is subjected to a workload. Whenever, a new server is put into service it is subjected to load testing, a form of performance testing, to identify the behaviour of the system under specific workload and also to identify the bottlenecks in the system.

In the case of a fileserver, the responsiveness can be measured in terms of throughput and latency when multiple users are performing concurrent I/O. Also, it is important to find out the number of users the system can successfully support without hitting a bottleneck.

Fileserver capacity tool (FSCT) is an excellent tool to determine how many users a file server can support. It is one among many tools that Microsoft developed for its internal usage and later made it available for the technical community at large It is a command line based tool and it works by running a set of workloads on the file server. The default workload is “HomeFolders” which runs some basic operations like creating files, directories, renaming and editing the files etc. It tries to determine the server’s behaviour assuming the server’s primary function is to store the user’s home directory. The scenarios are stored in .xml files and can be changed to run with a different configuration than the pre-configured one. The operations are performed from multiple clients simultaneously and the performance in terms of throughput, latency is calculated. As we can configure the test to run with n number of users, we can also calculate the maximum number of users that the fileserver supports. The server can be a Windows or non-windows server.

The FSCTStepbyStepGuide.doc available with the build has very detailed steps about configuring the machines to start the testing. In addition to the steps mentioned in the document the following steps can be followed to run the tool against a non-microsoft server.

Following Requirements are needed to run the tool against a non-windows server:

  • Domain Controller: Windows Server Active Directory Infrastructure
  • File Controller: Windows Server 2k8r2 or higher with at least 1GB free space
  • Client: Windows7 or higher with at least 1GB free space
  • Server: The non-windows server which we will be testing.

On all Windows machines following changes need to be done:

1) Disable UAC

2) Disable UAC remote restrictions on all Windows machines.

3) Ensure following services are running on the Domain controller and File controller:

  • DNS Client
  • Function Discovery Resource Publication
  • SSDP Discovery
  • UPnP Device Host

4) On all Windows machines, Turn On Network discovery and File and Printer sharing.

FSCT configures the server by using the hostname of server and sharename as fsroot followed by a number, if the sharename is not fsroot the preparation phase of server will fail.

Although, you can always rename your sharename to fsroot1, it might not be a good solution. An excellent way to address this limitation is

  • Create a softlink of the share with name fsroot1
  • Add the entry in /etc/samba/smb.conf
  • Restart smbd and nmbd service

Once the machines have been configured you can:

  • Run FSCT to prepare the domain controller, file controller, client and server to run the test.

Test Automation Framework using Python

Ryussi created a test automation framework to simplify and speed up the release process of MoSMB (SMB 3.0 server) and find regression issues during every release.

Main features of the framework

  • A highly modularized test framework with ability to test configuration using both individual and global parameters.
  • Reliability, Robustness and Regression  Test Automation
  • Perform simulated testing of MS Office Apps & Media Apps using Synthetic IO Tool
  • Ability to run tests for long duration in days, weeks and even longer.
  • Automation for Data Integrity
  • Filesystem metadata/data related Data integrity testing using MD5 checksum & other in-build methods provided by synthetic tools
  • Performance Automation which provides ability to perform, generate & extract performance parameters such as IOPS/throughput & latency & generate Excel graphs directly from these values

testAutomation_python

 

Benefits

  • Ability to select test type and sub-set of tests to execute for a given test cycle.
  • Fully automated controller node which propagates to worker nodes.
  • Light weight controller (Docker container) to consolidate all test results at a single location.
  • Easily scalable framework which allows addition of test cases with plug-n-play architecture.

Ryussi Advantage:

At Ryussi, we’ve addressed most of the testing automation challenges by adopting the appropriate derivatives of the solutions discussed above & have successfully delivered quality testing as per the needs of our customers.

  1. Leverage the existing knowledge within the organization to support testing
  2. Creation of new test automation frameworks to suit to user requirements that perform functional regression, stress, error-Injection, limit, and data path testing.
  3. Use of Tables, Graphs & Charts to create performance charting for publishing and also knowledge of other 3rd party vendor neutral testing techniques for generating performance numbers under typical workloads.
  4. Ability to identify weakly developed feature/function and focus on specific areas to find maximum issues early during the test cycle rather than later.
  5. In order to adapt to the changing requirements of the customers & quickly come-up with test strategy to address the change.

To assist you in creating your own NAS automation framework, please contact us as sales@ryussi.com

NAS QA Challenges

Storage QA, especially Network Attached Storage, (NAS) QA, poses many challenges for the OEM/ODM. It is not difficult by itself, but it needs a wide array of expertise like File System knowledge, Protocol functionality, understanding differences between various versions of the Protocol and interoperability between different protocol types & the underlying File System.

Moreover, it needs relevant knowledge of various tools & techniques that can effectively cover all the possible use cases.

All these need to be done before the product is released either to its Beta Users or for GA release of the software/firmware/hardware.

NASQAChallenge

 

Types of Testing:

  1. Functionality Testing of the features offered.
  2. Protocol validation, a.k.a conformance (to RFC) testing.
  3. Interoperability testing across matrix of major OS for verification of functionality & conformance.
  4. Compatibility testing against legacy version of the protocol versions.
  5. Scalability testing.
  6. Data Path testing by doing data ingestion of different types, size, and combination.
  7. Control Path testing, a.k.a Configuration Testing
  8. Out-of-Box (OOB) Testing for getting end-user perspective.
  9. User Interface Testing of CLI, GUI, and API if applicable.
  10. Stress/Load testing for identifying stability across various load condition.
  11. Error injection testing for identifying recoverability aspect of the system.
  12. Limit or Boundary Analysis Testing to identify limits of the overall system.
  13. File System interop with the network filesystem protocol; ext3/4 (POSIX), NTFS. ZFS, Reiser, HPFS or any other proprietary File System against SMB 2.x/3.x and NFS 3/4.x
  14. File System and File System Protocol locking semantics testing.
  15. 3rd Party Authentication/Authorization testing such as NIS+/AD/LDAP/Kerberos, etc;
  16. Solution Testing a.k.a. Use Case Testing, such as use in simple file serving, host virtualization, Backup & DR, High Performance Computing (HPC), etc.
  17. Performance testing for publishing to Solution White Papers, Technical Journals, or online vendor neutral charting of performance numbers under typical workloads.

 

Ryussi Advantage:

At Ryussi, we’ve addressed most of the NAS testing challenges by adopting the appropriate derivatives of the solutions discussed above & have successfully delivered quality testing as per the needs of our customers.

 

  1. Leverage the existing knowledge within the organization to support testing
  2. Use of existing Test Automation frameworks to perform functional regression, Stress, Error-Injection, Limit, and data path testing.
  3. Use of readymade open source tools to carry out the above testing using automation framework & also perform other complex testing such as file-locking, interop, scalability, and solution testing.
  4. Use of Tables, Graphs & Charts to create performance charting for publishing and also knowledge of other 3rd party vendor neutral testing techniques for generating performance numbers under typical workloads.
  5. Ability to identify weakly developed feature/function and focus on specific areas to find maximum issues early during the test cycle rather than later.
  6. In order to adapt to the changing requirements of the customers & quickly come-up with test strategy to address the change.