Summary of hardware/software used in tests:
- All tests are performed on Dell Vostro 3360 notebook in order to control identical of hardware
- Vmware Workstation 8 is used to run both guest VM servers
- Windows 2008 R2 Datacenter (trial) is used for both guest VM servers
- One VM guest server is running directly on Dell Vostro 3360 notebook’s HDD to simulate HDD performance (2.5″ 5400 RPM), refer as VM-HDD
- One VM guest server is running from SSD (connected to Dell Vostro 3360 notebook via external harddisk box with USB 3.0) to simulate SSD performance (2.5″ consumer grade SSD), refer as VM-SSD
Both VM guest servers have been configured as below:
| Memory | 2GB |
| Processors | 2 |
| Hard Disk (SCSI) | 30GB (Pre-allocated) |
| Network Adapter | Host-only (to exclude network latency) |
| Display | Auto Detect |
| Shared Folders | Disabled |
- OC 3.1.3 Community is installed and configured as https access only on both VM guest servers
- All tests are performed by Firefox 18.0.1 installed on individual VM guest server in order to exclude network latency out of the tests
- OC 3.1.3 database size is 236MB
- PostgreSQL’s maintenance commands (VACUUM, Reindex) have been applied on both VM guest servers database before perform the tests
- OC related components version info: Apache Tomcat 6.0.36, jdk 6u38 (x64) and PostgreSQL 8.4.15
Test Results – Data Entry
| Storage Type |
Time used in seconds (entry only first page of individual event). Lower is better.
|
|||||||
|
Event 1
|
Event 2
|
Event 3
|
Event 4
|
Event 5
|
Event 6
|
Event 7
|
Total
|
|
| VM-HDD |
3.497
|
1.943
|
1.882
|
1.404
|
1.336
|
1.406
|
1.397
|
12.865
|
| VM-SSD |
1.893
|
1.41
|
1.425
|
1.225
|
1.121
|
1.158
|
1.099
|
9.331
|
| SSD Performance Gain % |
45.868
|
27.432
|
24.283
|
12.749
|
16.093
|
17.639
|
21.331
|
27.470
|
Comment: On average, SSD performance gain over HDD is around 27.47% which clearly shows shorter time to save entries data.
Test Results – Generic Dataset
| Dataset File Type [uncompress file size/compressed (.zip) file size] |
Time used in seconds. Lower is better.
|
||
|
VM-HDD
|
VM-SSD
|
SSD Performance Gain %
|
|
| CDISC ODM XML 1.3 Full with OpenClinica extensions [4,593KB/25 KB] |
948
|
1019
|
-7.49
|
| CDISC ODM XML 1.3 Clinical Data with OpenClinica extensions [1,502KB/117KB] |
939
|
1001
|
-6.60
|
| CDISC ODM XML 1.3 Clinical Data [1,036KB/107KB] |
1005
|
964
|
4.08
|
| CDISC ODM XML 1.2 Clinical Data with OpenClinica extensions [1,451KB/115KB] |
940
|
966
|
-2.77
|
| CDISC ODM XML 1.2 Clinical Data [1,035KB/107KB] |
970
|
961
|
0.93
|
| Excel Spreadsheet [64KB/15KB] |
1238
|
1145
|
7.51
|
| Tab-delimited Text [64KB/14KB] |
1251
|
1173
|
6.24
|
| SPSS data and syntax [135KB/24KB] |
1224
|
1180
|
3.59
|
| Total |
8515
|
8409
|
1.24
|
Comment: On average, SSD performance gain over HDD is just 1.24%. SSD does not have much advantage due to CPU usage and file write operations for processing datasets. SSD is even slower than HDD in some tests above.
Additional Test Results Iometer (www.iometer.org)
| Details |
VM-HDD
|
VM-SSD | SSD Performance Gain % | VM-SAS RIAD 10 | SAS RAID 10 Performance Gain % |
| Read I/O per second (higher is better) |
277.95
|
1539.68
|
553.94
|
1022.58
|
367.90
|
| Write I/O per second (higher is better) |
272.42
|
1520.15
|
558.02
|
1042.01
|
382.50
|
| Total I/O per second (higher is better) |
550.37
|
3059.83
|
559.96
|
2064.60
|
375.13
|
| Read MBs per second (higher is better) |
3.57
|
19.75
|
553.22
|
13.21
|
370.03
|
| Write MBs per second (higher is better) |
3.48
|
19.55
|
561.78
|
13.37
|
384.20
|
| Total MBs per second (higher is better) |
7.05
|
39.29
|
557.30
|
26.58
|
377.02
|
| Average Read Response Time / ms (lower is better) |
1.809
|
0.1824
|
991.78
|
0.5563
|
325.18
|
| Average Write Response Time / ms (lower is better) |
1.8203
|
0.4711
|
386.39
|
0.4081
|
446.04
|
| Average I/O Response Time / ms (lower is better) |
1.8146
|
0.3258
|
556.97
|
0.4815
|
376.86
|
| Maximum Read Response Time / ms (lower is better) |
384.3666
|
47.3384
|
811.96
|
131.6158
|
292.03
|
| Maximum Write Response Time / ms (lower is better) |
114.2165
|
50.6832
|
225.35
|
206.3535
|
-55.35
|
| Maximum I/O Response Time / ms (lower is better) |
384.3666
|
50.6832
|
758.37
|
206.3535
|
186.27
|
| Test period 2 minutes, one configuration has been used inside IometerVM-SAS RAID 10, Test on a test server |
Average Gain %
|
589.25
|
Average Gain %
|
318.98
|
|
Comments:
- SSD enterprise grade may not give much higher performance over SSD consumer grade, but it provides higher endurance on server type workloads. SSD consumer grade with server type workloads may produce poor results.
- SSD enterprise grade is much more expensive than SSD consumer grade. If you are planning to use SSD on the server, SSD enterprise grade is highly recommended (however, it’s up to you if you are willing to spend less and accept to risk your data)
- For people who already used SAS disks with RAID 10 (hardware RAID), switch to SSD (enterprise grade) may not give you greater benefits, especially when looking at cost and space limitations
- Internet connection speed/network latency is a key factor of for OpenClinica. Due to the inability to control this over test time, it was excluded from this list.
- Due to organization’s confidential information, I cannot share the actual CRFs used in tests. The CRFs generally had minimal text, calendar, and option box with up to 20-30 items.
Content and Testing Results supplied by:
Montri Ridjaibun
Clinical Trials Database/Software Architect Manager
Mahidol-Oxford Tropical Medicine Research Unit (MORU)
Bangkok/Thailand