Saturday, May 7, 2011

QA Dept: AidAim initial benchmark results

Correspondent from New York, 
Correspondent from Australia,
Your Reviewer
Article to follow soon


Truth in Advertising Dept
When you claim that you are the best, you have the most features, it may be prudent to have your claims independently tested and verified.


For example, if you claim to be the best product in your market segment, lot of corporations have their own in-house technical support who will also test the products to be installed on hundreds of corporate computers. If the in-house technical support cannot install the product, or run into 101 technical support issues, then your product will probably be thrown out.


Your reviewer et al. will be reviewing AidAim claims with high-performance database. Your reviewer is using full version (with sources) of AidAim, NexusDB and ElevateDB to run this initial test. We went one step further than normal to switch four databases with a custom-developed software against a 25-users test-environment.


Initial Opinions
Your reviewer et al. were unable to find the original sources which AidAim used to test the product with, so we went ahead with what was already available and extended it. We took out the EasyTable because it was not worthy, even by AidAim standards to be benchmarked and replaced it with ElevateDB and added NexusDB.

After coding for a few days to get optimal results (Delphi XE, Release build, StringFormat checking off, FastMM4 4.97) with default component installation and ran this test result that AidAim uses.


Lower scores means better results


In every test that the AidAim developers used to test their product (except the LocateByID test), both ElevateDB and NexusDB posted faster results than Accuracer itself. The most astonishing being NexusDB results which, the AidAim developers used to pester the fact they were slower than Accuracer itself. To check the claim about no RecNo issue, your reviewer enabled this feature and went ahead and checked it.


We thought it was a joke that NexusDB had such fast results, so we ran the benchmark six or seven times and took a screenshot (above).


The testing environment here uses latest hardware: Core i7-2600K overclocked at 3.6Ghz, 8GB Ram,  INTEL SSD-SC2MH120A2, dual-monitor, Windows 2008R2 x64bit environment.


If you are tired of AidAim in-your-face obnoxious advertising, they really need to stop this kind of advertising they are doing: It will only hurt them very deeply that they keep claiming that their product is the fastest, the best, when, these claims are independently tested - are false, misleading and dubious at best.


More to come.

Update1:
Parts of a newer article mentions about Accuracer (link)

7 comments:

Michael Bunny said...

Nexus DB is surprisingly fast. I did not benchmark but from experience the overall behavior of this database is exceptional.

Louis said...

Does anyone know how Nexus compares with DISQLite3?

Delphi Haters said...

You mean Ralf Junker's Delphi Inspiration SQLLite3?


There will be 3 tests for this, one in Visual C++, another in Intel C++, and last Delphi XE.


Give it a week or so.

Louis said...

Yup. Ralf's is the one.

Over 4 years ago, there was this discussion which seemed to make DISQLite3 the fastest: http://www.delphigroups.info/2/12/264047.html I'm very interested in seeing if that still is the case.

Disclaimer: I have no connection Ralf Junker or to DISQLite3. I'm just in the process of evaluating single-file Delphi databases for my genealogy application. I need something to handle very large datasets as quickly as possible.

Louis said...

Not sure, but was this Embarcadero Forum post the AidAim claims your post is talking about?

Delphi Haters said...

Hi, yes.

They want to prove their product is better, but we tested it independently. The results show otherwise.

Delphi Haters said...

Russian forum post link:
http://www.sql.ru/forum/actualthread.aspx?bid=20&tid=872871