數據庫基準測試（Database Benchmarking） 說明

1   TPC概述

對數據庫和服務器進行壓力測試或做對照的時候，都是常常會使用HammerDB 或 Benchmark之類的工具，而且最經常使用TPCC的標準進行測試。

    這里說的TPCC，前3個字母是：TPC 是Transaction Processing Council（事務處理委員會），其提供了量化的方法和標準。

    所以我們說的TPCC就是其中1種測試方法。  

    TPC的官網是：http://www.tpc.org/

從圖上可以看出，除TPC-C 以外，還有很多其他的方法，包括廢棄的TPC-A,TPC-B,TPC-D,TPC-R,TPC-W。

關于不同的測試方法的區分可以直接點擊官網的說明。

2   Oracle 官網對Benchmarking的說明

下文直接引自Oracle 的網站，原文地址已不存在，文章講了database benchmarking的緣由、標準，和幾款丈量工具軟件。

2.1             Reasons for Benchmarking

Benchmarks are performed for variousreasons. However, benchmarks are primarily used:

* To compare different hardware configurations

Benchmarks can be used to compare the relative performance ofdifferent hardware running the same application. This isgenerally used to directly compare hardware configurations between two hardwarevendors.

* To compare different database vendor software

By running the same benchmark usingdifferent database software on the same machine, one can easily compare betweendifferent database vendors. This is generally usedto make a price/performance decision between vendors such asOracle, Microsoft, IBM, etc.

* To compare different database software releases

Similar to the above, one can use different versions of the samevendor's database software to compare the one they want to use or checkfor performance regressions due to upgrades (i.e. 10g vs. 11g).

2.2             Real World Benchmarks

While there areseveral industry standard benchmarks, most of the time, they don'treflect a company's actual workload. As such, you will often findcustom benchmarks being performed. The goals of a real-world benchmark are thesame reasons mentioned above.

2.3             Industry Standard DatabaseBenchmarks

Industrystandard benchmarks are generally used by businesses to compare differenthardware and software system performance forpurchase-related reasons. While industry standard benchmarks are modelled afterreal-world workloads, they rarely reflect a company's realworkload. However, it is a quick way to compare the performance of varioushardware and software combinations operating in a well-defined scenario. Themajor industry standard benchmarks are listed below.

2.4             Transaction Processing Council(TPC)

The TPC is anon-profit corporation which supports a consortium of hardware and databasesoftware vendors devoted to defining transaction processing anddatabase-related benchmarks. The primary goal behind TPC benchmarks is thedefinition of functional requirements which can be run on any database,regardless of the hardware or operating system. This allows vendors toimplement their own benchmark kits in order to satisfythe functional requirements. Similarly, after having publicly submitted proofthat a benchmark was conducted according to the specification, end-users havemore of an assurance that what they are presented with is avalid, apples-to-apples comparison.

2.4.1 TPC-B

The TPC-B benchmark stresses databases and is characterizedby significant disk input/output, moderate system, application execution time,and transaction integrity.  Thisbenchmark targets database management systems (DBMS) batch applications, andback-end database servers.  TPC-B is notan OLTP benchmark.

2.4.2 TPC Benchmark C (TPC-C)

The TPC-C simulates an order-entryenvironment where apopulation of terminal operators executes transactions against a database. Thebenchmark is comprised of transactions which include entering and deliveringorders, recording payments, checking the status of orders, and monitoring thelevel of stock at the warehouses. The most frequent transaction consists ofentering a new order which, on average, iscomprised of ten different items. Each warehouse tries to maintain stock for the 100,000 itemsin the Company's catalog and fill orders from thatstock. The performance metric reported by TPC-C measures the number of ordersthat can be fully processed per minute and is expressed in tpm-C. The TPC-Cwill soon be deprecated in favor of the TPC-E.

2.4.3 TPC-D

The TPC-D benchmark represents a broad range ofdecision support (DS) applications that require complex, and long runningqueries against large complex data structures. Real-world business questions were written against this model, andresulted in 17 complex queries.

2.4.4 TPC Benchmark E (TPC-E)

The TPC-E benchmark simulates the OLTP workload of a brokeragefirm. The focus of the benchmark is the centraldatabase that executes transactions related to the firm’s customer accounts.The TPC-E metric is given intransactions per second (tps). It specifically refers to the number ofTrade-Result transactions the server can sustain over a period of time.

2.4.5 TPC Benchmark H (TPC-H)

TheTPC-H is adecision support benchmark which consists of several business-oriented ad-hocqueries and concurrent data modifications. This benchmark illustrates decisionsupport systems that examine large volumes of data, execute queries with a highdegree of complexity, and give answers to critical business questions.

The performancemetric reported by TPC-H is called the TPC-H Composite Query-per-HourPerformance Metric (QphH@Size), and reflects multiple aspects of the capabilityof the system to process queries.

2.5             Standard Performance EvaluationCorporation (SPEC)

2.5.1 Free and Open Source Benchmark Kits

As the benchmarkkits used by various database vendors are proprietary, several open sourcedatabase benchmark kits have been written to implement not only the industrystandard workloads, but others as well.

* Open Source Development Labs Database Test Suite

The OSDL DBT suite is the mostcomprehensive of all open source benchmark kits. While its benchmarkimplementations are based on TPC standards, they differ in someareas and are not certified; making a comparison between a certified TPC-Cresult and OSDL's implementation of TPC-C (DBT⑵) impossible. As the originalOSDL DBT suite lacks Oracle support, Oracle has added and released it as partof the Oracle Linux Test kit.

* PolePosition

PolePosition is an open sourcebenchmark test suite to compare database engines and object-relationalmapping technology.

* jTPCC

jTPCC is an open source Java implementation ofthe TPC-C benchmark. While it claims compliance with TPC-C, it is not.

* BenchmarkSQL

BenchmarkSQL is an open source,easy to use JDBC benchmark which closely resembles the TPC-C standard for OLTP;it was originally based on jTPCC.

* Bristlecone

Bristlecone is a simpleSELECT+INSERT/UPDATE/DELETE throughput and response-time benchmark written in Java.

* FinTime

FinTime is an open source set ofdata and queries which reflects the needs of financial analysts who arestudying patterns in stock market data, but it should appealto the designers of any system that has pretensions of handling ordered datawell.

* Java TPC-W Implementation

This open-source implementation is basedon the TPC-W specification version 1.0.1. Itstrays from the official benchmark specification in a fewminor areas, which are specified in documentationincluded with the distribution. This version itself lacks Oracle support, butit can be found on the web.

* Swingbench

Swingbench is a free loadgenerator (and benchmarks) designed to stress test an Oracle database(9i,10g,11g).

* OpenLink ODBC Bench

OpenLink ODBC Bench is an open-sourceODBC Benchmarking tool providing real-time comparative benchmarking for ODBCDrivers, Database Engines, and Operating Systems combinations. The Benchmarks in thisapplication are loosely based on the TPC-A and TPC-C standard benchmarks, withmodifications to specifically test the performance of an ODBC Driver and/orDatabase Engine in a client/server environment.

* OpenLink JDBC Bench

Similar to OpenLink's ODBC Bench, this is a JDBC version.

* TPCC-UVa

A free, open-source implementation of the TPC-C Benchmark.

* Hammerora

An open source load generation tool.

2.5.2 Commercial Benchmark Kits

There are several industry standard benchmarks.

* Official Oracle Benchmark Kits

Similar to Microsoft and IBM, the official benchmark kits used byOracle can only be obtained under certain conditions. For more information,contact your Oracle Consulting/Partner/Support representative.

* Real Application Testing

While many wouldn't think of Real Application Testing asa traditional benchmark kit, it does perform workload execution primarily forthe real world benchmarking case.

* Quest Software'sBenchmark Factory for Databases

Benchmark Factory for Databases is aworkload simulation and scalability testing tool that stress tests yourenvironment by simulating users and transactions on the database. You caneither replay production workload or use synthetic workload to test the limitsof your database.

* SPEC jAppServer Benchmark Kit

This can be acquired directly from SPEC.

2.6             Benchmarking-related Books

There are several good books regarding benchmarking. The best I'vefound include the following:

* The Benchmark Handbook: For Database andTransaction Processing Systems (Jim Gray)

A great, albeit outdated book. While several of the benchmarks itdiscusses have been deprecated, it presents good concepts and backgroundinformation regarding benchmarking.

* Performance Analysis of Transaction ProcessingSystems (Wilbur H. Highleyman)

While out of print, this bookpresents both plain-text and math-heavy concepts of queueing and performanceanalysis techniques oriented for OLTP.

* Database Benchmarking: Practical Methods forOracle & SQL Server

This book explores all aspects of database benchmarking and showsyou a real-world approach that ensures that you are prepared forwhatever the future brings to your mission-critical database.

--------------------------------------------------------------------------------------------

版權所有，文章制止轉載，否則追究法律責任!

AboutDave:

--------------------------------------------------------------------------------------------

QQ:     251097186

Email:    tianlesoftware@gmail.com

Blog:    http://blog.csdn.net/tianlesoftware

Weibo:    http://weibo.com/tianlesoftware

Twitter:  http://twitter.com/tianlesoftware

Facebook: http://www.facebook.com/tianlesoftware

Linkedin: http://cn.linkedin.com/in/tianlesoftware

Dave 的QQ群：

--------------------------------------------------------------------------------------------

注意:加群必須注明表空間和數據文件關系 | 不要重復加群

CNDBA_1: 104207940 (滿)    CNDBA_2: 62697716 (滿)   CNDBA_3: 283816689

CNDBA_4: 391125754   CNDBA_5: 62697850    CNDBA_6: 62697977   CNDBA_7: 142216823（滿）

生活不易，碼農辛苦
如果您覺得本網站對您的學習有所幫助,可以手機掃描二維碼進行捐贈