澳门新萄京官方网站-www.8455.com-澳门新萄京赌场网址

澳门新萄京官方网站OLTP基准测验,Tookit工具包之

2019-09-16 作者:数据库网络   |   浏览(189)

 

 

Chapter 6: Benchmarking & Tuning

As the first four chapters of this book were all about performance as a coding pattern (asynchrony and concurrency), and Chapter 5 was about performance at the macro program architecture level, this chapter goes after the topic of performance at the micro level, focusing on single expressions/statements.

One of the most common areas of curiosity -- indeed, some developers can get quite obsessed about it -- is in analyzing and testing various options for how to write a line or chunk of code, and which one is faster.

We're going to look at some of these issues, but it's important to understand from the outset that this chapter is not about feeding the obsession of micro-performance tuning, like whether some given JS engine can run a faster than a . The more important goal of this chapter is to figure out what kinds of JS performance matter and which ones don't, and how to tell the difference.

But even before we get there, we need to explore how to most accurately and reliably test JS performance, because there's tons of misconceptions and myths that have flooded our collective cult knowledge base. We've got to sift through all that junk to find some clarity.

Load and Performance Test Tools

BrowserMob - On-demand, self-service, low-cost, pay-as-you-go service enables simulation of large volumes of real browsers hitting a website. Utilizes Amazon Web Services, Selenium. Uses real browsers for each virtual user so that traffic is realistic, AJAX & Flash support is automatic. Browser screen shots of errors included in reports.

Load Impact - Online load testing service from Gatorhole/loadimpact.com for load- and stress- testing of your website over the Internet; access to our distributed network of load generator nodes - server clusters with very fast connections to enable simulation of tens of thousands of users accessing your website concurrently. Free low level load tests for 1-50 simulated users; higher levels have monthly fees.

Pylot - Open source tool by Corey Goldberg for generating concurrent http loads. Define test cases in an XML file - specify requests - url, method, body/payload, etc - and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support.Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, rampup time, test duration. Real-time stats and error reporting are displayed.

AppLoader - Load testing app from NRG Global for web and other applications accessible from a Windows desktop; generates load from the end user's perspective. Protocol independent and supports a wide variety of enterprise class applications. Integrateswith their Chroniker monitoring suite so results of load testing can be correlated with system behavior as load is increased. Runs from Win platforms.

fwptt - Open source tool by Bogdan Damian for load testing web applications. Capabilities include handling of Ajax. Generates tests in C#. For Windows platforms

JCrawler - An open-source stress-testing tool for web apps; includes crawling/exploratory features. User can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load onthe web application. Load parameters (hits/sec) are configurablevia central XML file; fires up as many threads as needed to keep load constant; includes self-testing unit tests. Handles http redirects and cookies; platform independent.

vPerformer - Performance and load testing tool from Verisium Inc. to assess the performance and scalability of web apps. Use recorded scripts or customized scripts using Javascript.Targeted platforms: Windows

Curl-Loader - Open-source tool written in 'C', simulating application load and behavior of tens of thousand HTTP/HTTPS and FTP/FTPS clients, each with its own source IP-address. In contrast to other tools curl-loader is using real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of openssl. Activities of each virtual client are logged and collected statistics include information about: resolving, connection establishment, sending of requests, receiving responses, headers and data received/sent, errors from network, TLS/SSL and application (HTTP, FTP) level events and errors.

RealityLoad XF On-Demand Load Testing -An on-demand load testing service (no licenses) from Gomez.com. Leverages Gomez' peer panel, which consists of over 15,000 end-user desktop testing locations distributed across the world, to provide distributed load tests that accurately reproduce the network and latency characteristics encountered by real users in a live environment.

OpNet LoadScaler -Load test tool from OpNet Technologies Inc. Create tests without programming; generate loads against web applications, and other services including Web Services, FTP, and Email. Record end-user browser activity in the OPNET TestCreatorTM authoring environment to automatically generate test scripts in industry-standard JavaScript. Modify, extend and debug tests with the included JavaScript editor. Alternatively, drag and drop icons onto the test script tree. No knowledge of a scripting language is required to customize test scripts.

StressTester - Enterprise load and performance testing tool for web applications from Reflective Solutions Ltd. Advanced user journey modeling, scalable load, system resources monitors and results analysis. No scripting required. Suitable for any Web, JMS, IP or SQL Application. OS independent.

The Grinder -A Java-based load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.

Proxy Sniffer - Web load and stress testing tool from from Ingenieurbüro David Fischer GmbHCapabilities include: HTTP/S Web Session Recorder that can be used with any web browser; recordings can then be used to automatically create optimized Java-based load test programs; automatic protection from "false positive" results by examining actual web page content; detailed Error Analysis using saved error snapshots; real-time statistics.

Testing Master - Load test tool from Novosoft, capabilities include IP spoofing, multiple simultaneous test cases and website testing features for sites with dynamic content and secure HTTPS pages.

Funkload - Web load testing, stress testing, and functional testing tool written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit;https support; produces detailed reports in ReST, HTML, or PDF.

Avalanche - Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Simulates as many as two million concurrently-connected users with unique IP addresses, emulates multiple Web browsers, supports Web Services testing Supports HTTP 1.0/1.1, SSL, FTP, RTSP/ RTP, MS Win Media, SMTP, POP3, DNS, Telnet, and Video on Demand over Multicast protocols.

Loadea -Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.

LoadManager -Load, Stress, Stability and Performance testing tool from Alvicom. Runs on all platforms supported by Eclipse and Java.

QEngine Web Performance Testing -Automated testing tool from AdventNet for performance testing (load and stress testing) of web applications and web services. For Linux and Windows.

NeoLoad -Load testing tool for web applications from Neotys with clear and intuitive graphical interface, no scripting/fast learning curve, clear and comprehensive reports and test results. Can design complexscenarios to handle real world applications. Features includedata replacement, data extraction, SOAP support, system monitoring(Windows, Linux, IIS, Apache, WebLogic, Websphere...), SSL recording, PDF/HTML/Word reporting, IP spoofing, and more. Multi-platform: Windows,Linux, Solaris.

Test Complete Enterprise -Automated test tool from AutomatedQA Corp. includes web load testing capabilities.

WebPartner Test and Performance Center -Test tool from WebPartner for stress tests, load performance testing, transaction diagnostics and website monitoring of HTTP/HTTPS web transactions and XML/SOAP/WSDL web services.

QTest -Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.

Test Perspective Load Test -Do-it-yourself load testing service from Keynote Systems for Webapplications. Utilizes Keynote's load-generating infrastructure on theInternet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web application infrastructure.

SiteTester1 - Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests,HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.

httperf -Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code.

NetworkTester -Tool (formerly called 'NetPressure') from Agilent Technologies uses real user traffic, including DNS, HTTP, FTP, NNTP, streaming media, POP3, SMTP, NFS, CIFS, IM, etc. - through access authentication systems such as PPPOE, DHCP, 802.1X, IPsec, as necessary. Unlimited scalability; GUI-driven management station; no scripting; open API. Errors isolated and identified in real-time; traffic monitored at every step in a protocol exchange (such as time of DNS lookup, time to logon to server, etc.). All transactions logged, and detailed reporting available.

WAPT - Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.

Visual Studio Team System 2008 Test Edition -A suite of testing tools for Web applications and services that are integrated into the Microsoft Visual Studio environment. These enable testers to author, execute, and manage tests and related work items all from within Visual Studio.

OpenLoad -Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum hardware.

Apache JMeter -Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.

TestMaker -Free open source utility maintained by PushToTest.com and Frank Cohen, for performance, scalability, and functional testing of Web application. Features test authoring of Web applications, Rich Internet Applications (RIA) using Ajax, Service Oriented Architecture, and Business Process Management environments. Integrates Selenium, soapUI, TestGen4Web, and HTMLUnit to make test development faster/easier. Repurposes tests from these tools into load and performance tests, functional tests, and business service monitors with no coding. Repurposes unit tests written in Java, Jython, JRuby, Groovy, and other dynamic scripting languages. Runs on any platform.

SiteStress -Remote, consultative load testing service by Webmetrics. Simulates end-user activity against designated websites for performance and infrastructure reliability testing. Can generate an infinitely scalable user load from GlobalWatch Network, and provide performance reporting, analysis, and optimization recommendations.

Siege -Open source stress/regression test and benchmark utility; supports basic authentication, cookies, HTTP and HTTPS protocols.Enables testing a web server with a configurable number of concurrent simulated users. Stress a single URL with a specified number of simulated users or stress multiple URL's simultaneously. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein's torture.pl, but allows stressing many URLs simultaneously. Distributed under terms of the GPL; written in C; for UNIX and related platforms.

JBlitz - Load, performance and functional test tool from Clan Productions. Runs multiple concurrent virtual users.to simulate heavy load. Validates each response using plain text or regular expression searches, or by calling out to your own custom code. Full Java API. For testing and 'bullet-proofing' server side software - ASPs, JSPs, servlets, EJBs, Perl / PHP / C / C / CGI scripts etc.

WebServer Stress Tool -Web stress test tool from Paessler AG handles proxies, passwords, user agents, cookies, AAL.

Web Polygraph -Freely available benchmarking tool for caching proxies,origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C source available; binaries avail for Windows.

OpenSTA -'Open System Testing Architecture' is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.

PureLoad -Java-based multi-platform performance testing and analysis toolfrom Minq Software. Includes 'Comparer' and 'Recorder' capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.

ApacheBench -Perl API for Apache benchmarking and regression testing.Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.

Torture -Bare-bones Perl script by Lincoln Stein for testingweb server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.

WebSpray -Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. For Windows.

eValid LoadTest -Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.

WebPerformance Load Tester -Load test tool emphasizing ease-of-use, from WebPerformance Inc.Supports all browsers and web servers; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. For Windows, Linux, Solaris, most UNIX variants.

WebSuite -A collection of load testing, capture/playback, and related tools from Technovations for performance testing of web sites.Modules include WebCorder, Load Director, Report Generator, Batch,Manager, and others. WebSizr load testing tool supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. For Windows.

FORECAST - Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C scripting; windows browser or network recording/playback. Supports binary encoded data such as Adobe Flex/AMF, Serialised Java objects etc.SSL; supports NTLM, kerberos, proxies, authentication, redirects, certificates, cookies, caching, bandwidth limitation and page validation. Virtual user data can be parameterized. Works with a wide variety of platforms.

http-Load -Free load test application from ACME Labs to generate web server loads, from ACME Software. Handles HTTP and HTTPS; for Unix.

QALoad -Compuware's tool for load/stress testing of web, database,and character-based systems. Supports HTTP, SSL, SOAP, XML, Streaming Media. Works with a variety of databases, middleware, ERP.

Microsoft WCAT load test tool -Web load test tool from Microsoft for load testingof MS IIS servers; other MS stress tools also listed.

IBM Rational Performance Tester -Performance testing tool from IBM/Rational; has optional extensions to Seibel applications and SAP Solutions. Supports Windows, Linux and z/OS as distributed controller agents; provides high-level and detailed views of tests.

SilkPerformer -Enterprise-class load-testing tool from Borland (formerly Segue). Can simulate thousands of users working with multiple protocols andcomputing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity.

Radview's WebLoad -Load testing tool from Radview Software, also available as part of theirTestView web testing suite. Basic version available as open source, add-ons available for purchase. Capabilities include over 75 Performance Metrics; can view global or detailed account of transaction successes/failures on individual Virtual Client level, assisting in capturing intermittent errors; allows comparing of running test vs. past test metrics. Test scripting via visual tool or Javascript. Wizard for automating non-GUI-based services testing; DoS security testing.

Loadrunner -HP's (formerly Mercury's) load/stress testing tool for web and other applications; supports a wide variety of application environments, platforms, anddatabases. Large suite of network/app/server monitors to enableperformance measurement of each tier/server/componentand tracing of bottlenecks.

作者:Grig Gheorghiu
小编联系格局:

Preface

Preface

Benchmarking

OK, time to start dispelling some misconceptions. I'd wager the vast majority of JS developers, if asked to benchmark the speed (execution time) of a certain operation, would initially go about it something like this:

var start = (new Date()).getTime(); // or `Date.now()`

// do some operation

var end = (new Date()).getTime();

console.log( "Duration:", (end - start) );

Raise your hand if that's roughly what came to your mind. Yep, I thought so. There's a lot wrong with this approach, but don't feel bad; we've all been there.

What did that measurement tell you, exactly? Understanding what it does and doesn't say about the execution time of the operation in question is key to learning how to appropriately benchmark performance in JavaScript.

If the duration reported is 0, you may be tempted to believe that it took less than a millisecond. But that's not very accurate. Some platforms don't have single millisecond precision, but instead only update the timer in larger increments. For example, older versions of windows (and thus IE) had only 15ms precision, which means the operation has to take at least that long for anything other than 0 to be reported!

Moreover, whatever duration is reported, the only thing you really know is that the operation took approximately that long on that exact single run. You have near-zero confidence that it will always run at that speed. You have no idea if the engine or system had some sort of interference at that exact moment, and that at other times the operation could run faster.

What if the duration reported is 4? Are you more sure it took about four milliseconds? Nope. It might have taken less time, and there may have been some other delay in getting either start or end timestamps.

More troublingly, you also don't know that the circumstances of this operation test aren't overly optimistic. It's possible that the JS engine figured out a way to optimize your isolated test case, but in a more real program such optimization would be diluted or impossible, such that the operation would run slower than your test.

So... what do we know? Unfortunately, with those realizations stated, we know very little. Something of such low confidence isn't even remotely good enough to build your determinations on. Your "benchmark" is basically useless. And worse, it's dangerous in that it implies false confidence, not just to you but also to others who don't think critically about the conditions that led to those results.

I recently got some comments/questions related to my previous blog entry on performance vs. load vs. stress testing. Many people are still confused as to exactly what the difference is between performance and load testing. I've been thinking more about it and I'd like to propose the following question as a litmus test[试金石] to distinguish between these two types of testing: are you actively profiling your application code and/or monitoring the server(s) running your application? If the answer is yes, then you're engaged in performance testing. If the answer is no, then what you're doing is load testing.

 

 

Repetition

"OK," you now say, "Just put a loop around it so the whole test takes longer." If you repeat an operation 100 times, and that whole loop reportedly takes a total of 137ms, then you can just divide by 100 and get an average duration of 1.37ms for each operation, right?

Well, not exactly.

A straight mathematical average by itself is definitely not sufficient for making judgments about performance which you plan to extrapolate to the breadth of your entire application. With a hundred iterations, even a couple of outliers (high or low) can skew the average, and then when you apply that conclusion repeatedly, you even further inflate the skew beyond credulity.

Instead of just running for a fixed number of iterations, you can instead choose to run the loop of tests until a certain amount of time has passed. That might be more reliable, but how do you decide how long to run? You might guess that it should be some multiple of how long your operation should take to run once. Wrong.

Actually, the length of time to repeat across should be based on the accuracy of the timer you're using, specifically to minimize the chances of inaccuracy. The less precise your timer, the longer you need to run to make sure you've minimized the error percentage. A 15ms timer is pretty bad for accurate benchmarking; to minimize its uncertainty (aka "error rate") to less than 1%, you need to run your each cycle of test iterations for 750ms. A 1ms timer only needs a cycle to run for 50ms to get the same confidence.

But then, that's just a single sample. To be sure you're factoring out the skew, you'll want lots of samples to average across. You'll also want to understand something about just how slow the worst sample is, how fast the best sample is, how far apart those best and worse cases were, and so on. You'll want to know not just a number that tells you how fast something ran, but also to have some quantifiable measure of how trustable that number is.

Also, you probably want to combine these different techniques (as well as others), so that you get the best balance of all the possible approaches.

That's all bare minimum just to get started. If you've been approaching performance benchmarking with anything less serious than what I just glossed over, well... "you don't know: proper benchmarking."

[译:小编上一篇公布在友好博客上的稿子关于品质、负载、压力的测量试验小说收到了部分主题材料和评价,比相当多人对哪些精确的区分品质测验与负载测验还是很迷惑。为此笔者合计了一段时间,並且愿意将如下的难点做为试金石来分别出这两类测量检验:你是积极的深入分析你的程序代码或然监测正运转你程序服务器的服务器吗?假若答案是必定的,则你正致力质量测量检验。假设答案是还是不是定的,则你做的是负载测量试验]

    In order to know clearly about the real performance threshold of database server,we usually do database benchmarks.What's it?It's similar with preasure test of database in the whole system.The difference is that the database benchmark may not care about the real business performance but the direct performance with virtual data.It's simple and convenient to implement whereas what is more complex in presure test.The items what we really care about in database benchmark is TPS/QPS/RT(Response Time)/Concurrency,etc.I'm gonna use sysbench(one of the benchmark test tools) to do some basic oltp read write test of MySQL database today.

    We have a lot of methods to diagnose problems in our system such as strace,pstack,gstack,gdb,pt-pmp,etc.But sometimes there will be some fitful performance issues which are not so easy to trace.Thus,pt-stalk may help us in diagnosing these kind of problems.

Benchmark.js

Any relevant and reliable benchmark should be based on statistically sound practices. I am not going to write a chapter on statistics here, so I'll hand wave around some terms: standard deviation, variance, margin of error. If you don't know what those terms really mean -- I took a stats class back in college and I'm still a little fuzzy on them -- you are not actually qualified to write your own benchmarking logic.

Luckily, smart folks like John-David Dalton and Mathias Bynens do understand these concepts, and wrote a statistically sound benchmarking tool called Benchmark.js (http://benchmarkjs.com/). So I can end the suspense by simply saying: "just use that tool."

I won't repeat their whole documentation for how Benchmark.js works; they have fantastic API Docs (http://benchmarkjs.com/docs) you should read. Also there are some great (http://calendar.perfplanet.com/2010/bulletproof-javascript-benchmarks/) writeups (http://monsur.hossa.in/2012/12/11/benchmarkjs.html) on more of the details and methodology.

But just for quick illustration purposes, here's how you could use Benchmark.js to run a quick performance test:

function foo() {
    // operation(s) to test
}

var bench = new Benchmark(
    "foo test",             // test name
    foo,                    // function to test (just contents)
    {
        // ..               // optional extra options (see docs)
    }
);

bench.hz;                   // number of operations per second
bench.stats.moe;            // margin of error
bench.stats.variance;       // variance across samples
// ..

There's lots more to learn about using Benchmark.js besides this glance I'm including here. But the point is that it's handling all of the complexities of setting up a fair, reliable, and valid performance benchmark for a given piece of JavaScript code. If you're going to try to test and benchmark your code, this library is the first place you should turn.

We're showing here the usage to test a single operation like X, but it's fairly common that you want to compare X to Y. This is easy to do by simply setting up two different tests in a "Suite" (a Benchmark.js organizational feature). Then, you run them head-to-head, and compare the statistics to conclude whether X or Y was faster.

Benchmark.js can of course be used to test JavaScript in a browser (see the "jsPerf.com" section later in this chapter), but it can also run in non-browser environments (Node.js, etc.).

One largely untapped potential use-case for Benchmark.js is to use it in your Dev or QA environments to run automated performance regression tests against critical path parts of your application's JavaScript. Similar to how you might run unit test suites before deployment, you can also compare the performance against previous benchmarks to monitor if you are improving or degrading application performance.

Another way to look at it is to see whether you're doing more of a white-box type testing as opposed to black-box testing. In the white-box approach, testers, developers, system administrators and DBAs work together in order to instrument the application code and the database queries (via specialized profilers for example), and the hardware/operating system of the server(s) running the application and the database (via monitoring tools such as vmstat, iostat, top or Windows PerfMon). All these activities belong to performance testing.

 

 

Setup/Teardown

In the previous code snippet, we glossed over the "extra options" { .. } object. But there are two options we should discuss: setup and teardown.

These two options let you define functions to be called before and after your test case runs.

It's incredibly important to understand that your setup and teardown code does not run for each test iteration. The best way to think about it is that there's an outer loop (repeating cycles), and an inner loop (repeating test iterations). setup and teardown are run at the beginning and end of each outer loop (aka cycle) iteration, but not inside the inner loop.

Why does this matter? Let's imagine you have a test case that looks like this:

a = a   "w";
b = a.charAt( 1 );

Then, you set up your test setup as follows:

var a = "x";

Your temptation is probably to believe that a is starting out as "x" for each test iteration.

But it's not! It's starting a at "x" for each test cycle, and then your repeated "w" concatenations will be making a larger and larger a value, even though you're only ever accessing the character "w" at the 1 position.

Where this most commonly bites you is when you make side effect changes to something like the DOM, like appending a child element. You may think your parent element is set as empty each time, but it's actually getting lots of elements added, and that can significantly sway the results of your tests.

[译:从另一方面思索它的形式是是还是不是你相对于黑盒测量试验做了越来越多的白盒测量试验的干活。在白盒测量检验里,测量检验人士,开采职员,系统管理员,数据库管理员在联合工作为了兑现程序和数据库查询(例如通过特殊的模具),并且应用程序和多少运维在服务器(组)上的硬件/操作系统上(通过监视工具如vmstat, iostat, top or Windows PerfMon等).所以这个动作归属为质量测验。]

Introduce

Introduce

Context Is King

Don't forget to check the context of a particular performance benchmark, especially a comparison between X and Y tasks. Just because your test reveals that X is faster than Y doesn't mean that the conclusion "X is faster than Y" is actually relevant.

For example, let's say a performance test reveals that X runs 10,000,000 operations per second, and Y runs at 8,000,000 operations per second. You could claim that Y is 20% slower than X, and you'd be mathematically correct, but your assertion doesn't hold as much water as you'd think.

Let's think about the results more critically: 10,000,000 operations per second is 10,000 operations per millisecond, and 10 operations per microsecond. In other words, a single operation takes 0.1 microseconds, or 100 nanoseconds. It's hard to fathom just how small 100ns is, but for comparison, it's often cited that the human eye isn't generally capable of distinguishing anything less than 100ms, which is one million times slower than the 100ns speed of the X operation.

Even recent scientific studies showing that maybe the brain can process as quick as 13ms (about 8x faster than previously asserted) would mean that X is still running 125,000 times faster than the human brain can perceive a distinct thing happening. X is going really, really fast.

But more importantly, let's talk about the difference between X and Y, the 2,000,000 operations per second difference. If X takes 100ns, and Y takes 80ns, the difference is 20ns, which in the best case is still one 650-thousandth of the interval the human brain can perceive.

What's my point? None of this performance difference matters, at all!

But wait, what if this operation is going to happen a whole bunch of times in a row? Then the difference could add up, right?

OK, so what we're asking then is, how likely is it that operation X is going to be run over and over again, one right after the other, and that this has to happen 650,000 times just to get a sliver of a hope the human brain could perceive it. More likely, it'd have to happen 5,000,000 to 10,000,000 times together in a tight loop to even approach relevance.

While the computer scientist in you might protest that this is possible, the louder voice of realism in you should sanity check just how likely or unlikely that really is. Even if it is relevant in rare occasions, it's irrelevant in most situations.

The vast majority of your benchmark results on tiny operations -- like the x vs x myth -- are just totally bogus for supporting the conclusion that X should be favored over Y on a performance basis.

In practice though the 2 terms are often used interchangeably, and I am as guilty as anyone else of doing this, since I called one of my recent blog entries "HTTP performance testingwith httperf, autobench and openload" instead of calling it more precisely "HTTP load testing". I didn't have access to the application code or the servers hosting the applications I tested, so I wasn't really doing performance testing, only load testing.

 

 

Engine Optimizations

You simply cannot reliably extrapolate that if X was 10 microseconds faster than Y in your isolated test, that means X is always faster than Y and should always be used. That's not how performance works. It's vastly more complicated.

For example, let's imagine (purely hypothetical) that you test some microperformance behavior such as comparing:

var twelve = "12";
var foo = "foo";

// test 1
var X1 = parseInt( twelve );
var X2 = parseInt( foo );

// test 2
var Y1 = Number( twelve );
var Y2 = Number( foo );

If you understand what parseInt(..) does compared to Number(..), you might intuit that parseInt(..) potentially has "more work" to do, especially in the foo case. Or you might intuit that they should have the same amount of work to do in the foo case, as both should be able to stop at the first character "f".

Which intuition is correct? I honestly don't know. But I'll make the case it doesn't matter what your intuition is. What might the results be when you test it? Again, I'm making up a pure hypothetical here, I haven't actually tried, nor do I care.

Let's pretend the test comes back that X and Y are statistically identical. Have you then confirmed your intuition about the "f" character thing? Nope.

It's possible in our hypothetical that the engine might recognize that the variables twelve and foo are only being used in one place in each test, and so it might decide to inline those values. Then it may realize that Number( "12" ) can just be replaced by 12. And maybe it comes to the same conclusion with parseInt(..), or maybe not.

Or an engine's dead-code removal heuristic could kick in, and it could realize that variables X and Y aren't being used, so declaring them is irrelevant, so it doesn't end up doing anything at all in either test.

And all that's just made with the mindset of assumptions about a single test run. Modern engines are fantastically more complicated than what we're intuiting here. They do all sorts of tricks, like tracing and tracking how a piece of code behaves over a short period of time, or with a particularly constrained set of inputs.

What if the engine optimizes a certain way because of the fixed input, but in your real program you give more varied input and the optimization decisions shake out differently (or not at all!)? Or what if the engine kicks in optimizations because it sees the code being run tens of thousands of times by the benchmarking utility, but in your real program it will only run a hundred times in near proximity, and under those conditions the engine determines the optimizations are not worth it?

And all those optimizations we just hypothesized about might happen in our constrained test but maybe the engine wouldn't do them in a more complex program (for various reasons). Or it could be reversed -- the engine might not optimize such trivial code but may be more inclined to optimize it more aggressively when the system is already more taxed by a more sophisticated program.

The point I'm trying to make is that you really don't know for sure exactly what's going on under the covers. All the guesses and hypothesis you can muster don't amount to hardly anything concrete for really making such decisions.

Does that mean you can't really do any useful testing? Definitely not!

What this boils down to is that testing not real code gives you not real results. In so much as is possible and practical, you should test actual real, non-trivial snippets of your code, and under as best of real conditions as you can actually hope to. Only then will the results you get have a chance to approximate reality.

Microbenchmarks like x vs x are so incredibly likely to be bogus, we might as well just flatly assume them as such.

[译:在实施中那二种术语平日被轮番使用,作者也愧对的如此做过,因为作者在近些日子的一篇"HTTP performance testingwith httperf, autobench and openload" 的篇章题目应该更贴切的轮换为"HTTP load testing",作者并未有接触被测量试验程序的源码或然服务器主机,所以自个儿从没真正的做品质测量检验而仅是负载测验]

    sysbench is a common tool used in database bentchmark to test MySQL database(of cource,it supports PostgreSQL and Oracle,too).It provide many test case through several built-in lua scripts by specifying option of "testname".We can use the tool to get performance statistics about CPU,IO,Memory,etc.

    pt-stalk is a tool to collect detail diagnostic data base on triggers you specified such as gdb,oprofile,strace,tcpdump.The trigger is not the conception of trigger in database.They're different at all.pt-stalk provides various options to collect comprehensive data you need.It's really useful and helpful in performance diagnosing.Let's see some details of it.

jsPerf.com

While Benchmark.js is useful for testing the performance of your code in whatever JS environment you're running, it cannot be stressed enough that you need to compile test results from lots of different environments (desktop browsers, mobile devices, etc.) if you want to have any hope of reliable test conclusions.

For example, Chrome on a high-end desktop machine is not likely to perform anywhere near the same as Chrome mobile on a smartphone. And a smartphone with a full battery charge is not likely to perform anywhere near the same as a smartphone with 2% battery life left, when the device is starting to power down the radio and processor.

If you want to make assertions like "X is faster than Y" in any reasonable sense across more than just a single environment, you're going to need to actually test as many of those real world environments as possible. Just because Chrome executes some X operation faster than Y doesn't mean that all browsers do. And of course you also probably will want to cross-reference the results of multiple browser test runs with the demographics of your users.

There's an awesome website for this purpose called jsPerf (http://jsperf.com). It uses the Benchmark.js library we talked about earlier to run statistically accurate and reliable tests, and makes the test on an openly available URL that you can pass around to others.

Each time a test is run, the results are collected and persisted with the test, and the cumulative test results are graphed on the page for anyone to see.

When creating a test on the site, you start out with two test cases to fill in, but you can add as many as you need. You also have the ability to set up setup code that is run at the beginning of each test cycle and teardown code run at the end of each cycle.

Note: A trick for doing just one test case (if you're benchmarking a single approach instead of a head-to-head) is to fill in the second test input boxes with placeholder text on first creation, then edit the test and leave the second test blank, which will delete it. You can always add more test cases later.

You can define the initial page setup (importing libraries, defining utility helper functions, declaring variables, etc.). There are also options for defining setup and teardown behavior if needed -- consult the "Setup/Teardown" section in the Benchmark.js discussion earlier.

The black box approach is to run client load tools against the application in order to measure its responsiveness. Such tools range from lightweight, command-line driven tools such as httperf, openload, siege, Apache Flood, to more heavy duty tools such as OpenSTA, The Grinder, JMeter. This type of testing doesn't look at the internal behavīor of the application, nor does it monitor the hardware/OS resources on the server(s) hosting the application. If this sounds like the type of testing you're doing, then I call it load testing.

 

 

Sanity Check

jsPerf is a fantastic resource, but there's an awful lot of tests published that when you analyze them are quite flawed or bogus, for any of a variety of reasons as outlined so far in this chapter.

Consider:

// Case 1
var x = [];
for (var i=0; i<10; i  ) {
    x[i] = "x";
}

// Case 2
var x = [];
for (var i=0; i<10; i  ) {
    x[x.length] = "x";
}

// Case 3
var x = [];
for (var i=0; i<10; i  ) {
    x.push( "x" );
}

Some observations to ponder about this test scenario:

  • It's extremely common for devs to put their own loops into test cases, and they forget that Benchmark.js already does all the repetition you need. There's a really strong chance that the for澳门新萄京官方网站OLTP基准测验,Tookit工具包之pt。 loops in these cases are totally unnecessary noise.

  • The declaring and initializing of x is included in each test case, possibly unnecessarily. Recall from earlier that if x = [] were in the setup code, it wouldn't actually be run before each test iteration, but instead once at the beginning of each cycle. That means x would continue growing quite large, not just the size 10 implied by the for loops.

    So is the intent to make sure the tests are constrained only to how the JS engine behaves with very small arrays (size 10澳门新萄京官方网站,)? That could be the intent, but if it is, you have to consider if that's not focusing far too much on nuanced internal implementation details.

    On the other hand, does the intent of the test embrace the context that the arrays will actually be growing quite large? Is the JS engines' behavior with larger arrays relevant and accurate when compared with the intended real world usage?

  • Is the intent to find out how much x.length or x.push(..) add to the performance of the operation to append to the x array? OK, that might be a valid thing to test. But then again, push(..) is a function call, so of course it's going to be slower than [..] access. Arguably, cases 1 and 2 are fairer than case 3.

Here's another example that illustrates a common apples-to-oranges flaw:

// Case 1
var x = ["John","Albert","Sue","Frank","Bob"];
x.sort();

// Case 2
var x = ["John","Albert","Sue","Frank","Bob"];
x.sort( function mySort(a,b){
    if (a < b) return -1;
    if (a > b) return 1;
    return 0;
} );

Here, the obvious intent is to find out how much slower the custom mySort(..) comparator is than the built-in default comparator. But by specifying the function mySort(..) as inline function expression, you've created an unfair/bogus test. Here, the second case is not only testing a custom user JS function, but it's also testing creating a new function expression for each iteration.

Would it surprise you to find out that if you run a similar test but update it to isolate only for creating an inline function expression versus using a pre-declared function, the inline function expression creation can be from 2% to 20% slower!?

Unless your intent with this test is to consider the inline function expression creation "cost," a better/fairer test would put mySort(..)'s declaration in the page setup -- don't put it in the test setup as that's unnecessary redeclaration for each cycle -- and simply reference it by name in the test case: x.sort(mySort).

Building on the previous example, another pitfall is in opaquely avoiding or adding "extra work" to one test case that creates an apples-to-oranges scenario:

// Case 1
var x = [12,-14,0,3,18,0,2.9];
x.sort();

// Case 2
var x = [12,-14,0,3,18,0,2.9];
x.sort( function mySort(a,b){
    return a - b;
} );

Setting aside the previously mentioned inline function expression pitfall, the second case's mySort(..) works in this case because you have provided it numbers, but would have of course failed with strings. The first case doesn't throw an error, but it actually behaves differently and has a different outcome! It should be obvious, but: a different outcome between two test cases almost certainly invalidates the entire test!

But beyond the different outcomes, in this case, the built in sort(..)'s comparator is actually doing "extra work" that mySort() does not, in that the built-in one coerces the compared values to strings and does lexicographic comparison. The first snippet results in [-14, 0, 0, 12, 18, 2.9, 3] while the second snippet results (likely more accurately based on intent) in [-14, 0, 0, 2.9, 3, 12, 18].

So that test is unfair because it's not actually doing the same task between the cases. Any results you get are bogus.

These same pitfalls can even be much more subtle:

// Case 1
var x = false;
var y = x ? 1 : 2;

// Case 2
var x;
var y = x ? 1 : 2;

Here, the intent might be to test the performance impact of the coercion to a Boolean that the ? : operator will do if the x expression is not already a Boolean (see the Types & Grammar title of this book series). So, you're apparently OK with the fact that there is extra work to do the coercion in the second case.

The subtle problem? You're setting x's value in the first case and not setting it in the other, so you're actually doing work in the first case that you're not doing in the second. To eliminate any potential (albeit minor) skew, try:

// Case 1
var x = false;
var y = x ? 1 : 2;

// Case 2
var x = undefined;
var y = x ? 1 : 2;

Now there's an assignment in both cases, so the thing you want to test -- the coercion of x or not -- has likely been more accurately isolated and tested.

[译:在黑盒测验进程中是运营客商负载工具来测验程序的反射。 此类工具从轻量级,命令行驱动工具如httperf, openload, siege, Apache Flood 到更重量级工具如OpenSTA, The Grinder, JMeter. 此类测量试验并不关切程序的内在运营, 也不监视运维在服务器(群)上的硬件/操作系统的财富。假设听上去是此类的您正在拓宽的办事,则自个儿称它为负载测量试验。]

Procedure

Procedure

Writing Good Tests

Let me see if I can articulate the bigger point I'm trying to make here.

Good test authoring requires careful analytical thinking about what differences exist between two test cases and whether the differences between them are intentional or unintentional.

Intentional differences are of course normal and OK, but it's too easy to create unintentional differences that skew your results. You have to be really, really careful to avoid that skew. Moreover, you may intend a difference but it may not be obvious to other readers of your test what your intent was, so they may doubt (or trust!) your test incorrectly. How do you fix that?

Write better, clearer tests. But also, take the time to document (using the jsPerf.com "Description" field and/or code comments) exactly what the intent of your test is, even to the nuanced detail. Call out the intentional differences, which will help others and your future self to better identify unintentional differences that could be skewing the test results.

Isolate things which aren't relevant to your test by pre-declaring them in the page or test setup settings so they're outside the timed parts of the test.

Instead of trying to narrow in on a tiny snippet of your real code and benchmarking just that piece out of context, tests and benchmarks are better when they include a larger (while still relevant) context. Those tests also tend to run slower, which means any differences you spot are more relevant in context.

I think part of the confusion is that no matter how you look at these two types of testing, they have one common element: the load testing part. Even when you're profiling the application and monitoring the servers (hence doing performance testing), you still need to run load tools against the application, so from that perspective you're doing load testing.

 

 

Microperformance

OK, until now we've been dancing around various microperformance issues and generally looking disfavorably upon obsessing about them. I want to take just a moment to address them directly.

The first thing you need to get more comfortable with when thinking about performance benchmarking your code is that the code you write is not always the code the engine actually runs. We briefly looked at that topic back in Chapter 1 when we discussed statement reordering by the compiler, but here we're going to suggest the compiler can sometimes decide to run different code than you wrote, not just in different orders but different in substance.

Let's consider this piece of code:

var foo = 41;

(function(){
    (function(){
        (function(baz){
            var bar = foo   baz;
            // ..
        })(1);
    })();
})();

You may think about the foo reference in the innermost function as needing to do a three-level scope lookup. We covered in the Scope & Closures title of this book series how lexical scope works, and the fact that the compiler generally caches such lookups so that referencing foo from different scopes doesn't really practically "cost" anything extra.

But there's something deeper to consider. What if the compiler realizes that foo isn't referenced anywhere else but that one location, and it further notices that the value never is anything except the 41 as shown?

Isn't it quite possible and acceptable that the JS compiler could decide to just remove the foo variable entirely, and inline the value, such as this:

(function(){
    (function(){
        (function(baz){
            var bar = 41   baz;
            // ..
        })(1);
    })();
})();

Note: Of course, the compiler could probably also do a similar analysis and rewrite with the baz variable here, too.

When you begin to think about your JS code as being a hint or suggestion to the engine of what to do, rather than a literal requirement, you realize that a lot of the obsession over discrete syntactic minutia is most likely unfounded.

Another example:

function factorial(n) {
    if (n < 2) return 1;
    return n * factorial( n - 1 );
}

factorial( 5 );     // 120

Ah, the good ol' fashioned "factorial" algorithm! You might assume that the JS engine will run that code mostly as is. And to be honest, it might -- I'm not really sure.

But as an anecdote, the same code expressed in C and compiled with advanced optimizations would result in the compiler realizing that the call factorial(5) can just be replaced with the constant value 120, eliminating the function and call entirely!

Moreover, some engines have a practice called "unrolling recursion," where it can realize that the recursion you've expressed can actually be done "easier" (i.e., more optimally) with a loop. It's possible the preceding code could be rewritten by a JS engine to run as:

function factorial(n) {
    if (n < 2) return 1;

    var res = 1;
    for (var i=n; i>1; i--) {
        res *= i;
    }
    return res;
}

factorial( 5 );     // 120

Now, let's imagine that in the earlier snippet you had been worried about whether n * factorial(n-1) or n *= factorial(--n) runs faster. Maybe you even did a performance benchmark to try to figure out which was better. But you miss the fact that in the bigger context, the engine may not run either line of code because it may unroll the recursion!

Speaking of --, --n versus n-- is often cited as one of those places where you can optimize by choosing the --n version, because theoretically it requires less effort down at the assembly level of processing.

That sort of obsession is basically nonsense in modern JavaScript. That's the kind of thing you should be letting the engine take care of. You should write the code that makes the most sense. Compare these three for loops:

// Option 1
for (var i=0; i<10; i  ) {
    console.log( i );
}

// Option 2
for (var i=0; i<10;   i) {
    console.log( i );
}

// Option 3
for (var i=-1;   i<10; ) {
    console.log( i );
}

Even if you have some theory where the second or third option is more performant than the first option by a tiny bit, which is dubious at best, the third loop is more confusing because you have to start with -1 for i to account for the fact that i pre-increment is used. And the difference between the first and second options is really quite irrelevant.

It's entirely possible that a JS engine may see a place where i is used and realize that it can safely replace it with the i equivalent, which means your time spent deciding which one to pick was completely wasted and the outcome moot.

Here's another common example of silly microperformance obsession:

var x = [ .. ];

// Option 1
for (var i=0; i < x.length; i  ) {
    // ..
}

// Option 2
for (var i=0, len = x.length; i < len; i  ) {
    // ..
}

澳门新萄京官方网站OLTP基准测验,Tookit工具包之pt。The theory here goes that you should cache the length of the x array in the variable len, because ostensibly it doesn't change, to avoid paying the price of x.length being consulted for each iteration of the loop.

If you run performance benchmarks around x.length usage compared to caching it in a len variable, you'll find that while the theory sounds nice, in practice any measured differences are statistically completely irrelevant.

In fact, in some engines like v8, it can be shown (http://mrale.ph/blog/2014/12/24/array-length-caching.html) that you could make things slightly worse by pre-caching the length instead of letting the engine figure it out for you. Don't try to outsmart your JavaScript engine, you'll probably lose when it comes to performance optimizations.

[译:小编想有部分的吸引是不管你什么样对待这两类测量试验,它们有一对通用的因素:负载测量检验部分。乃至当您正模压程序并监视服务器(由此开展品质测量试验),你照旧要求去运行负载工具测量检验运维程序,所以从那一个观念上您正在展开负荷测验。]

Official web site:

Usage

Not All Engines Are Alike

The different JS engines in various browsers can all be "spec compliant" while having radically different ways of handling code. The JS specification doesn't require anything performance related -- well, except ES6's "Tail Call Optimization" covered later in this chapter.

The engines are free to decide that one operation will receive its attention to optimize, perhaps trading off for lesser performance on another operation. It can be very tenuous to find an approach for an operation that always runs faster in all browsers.

There's a movement among some in the JS dev community, especially those who work with Node.js, to analyze the specific internal implementation details of the v8 JavaScript engine and make decisions about writing JS code that is tailored to take best advantage of how v8 works. You can actually achieve a surprisingly high degree of performance optimization with such endeavors, so the payoff for the effort can be quite high.

Some commonly cited examples (https://github.com/petkaantonov/bluebird/wiki/Optimization-killers) for v8:

  • Don't pass the arguments variable from one function to any other function, as such "leakage" slows down the function implementation.
  • Isolate a try..catch in its own function. Browsers struggle with optimizing any function with a try..catch in it, so moving that construct to its own function means you contain the de-optimization harm while letting the surrounding code be optimizable.

But rather than focus on those tips specifically, let's sanity check the v8-only optimization approach in a general sense.

Are you genuinely writing code that only needs to run in one JS engine? Even if your code is entirely intended for Node.js right now, is the assumption that v8 will always be the used JS engine reliable? Is it possible that someday a few years from now, there's another server-side JS platform besides Node.js that you choose to run your code on? What if what you optimized for before is now a much slower way of doing that operation on the new engine?

Or what if your code always stays running on v8 from here on out, but v8 decides at some point to change the way some set of operations works such that what used to be fast is now slow, and vice versa?

These scenarios aren't just theoretical, either. It used to be that it was faster to put multiple string values into an array and then call join("") on the array to concatenate the values than to just use concatenation directly with the values. The historical reason for this is nuanced, but it has to do with internal implementation details about how string values were stored and managed in memory.

As a result, "best practice" advice at the time disseminated across the industry suggesting developers always use the array join(..) approach. And many followed.

Except, somewhere along the way, the JS engines changed approaches for internally managing strings, and specifically put in optimizations for concatenation. They didn't slow down join(..) per se, but they put more effort into helping usage, as it was still quite a bit more widespread.

Note: The practice of standardizing or optimizing some particular approach based mostly on its existing widespread usage is often called (metaphorically) "paving the cowpath."

Once that new approach to handling strings and concatenation took hold, unfortunately all the code out in the wild that was using array join(..) to concatenate strings was then sub-optimal.

Another example: at one time, the Opera browser differed from other browsers in how it handled the boxing/unboxing of primitive wrapper objects (see the Types & Grammar title of this book series). As such, their advice to developers was to use a String object instead of the primitive string value if properties like length or methods like charAt(..) needed to be accessed. This advice may have been correct for Opera at the time, but it was literally completely opposite for other major contemporary browsers, as they had optimizations specifically for the string primitives and not their object wrapper counterparts.

I think these various gotchas are at least possible, if not likely, for code even today. So I'm very cautious about making wide ranging performance optimizations in my JS code based purely on engine implementation details, especially if those details are only true of a single engine.

The reverse is also something to be wary of: you shouldn't necessarily change a piece of code to work around one engine's difficulty with running a piece of code in an acceptably performant way.

Historically, IE has been the brunt of many such frustrations, given that there have been plenty of scenarios in older IE versions where it struggled with some performance aspect that other major browsers of the time seemed not to have much trouble with. The string concatenation discussion we just had was actually a real concern back in the IE6 and IE7 days, where it was possible to get better performance out of join(..) than .

But it's troublesome to suggest that just one browser's trouble with performance is justification for using a code approach that quite possibly could be sub-optimal in all other browsers. Even if the browser in question has a large market share for your site's audience, it may be more practical to write the proper code and rely on the browser to update itself with better optimizations eventually.

"There is nothing more permanent than a temporary hack." Chances are, the code you write now to work around some performance bug will probably outlive the performance bug in the browser itself.

In the days when a browser only updated once every five years, that was a tougher call to make. But as it stands now, browsers across the board are updating at a much more rapid interval (though obviously the mobile world still lags), and they're all competing to optimize web features better and better.

If you run across a case where a browser does have a performance wart that others don't suffer from, make sure to report it to them through whatever means you have available. Most browsers have open public bug trackers suitable for this purpose.

Tip: I'd only suggest working around a performance issue in a browser if it was a really drastic show-stopper, not just an annoyance or frustration. And I'd be very careful to check that the performance hack didn't have noticeable negative side effects in another browser.

As far as I'm concerned, these definitions don't have much value in and of themselves. What matters most is to have a well-established procedure for tuning the application and the servers so that you can meet your users' or your business customers' requirements. This procedure will use elements of all the types of testing mentioned here and in my previous entry: load, performance and stress testing.

https://github.com/akopytov/sysbench

1 pt-stalk [OPTIONS]

Big Picture

Instead of worrying about all these microperformance nuances, we should instead be looking at big-picture types of optimizations.

How do you know what's big picture or not? You have to first understand if your code is running on a critical path or not. If it's not on the critical path, chances are your optimizations are not worth much.

Ever heard the admonition, "that's premature optimization!"? It comes from a famous quote from Donald Knuth: "premature optimization is the root of all evil.". Many developers cite this quote to suggest that most optimizations are "premature" and are thus a waste of effort. The truth is, as usual, more nuanced.

Here is Knuth's quote, in context:

Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. [emphasis added]

(http://web.archive.org/web/20130731202547/http://pplab.snu.ac.kr/courses/adv_pl05/papers/p261-knuth.pdf, Computing Surveys, Vol 6, No 4, December 1974)

I believe it's a fair paraphrasing to say that Knuth meant: "non-critical path optimization is the root of all evil." So the key is to figure out if your code is on the critical path -- you should optimize it! -- or not.

I'd even go so far as to say this: no amount of time spent optimizing critical paths is wasted, no matter how little is saved; but no amount of optimization on noncritical paths is justified, no matter how much is saved.

If your code is on the critical path, such as a "hot" piece of code that's going to be run over and over again, or in UX critical places where users will notice, like an animation loop or CSS style updates, then you should spare no effort in trying to employ relevant, measurably significant optimizations.

For example, consider a critical path animation loop that needs to coerce a string value to a number. There are of course multiple ways to do that (see the Types & Grammar title of this book series), but which one if any is the fastest?

var x = "42";   // need number `42`

// Option 1: let implicit coercion automatically happen
var y = x / 2;

// Option 2: use `parseInt(..)`
var y = parseInt( x, 0 ) / 2;

// Option 3: use `Number(..)`
var y = Number( x ) / 2;

// Option 4: use ` ` unary operator
var y =  x / 2;

// Option 5: use `|` unary operator
var y = (x | 0) / 2;

Note: I will leave it as an exercise to the reader to set up a test if you're interested in examining the minute differences in performance among these options.

When considering these different options, as they say, "One of these things is not like the others." parseInt(..) does the job, but it also does a lot more -- it parses the string rather than just coercing. You can probably guess, correctly, that parseInt(..) is a slower option, and you should probably avoid it.

Of course, if x can ever be a value that needs parsing, such as "42px" (like from a CSS style lookup), then parseInt(..) really is the only suitable option!

Number(..) is also a function call. From a behavioral perspective, it's identical to the unary operator option, but it may in fact be a little slower, requiring more machinery to execute the function. Of course, it's also possible that the JS engine recognizes this behavioral symmetry and just handles the inlining of Number(..)'s behavior (aka x) for you!

But remember, obsessing about x versus x | 0 is in most cases likely a waste of effort. This is a microperformance issue, and one that you shouldn't let dictate/degrade the readability of your program.

While performance is very important in critical paths of your program, it's not the only factor. Among several options that are roughly similar in performance, readability should be another important concern.

[译:到近日自身关切的,这一个概念本人不享有价值.难点的根本是让二个杰出构建的经过用于调节和测验应用程序和服务器以使自个儿能够满意用你客商仍然商业顾客的须要.这一个进度将会接纳具备在那涉及过的测量检验项目和自身上一篇小说:负载,品质和压力测量试验]

 

 

Tail Call Optimization (TCO)

As we briefly mentioned earlier, ES6 includes a specific requirement that ventures into the world of performance. It's related to a specific form of optimization that can occur with function calls: tail call optimization.

Briefly, a "tail call" is a function call that appears at the "tail" of another function, such that after the call finishes, there's nothing left to do (except perhaps return its result value).

For example, here's a non-recursive setup with tail calls:

function foo(x) {
    return x;
}

function bar(y) {
    return foo( y   1 );    // tail call
}

function baz() {
    return 1   bar( 40 );   // not tail call
}

baz();                      // 42

foo(y 1) is a tail call in bar(..) because after foo(..) finishes, bar(..) is also finished except in this case returning the result of the foo(..) call. However, bar(40) is not a tail call because after it completes, its result value must be added to 1 before baz() can return it.

Without getting into too much nitty-gritty detail, calling a new function requires an extra amount of reserved memory to manage the call stack, called a "stack frame." So the preceding snippet would generally require a stack frame for each of baz(), bar(..), and foo(..) all at the same time.

However, if a TCO-capable engine can realize that the foo(y 1) call is in tail position meaning bar(..) is basically complete, then when calling foo(..), it doesn't need to create a new stack frame, but can instead reuse the existing stack frame from bar(..). That's not only faster, but it also uses less memory.

That sort of optimization isn't a big deal in a simple snippet, but it becomes a much bigger deal when dealing with recursion, especially if the recursion could have resulted in hundreds or thousands of stack frames. With TCO the engine can perform all those calls with a single stack frame!

Recursion is a hairy topic in JS because without TCO, engines have had to implement arbitrary (and different!) limits to how deep they will let the recursion stack get before they stop it, to prevent running out of memory. With TCO, recursive functions with tail position calls can essentially run unbounded, because there's never any extra usage of memory!

Consider that recursive factorial(..) from before, but rewritten to make it TCO friendly:

function factorial(n) {
    function fact(n,res) {
        if (n < 2) return res;

        return fact( n - 1, n * res );
    }

    return fact( n, 1 );
}

factorial( 5 );     // 120

This version of factorial(..) is still recursive, but it's also optimizable with TCO, because both inner fact(..) calls are in tail position.

Note: It's important to note that TCO only applies if there's actually a tail call. If you write recursive functions without tail calls, the performance will still fall back to normal stack frame allocation, and the engines' limits on such recursive call stacks will still apply. Many recursive functions can be rewritten as we just showed with factorial(..), but it takes careful attention to detail.

One reason that ES6 requires engines to implement TCO rather than leaving it up to their discretion is because the lack of TCO actually tends to reduce the chances that certain algorithms will be implemented in JS using recursion, for fear of the call stack limits.

If the lack of TCO in the engine would just gracefully degrade to slower performance in all cases, it wouldn't probably have been something that ES6 needed to require. But because the lack of TCO can actually make certain programs impractical, it's more an important feature of the language than just a hidden implementation detail.

ES6 guarantees that from now on, JS developers will be able to rely on this optimization across all ES6 compliant browsers. That's a win for JS performance!

Here's one example of such a procedure. Let's say you're developing a Web application with a database back-end that needs to support 100 concurrent users, with a response time of less than 3 seconds. How would you go about testing your application in order to make sure these requirements are met?

Git clone address:

**Common Parameters**

Review

Effectively benchmarking performance of a piece of code, especially to compare it to another option for that same code to see which approach is faster, requires careful attention to detail.

Rather than rolling your own statistically valid benchmarking logic, just use the Benchmark.js library, which does that for you. But be careful about how you author tests, because it's far too easy to construct a test that seems valid but that's actually flawed -- even tiny differences can skew the results to be completely unreliable.

It's important to get as many test results from as many different environments as possible to eliminate hardware/device bias. jsPerf.com is a fantastic website for crowdsourcing performance benchmark test runs.

Many common performance tests unfortunately obsess about irrelevant microperformance details like x versus x. Writing good tests means understanding how to focus on big picture concerns, like optimizing on the critical path, and avoiding falling into traps like different JS engines' implementation details.

Tail call optimization (TCO) is a required optimization as of ES6 that will make some recursive patterns practical in JS where they would have been impossible otherwise. TCO allows a function call in the tail position of another function to execute without needing any extra resources, which means the engine no longer needs to place arbitrary restrictions on call stack depth for recursive algorithms.

[译:举一个此类程序的事例。 如果你正在开采三个web应用程序和一台终端服务器且需求援救100名并发客户,且响应时间低于3妙。 你什么样举行测量试验使它达符合这么些须求呢?]

https://github.com/akopytov/sysbench.git

 1 --collect //Collect diagostic data based on spedified triggers.
 2 --collect-gdb //Collect gdb data.It prints stack traces from all threads.
 3 --collect-oprofile //Collect oprofile data.
 4 --collect-strace //Collect strace data.Do not specify it when "--collect-gdb" has been used.
 5 --collect-tcpdump //Collect tcpdump data.
 6 --cycles //Limite the times when triggering condition occurs(default "5").
 7 --dest //Specify the position to store diagnostic data(default "/var/lib/pt-stalk").
 8 --disk-bytes-free //Prevent the lack of least disk space.It does not collect data until it has the necessary disk space(default "100M",valid surffixes of unit is k,M,G and T).
 9 --disk-pct-free //It's similar with "--disk-bytes-free" but specify the percentage of disk space.
10 --function //Specify the contents to watch for the triggers(default "status",other value is "processlist" or <yourfilename>).
11 --iterations //Limit the times to collect.It will run forever if not give a specific value.
12 --log //Specify the postion to record logs(default "/var/log/pt-stalk.log").It only generates when deamonized.
13 --match //The pattern to specify when using "--function processlist".
14 --mysql-only //Merely collect the MySQL relevant diagnostic data but disk space is the exception.
15 --retention-time //Specify the purge days of diagnostic data(default "30").
16 --run-time //Specify how many seconds to collect diagnostic data(default "30").It should not be longer than the value of "--sleep".
17 --sleep //Specify how many seconds to sleep after last collection.It used to prevent too many operations of collection(default "300").
18 --stalk //Watch and wait for trigger occur(default "yes").
19 --no_stalk //The option can be specify if you want to collect diagnostic data immediately without waiting for trigger to occur.
20 --threshold //Specify the maximum value for collecting(default "25").
21 --variable //Specify the variables to compared against(default "Threads_running").
  1. Start with 1 Web/Application server connected to 1 Database server. If you can, put both servers behind a firewall, and if you're thinking about doing load balancing down the road, put the Web server behind the load balancer. This way you'll have one each of different devices that you'll use in a real production environment.

 

 

[译:1 开始运用web恐怕程序服务器连接到一台数据库服务器。假如您能够的话,将两台服务置于防火墙后,固然你挂念那一个格局做负载均衡将WEB服务器置于负载器前边。这种办法将您将会使您的每一个不等的器具都在二个安分守己的成品景况中。]

Download it from github.

Examples

  1. Run a load test against the Web server, starting with 10 concurrent users, each user sending a total of 1000 requests to the server. Step up the number of users in increments of 10, until you reach 100 users.
1 [root@zlm1 12:12:16 ~]
2 #cd /vagrant
3 
4 [root@zlm1 12:12:20 ~]
5 #wget https://github.com/akopytov/sysbench/archive/1.0.zip -O "sysbench-1.0.zip"

 

[译:2 对一台WEB服务器运维贰个载荷测量试验,以11个冒出客户开头,种种顾客向服务器发出一千个央求,设置客商增量为10,直到达到99个顾客甘休.]

 

Generate benchmark with sysbench.

  1. While you're blasting the Web server, profile your application and your database to see if there are any hot spots[是或不是译成拐点比较好] in your code/SQL queries/stored procedures that you need to optimize. I realize I'm glossing over important details here, but this step is obviously highly dependent on your particular application.

**Install sysbench.**

 1 [root@zlm2 08:07:54 ~/sysbench-1.0/src/lua]
 2 #sysbench oltp_read_write.lua --mysql-host=192.168.1.101 --mysql-port=3306 --mysql-user=zlm --mysql-password=aaron8219 --mysql-db=sysbench --tables=10 --table-size=100000 --mysql-storage-engine=innodb cleanup
 3 sysbench 1.0.15 (using bundled LuaJIT 2.1.0-beta2)
 4 
 5 Dropping table 'sbtest1'...
 6 Dropping table 'sbtest2'...
 7 Dropping table 'sbtest3'...
 8 Dropping table 'sbtest4'...
 9 Dropping table 'sbtest5'...
10 Dropping table 'sbtest6'...
11 Dropping table 'sbtest7'...
12 Dropping table 'sbtest8'...
13 Dropping table 'sbtest9'...
14 Dropping table 'sbtest10'...
15 
16 [root@zlm2 08:08:08 ~/sysbench-1.0/src/lua]
17 #sysbench oltp_read_write.lua --mysql-host=192.168.1.101 --mysql-port=3306 --mysql-user=zlm --mysql-password=aaron8219 --mysql-db=sysbench --tables=10 --table-size=100000 --mysql-storage-engine=innodb prepare
18 sysbench 1.0.15 (using bundled LuaJIT 2.1.0-beta2)
19 
20 Creating table 'sbtest1'...
21 Inserting 100000 records into 'sbtest1'
22 Creating a secondary index on 'sbtest1'...
23 Creating table 'sbtest2'...
24 Inserting 100000 records into 'sbtest2'
25 Creating a secondary index on 'sbtest2'...
26 Creating table 'sbtest3'...
27 Inserting 100000 records into 'sbtest3'
28 Creating a secondary index on 'sbtest3'...
29 Creating table 'sbtest4'...
30 Inserting 100000 records into 'sbtest4'
31 Creating a secondary index on 'sbtest4'...
32 Creating table 'sbtest5'...
33 Inserting 100000 records into 'sbtest5'
34 Creating a secondary index on 'sbtest5'...
35 Creating table 'sbtest6'...
36 Inserting 100000 records into 'sbtest6'
37 Creating a secondary index on 'sbtest6'...
38 Creating table 'sbtest7'...
39 Inserting 100000 records into 'sbtest7'
40 Creating a secondary index on 'sbtest7'...
41 Creating table 'sbtest8'...
42 Inserting 100000 records into 'sbtest8'
43 Creating a secondary index on 'sbtest8'...
44 Creating table 'sbtest9'...
45 Inserting 100000 records into 'sbtest9'
46 Creating a secondary index on 'sbtest9'...
47 Creating table 'sbtest10'...
48 Inserting 100000 records into 'sbtest10'
49 Creating a secondary index on 'sbtest10'...
50 
51 [root@zlm2 08:09:23 ~/sysbench-1.0/src/lua]
52 #sysbench oltp_read_write.lua --mysql-host=192.168.1.101 --mysql-port=3306 --mysql-user=zlm --mysql-password=aaron8219 --mysql-db=sysbench --threads=3 --time=3600 --report-interval=10 --rand-type=uniform run
53 sysbench 1.0.15 (using bundled LuaJIT 2.1.0-beta2)
54 
55 Running the test with following options:
56 Number of threads: 3
57 Report intermediate results every 10 second(s)
58 Initializing random number generator from current time
59 
60 
61 Initializing worker threads...
62 
63 Threads started!
64 
65 [ 10s ] thds: 3 tps: 258.68 qps: 5176.49 (r/w/o: 3624.11/1034.82/517.56) lat (ms,95%): 15.83 err/s: 0.00 reconn/s: 0.00
66 [ 20s ] thds: 3 tps: 286.25 qps: 5726.15 (r/w/o: 4008.67/1144.89/572.60) lat (ms,95%): 15.00 err/s: 0.00 reconn/s: 0.00
67 [ 30s ] thds: 3 tps: 270.82 qps: 5416.13 (r/w/o: 3790.80/1083.69/541.64) lat (ms,95%): 16.12 err/s: 0.00 reconn/s: 0.00
68 [ 40s ] thds: 3 tps: 280.98 qps: 5619.75 (r/w/o: 3934.26/1123.53/561.97) lat (ms,95%): 16.12 err/s: 0.00 reconn/s: 0.00
69 [ 50s ] thds: 3 tps: 298.42 qps: 5968.38 (r/w/o: 4177.83/1193.70/596.85) lat (ms,95%): 14.46 err/s: 0.00 reconn/s: 0.00
70 [ 60s ] thds: 3 tps: 278.88 qps: 5578.20 (r/w/o: 3904.92/1115.52/557.76) lat (ms,95%): 15.83 err/s: 0.00 reconn/s: 0.00
71 [ 70s ] thds: 3 tps: 280.91 qps: 5617.66 (r/w/o: 3932.21/1123.63/561.82) lat (ms,95%): 15.83 err/s: 0.00 reconn/s: 0.00
72 [ 80s ] thds: 3 tps: 281.68 qps: 5632.83 (r/w/o: 3942.77/1126.71/563.35) lat (ms,95%): 16.12 err/s: 0.00 reconn/s: 0.00
73 [ 90s ] thds: 3 tps: 281.60 qps: 5631.55 (r/w/o: 3942.07/1126.39/563.10) lat (ms,95%): 16.12 err/s: 0.00 reconn/s: 0.00
74 [ 100s ] thds: 3 tps: 287.62 qps: 5753.86 (r/w/o: 4028.02/1150.49/575.35) lat (ms,95%): 15.55 err/s: 0.00 reconn/s: 0.00
75 [ 110s ] thds: 3 tps: 308.99 qps: 6180.45 (r/w/o: 4326.12/1236.35/617.97) lat (ms,95%): 13.95 err/s: 0.00 reconn/s: 0.00
76 ... //Omitted.

[译:3 当您正在冲击WEB 服务器时,监视描绘你的应用程序和您的数据库,观望在您的代码或存款和储蓄进度是不是有拐点供给你优化. 小编明白笔者在这遮盖了过多细节,但此一步十分的大程序程度上有赖于特定的采纳程序. ]

 1 [root@zlm1 12:15:26 /vagrant]
 2 #cp -R sysbench-1.0 ~
 3 
 4 [root@zlm1 12:16:02 ~]
 5 #cd sysbench-1.0/
 6 
 7 [root@zlm1 12:16:07 ~/sysbench-1.0]
 8 #./autogen.sh 
 9 ./autogen.sh: running `libtoolize --copy --force' 
10 libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `config'.
11 libtoolize: copying file `config/ltmain.sh'
12 libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
13 libtoolize: copying file `m4/libtool.m4'
14 libtoolize: copying file `m4/ltoptions.m4'
15 libtoolize: copying file `m4/ltsugar.m4'
16 libtoolize: copying file `m4/ltversion.m4'
17 libtoolize: copying file `m4/lt~obsolete.m4'
18 ./autogen.sh: running `aclocal -I m4' 
19 ./autogen.sh: running `autoheader' 
20 ./autogen.sh: running `automake -c --foreign --add-missing' 
21 configure.ac:59: installing 'config/ar-lib'
22 configure.ac:45: installing 'config/compile'
23 configure.ac:27: installing 'config/config.guess'
24 configure.ac:27: installing 'config/config.sub'
25 configure.ac:32: installing 'config/install-sh'
26 configure.ac:32: installing 'config/missing'
27 src/Makefile.am: installing 'config/depcomp'
28 parallel-tests: installing 'config/test-driver'
29 ./autogen.sh: running `autoconf' 
30 Libtoolized with: libtoolize (GNU libtool) 2.4.2
31 Automade with: automake (GNU automake) 1.13.4
32 Configured with: autoconf (GNU Autoconf) 2.69
33 
34 [root@zlm1 12:16:18 ~/sysbench-1.0]
35 #./configure && make && make install
36 //Omitted.
37 [root@zlm1 12:18:40 ~/sysbench-1.0]
38 #sysbench --version
39 sysbench: error while loading shared libraries: libmysqlclient.so.20: cannot open shared object file: No such file or directory
40 
41 [root@zlm1 12:19:14 ~/sysbench-1.0]
42 #whereis sysbench
43 sysbench: /usr/local/bin/sysbench
44 
45 [root@zlm1 12:19:27 ~/sysbench-1.0]
46 #ldd /usr/local/bin/sysbench
47     linux-vdso.so.1 =>  (0x00007fff2abfe000)
48     libmysqlclient.so.20 => not found //Only lack this lib.
49     libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f86e2986000)
50     librt.so.1 => /lib64/librt.so.1 (0x00007f86e277e000)
51     libdl.so.2 => /lib64/libdl.so.2 (0x00007f86e257a000)
52     libaio.so.1 => /lib64/libaio.so.1 (0x00007f86e2377000)
53     libm.so.6 => /lib64/libm.so.6 (0x00007f86e2075000)
54     libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f86e1e5f000)
55     libc.so.6 => /lib64/libc.so.6 (0x00007f86e1a9d000)
56     /lib64/ld-linux-x86-64.so.2 (0x00007f86e2bab000)
57 
58 [root@zlm1 12:19:41 ~/sysbench-1.0]
59 #find / -name libmysqlclient //There's no result found.
60 
61 [root@zlm1 12:21:00 ~/sysbench-1.0]
62 #

 

  1. Also monitor both servers (Web/App and Database) via command line utilities mentioned before (top, vmstat, iostat, netstat, Windows PerfMon). These utilities will let you know what's going on with the servers in terms of hardware resources. Also monitor the firewall and the load balancer (many times you can do this via SNMP) -- but these devices are not likely to be a bottleneck at this level, since they usualy can deal with thousands of connections before they hit a limit, assuming they're hardware-based and not software-based.

 

**Collect diagnostic data using pt-stalk.**

[译:4 一致通过前边提过的命令行工具来监视应用程序和数据. 那些工具会使您打探就硬件能源上的局地情形. 还要监视防火墙和负载器(大多情况下你能够行使SNMP)--但那一个设施在这么些范围上不会成为一个瓶颈,因为它们平日能够处理数千个一而再在完毕限制前,要是它们是硬件加强并非软件加强的.(失常)]

**Solve the lack of mysqlclient.so.20 file.**

 1 [root@zlm2 08:14:36 /data/mysql/mysql3306]
 2 #pt-stalk --host localhost --port 3306 --user root --password Passw0rd --collect-gdb --cycles 1 --variable Threads_connect --threshold 2
 3 mysql: [Warning] Using a password on the command line interface can be insecure.
 4 2018_07_09_08_15_46 Starting /usr/bin/pt-stalk --function=status --variable=Threads_connect --threshold=2 --match= --cycles=1 --interval=1 --iterations= --run-time=30 --sleep=300 --dest=/var/lib/pt-stalk --prefix= --notify-by-email= --log=/var/log/pt-stalk.log --pid=/var/run/pt-stalk.pid --plugin=
 5 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
 6 2018_07_09_08_15_46 Detected value is empty; something failed?  Trigger exit status: 0
 7 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
 8 2018_07_09_08_15_47 Detected value is empty; something failed?  Trigger exit status: 0
 9 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
10 2018_07_09_08_15_48 Detected value is empty; something failed?  Trigger exit status: 0
11 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
12 2018_07_09_08_15_50 Detected value is empty; something failed?  Trigger exit status: 0
13 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
14 2018_07_09_08_15_51 Detected value is empty; something failed?  Trigger exit status: 0
15 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
16 2018_07_09_08_15_52 Detected value is empty; something failed?  Trigger exit status: 0
17 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
18 2018_07_09_08_15_53 Detected value is empty; something failed?  Trigger exit status: 0
19 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
20 2018_07_09_08_15_54 Detected value is empty; something failed?  Trigger exit status: 0
21 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
22 2018_07_09_08_15_55 Detected value is empty; something failed?  Trigger exit status: 0
23 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
24 2018_07_09_08_15_56 Detected value is empty; something failed?  Trigger exit status: 0
25 ^C2018_07_09_08_15_57 Caught signal, exiting
26 2018_07_09_08_15_57 All subprocesses have finished
27 2018_07_09_08_15_57 Exiting because OKTORUN is false
28 2018_07_09_08_15_57 /usr/bin/pt-stalk exit status 1
29 
30 //The value of "--variable" should be "Threads_connected".
31 //Modify the correct value and run it again.
32 
33 [root@zlm2 08:15:57 /data/mysql/mysql3306]
34 #pt-stalk --host localhost --port 3306 --user root --password Passw0rd --collect-gdb --cycles 1 --variable Threads_connected --threshold 2
35 mysql: [Warning] Using a password on the command line interface can be insecure.
36 2018_07_09_08_19_39 Starting /usr/bin/pt-stalk --function=status --variable=Threads_connected --threshold=2 --match= --cycles=1 --interval=1 --iterations= --run-time=30 --sleep=300 --dest=/var/lib/pt-stalk --prefix= --notify-by-email= --log=/var/log/pt-stalk.log --pid=/var/run/pt-stalk.pid --plugin=
37 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
38 2018_07_09_08_19_39 Check results: status(Threads_connected)=5, matched=yes, cycles_true=1
39 2018_07_09_08_19_39 Collect 1 triggered
40 2018_07_09_08_19_39 Collect 1 PID 5921
41 2018_07_09_08_19_39 Collect 1 done
42 2018_07_09_08_19_39 Sleeping 300 seconds after collect
43 mysqladmin: [Warning] Using a password on the command line interface can be insecure.
44 2018_07_09_08_24_39 Check results: status(Threads_connected)=6, matched=yes, cycles_true=1
45 2018_07_09_08_24_39 Collect 2 triggered
46 2018_07_09_08_24_39 Collect 2 PID 7398
47 2018_07_09_08_24_39 Collect 2 done
48 2018_07_09_08_24_40 Sleeping 300 seconds after collect
49 ^C2018_07_09_08_25_56 Caught signal, exiting //Terminate to collect diagnostic data by "Ctrl C"
50 2018_07_09_08_25_56 Waiting up to 90 seconds for subprocesses to finish...
51 2018_07_09_08_25_56 Exiting because OKTORUN is false
52 2018_07_09_08_25_56 /usr/bin/pt-stalk exit status 1

5.This is one of the most important steps in the whole procedure. It's not easy to make sense of the output of these monitoring tools, you need somebody who has a lot of experience in system/network architecture and administration. On Sun/Solaris platforms, there is a tool called the SE Performance Toolkit that tries to alleviate this task via built-in heuristics that kick in when certain thresholds are reached and tell you exactly what resource is being taxed.

 1 [root@zlm1 12:24:27 ~/sysbench-1.0]
 2 #vim /etc/ld.so.conf.d/mysql.conf
 3 
 4 [root@zlm1 12:26:17 ~/sysbench-1.0]
 5 #cat /etc/ld.so.conf.d/mysql.conf
 6 /usr/local/mysql/lib
 7 
 8 [root@zlm1 12:26:22 ~/sysbench-1.0]
 9 #ldconfig
10 
11 [root@zlm1 12:26:41 ~/sysbench-1.0]
12 #ldd /usr/local/bin/sysbench
13     linux-vdso.so.1 =>  (0x00007fff767fe000)
14     libmysqlclient.so.20 => /usr/local/mysql/lib/libmysqlclient.so.20 (0x00007fb2ca8a8000)
15     libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb2ca68b000)
16     librt.so.1 => /lib64/librt.so.1 (0x00007fb2ca483000)
17     libdl.so.2 => /lib64/libdl.so.2 (0x00007fb2ca27f000)
18     libaio.so.1 => /lib64/libaio.so.1 (0x00007fb2ca07d000)
19     libm.so.6 => /lib64/libm.so.6 (0x00007fb2c9d7a000)
20     libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fb2c9b64000)
21     libc.so.6 => /lib64/libc.so.6 (0x00007fb2c97a3000)
22     libstdc  .so.6 => /lib64/libstdc  .so.6 (0x00007fb2c949b000)
23     /lib64/ld-linux-x86-64.so.2 (0x00007fb2caec7000)
24 
25 [root@zlm1 12:26:53 ~/sysbench-1.0]
26 #sysbench --version
27 sysbench 1.0.14

 

[译:5  在全体进度中那是一个最首要的步骤,领会那么些监视工具的出口并不便于,要求某一个人具备很深的系统/网络结议和治温病条辨验. 在SUN/Solais 平台上有一款名称为SE Performace Tookit的工具它能够操作那类职业经过松手探究当有个别运营项已经达到规定的标准了并公告你财富真正的负载.]

 

**Check the value of "show global status ... ".**

  1. Let's say your Web server's reply rate[回应速度] starts to level off around 50 users. Now you have a repeatable condition that you know causes problems. All the profiling and monitoring you've done in step 3, should have already given you a good idea about hot spots in your applicationm about SQL queries that are not optimized properly, about resource status at the hardware/OS level.

**Lua scripts of sysbench.**

 1 (root@localhost mysql3306.sock)[sysbench]>show global status like '%Threads%';
 2  ------------------------ ------- 
 3 | Variable_name          | Value |
 4  ------------------------ ------- 
 5 | Delayed_insert_threads | 0     |
 6 | Slow_launch_threads    | 0     |
 7 | Threads_cached         | 1     |
 8 | Threads_connected      | 5     | //Threads_connected has been up to 5.
 9 | Threads_created        | 5     |
10 | Threads_running        | 1     |
11  ------------------------ ------- 
12 6 rows in set (0.04 sec)
13 
14 
15 (root@localhost mysql3306.sock)[sysbench]>show global status like '%Threads%';
16  ------------------------ ------- 
17 | Variable_name          | Value |
18  ------------------------ ------- 
19 | Delayed_insert_threads | 0     |
20 | Slow_launch_threads    | 0     |
21 | Threads_cached         | 3     |
22 | Threads_connected      | 6     | //Second time,Threads_connected has been up to 6.
23 | Threads_created        | 9     |
24 | Threads_running        | 1     |
25  ------------------------ ------- 
26 6 rows in set (0.01 sec)

[译:6  假诺说你的WEB 服务器的响应率以保险在56人左右开启.以后你有了三个能够复用的唤起难点的条件.全数在第三步进行的剖释和监视,应当已经给你提供了贰个很好的想法关于在您的应用程序和SQL查询的拐点当其未有适当优化及硬件/操作系统等第上的能源状态.]

 1 [root@zlm1 13:24:38 ~/sysbench-1.0/src/lua]
 2 #ls -l
 3 total 124
 4 -rwxr-xr-x 1 root root  1452 Jul  1 12:15 bulk_insert.lua
 5 drwxr-xr-x 2 root root  4096 Jul  1 12:18 internal
 6 -rw-r--r-- 1 root root 25855 Jul  1 12:17 Makefile
 7 -rwxr-xr-x 1 root root  1219 Jul  1 12:15 Makefile.am
 8 -rw-r--r-- 1 root root 25401 Jul  1 12:16 Makefile.in
 9 -rwxr-xr-x 1 root root 14369 Jul  1 12:15 oltp_common.lua
10 -rwxr-xr-x 1 root root  1290 Jul  1 12:15 oltp_delete.lua
11 -rwxr-xr-x 1 root root  2415 Jul  1 12:15 oltp_insert.lua
12 -rwxr-xr-x 1 root root  1265 Jul  1 12:15 oltp_point_select.lua
13 -rwxr-xr-x 1 root root  1649 Jul  1 12:15 oltp_read_only.lua
14 -rwxr-xr-x 1 root root  1824 Jul  1 12:15 oltp_read_write.lua
15 -rwxr-xr-x 1 root root  1118 Jul  1 12:15 oltp_update_index.lua
16 -rwxr-xr-x 1 root root  1127 Jul  1 12:15 oltp_update_non_index.lua
17 -rwxr-xr-x 1 root root  1440 Jul  1 12:15 oltp_write_only.lua
18 -rwxr-xr-x 1 root root  1919 Jul  1 12:15 select_random_points.lua
19 -rwxr-xr-x 1 root root  2118 Jul  1 12:15 select_random_ranges.lua

 

At this point, the developers need to take back the profiling measurements and tune the code and the database queries. The system administrators can also increase server performance simply by throwing more hardware at the servers -- especially more RAM at the Web/App server in my experience, the more so if it's Java-based.

 

**Check the output files in default directory("/var/lib/pt-stalk").**

[译:7 在那一点上,开辟职员需求回溯随管理]

**Usage**

 1 [root@zlm2 08:31:41 /var/lib/pt-stalk]
 2 #ls -lrt
 3 total 3928
 4 -rw-r--r-- 1 root root     383 Jul  9 08:19 2018_07_09_08_19_39-trigger
 5 -rw-r--r-- 1 root root    9845 Jul  9 08:19 2018_07_09_08_19_39-pmap
 6 -rw-r--r-- 1 root root   16554 Jul  9 08:19 2018_07_09_08_19_39-variables
 7 -rw-r--r-- 1 root root    4352 Jul  9 08:19 2018_07_09_08_19_39-innodbstatus1
 8 -rw-r--r-- 1 root root     384 Jul  9 08:19 2018_07_09_08_19_39-mutex-status1
 9 -rw-r--r-- 1 root root    8507 Jul  9 08:19 2018_07_09_08_19_39-ps
10 -rw-r--r-- 1 root root   10356 Jul  9 08:19 2018_07_09_08_19_39-lsof
11 -rw-r--r-- 1 root root    5280 Jul  9 08:19 2018_07_09_08_19_39-opentables1
12 -rw-r--r-- 1 root root    8204 Jul  9 08:19 2018_07_09_08_19_39-top
13 -rw-r--r-- 1 root root   28343 Jul  9 08:19 2018_07_09_08_19_39-sysctl
14 -rw-r--r-- 1 root root     139 Jul  9 08:20 2018_07_09_08_19_39-disk-space
15 -rw-r--r-- 1 root root 1084769 Jul  9 08:20 2018_07_09_08_19_39-mysqladmin
16 -rw-r--r-- 1 root root    2782 Jul  9 08:20 2018_07_09_08_19_39-vmstat
17 -rw-r--r-- 1 root root   24750 Jul  9 08:20 2018_07_09_08_19_39-procstat
18 -rw-r--r-- 1 root root   14403 Jul  9 08:20 2018_07_09_08_19_39-diskstats
19 -rw-r--r-- 1 root root   72469 Jul  9 08:20 2018_07_09_08_19_39-procvmstat
20 -rw-r--r-- 1 root root   51960 Jul  9 08:20 2018_07_09_08_19_39-netstat_s
21 -rw-r--r-- 1 root root  296850 Jul  9 08:20 2018_07_09_08_19_39-slabinfo
22 -rw-r--r-- 1 root root   33210 Jul  9 08:20 2018_07_09_08_19_39-interrupts
23 -rw-r--r-- 1 root root   48390 Jul  9 08:20 2018_07_09_08_19_39-netstat
24 -rw-r--r-- 1 root root   37260 Jul  9 08:20 2018_07_09_08_19_39-meminfo
25 -rw-r--r-- 1 root root   17100 Jul  9 08:20 2018_07_09_08_19_39-df
26 -rw-r--r-- 1 root root   40753 Jul  9 08:20 2018_07_09_08_19_39-processlist
27 -rw-r--r-- 1 root root    4555 Jul  9 08:20 2018_07_09_08_19_39-innodbstatus2
28 -rw-r--r-- 1 root root   74476 Jul  9 08:20 2018_07_09_08_19_39-transactions
29 -rw-r--r-- 1 root root       5 Jul  9 08:20 2018_07_09_08_19_39-hostname
30 -rw-r--r-- 1 root root     385 Jul  9 08:20 2018_07_09_08_19_39-mutex-status2
31 -rw-r--r-- 1 root root    5280 Jul  9 08:20 2018_07_09_08_19_39-opentables2
32 -rw-r--r-- 1 root root     325 Jul  9 08:20 2018_07_09_08_19_39-vmstat-overall
33 -rw-r--r-- 1 root root   24763 Jul  9 08:20 2018_07_09_08_19_39-output
34 
35 //There're a series of files(starts with "tigger" and ends with "output") each time we collect the diagnostic data.
36 
37 -rw-r--r-- 1 root root     383 Jul  9 08:24 2018_07_09_08_24_39-trigger
38 -rw-r--r-- 1 root root   10251 Jul  9 08:24 2018_07_09_08_24_39-pmap
39 -rw-r--r-- 1 root root   16554 Jul  9 08:24 2018_07_09_08_24_39-variables
40 -rw-r--r-- 1 root root    4353 Jul  9 08:24 2018_07_09_08_24_39-innodbstatus1
41 -rw-r--r-- 1 root root    8583 Jul  9 08:24 2018_07_09_08_24_39-ps
42 -rw-r--r-- 1 root root     386 Jul  9 08:24 2018_07_09_08_24_39-mutex-status1
43 -rw-r--r-- 1 root root   28343 Jul  9 08:24 2018_07_09_08_24_39-sysctl
44 -rw-r--r-- 1 root root   10534 Jul  9 08:24 2018_07_09_08_24_39-lsof
45 -rw-r--r-- 1 root root    5280 Jul  9 08:24 2018_07_09_08_24_39-opentables1
46 -rw-r--r-- 1 root root    8053 Jul  9 08:24 2018_07_09_08_24_39-top
47 -rw-r--r-- 1 root root     139 Jul  9 08:25 2018_07_09_08_24_39-disk-space
48 -rw-r--r-- 1 root root    2783 Jul  9 08:25 2018_07_09_08_24_39-vmstat
49 -rw-r--r-- 1 root root 1084769 Jul  9 08:25 2018_07_09_08_24_39-mysqladmin
50 -rw-r--r-- 1 root root   24899 Jul  9 08:25 2018_07_09_08_24_39-procstat
51 -rw-r--r-- 1 root root   48390 Jul  9 08:25 2018_07_09_08_24_39-netstat
52 -rw-r--r-- 1 root root  296850 Jul  9 08:25 2018_07_09_08_24_39-slabinfo
53 -rw-r--r-- 1 root root   52230 Jul  9 08:25 2018_07_09_08_24_39-netstat_s
54 -rw-r--r-- 1 root root   33210 Jul  9 08:25 2018_07_09_08_24_39-interrupts
55 -rw-r--r-- 1 root root   37260 Jul  9 08:25 2018_07_09_08_24_39-meminfo
56 -rw-r--r-- 1 root root   14460 Jul  9 08:25 2018_07_09_08_24_39-diskstats
57 -rw-r--r-- 1 root root   17100 Jul  9 08:25 2018_07_09_08_24_39-df
58 -rw-r--r-- 1 root root   72914 Jul  9 08:25 2018_07_09_08_24_39-procvmstat
59 -rw-r--r-- 1 root root   44370 Jul  9 08:25 2018_07_09_08_24_39-processlist
60 -rw-r--r-- 1 root root    4445 Jul  9 08:25 2018_07_09_08_24_39-innodbstatus2
61 -rw-r--r-- 1 root root   76282 Jul  9 08:25 2018_07_09_08_24_39-transactions
62 -rw-r--r-- 1 root root       5 Jul  9 08:25 2018_07_09_08_24_39-hostname
63 -rw-r--r-- 1 root root     387 Jul  9 08:25 2018_07_09_08_24_39-mutex-status2
64 -rw-r--r-- 1 root root    5280 Jul  9 08:25 2018_07_09_08_24_39-opentables2
65 -rw-r--r-- 1 root root     325 Jul  9 08:25 2018_07_09_08_24_39-vmstat-overall
66 -rw-r--r-- 1 root root   24763 Jul  9 08:25 2018_07_09_08_24_39-output
67 
68 [root@zlm2 08:33:37 /var/lib/pt-stalk]
69 #cat 2018_07_09_08_24_39-trigger
70 2018_07_09_08_24_39 Check results: status(Threads_connected)=6, matched=yes, cycles_true=1
71 2018_07_09_08_24_39 pt-stalk ran with --function=status --variable=Threads_connected --threshold=2 --match= --cycles=1 --interval=1 --iterations= --run-time=30 --sleep=300 --dest=/var/lib/pt-stalk --prefix= --notify-by-email= --log=/var/log/pt-stalk.log --pid=/var/run/pt-stalk.pid --plugin=
72 
73 //The trigger file records the options we have used.
74 
75 [root@zlm2 08:34:17 /var/lib/pt-stalk]
76 #cat 2018_07_09_08_24_39-vmstat-overall
77 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
78  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
79 18  2   9692  60716      0 583316    0    0    11   139   92  597  4  1 96  0  0
80  4  0  10440  78828      0 569672    0   25    89  1965 1455 12046 80 20  0  0  0
81  
82  //The vmstat-overall records the vmstat output.
83  //each file contains the relevant diagnostic data about what their name called.I'm not going to demonstrate all of them.
  1. Let's say the application/database code, as well as the hardware/OS environment have been tuned to the best of everybody's abilities. You re-run the load test from step 2 and now you're at 75 concurrent users before performance starts to degrade.
1 sysbench [options]... [testname] [command]
2 Commands implemented by most tests: prepare run cleanup help

 

At this point, there's not much you can do with the existing setup. It's time to think about scaling the system horizontally, by adding other Web servers in a load-balanced Web server farm[web 服务群组], or adding other database servers. Or maybe do content caching, for example with Apache mod_cache. Or maybe adding an external caching server[高速缓存服务器] such as Squid.

 

**Take care of you remain disk space,pt-stalk won't run anymore if you're out of space.**

One very important product of this whole procedure is that you now have a baseline number for your application for this given "staging" hardware environment. You can use the staging setup for nightly peformance testing runs that will tell you whether changes in your application/database code caused an increase or a decrease in performance.[实在的触及到了开支规模的本金意义]

**Parameters**

 1 [root@zlm2 08:25:56 /data/mysql/mysql3306]
 2 #pt-stalk --host localhost --port 3306 --user root --password Passw0rd --collect --cycles 1 --variable Threads_connected --threshold 2 --sleep 30 --demonize
 3 Cannot open /tmp/pt-stalk.8940.FSboRq/po/daemonize: No space left on device at -e line 13, <$fh> chunk 274.
 4 No long attribute in option spec /tmp/pt-stalk.8940.FSboRq/po/ask-pass
 5 
 6 [root@zlm2 08:51:27 /data/mysql/mysql3306]
 7 #df -h
 8 ]Filesystem               Size  Used Avail Use% Mounted on
 9 /dev/mapper/centos-root  8.4G  8.4G   20K 100% / //The root of linux has been out of space.
10 devtmpfs                 488M     0  488M   0% /dev
11 tmpfs                    497M     0  497M   0% /dev/shm
12 tmpfs                    497M  6.6M  491M   2% /run
13 tmpfs                    497M     0  497M   0% /sys/fs/cgroup
14 /dev/sda1                497M  118M  379M  24% /boot
15 none                      87G   81G  5.9G  94% /vagrant
  1. Repeat above steps in a "real" production environment before you actually launch your application.
 1 //Common Options.
 2 --threads //Specify the threads to use(default 1).
 3 --time //Specify the execution time in seconds(default 10s).
 4 --events //Specify the number of events(default 0).
 5 --rate //Speicify the transactions rate(default 0).
 6 --db-driver //Specify the database driver(default mysql).
 7 --rand-type //Specify the random numbers distribution(defautl "special",others "uniform","gaussian","pareto").
 8 --warmup-time //Specify the time to warmup before real execution to avoid inaccurate result(default 0).
 9 --report-interval //Specify the time about the report interval(default 0,means disable intermediate report).
10 --table //Specify the number of tables in test database.
11 --table-size //Specify the number of records in each test table.
12 
13 //MySQL Driver Options.
14 --mysql-host //Specify the host(default localhost).
15 --mysql-port //Specify the port(default 3306).
16 --mysql-socket //Specify the socket.
17 --mysql-user //Default sbtest.
18 --mysql-password //Specify the password of user.
19 --mysql-db //Specify the test db(default sbtest).
20 --mysql-dry-run //It does not really run but treat MySQL client API calls are successful.
21 
22 //Command Options(depends on "testname").
23 prepare //creates necessary files or data that the peticular test need.
24 run //actually runs the specific test which is specified by "testname" option.
25 cleanup //Removes all the test files or data which are relevant with.
26 help //Shows the information of particular test.

 

All this discussion assumed you want to get performance/benchmarking numbers for your application. If you want to actually discover bugs and to see if your application fails and recovers gracefully, you need to do stress testing. Blast your Web server with double the number of users for example. Unplug network cables randomly (or shut down/restart switch ports via SNMP). Take out a disk from a RAID array. That kind of thing.

 

Summary

The conclusion? At the end of the day, it doesn't really matter what you call your testing, as long as you help your team deliver what it promised in terms of application functionality and performance. Performance testing in particular is more art than science, and many times the only way to make progress in optimizing and tuning the application and its environment is by trial-and-error[反复试验] and perseverance. Having lots of excellent open source tools also helps a lot.   

Example

  • pt-stalk is another tool in diagnosing system performance and similar with pt-pmp but not the same.
  • pt-stalk will generate many statistic files which can really help you collecting information in almost every aspect.
  • pt-stalk has four main triggers:gdb,oprofile,strace,tcpdump.It's flexible to collect data with diffrent dimensionality.

 

 

 

 

Prepare

 1 [root@zlm1 14:31:03 ~/sysbench-1.0/src/lua]
 2 #sysbench oltp_read_write.lua --mysql-host=192.168.56.100 --mysql-port=3306 --mysql-user=zlm --mysql-password=aaron8219 --mysql-db=sysbench --tables=10 --table-size=100000 --mysql-storage-engine=innodb prepare
 3 sysbench 1.0.14 (using bundled LuaJIT 2.1.0-beta2)
 4 
 5 Creating table 'sbtest1'...
 6 Inserting 100000 records into 'sbtest1'
 7 Creating a secondary index on 'sbtest1'...
 8 Creating table 'sbtest2'...
 9 Inserting 100000 records into 'sbtest2'
10 Creating a secondary index on 'sbtest2'...
11 Creating table 'sbtest3'...
12 Inserting 100000 records into 'sbtest3'
13 Creating a secondary index on 'sbtest3'...
14 Creating table 'sbtest4'...
15 Inserting 100000 records into 'sbtest4'
16 Creating a secondary index on 'sbtest4'...
17 Creating table 'sbtest5'...
18 Inserting 100000 records into 'sbtest5'
19 Creating a secondary index on 'sbtest5'...
20 Creating table 'sbtest6'...
21 Inserting 100000 records into 'sbtest6'
22 Creating a secondary index on 'sbtest6'...
23 Creating table 'sbtest7'...
24 Inserting 100000 records into 'sbtest7'
25 Creating a secondary index on 'sbtest7'...
26 Creating table 'sbtest8'...
27 Inserting 100000 records into 'sbtest8'
28 Creating a secondary index on 'sbtest8'...
29 Creating table 'sbtest9'...
30 Inserting 100000 records into 'sbtest9'
31 Creating a secondary index on 'sbtest9'...
32 Creating table 'sbtest10'...
33 Inserting 100000 records into 'sbtest10'
34 Creating a secondary index on 'sbtest10'...
35 
36 //Create 10 tables with 10w rows each.
37 [root@zlm1 14:32:59 ~/sysbench-1.0/src/lua]
38 #

 

 Run

 1 [root@zlm1 14:37:31 ~/sysbench-1.0/src/lua]
 2 #sysbench oltp_read_write.lua --mysql-host=192.168.56.100 --mysql-port=3306 --mysql-user=zlm --mysql-password=aaron8219 --mysql-db=sysbench --threads=3 --time=60 --warmup-time=30 --report-interval=10 --rand-type=uniform run
 3 sysbench 1.0.14 (using bundled LuaJIT 2.1.0-beta2)
 4 
 5 invalid option: --warmup-time=30 //It's not supported in my version.
 6 
 7 [root@zlm1 14:37:34 ~/sysbench-1.0/src/lua]
 8 #sysbench oltp_read_write.lua --mysql-host=192.168.56.100 --mysql-port=3306 --mysql-user=zlm --mysql-password=aaron8219 --mysql-db=sysbench --threads=3 --time=60 --report-interval=10 --rand-type=uniform run
 9 sysbench 1.0.14 (using bundled LuaJIT 2.1.0-beta2)
10 
11 Running the test with following options:
12 Number of threads: 3 //There're three threads.
13 Report intermediate results every 10 second(s) //Show report every 10s.
14 Initializing random number generator from current time
15 
16 
17 Initializing worker threads...
18 
19 Threads started!
20 
21 [ 10s ] thds: 3 tps: 134.07 qps: 2684.74 (r/w/o: 1879.80/536.59/268.34) lat (ms,95%): 48.34 err/s: 0.00 reconn/s: 0.00 //Notice the low value here.
22 [ 20s ] thds: 3 tps: 273.74 qps: 5473.50 (r/w/o: 3831.29/1094.64/547.57) lat (ms,95%): 15.00 err/s: 0.00 reconn/s: 0.00
23 [ 30s ] thds: 3 tps: 273.30 qps: 5467.40 (r/w/o: 3827.60/1093.20/546.60) lat (ms,95%): 14.73 err/s: 0.00 reconn/s: 0.00
24 [ 40s ] thds: 3 tps: 250.50 qps: 5009.82 (r/w/o: 3506.61/1002.20/501.00) lat (ms,95%): 17.95 err/s: 0.00 reconn/s: 0.00
25 [ 50s ] thds: 3 tps: 261.07 qps: 5222.57 (r/w/o: 3656.33/1044.09/522.15) lat (ms,95%): 16.41 err/s: 0.00 reconn/s: 0.00
26 [ 60s ] thds: 3 tps: 264.52 qps: 5289.24 (r/w/o: 3702.14/1058.07/529.03) lat (ms,95%): 15.55 err/s: 0.00 reconn/s: 0.00
27 SQL statistics:
28     queries performed:
29         read:                            203980 //Executions of writing.
30         write:                           58280 //Executions of reading.
31         other:                           29140 //Operations excetp CURD.
32         total:                           291400
33     transactions:                        14570  (242.76 per sec.) //TPS is 242.
34     queries:                             291400 (4855.30 per sec.) //QPS is 4855.
35     ignored errors:                      0      (0.00 per sec.)
36     reconnects:                          0      (0.00 per sec.)
37 
38 General statistics:
39     total time:                          60.0155s
40     total number of events:              14570 //Total transactions.
41 
42 Latency (ms):
43          min:                                    3.17
44          avg:                                   12.35
45          max:                                  887.76
46          95th percentile:                       16.41 //Response time of former 95%
47          sum:                               179957.65
48 
49 Threads fairness:  //Statistics of concurrency.
50     events (avg/stddev):           4856.6667/21.23
51     execution time (avg/stddev):   59.9859/0.00
52 
53 
54 [root@zlm1 14:39:06 ~/sysbench-1.0/src/lua]
55 #

 

 summary

  • sysbench is more powerful than mysqlslap(built-in program of MySQL).
  • Be ware of  the diffrence between database benchmark test and presure test.
  • There're also some other tools can be used to test database performance such as mysql-tpcc,YCSB,etc.
  • Database benchmark really helps us to have a explicit overview about our newly arrived server.More test is necessary.

 

本文由澳门新萄京官方网站发布于数据库网络,转载请注明出处:澳门新萄京官方网站OLTP基准测验,Tookit工具包之

关键词: