2015年2月6日 星期五

【敏捷迭代式開發中的測試】第三篇:有哪些測試?


這篇文章只是摘錄 《 Software-Testing-And-Continuous-Quality-Improvement-Third-Edition》
Page 43 ~ 50 。


table 3.1 testing technique Categories
Technique
Manual
Automated
Static
Dynamic
Functional
Structural
Acceptance testing
x
x
x
x
Ad hoc testing
x
x
Alpha testing
x
x
x
Basis path testing
x
x
x
Beta testing
x
x
x
Black-box testing
x
x
x
Bottom-up testing
x
x
x
Boundary value testing
x
x
x
Branch coverage testing
x
x
x
Branch/condition coverage
x
x
x
Cause–effect graphing
x
x
x
Comparison testing
x
x
x
x
x
Compatibility testing
x
x
x
Condition coverage testing
x
x
x
CRUD (create, read, update, and delete) testing
x
x
x
Database testing
x
x
x
Decision tables
x
x
x
Desk checking
x
x
x
End-to-end testing
x
x
x
Equivalence partitioning
x
x
Exception testing
x
x
x
Exploratory testing
x
x
x
Free-form testing
x
x
x
Gray-box testing
x
x
x
x
Histograms
x
x
Incremental integration testing
x
x
x
x
Inspections
x
x
x
x
Integration testing
x
x
x
x
JADs (joint application designs)
x
x
x
Load testing
x
x
x
x
Mutation testing
x
x
x
x
© 2009 by Taylor & Francis Group, LLC
Continued
44 Software Testing and Continuous Quality Improvement table 3.1 testing technique Categories (Continued)
Technique
Manual
Automated
Static
Dynamic
Functional
Structural
Orthogonal array testing
x
x
x
Pareto analysis
x
x
Performance testing
x
x
x
x
x
Positive and negative testing
x
x
x
Prior defect history testing
x
x
x
Prototyping
x
x
x
Random testing
x
x
x
Range testing
x
x
x
Recovery testing
x
x
x
x
Regression testing
x
x
Risk-based testing
x
x
x
Run charts
x
x
x
Sandwich testing
x
x
x
Sanity testing
x
x
x
x
Security testing
x
x
x
State transition testing
x
x
x
Statement coverage testing
x
x
x
Statistical profile testing
x
x
x
Stress testing
x
x
x
Structured walkthroughs
x
x
x
x
Syntax testing
x
x
x
x
System testing
x
x
x
x
Table testing
x
x
x
Thread testing
x
x
x
Top-down testing
x
x
x
x
Unit testing
x
x
x
x
Usability testing
x
x
x
x
User acceptance testing
x
x
x
x
White-box testing
x
x
x
© 2009 by Taylor & Francis Group, LLC
Overview of Testing Techniques 45 table 3.2 testing technique descriptions
Technique
Brief Description
Acceptance testing
Final testing based on the end-user/customer specifications, or based on use by end users/ customers over a defined period of time
Ad hoc testing
Similar to exploratory testing, but often taken to mean that the testers have significant understanding of the software before testing it
Alpha testing
Testing of an application when development is nearing completion; minor design changes may still be made as a result of such testing. Typically done by end users or others, not by programmers or testers
Basis path testing
Identifying tests based on flow and paths of a program or system
Beta testing
Testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end users or others, not by programmers or testers
Black-box testing
Testing cases generated based on the system’s functionality
Bottom-up testing
Integrating modules or programs starting from the bottom
Boundary value testing
Testing cases generated from boundary values of equivalence classes
Branch coverage testing
Verifying that each branch has true and false outcomes at least once
Branch/condition coverage testing
Verifying that each condition in a decision takes on all possible outcomes at least once
Cause–effect graphing
Mapping multiple simultaneous inputs that may affect others, to identify their conditions to test
Comparison testing
Comparing software weaknesses and strengths to competing products
© 2009 by Taylor & Francis Group, LLC
Continued
46 Software Testing and Continuous Quality Improvement table 3.2 testing technique descriptions (Continued)
Technique
Brief Description
Compatibility testing
Testing how well software performs in a particular hardware/software/operating system/network environment
Condition coverage testing
Verifying that each condition in a decision takes on all possible outcomes at least once
CRUD testing
Building a CRUD matrix and testing all object creations, reads, updates, and deletions
Database testing
Checking the integrity of database field values
Decision tables
Table showing the decision criteria and the respective actions
Desk checking
Developer reviews code for accuracy
End-to-end testing
Similar to system testing; the “macro” end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate
Equivalence partitioning
Each input condition is partitioned into two or more groups. Test cases are generated from representative valid and invalid classes
Exception testing
Identifying error messages and exception- handling processes and conditions that trigger them
Exploratory testing
Often taken to mean a creative, informal software test that is not based on formal test plans or test cases; testers may be learning the software as they test it
Free-form testing
Ad hoc or brainstorming using intuition to define test cases
Gray-box testing
A combination of black-box and white-box testing to take advantage of both
Histograms
A graphical representation of measured values organized according to the frequency of occurrence; used to pinpoint hot spots
© 2009 by Taylor & Francis Group, LLC
Overview of Testing Techniques 47 table 3.2 testing technique descriptions (Continued)
Technique
Brief Description
Incremental integration testing
Continuous testing of an application as new functionality is added; requires that various aspects of an application’s functionality be independent enough to work separately before all parts of the program are completed, or that test drivers be developed as needed; done by programmers or by testers
Inspections
Formal peer review that uses checklists, entry criteria, and exit criteria
Integration testing
Testing of combined parts of an application to determine if they function together correctly. The “parts” can be code modules, individual applications, or client/server applications on a network. This type of testing is especially relevant to client/server and distributed systems
JADs
Technique that brings users and developers together to jointly design systems in facilitated sessions
Load testing
Testing an application under heavy loads, such as testing of a Web site under a range of loads to determine at what point the system’s response time degrades or fails
Mutation testing
A method for determining if a set of test data or test cases is useful, by deliberately introducing various code changes (“bugs”) and retesting with the original test data/cases to determine if the bugs are detected. Proper implementation requires large computational resources
Orthogonal array testing
Mathematical technique to determine which variations of parameters need to be tested
Pareto analysis
Analyze defect patterns to identify causes and sources
Performance testing
Term often used interchangeably with stress and load testing. Ideally, performance testing (and any other type of testing) is defined in requirements documentation or QA or Test Plans
© 2009 by Taylor & Francis Group, LLC
Continued
48 Software Testing and Continuous Quality Improvement table 3.2 testing technique descriptions (Continued)
Technique
Brief Description
Positive and negative testing
Testing the positive and negative values for all inputs
Prior defect history testing
Test cases are created or rerun for every defect found in prior tests of the system
Prototyping
General approach to gather data from users by building and demonstrating to them some part of a potential application
Random testing
Technique involving random selection from a specific set of input values where any value is as likely as any other
Range testing
For each input, identifies the range over which the system behavior should be the same
Recovery testing
Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems
Regression testing
Testing a system in light of changes made during a development spiral, debugging, maintenance, or the development of a new release
Risk-based testing
Measures the degree of business risk in a system to improve testing
Run charts
A graphical representation of how a quality characteristic varies with time
Sandwich testing
Integrating modules or programs from the top and bottom simultaneously
Sanity testing
Typically, an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effort. For example, if the new software is crashing systems every five minutes, bogging down systems to a crawl, or destroying databases, the software may not be in a “sane” enough condition to warrant further testing in its current state
Security testing
Testing how well the system protects against unauthorized internal or external access, willful damage, etc.; may require sophisticated testing techniques
© 2009 by Taylor & Francis Group, LLC
Overview of Testing Techniques 49 table 3.2 testing technique descriptions (Continued)
Technique
Brief Description
State transition testing
Technique in which the states of a system are first identified, and then test cases written to test the triggers causing a transition from one state to another
Statement coverage testing
Every statement in a program is executed at least once
Statistical profile testing
Statistical techniques are used to develop a usage profile of the system that helps define transaction paths, conditions, functions, and data tables
Stress testing
Term often used interchangeably with load and performance testing. Also used to describe such tests as system functional testing while under unusually heavy loads, heavy repetition of certain actions or inputs, input of large numerical values, or large complex queries to a database system
Structured walkthroughs
A technique for conducting a meeting at which project participants examine a work product for errors
Syntax testing
Data-driven technique to test combinations of input syntax
System testing
Black-box type testing that is based on overall requirements specifications; covers all combined parts of a system
Table testing
Testing access, security, and data integrity of table entries
Thread testing
Combining individual units into threads of functionality that together accomplish a function or set of functions
Top-down testing
Integrating modules or programs starting from the top
© 2009 by Taylor & Francis Group, LLC
Continued

50 Software Testing and Continuous Quality Improvement table 3.2 testing technique descriptions (Continued)
Technique
Brief Description
Unit testing
The most “micro” scale of testing; to test particular functions or code modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. Not always easily done unless the application has a well-designed architecture with tight code; may require developing test driver modules or test harnesses
Usability testing
Testing for “user-friendliness.” Clearly, this is subjective, and will depend on the targeted end user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers
User acceptance testing
Determining if software is satisfactory to an end user or customer
White-box testing
Test cases are defined by examining the logic paths of a system

2015年2月5日 星期四

【敏捷迭代式開發中的測試】第二篇:夥伴.品質.溝通


從前一篇文章中:『【敏捷迭代式開發中的測試】第一篇:有關測試的真實劇本序章』提及了測試人員在敏捷開發流程中所扮演的角色及人員素質。在這一篇文章裡,將進一步細化說明測試人員的關鍵工作成功因素及工作目標。

一、夥伴意識

相較於瀑布式開發中,將測試列為開發後的工作環節之一,在迭代式開發中測試人員的角色顯示與開發與需求方都更為親暱地多。

瀑布式開發將「需求」發展至「產出物」的一系列工作事項,以功能區分及線性流程串起來。需求蒐集及分析、系統分析、系統設計、系統開發、系統測試,及至驗收,都可能是以一個實體部門進行負責。完美的情形下,每個環節的交付物是如鎖練般地被串接,也或者像是線上地圖系統可以從整個地球的五大洲圖逐一細化到指定國定、城市、鄉鎮、道路、巷弄一樣的能不斷的 refine。

當然,這種完美並不存在,所以對於每個環節的部門或人員來說,上游的產出物或導致本身環節的產出物不正確、與需求目標產生偏離、或是根本無法產出。除非,在這個環節去償還上一個環節所欠的債!分析的債、設計的債、溝通的債、文件的債…債的類型很多,共同之處在於:還債的人大多和欠債的人不一樣。

還債行為會導致組織內的夥伴意識崩潰。因為欠債的人有時甚至不知道欠下債的人是自己,還進一步地怪罪幫忙還債的人為什麼無法如預期地進行產出。如果下游將欠債的事實告知上游,就會變會上游必須暫停手邊的工作、去償還自己積欠的債務,而此時就代表:在該時間點,上游原先預定的工作將產生延遲,而下游突然間變得沒事做。

在這種總體時間及產出物於各環節中不平衡的狀況下,責任的歸屬問題、專案成敗關鍵的茅頭就會很人性地讓整個組織容易處於敵對、相互隔離、彼此試探…等內耗。那循環儼然是不良的、參雜了很多人性及政治因素,一旦與風氣、權力個人相結合後,就會成為組織中極為難解的問題。

迭代式開發,更仔細地說,是「重複漸增」式的系統演化方式,要求的是測試人員成為開發人員的夥伴。當開發人員專注於功能的構建時,測試人員使用外部環視的角度為其提示品質相關的問題,讓開發的進展與問題的浮現腳步一致。這樣的好處是:問題在剛開始產生時,多半很容易修正!而測試人員應該是開發人員的另一雙眼鏡,讓開發人員隨時能輕易地改正自己犯下的錯誤、沒正視到的盲點。

也由於測試人員能隨時協助開發人員關注品質,所以開發人員能更專注、更放心地把心力放在功能的實踐上,而不需自己三不五時地必須去確認、質疑某些質量需求、思考是否要對現有的設計想法進行修正。

記得,測試人員應該和開發人員是好夥伴,就像左右手一樣,應該能共同地把好東西做出來!

二、品質

相較於專注於如何把真正的需求進行蒐集、分析、系統化、及系統分析、設計及整合的人員來說,測試員會幾近參與上述所有流程,因為他們需要做整體的檢視,並且條列出符合環境限制及真實需求的品質定義才行。在執行層面,不論他們做了什麼,都是為了維護該品質定義清單中的每一項。

測試人員在品質這一項工作上的探究,基本上和架構師的工作是有層次相關的。首先,除了要留意 context limits (這包括客戶及自身組織)及功能性需求 (functional requirement)外,品質的另一個名字還可以稱之為「非功能性需求」(non-functional requirement)。

依需求/產出物的要求、不同行業別、組織限制的情形下,所需留意的非功能性需求有很大的不同。但是大抵來說,要有以下起碼的認知:非功能性需求的各項是經常彼此產生矛盾、需要權衡的。例如:易用性和安全性。要有什麼樣的取捨?這可能由客戶決定、依環境限制進行妥協,或是由架構師視系統的長遠目標進行權衡。但是,真正在整個流程中不斷關注非功能性需求決議、並保持該決議沒有被違背的人,就是測試人員。

有關非功能需求有哪些?筆者偷個懶,給個 wiki 的定義,讓讀者去發揮吧!

http://zh.wikipedia.org/wiki/非功能性需求


三、溝通

當我們發現測試人員,依上述所言,需要:

1. 全程參與需求至產出的所有流程。
2. 需要瞭解客戶、架構師對品質的主要訴求、取捨以及環境上有哪些限制?
3. 必須和開發人員 (乃至上述的所有角色) 都要能成為良佳的夥伴關係。

那麼,測試人員在溝通上的能力就應該備受重視!

測試人員要能和開發人員進行:

  1. 有效的溝通:提供明確的輸入及輸出限制及回應品質。
  2. 加分的溝通:隨時讓開發人員了解其忽略或沒留意到的地方,讓開發人員對整體需求有更具體的方向感。
  3. 體貼的溝通:能理解開發人員的難處,並將其反應在品質控管的結果中。
測試人員要能和需求方協議出合理的品質要求,就必須進行:
  1. 務實的溝通:環境的限制、技術的範圍、資源的有限性都是必須務實的原因。當其他人過於關注需求的想望與功能的實現時,測試人員要能即時提出務實的中斷點,讓全場的人得以回到地面上再進行討論。
  2. 雙贏的溝通:由於測試人員能站在務實的角度去觀察系統整體的進展,所以也有提出讓系統需求及開發方創造雙贏的餘裕及職責。
綜合上述,測試人員在溝通上的能力及個性或其它天賦上的要求是不低的。你身邊已經有一個好夥伴了嗎?那麼,就恭禧你了!