Lewis, human factors, speech human factors, usability, speech usability

James R. Lewis, PhD, CHFP

Last updated: 01/28/2012

This is my personal professional website. I've worked as a human factors engineer for IBM since 1981, and am currently in the IBM Software Group (Application and Integration Middleware Software) in Boca Raton, FL, specializing in the design and evaluation of speech user interfaces. For education and employment details, see my Professional Summary.

PROFESSIONAL PUBLICATIONS

Books

Sauro, J., and Lewis, J. R. (2012). Quantifying the user experience: Practical statistics for user research. Burlington, MA: Morgan Kaufmann.
Lewis, J. R. (2011). Practical speech user interface design. Boca Raton, FL: Taylor & Francis.
Book Chapters

Lewis, J. R. (2012). Usability testing. In G. Salvendy (Ed.), Handbook of Human Factors and Ergonomics, 4th ed. (pp. 1267-1312). New York, NY: John Wiley.

Lewis, J. R. (2011). Human factors engineering. In P. A. LaPlante (Ed.), Encyclopedia of Software Engineering (pp. 383-394). New York, NY: Taylor & Francis.

Lewis, J. R., Commarford, P. M., Kennedy, P. J., and Sadowski, W. J. (2008). Handheld electronic devices. In C. Melody Carswell (Ed.), Reviews of Human Factors and Ergonomics, Vol. 4 (pp. 105-148). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R. (2006). Usability testing. In G. Salvendy (Ed.), Handbook of Human Factors and Ergonomics (pp. 1275-1316). New York, NY: John Wiley.

Turner, C. W., Lewis, J. R., and Nielsen, J. (2006). Determining usability test sample size. In W. Karwowski (Ed.), International Encyclopedia of Ergonomics and Human Factors (pp. 3084-3088). Boca Raton, FL: CRC Press.

Lewis, J. R., Potosnak, K. M., and Magyar, R. L. (1997). Keys and keyboards. In M. Helandar, T. K. Landauer, and P. Prabhu (Eds.), Handbook of Human-Computer Interaction (pp. 1285-1315). Amsterdam: Elsevier.

Articles in Refereed Journals

Millet, B., Asfour, S., and Lewis, J. R. (2009). Selection-based virtual keyboard prototypes and data collection application. Behavior Research Methods, 41, 951-956.

Commarford, P. M., Lewis, J. R., Smither, J. A., and Gentzler, M. D. (2008). A comparison of broad versus deep auditory menu structures. Human Factors, 50, 77-89.

Lewis, J. R., and Sauro, J. (2006). When 100% really isn’t 100%: Improving the accuracy of small-sample estimates of completion rates. Journal of Usability Studies, 3, 136-150.

Commarford, P. M., and Lewis, J. R. (2004). Models of throughput rates for dictation and voice spelling for handheld devices. International Journal of Speech Technology, 7, 69-79.

Lewis, J. R. (2004). Evaluating the potential effectiveness of automatic document analysis. International Journal of Speech Technology, 7, 35-43.

Lewis, J. R., and Commarford, P. M. (2003). Developing a voicespelling alphabet for PDAs. IBM Systems Journal, 42, 624-638.

Polkosky, M. D., and Lewis, J. R. (2003). Expanding the MOS: Development and psychometric evaluation of the MOS-R and MOS-X. International Journal of Speech Technology, 6, 161-182.

Lewis, J. R. (2002). Psychometric evaluation of the PSSUQ using data from five years of usability studies. International Journal of Human-Computer Interaction, 14, 463-488.

Polkosky, M. D., and Lewis, J. R. (2002). Effect of auditory waiting cues on time estimation in speech recognition telephony applications. International Journal of Human-Computer Interaction, 14, 423-446.

Lewis, J. R. (2001). Evaluation of procedures for adjusting problem-discovery rates estimated from small samples. International Journal of Human-Computer Interaction, 13, 445-479.

Lewis, J. R. (2001). Introduction: Some current issues in usability evaluation. International Journal of Human-Computer Interaction, 13, 343-349.

Lewis, J. R. (2001). Voicing psychology at Big Blue. APS Observer, 14(1), 6, 25.

Milenkovic, M., Happ, A. J., and Lewis, J. R. (1997). Design decisions for a voice navigation system. International Journal of Speech Technology, 2, 71-79.

Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7, 57-78.

Lewis, J. R. (1994). Sample sizes for usability studies: Additional considerations. Human Factors, 36, 368-378.

Lewis, J. R. (1993). Multipoint scales: Mean and median differences and observed significance levels. International Journal of Human-Computer Interaction, 5, 383-392.
Lewis, J. R. (1993). Pairs of Latin squares that produce digram-balanced Greco-Latin designs: A BASIC program. Behavior Research Methods, Instruments, & Computers, 25, 414-415.
Download the Latbuild program: LATBUILD.BAS (zipped).
Download the license agreement: Latbuild License.

Lewis, J. R. (1991). An after- scenario questionnaire for computer usability studies: Psychometric Evaluation Over Three Trials. SIGCHI Bulletin, 23 (4), 79.

Lewis, J. R. (1991). Psychometric evaluation of an after- scenario questionnaire for computer usability studies: The ASQ. SIGCHI Bulletin, 23 (1), 78-81.

Lewis, J. R. (1987). Slot vs. insertion magnetic stripe readers: User performance and preference. Human Factors, 29, 461-464.
Papers Included in Published Proceedings of a Professional Conference
Sauro, J., and Lewis, J. R. (2011). When designing usability questionnaires, does it hurt to be positive? In Proceedings of CHI 2011 (pp. 2215-2223). Vancouver, Canada: Association for Computing Machinery. Nominated for Best of CHI award.

Millet, B., Asfour, S., Lewis, J. R., and Onar-Thomas, A. (2010). Five-key text entry: An experimental comparison of novice performance with three keyboard layouts. In Proceedings of the 3rd International Conference on Applied Human Factors and Ergonomics (pp. 1-10). Miami, FL: USA Publishing.

Sauro, J., and Lewis, J. R. (2010). Average task times in usability tests: What to report? In Proceedings of CHI 2010 (pp. 2347-2350). Atlanta, GA: Association for Computing Machinery.

Lewis, J. R., and Sauro, J. (2009). The factor structure of the System Usability Scale. In M. Kurosu, (Ed.), Proceedings of HCII 2009, Human-Centered Design (pp. 94-103). Berlin, Germany: Springer-Verlag.

Sauro, J., and Lewis, J. R. (2009). Correlations among prototypical usability metrics: Evidence for the construct of usability. In Proceedings of CHI 2009 (pp. 1609-1618). Boston, MA: Association for Computing Machinery. Nominated for Best of CHI award.

Millet, B., Asfour, S., and Lewis, J. R. (2008). Designing a hybrid layout for a five-key text entry technique. In Proceedings of the 20th Annual International Occupational Ergonomics and Safety Conference (pp. 156-162). Chicago, IL: International Society of Occupational Ergonomics and Safety.

Lewis, J. R. (2006). Effectiveness of various automated readability measures for the competitive evaluation of user documentation. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting (pp. 624-628). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R. (2006). Effect of level of problem description on problem discovery rates: Two case studies. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting (pp. 2567-2571). Santa Monica, CA: Human Factors and Ergonomics Society.

Kotan, C., and Lewis, J. R. (2006). Investigation of confirmation strategies for speech recognition applications. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting (pp. 728-732). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R., Commarford, P. M., and Kotan, C. (2006). Web-based comparison of two styles of auditory presentation: All TTS versus rapidly mixed TTS and recordings. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting (pp. 723-727). Santa Monica, CA: Human Factors and Ergonomics Society.

Sauro, J., and Lewis, J. R. (2005). Estimating completion rates from small samples using binomial confidence intervals: Comparisons and recommendations. In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting (pp. 2100-2104). Santa Monica, CA: Human Factors and Ergonomics Society.

Commarford, P. M., and Lewis, J. R. (2005). Optimizing the pause length before presentation of global navigation commands. In Proceedings of HCI International 2005: Volume 2 - The Management of Information: E-Business, the Web, and Mobile Computing (pp. 1-7). St. Louis, MO: Mira Digital Publication

Lewis, J. R. (2005). Accuracy error span evaluations: Implications for user interface design of speech dictation and handwriting recognition applications. In Proceedings of HCI International 2005: Vol. 6 - Human Factors Issues in Human-Computer Interaction (pp. 1-8). St. Louis, MO: Mira Digital Publication

Lewis, J. R. (2005). Frequency distributions for names and unconstrained words associated with the letters of the English alphabet. In Proceedings of HCI International 2005: Posters (pp. 1-5). St. Louis, MO: Mira Digital Publication

Lewis, J. R. (2004). Selection accuracy with pen selection slots. In Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting (pp. 783-787). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R. (2004). Effect of speaker and sampling rate on MOS-X ratings of concatenative TTS voices. In Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting (pp. 759-763). Santa Monica, CA: Human Factors and Ergonomics Society.

Turner, C. W., Nielsen, J., and Lewis, J. R. (2002). Current issues in the determination of usability test sample size: How many users is enough? In Proceedings of Usability Professionals Association UPA '02.
Download the associated J. Lewis Powerpoint presentation: SAMSIZE-UPA02.PPT.

Lewis, J. R. (2001). Psychometric properties of the Mean Opinion Scale. In Proceedings of HCI International 2001: Usability Evaluation and Interface Design (pp. 149-153). Mahwah, NJ: Lawrence Erlbaum.

Wang, H., and Lewis, J. R. (2001). Intelligibility and acceptability of short phrases generated by embedded text-to-speech engines. In Proceedings of HCI International 2001: Usability Evaluation and Interface Design (pp. 144-148). Mahwah, NJ: Lawrence Erlbaum.

Lewis, J. R. (1999). Effect of error correction strategy on speech dictation throughput. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 457-461). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R., LaLomia, M. J., and Kennedy, P. J. (1999). Evaluation of typing key layouts for stylus input. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 420-424). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R., Kennedy, P. J., and LaLomia, M. J., (1999). Development of a digram-based typing key layout for single-finger/stylus input. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 415-419). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R. (1999). Input rates and user preference for three small-screen input methods: Standard keyboard, predictive keyboard, and handwriting. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 425-428). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R., Allard, D. J., and Hudson, H. D. (1999). Predictive keyboard design study: Effects of word populations, number of displayed letters, and number of transitional probability tables. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 429-432). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R. (1999). Information for PDA application design: Calendar entry and name length statistics. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 467-470). Santa Monica, CA: Human Factors and Ergonomics Society.

Lewis, J. R. (1999). Trade-offs in the design of the IBM computer usability satisfaction questionnaires. In H. Bullinger and J. Ziegler (eds.)., Human-Computer Interaction: Ergonomics and User Interfaces – Vol. I (pp. 1023-1027). Mahwah, NJ: Lawrence Erlbaum.

Lewis, J. R. (1996). Binomial confidence intervals for small sample usability studies. In G. Salvendy and A. Ozok (eds.), Advances in Applied Ergonomics: Proceedings of the 1st International Conference on Applied Ergonomics -- ICAE '96 (pp. 732-737). Istanbul, Turkey: USA Publishing.

Lewis, J. R. (1996). Reaping the benefits of modern usability evaluation: The Simon story. In G. Salvendy and A. Ozok (eds.), Advances in Applied Ergonomics: Proceedings of the 1st International Conference on Applied Ergonomics -- ICAE '96 (pp. 752-757). Istanbul, Turkey: USA Publishing.

Lewis, J. R. (1993). Problem discovery in usability studies: A model based on the binomial probability formula. In Proceedings of the Fifth International Conference on Human-Computer Interaction (pp. 666-671). Orlando, FL: Elsevier.

Lewis, J. R. (1992). Psychometric evaluation of the post-study system usability questionnaire: The PSSUQ. In Proceedings of the Human Factors Society 36th Annual Meeting (pp. 1259-1263). Atlanta, GA: Human Factors Society.

Lewis, J. R. (1991). A rank-based method for the usability comparison of competing products. In Proceedings of the Human Factors Society 35th Annual Meeting (pp. 1312-1316). San Francisco, CA: Human Factors Society.

Loricchio, D. F. and Lewis, J. R. (1991). User assessment of standard and reduced-size numeric keypads. In Proceedings of the Human Factors Society 35th Annual Meeting (pp. 251-252). San Francisco, CA: Human Factors Society.

Lewis, J. R. (1990). The Iowa Silent Reading Test's reading comprehension section: Local norms and predictive validity for usability studies. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 922-926). Orlando, FL: Human Factors Society.

Lewis, J. R., Henry, S. C., and Mack, R. L. (1990). Integrated office software benchmarks: A case study. In Human-Computer Interaction - INTERACT '90, Proceedings of the Third IFIP Conference on Human-Computer Interaction (pp. 337-343). Cambridge, England: Elsevier Science Publishers.

Lewis, J. R. (1989). Pairs of Latin squares to counterbalance sequential effects and pairing of conditions and stimuli. In Proceedings of the Human Factors Society 33rd Annual Meeting (pp. 1223-1227). Denver, CO: Human Factors Society.

Lewis, J. R. and Alfonso, P. (1989). Developing the IBM Personal System/2 mouse: An industrial design/human factors collaboration. In Proceedings of the Sixth Symposium on Human Factors and Industrial Design in Consumer Products - Interface 89 (pp. 263-267).
Kennedy, P. J., Lewis, J. R., and Alfonso, P. (1989). Industrial design and human factors contributions to the design of IBM's PS/2. In Proceedings of the Sixth Symposium on Human Factors and Industrial Design in Consumer Products - Interface 89 (pp. 221-224). Pittsburgh, PA: Human Factors Society.

Snyder, K. M. and Lewis, J. R. (1989). Cognitive representations of DOS commands as a function of expertise. In Proceedings of the 22nd Annual Hawaii International Conference on System Science (pp. 447-456). Kailua-Kona, Hawaii: Computer Society of the IEEE.

Lewis, J. R. (1986). Power switches: Some user expectations and preferences. In Proceedings of the Human Factors Society 30th Annual Meeting (pp. 895-899). Dayton, OH: Human Factors Society.

Kennedy, P. J. and Lewis, J. R. (1985). A method of analyzing personal computer use in an application environment. In Proceedings of the Human Factors Society 29th Annual Meeting (pp. 1057-1060). Baltimore, MD: Human Factors Society.

Snyder, K. M., Happ, A. J., Malcus, L., Paap, K. R., and Lewis, J. R. (1985). Using cognitive models to create menus. In Proceedings of the Human Factors Society 29th Annual Meeting (pp. 655-658). Baltimore, MD: Human Factors Society.

Lewis, J. R. (1984). Association of visually coded functions with an alternate key. In Proceedings of the Human Factors Society 28th Annual Meeting (pp. 973-976). San Antonio, TX: Human Factors Society.

Kennedy, P. J. and Lewis, J. R. (1983). Automated data collection. In Proceedings of the Human Factors Society 27th Annual Meeting (pp. 546-547). Norfolk, VA: Human Factors Society.

Happ, A. J. and Lewis, J. R. (1983). The effect of screen boundary, familiarity, and data type on user's decision to scroll or window. In Proceedings of the Human Factors Society 27th Annual Meeting (pp. 512-515). Norfolk, VA: Human Factors Society.

Lewis, J. R. (1982). Testing small system customer set-up. In Proceedings of the Human Factors Society 26th Annual Meeting (pp. 718-720). Dayton, OH: Human Factors Society.
Technical Reports
Lewis, J. R. (2007). Advantages and disadvantages of "press or say" speech user interfaces (Tech. Report BCR-UX-2007-0002). Boca Raton, FL: IBM Corp.

Lewis, J. R. (2007). Research-based user interface recommendations for natural language call routing (Tech. Report BCR-UX-2007-0001). Boca Raton, FL: IBM Corp.

Lewis, J. R. (2005). Usability testing (Tech. Report 29.3820). Raleigh, NC: IBM Corp.

Millet, B., and Lewis, J. R. (2004). Usability evaluation of prompt clarity for a language selection task (Tech. Report 29.3819). Raleigh, NC: IBM Corp.

Commarford, P. M., and Lewis, J. R. (2004). Word completion feature that recognizes implicit rejections (Tech. Report 29.3812). Raleigh, NC: IBM Corp.

Lewis, J. R. (2004). Actual versus estimated dictation accuracy (Tech. Report 29.3790). Raleigh, NC: IBM Corp.

Lewis, J. R. (2004). Sample size adequacy for usability studies: Current issues (Tech. Report 29.3789). Raleigh, NC: IBM Corp.

Lewis, J. R. (2004). Accuracy error span evaluations: Implications for user interface design of speech dictation and handwriting recognition applications (Tech. Report 29.3788). Raleigh, NC: IBM Corp.

Gomez, F., Agapi, C., and Lewis, J. R. (2004). Rapid VoiceXML developmeng using IBM’S graphical Call Flow Builder (Tech. Report 29.3787). Raleigh, NC: IBM Corp.

Lewis, J. R. (2004). Prototyping best-practice speech user interfaces with VoiceXML 2.0 and the IBM VoiceXML toolkit (Tech. Report 29.3732). Raleigh, NC: IBM Corp.

Michelini, V., Agapi, C., Mirt, M., Wang, F., Lewis, J. R., and Polkosky, M. D. (2004). A flexible method for manipulating fields in a directory assistance application (Tech. Report 29.3731). Raleigh, NC: IBM Corp.

Lewis, J. R., Commarford, P. M., and Kotan, C. A. (2003). Web-based comparison of two styles of Auditory presentation: All TTS versus rapidly mixed TTS and recordings (Tech. Report 29.3684). Raleigh, NC: IBM Corp.

Kotan, C. A., and Lewis, J. R. (2003). Comparison of two delayed confirmation strategies for speech recognition interactive voice response systems (Tech. Report 29.3675). Raleigh, NC: IBM Corp.

Lewis, J. R., and Sadowski, W. J. (2003). Analysis of reduction in passcode guessability as a function of conversion from alphanumeric to numeric codes (Tech. Report 29.3665). Raleigh, NC: IBM Corp.

Lewis, J. R. (2003). Preliminary comparison of immediate and delayed confirmation strategies as a function of number of corrections (Tech. Report 29.3663). Raleigh, NC: IBM Corp.

Lewis, J. R. (2003). Prototyping the conversion from a directed dialog speech user interface to a natural commands user interface (Tech. Report 29.3636). Raleigh, NC: IBM Corp.

Lewis, J. R. (2002). Effect of level of problem description on problem discovery rate: Two case studies (Tech. Report 29.3604). Raleigh, NC: IBM Corp.

Commarford, P. M., and Lewis, J. R. (2002). Models of throughput rates for dictation revisited: Consideration of speech processing speed (Tech. Report 29.3599). Raleigh, NC: IBM Corp.

Lewis, J. R. (2002). Prototyping best-practice speech user interfaces with VoiceXML and the IBM VoiceXML Toolkit (Tech. Report 29.3598). Raleigh, NC: IBM Corp.

Commarford, P. M, and Lewis, J. R. (2002). Optimizing the pause length before presentation of always-active navigation commands (Tech. Report 29.3594). Raleigh, NC: IBM Corp.

Lewis, J. R. (2002). Effect of voice and bandwidth on MOS-X ratings (Tech. Report 29.3550). Raleigh, NC: IBM Corp.

Commarford, P. M., and Lewis, J. R. (2002). Models of throughput rates for dictation and voice spelling for handheld computers (Tech. Report 29.3544). Raleigh, NC: IBM Corp.

Polkosky, M. D., and Lewis, J. R. (2002). Enhancement of the Mean Opinion Scale - Expanded (MOS-X) (Tech. Report 29.3542). Raleigh, NC: IBM Corp.

Lewis, J. R., and Commarford, P. M. (2002). Developing and tuning a voice spelling alphabet for devices with small displays (Tech. Report 29.3517). Raleigh, NC: IBM Corp.

Polkosky, M. D., and Lewis, J. R. (2002). Development and psychometric evaluation of an expanded Mean Opinion Scale (MOS-X) (Tech. Report 29.3499). Raleigh, NC: IBM Corp.

Wright, L. E., Hartley, M. W., and Lewis, J. R. (2002). Conditional probabilities for IBM Voice Browser 2.0 alpha and alphanumeric recognition (Tech. Report 29.3498). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). The accuracy wars: Journalists’ estimates of continuous speech product dictation accuracy from 1997-1999 (Tech. Report 29.3465). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). Estimating the effectiveness of automatic vocabulary expansion (Tech. Report 29.3452). Raleigh, NC: IBM Corp.

Polkosky, M. D., and Lewis, J. R. (2001). User preference for turntaking tones 2: Participant source issues and additional data (Tech. Report 29.3447). Raleigh, NC: IBM Corp.

Polkosky, M. D., and Lewis, J. R. (2001). Effect of ticking rate on user estimation of system response time: Part 2 (Tech. Report 29.3446). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). Frequency distributions for names and unconstrained words associated with the letters of the English alphabet (Tech. Report 29.3437). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). Web-based usability evaluation of ink-on-page indicators (Tech. Report 29.3428). Raleigh, NC: IBM Corp.

Polkosky, M. D., and Lewis, J. R. (2001). Effect of ticking rate on user estimation of system response time (Tech. Report 29.3425). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). Preference for and accuracy with 55- and 99-page ThinkScribe selection slots (Tech. Report 29.3424). Raleigh, NC: IBM Corp.

Hartley, M. W., and Lewis, J. R. (2001). Conditional probabilities for IBM Voice Browser recognition of letters of the alphabet (Tech. Report 29.3421). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). Psychometric evaluation of the CSUQ using data from five years of usability studies (Tech. Report 29.3418). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). The revised Mean Opinion Scale (MOS-R): Preliminary psychometric evaluation (Tech. Report 29.3414). Raleigh, NC: IBM Corp.

Polkosky, M. D., Sadowski, W. J., and Lewis, J. R. (2001). The uniqueness of initial words in news headlines (Tech. Report 29.3412). Raleigh, NC: IBM Corp.

Polkosky, M. D., and Lewis, J. R. (2001). The function of nonspeech audio in speech recognition applications: A review of the literature (Tech. Report 29.3405). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). Psychometric properties of the Mean Opinion Scale (Tech. Report 29.3403). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). Selection accuracy with a new pen-based selection device (Tech. Report 29.3396). Raleigh, NC: IBM Corp.

Sadowski, W. J., and Lewis, J. R. (2001). Usability evaluation of the IBM WebSphere WebVoice demo (Tech. Report 29.3387). Raleigh, NC: IBM Corp.

Lewis, J. R. (2001). Sample size estimation and use of substitute audiences (Tech. Report 29.3385). Raleigh, NC: IBM Corp.

Lewis, J. R. (2000). Evaluation of problem discovery rate adjustment procedures for sample sizes from two to ten (Tech. Report 29.3362). Raleigh, NC: IBM Corp.

Lewis, J. R. (2000). Reducing the overestimation of p in problem discovery usability studies: Normalization, regression, and a combination normalization/Good-Turing approach (Tech. Report 29.3361). Raleigh, NC: IBM Corp.

Lewis, J. R. (2000). Using discounting methods to reduce overestimation of p in problem discovery usability studies (Tech. Report 29.3359). Raleigh, NC: IBM Corp.

Lewis, J. R. (2000). Overestimation of p in problem discovery usability studies: How serious is the problem? (Tech. Report 29.3358). Raleigh, NC: IBM Corp.

Lewis, J. R. (2000). Validation of Monte Carlo estimation of problem discovery likelihood (Tech. Report 29.3357). Raleigh, NC: IBM Corp.

Sadowski, W. J., and Lewis, J. R. (2000). Wizard of Oz usability evaluation of the IBM WebSphere 'WebVoice' Demo (Tech. Report 29.3321). Raleigh, NC: IBM Corp.

Sadowski, W. J., and Lewis, J. R. (2000). Usability evaluation of speech user interfaces for three currency conversion prototypes (Tech. Report 29.3308). Raleigh, NC: IBM Corp.

Lewis, J. R. (2000). Testing natural language application recognition accuracy and habitability (Tech. Report 29.3297). Raleigh, NC: IBM Corp.

Lewis, J. R., Simone, J. E., and Bogacz, M. (2000). Designing common functions for speech-only user interfaces: Rationales, sample dialogs, potential uses for event counting, and sample grammars (Tech. Report 29.3287). Raleigh, NC: IBM Corp.

Lewis, J. R. (2000). Developing minimal comprehensive sets of test cases for natural command grammars (Tech. Report 29.3272). Raleigh, NC: IBM Corp.

Lewis, J. R. (1999). Streamlining a general test plan for competitive evaluation of dictation accuracy and throughput (Tech. Report 29.3158). Raleigh, NC: IBM Corp.

Lewis, J. R. (1997). A general plan for conducting human factors studies of competitive speech dictation accuracy and throughput (Tech. Report 29.2246). Raleigh, NC: IBM Corp.

Lewis, J. R. (1995). Reaping the benefits of modern usability evaluation: The Simon story (Tech. Report 54.948). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1995). A stimulus resource for lexical decision experiments (Tech. Report 54.904). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1995). Meta-analysis of preference for split versus standard keyboards: Findings from 1972 to 1993 (Tech. Report 54.899). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1995). The effects of standard typing experience and split keyboard experience on split keyboard experimental outcomes: Evidence from the split keyboard literature (Tech. Report 54.898). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1995). Input rates and user preference for three small-screen input methods: Standard keyboard, predictive keyboard, and handwriting (Tech. Report 54.889). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1994). Small-sample evaluation of the Paulson-Takeuchi approximation to the exact binomial confidence interval (Tech. Report 54.872). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1994). Direction stereotypes for setting dates and times (Tech. Report 54.867). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1994). Competitive usability evaluation of Sharp Wizard OZ-9600, Hewlett-Packard HP-95LX, and Angler (Tech. Report 54.866). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R., LaLomia, M. J., and Miranda, S. (1994). Competitive evaluation of Newton using personal communicator scenarios (Tech. Report 54.865). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1994). Speaking Simon: Simon's language guidelines and automated readability measures (Tech. Report 54.864). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R., Allard, D. J., and Hudson, H. D. (1994). Predictive keyboard design study: Effects of different word populations, number of displayed letters, and number of trigraph tables (Tech. Report 54.846). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1994). A critical literature review of human factors studies of split keyboards from 1926 to 1993 (Tech. Report 54.853). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1993). Name length statistics for touch-screen buttons (Tech. Report 54.810). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1993). An icon usability evaluation procedure with application to Personal Communicator common icons (Tech. Report 54.792). Boca Raton, FL: International Business Machines Corp.
Lewis, J. R. (1993). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use (Tech. Report 54.786). Boca Raton, FL: IBM Corp.
Note: The PSSUQ (Post-Study System Usability Questionnaire), CSUQ (Computer System Usability Questionnaire) and ASQ (After-Scenario Questionnaire) questionnaires are available in the preceding TR, which also provides details about their psychometric properties. To download the CSUQ or ASQ separately, use these links:

CSUQ
ASQ

Lewis, J. R. (1993). Relative importance of the components of the cloudiness count: Passive voice and infrequent words (Tech. Report 54.759). Boca Raton, FL: IBM Corp.

Lewis, J. R. (1993). Calendar entry statistics for computer calendar users (Tech. Report 54.754). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1993). The Kruskal-Wallis rank sum test with multiple comparisons: A BASIC program (Tech. Report 54.732). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1993). The Friedman rank sum test with multiple comparisons: A BASIC program (Tech. Report 54.733). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). Typing key layouts for single-finger or stylus input: Initial user preference and performance (Tech. Report 54.729). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). A study of the perceived value of color illustrations in documentation: Preliminary results (Tech. Report 54.719). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). Psychometric evaluation of the Computer System Usability Questionnaire: The CSUQ (Tech. Report 54.723). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). IBM employees' satisfaction with their computer systems: Initial norms for the Computer System Usability Questionnaire (Tech. Report 54.728). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). Sample sizes for usability studies: Additional considerations (Tech. Report 54.711). Boca Raton, FL: IBM Corp.

Lewis, J. R. (1992). An alternative digraph-based typing-key layout for single-finger or stylus input (Tech. Report 54.710). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). The kappa measure of interrater agreement: Two BASIC programs (Tech. Report 54.695). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). The Kolmogorov-Smirnov test for identical populations: A BASIC program (Tech. Report 54.697). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). The Friedman rank sum test with a correction for ties: A BASIC program (Tech. Report 54.696). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). Literature review of touch-screen research from 1980 to 1992 (Tech. Report 54.694). Boca Raton, FL: IBM Corp.

Lewis, J. R. (1992). Pairs of Latin squares that simultaneously counterbalance immediate sequential effects and condition-stimulus pairing: A BASIC program (Tech. Report 54.673). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R., Kennedy, P. J., and LaLomia, M. J. (1992). Improved typing key layouts for single-finger or stylus input (Tech. Report 54.692). Boca Raton, FL: International Business Machines Corp.

Kennedy, P. J., Pruitt, J. S., and Lewis, J. R. (1992). An evaluation of the readability of anti-aliased CRT typeface versus Adobe Type Manager bi-level typeface (Tech. Report 54.663). Boca Raton, FL: International Business Machines Corp.

Loricchio, D. F. and Lewis, J. R. (1992). Integrated pointing devices for the PS/2 keyboard: 16 mm trackball, 25 mm trackball, and the pointing stick (Tech. Report 54.698). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1992). Using scenario step counts to predict expert user time-on-task: A case study (Tech. Report 54.653). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R., and Tripi, B. (1992). Competitive evaluation of documentation: Surveys and automated readability measures (Tech. Report 54.647). Boca Raton, FL: IBM Corp.

Lewis, J. R. (1991). The predictive accuracy of design studies: A case study (Tech. Report 54.611). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1991). User satisfaction questionnaires for usability studies: 1991 manual of directions for the PSSUQ and the ASQ (Tech. Report 54.609). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1991). Legitimate use of small samples in usability studies: Three examples (Tech. Report 54.594). Boca Raton, FL: International Business Machines Corp.

Weild, N. E., Lewis, J. R., and Kennedy, P. J. (1991). Two tablet computer usability studies: Tethered versus untethered pens and varied tablet dimensions (Tech. Report 54.588). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1991). An after-scenario questionnaire for usability studies: Psychometric evaluation over three trials (Tech. Report 54.584). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R., Bloch, D. R. and Korn, P. A. (1991). Multimedia usability requirements: Findings from the Boca Raton Multimedia QFD customer visits. (Tech. Report 54.576). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1990). Sample sizes for observational usability studies: Tables based on the binomial probability formula (Tech. Report 54.571). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. and Pallo, S. (1990). Evaluation of graphic symbols for PHONE and LINE (Tech. Report 54.572). Boca Raton, FL: International Business Machines Corp.

King, L., Lee, R., and Lewis, J. R. (1990). Errata sheet effectiveness: A risk assessment (Tech. Report 54.567). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. and Ryan, K. S. (1990). User assessment of the value of a hard file in-use indicator for floor-standing personal computers (Tech. Report 54.539). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1990). A simple usability study pretest for dexterity: Reliability and revised local norms (Tech. Report 54.545). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1990). Psychometric evaluation of a post-study system usability questionnaire: The PSSUQ (Tech. Report 54.535). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1990). A simple usability pretest for dexterity: Local norms and predictive validity (Tech. Report 54.538). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1990). Psychometric evaluation of an after-scenario questionnaire for computer usability studies: The ASQ (Tech. Report 54.541). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1990). Using the reading comprehension section of the Iowa Silent Reading Test as a usability study pretest: Local norms and predictive validity (Tech. Report 54.537). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1989). The relative reliabilities of mean and median differences as indicators of statistically significant differences for 7-point scales (Tech. Report 54.532). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1988). Human factors study of an improved slot design for swipe motion magnetic stripe badge readers (Tech. Report 54.482). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1988). The PLANBUILDER system: System administrator's guide (Tech. Report 54.476). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1988). The PLANBUILDER system: PLAN diskette user's guide (Tech. Report 54.477). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1988). The PLANBUILDER system: The COMBINE program (Tech. Report 54.478). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R., Snyder, K. M., and Happ, A. J. (1988). Measuring the similarity of menu structures (Tech. Report 54.455). Boca Raton, FL: International Business Machines Corp.

Kennedy, P. J. and Lewis, J. R. (1988). Using Keysave to analyze personal computer use in an application environment (Tech. Report 54.458). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1987). Developing the Personal System/2 mouse: Human factors studies (Tech. Report 54.426). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1986). Analysis of a symbol evaluation methodology (Tech. Report 54.413). Boca Raton, FL: International Business Machines Corp.

Snyder, K. M., Happ, A. J., Paap, K., Malcus, L., Dyck, J., Rotella, J., and Lewis, J. R. (1986). Using cognitive networks to create menus (Tech. Report 54.405). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1986). User reaction to three personal computer design concepts (Tech. Report 54.398). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1985). Human factors study of two safety symbols (Tech. Report 54.396). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1985). Two and three button mice with TopView (Tech. Report 54.363). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1985). Personal computer key frequencies: A human factors field study with emphasis on the frequency of acquisition of the backspace and enter keys (Tech. Report 54.352). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1985). Pointing devices with TopView: The keyboard, mouse, and touchpad (Tech. Report 54.355). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1985). Human factors evaluation of four magnetic stripe readers for time and attendance use (Tech. Report 54.402). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1984). Development of a competitive printer usability database and application of a competitive usability analysis procedure (Tech. Report 54.310). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1984). Usability comparison of competitive products with ranks (Tech. Report 54.306). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1984). The Decision Data 6541-04 printer: A usability study (Tech. Report 54.305). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1984). The TI 810LQ printer: A usability study (Tech. Report 54.295). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1983). The DEC Letterprinter 100 printer: A usability study (Tech. Report 54.290). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1983). The IBM 3287 Model 2C printer: A usability study (Tech. Report 54.286). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1983). The IBM Model 4975 Model 2L printer: A usability study (Tech. Report 54.275). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. and Watson, A. L. (1983). The IBM 5224 Model 2 printer: A usability study (Tech. Report 54.272). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1983). The Centronix 353 printer: A usability study (Tech. Report 54.263). Boca Raton, FL: International Business Machines Corp.

Lewis, J. R. (1983). Customer setup and operator procedures: Guidelines for designing study and test plans (Tech. Report 54.215). Boca Raton, FL: International Business Machines Corp.