Members Weigh in on HFES Publications
By Thomas F. Sanquist & Kermit G. Davis

     The 2009 HFES Publications Survey was designed to allow the membership to assess the health and utility of our various print and digital communications media. In addition to soliciting feedback about the utility of the publications, the survey aimed to gather suggestions for improvement.

     The survey, created by the Publications Committee and implemented on the digital survey tool Zoomerang (which yielded quite a few human factors comments as well), was made accessible to members during the period September 10-October, 10, 2009. Responses to complete the online survey were invited from 3,999 current members by e-mail. We obtained 368 responses (325 complete, 64 partial), for a response rate of 9.2%. The last survey to gather information related to publications was conducted in 2001 (Linegang, Williams, & Moroney, 2002).

     HFES publications fall into several broad categories, including periodicals (Human Factors, HFES Bulletin, Ergonomics in Design, the HFES Annual Meeting proceedings, and Journal of Cognitive Engineering and Decision Making), the Reviews of Human Factors and Ergonomics series, standards, books, directories, the HFES Digital Library, and the Web site. Each of these communication tools serves somewhat different purposes for the Society.

     From the HFES staff perspective, gathering this information is timely, given that both Human Factors and Ergonomics in Design will have new editors next year: William S. Marras and Carol Stuart-Buttle, respectively. In addition, the Society is just ending the first year of its partnership with SAGE Publications for the publication of Human Factors.

     General issues of concern identified in the survey responses include publication costs, redundancy of digital and print delivery, the general trend toward migration of scientific publication to digital archives, "legacy" access to material for people who discontinue membership, design and usability of digital publication and Web site interfaces, and, most important, how well the various media serve the needs of our members. Of particular interest in this regard is the work process of members - what information do they need, what tools do they use, what sources do they search, and how do they work: all paper, all digital, or some combination?

Respondent Profile

     Respondents were predominantly PhDs (51%) and master's-level (36%) professionals; a smaller group of members have a bachelor's (12%) as the highest degree. There was a substantial spread in length of Society membership: 35% reported 0-5 years, 20% have 6-10 years (20%), 22% with 11-20 years, and 23% with more than 20 years' membership. This survey did not capture the employment sector of the respondents, although the general membership trend has been toward an increasing proportion of university/educational affiliations. This seems to be reflected by the large percentage of PhD respondents - about 10% higher than the overall membership proportion of PhDs (40%).

Quality of Current Publications and Communications Tools

     Quality was assessed through several items that asked respondents the extent to which they agreed or disagreed with the following statement: "I consider (name) among the best publications I read and use." For many of the publications, responses clustered in the "neither agree nor disagree" and "agree" categories. The lack of disagreement with this statement indicates that none of the publications is perceived to have noteworthy quality problems.

     Human Factors elicited the most agreement with this statement; 63% of the respondents agreed it is among the best they use and read, followed by 62% for the HFES Annual Meeting proceedings, 53% for Ergonomics in Design, 35% for Reviews of Human Factors and Ergonomics, and 26% for Journal of Cognitive Engineering and Decision Making. Although 40% of the respondents agreed that the HFES Digital Library-Archive was among the best they have used, this was counterbalanced by many suggestions for improvement (see below).

Usefulness of Current Publications and Communications Tools

     Several questions were posed to assess the usefulness of various publications to the work of members. The results of these questions are shown in Figure 1, which presents the data in descending order of utility to work.

Figure 1. Percentage of respondents rating publications useful or not useful for work.

     The results indicate that Human Factors is considered the most useful for members' work, followed by the HFES Bulletin, EID, the Annual Meeting proceedings, the HFES Digital Library-Archive, and the membership directory. For all these publications, the percentage of members who find the publication useful was substantially greater than that of members who rate them not useful.

     Another set of publications was perceived to be useful for about half the membership; the other half found them not useful, including standards and best practices, Reviews of Human Factors and Ergonomics, other HFES books, and the Journal of Cognitive Engineering and Decision Making. The latter group of publications is more niche-oriented, with specific target readerships, so these findings do not reflect negatively on quality but instead reflect more-focused constituencies.

     The survey respondents have tended not to purchase books published by HFES (such as Reviews of Human Factors and Ergonomics and Augmented Cognition) in the past 12 months (81%), principally because the content was not useful for their work or the price was judged as too high (40% for both categories).

Digital Interactions

     In contrast with the publications section of the 2001 membership survey, in which print publications were the preferred mode for knowledge and information (43% print, 40% digital), the current survey indicates that electronic versions of journals are the most preferred source (59% prefer electronic). This can be attributed to the continued trend toward electronic publications and improvements in access. Our publications - such as Human Factors, EID, and JCEDM - will certainly continue toward this mode of dissemination via e-mail notifications, online access, the Digital Library, and links among publications.

     The survey asked members about their preferred methods of searching the literature when they need information. They were asked to rate various search tools on a 1-5 scale from most to least preferred. The results in Figure 2 indicate that Google search tools are the most preferred (47%), exceeding by far any of the other methods rated. The most professionally specific content method - the HFES online publications via Ingenta or SAGE - was the least-preferred method, indicating a need to improve the usability of these tools in the future. The results suggest that even for highly specific technical information, Google is the primary entry point.

Figure 2. Most- and least-preferred literature search methods.

     In terms of work process, 84% of the respondents indicate that downloading full-text articles is their most-preferred method of seeking information. Viewing abstracts was the second-most-preferred method (55%), with other methods such as viewing tables of contents, lists of most-cited articles, and lists of related articles being much less preferred. Respondents indicated that if they receive HFES publications in electronic format, the preferred method is an e-mail notification linking to a Web-based table of contents permitting selected article downloads. The ability to read articles on a smart phone is desired by 19% of the respondents and not desired by 46%, with 36% expressing no preference.

     The HFES Digital Library-Archive is a resource that 48% of the respondents agree is valuable for their work: 41% expressed no preference and 11% disagreed. (The disparity between this result and that reported in Figure 1 represents the use of different scales. The question for Figure 1 results did not have a "no preference" category.) For the respondents who do not feel that the Digital Library is a valuable resource, 33% are not aware of it, and 45% consider the instructions too complex or have tried the access procedures without success. This latter finding is amplified by many open-ended comments suggesting that the HFES Digital Library-Archive could be improved by adopting some of the design features of the Association for Computing Machinery (ACM) portal for digital material access.

     The survey also assessed the extent to which scientific and research blogs are an important source of information for members. Of the respondents, 37% regularly read blogs, 45% do not, and 20% have no preference. (See "Blogging About Human Factors" and "What Science and Research Blogs Do Members Read?" also in this issue of the HFES Bulletin.) Open-ended responses regarding which blogs people read suggested some confusion as to what actually constitutes a blog (many responses listed online journals); the most frequently cited actual blog was The Human Factors Blog. Asked whether they would read a moderated blog if it were provided by HFES, 60% of the respondents said they would.

     Another indication of increased electronic communication is shown by the finding that 73% of the respondents use such social networking sites as LinkedIn, Facebook, MySpace, and Twitter.

Suggestions for Improvement

     Several questions addressed potential improvements in HFES publications and communication tools. Two of the most specific suggestions include a function to find "related articles" in digital media (62%) and automatic downloads of articles of interest (47%). Analysis of open-ended responses on suggestions for improvements in the various publication types suggests the following:

• providing more practice/applications-oriented content (16%),
• improving the ability to access the Digital Library (13%),
• making the membership directory completely digital (9%).

     Although these clusters seem like small proportions relative to the fixed-response survey questions, they do represent a preponderance of comments among 191 very diverse and often unique suggestions.

Observations and Conclusions

     The finding that some publications are valuable to 50% or fewer members raises a variety of questions concerning the level of Society resources that should be devoted to niche publications versus those that appear to serve the broader membership base. This issue will continue to be examined by the Scientific Communications and Publications Domain and the Publications Committee.

     The survey results suggest that the Society has followed the trend of migration toward digital interactions. A much larger proportion of the membership uses electronic journals as the primary source of information now than the number reported in the 2001 membership survey (59% vs. 40%). The finding that Google is the search method of choice, even for searching the scientific literature, may reflect a tendency to use Google as a "wide net" that can then lead to the use of specific digital interfaces, although the survey did not address that level of detail. Capabilities such as full-text downloading of specific articles and a desire for a "related articles" function represent design enhancements that can be proposed to SAGE and Ingenta.

     The HFES Digital Library-Archive appears to be a resource that provides value to members' work, but its usability could be significantly enhanced. Many comments suggested that we apply our own analysis and design methods to improve the site and pointed to the ACM portal as a favorable design model. Given the significantly increased use of digital resources in our work processes, this area warrants further attention.

     Finally, HFES continues to work toward enhancing the value of our publications to the membership. Through the diligent efforts of outgoing Human Factors editor Nancy Cooke, the Impact Factor of the journal has more than doubled, the time from submission to first review has decreased from 8.5 weeks to 7 weeks, and the time from acceptance to final publication has dropped from more than 10 months to less than 3 months in some cases. The turnaround time for other HFES journals (EID and JCEDM) will be targeted for improvement. The Publications Committee and Scientific Communications and Publications Domain leader will continue to consider various means for increasing practitioner-oriented material in all HFES publications, better modes of dissemination, and use of evaluation of these survey results to further improve all of our publications, thereby benefiting our members.

     These are exciting times for the Society's publications program, with many substantial changes in several of our publications. The new editors of Human Factors and Ergonomics in Design will be making significant changes to increase the impact of the journals. William Marras is planning to build on the success of Nancy Cooke and other previous editors to maintain Human Factors as the leading journal in the HF/E field. Carol Stuart-Buttle will be working with the EID editorial board to expand the outreach potential of the journal and expand the practitioner readership. New dissemination strategies are being adopted for Reviews of Human Factors and Ergonomics to allow more customized selection of the chapters and to expand the readership of that publication. Finally, JCEDM Editor Mica Endsley and the editorial board are working to enhance that journal's reach and impact.

Reference

     Linegang, M. P., Williams, H. L., & Moroney, W. F. (2002, December). The 2001 HFES membership survey: Findings. Human Factors and Ergonomics Society Bulletin, 45(12), 1.

Thomas F. Sanquist, a member of the HFES Publications Committee, is a research scientist with the Pacific Northwest National Laboratory, operated for the U.S. Department of Energy by Battelle Memorial Institute.

Kermit G. Davis, PhD, is the HFES Scientific Communications and Publications Domain Leader and associate professor at the University of Cincinnati.

Back to the Table of Contents for the December 2009 HFES Bulletin

Download a .pdf version of this issue

Archive of past HFES Bulletin issues (in .pdf format)