The opinions or views expressed in this article are those of the author and do not necessarily reflect the opinions, recommendations, or official position of the Translation Journal editors.
The first version of this article was to appear in the April 2006 issue of the Translation Journal. Before its intended publication I showed it to ATA President Marian Greenfield for corrections of any error of fact and for comments; she in turn showed it to members of the Board and other individuals involved in the Certification Program. None was able to pinpoint any error of fact; nevertheless, I edited a few passages where, according to the President, I was expressing my subjective opinions. In the last minute, I decided to withhold publication.
Why this article now?
More than nine years have passed since this article was originally written and ten years since my article promising Computerized Certification Exam (CCE) by early 2006 was published in the June 2005 issue of the ATA Chronicle. Those members of the American Translators Association (ATA) who have read that article may be wondering what progress, if any, has been made since then and why the Certification exam is still being given on paper as it has been for the past 42 years. This article is intended as a report to my fellow ATA members on the chronology of the CCE project and the actions of those involved in the related decisions, from my personal perspective. It also intends to point out certain flaws in the ATA Board's communication and decision-making process, which I was able to witness first-hand and did my best to correct during my service on the Board after the events described here. This article may also provide some useful clues to the leadership of ATA and other organizations for handling similar projects in the future.
I'm fully aware that many readers will ask: Why not let sleeping dogs lie? Why bring up now events that happened ten years ago? I didn't publish this article for ten years because I was expecting/hoping that the problems that have plagued the project would be solved and the keyboarded exam would finally become a reality. And I wouldn't publish it now if many of the root causes of what happened in 2005 didn't continue to exist today or if any of those who were involved in the decision-making on the CCE project then were willing to acknowledge the obvious now—that those decisions were nothing but a costly blunder. This article will win me no friends among those individuals, many of whom will resent what they may perceive as my airing ATA's dirty linen. I'm prepared to take their criticism if I can prevent the repetition of the mistakes that cost ATA well over a hundred thousand dollars and needlessly set back the Computerized Certification Exam project by many years.
In this article I'm describing the events on the basis of my personal experience and available documents. I'm making no attempt to guess the players' motives for acting or failing to act as they did. The reader may draw his or her own conclusions.
A little bit of history
The Certification (formerly known as Accreditation) program of the American Translators Association dates back to 1973 when the first tests were given in the only language combination offered at that time, German to English. Since that time, the program has expanded in both scope and quality under the leadership of a succession of volunteer Accreditation/Certification Chairs, and more recently a Certification Program Manager, who is a member of ATA Headquarters' salaried staff. New language combinations are being constantly added. As of this update, exams are being offered in 27 language combinations, including Chinese, Japanese, Arabic, Polish, Russian, Hungarian, Croatian, and others, with English always being either the source or the target language. The quality of passage selection and grading has been constantly improved over the years by introducing objective selection and grading criteria and grader training. Detailed information about the current state of ATA's Certification Program is available at the ATA's website http://atanet.org/certification/index.php.
One thing, however, has not changed in the past 42 years, during which the translation profession has gone from the typewriter to the Internet. While translators have been using computers in their work for the past 30 years or so, certification exams are still being given on paper. This state of affairs has not only contributed to increasingly distancing the conditions of the exam from those under which translators actually work, but has also caused difficulties, especially to Chinese and Japanese candidates, who have grown unaccustomed to hand-writing longer texts. Complaints about the certification program's failure to emulate translators' normal working environment have become increasingly frequent and insistent.
The first CCE Task Force
Past attempts to computerize the certification exams failed due to the technical difficulties encountered in ensuring the necessary security, while providing a user-friendly test environment. In October 2004, Certification Chair Lilian Van Vranken asked me to put together a Task Force to study the possibility of moving the exam to computers. Aware of the difficulty of the task, I reluctantly accepted the assignment and immediately started working on the project. A Task Force (TF) of enthusiastic and computer-savvy individuals was assembled, and a Yahoo Group was created to exchange information among the TF members and archive the relevant documents. All members of the Certification Committee (CC) were made members ex officio of the TF.
Since we knew we were navigating uncharted waters, we attacked the task before us without any preconceived ideas. We considered the use of computers owned by ATA, by the candidates, or by third parties, and the impact of each option on security, convenience, availability, and costs. We considered renting or purchasing computers. We even considered hybrid electronic-paper exams. We finally settled on using third-party computers in a computer lab-type proctored environment. This approach required: a) a service provider to supply and manage the necessary software for storing the exam and practice test passages by language combination and version, identifying the candidates and their language combinations, feeding the source text to the appropriate candidates together with the appropriate keyboard, receiving the finished translations, and preparing them for grading; and b) a number of sites equipped with Internet-connected computers for holding certification sittings.
Part a) was the technically more demanding task, since security, the variety of keyboards (including keyboards for pictogram-based and right-to-left scripts), and a host of other details such as timing, backups, continuity of service, and confidentiality had to be taken into account. In order to find a suitable provider, we searched the Web for organizations engaged in testing, in particular language testing, including universities and learning centers. We contacted organizations that had been considered during the previous attempts at CCE, consulted colleagues involved in testing, and asked others to contact their personal and professional acquaintances for suggestions. We also investigated the certification programs of sister translators' organizations. Most potential vendors we found during this process either showed no interest in a project of such a relatively small size or were eliminated after the initial discussions because they did not offer essay-type (as opposed to multiple-choice) exams, were not familiar with foreign-language computing, charged excessive fees for their services, or for other reasons. The search process followed by the TF is well-documented in the TF's Yahoo message archive.
By March 2005 we had narrowed down the choice of candidates to three: two commercial outfits and one university, all with computer-based translation testing systems in place, which qualified them not only to provide the necessary software, but also to host it on their computers and manage the entire system. Extensive discussions ensued with each of these candidates and within the TF. We identified those technical features not offered by the candidates we deemed necessary and those deemed desirable whose cost-benefit ratio justified the associated development expense. In April 2005 the TF members were asked to vote for one of the candidates on the basis of the technical features offered, cost, and cooperation shown during the discussions. The candidate unanimously chosen by the TF was also unanimously approved by the Certification Committee. The company selected had had a full-fledged translation testing system in commercial operation since 1996. However, this system lacked certain security features that were considered indispensable by the TF and were to be developed by the company after the ATA had made a commitment, i.e., made a down payment.
The decision was communicated and a detailed report submitted to ATA's Board of Directors, which met on April 22-23, 2005. On this occasion the Board voted "to allocate $4,334 [one-third of the down payment required by the selected vendor] to fund the development of a working software system to offer the ATA certification exam on computer" (Source: minutes of the ATA Board meeting of April 22-23, 2005). During the same weekend, the technical features of the system were presented and the interface demonstrated at a meeting of Certification Language Chairs, with the presence of some Board members, whose input was requested. Based on the input by those present at the meeting and others, further discussions with the chosen provider ensued, and the technical features of the system were fine-tuned.
The results of the TF's work were presented in the above-mentioned June 2005 The ATA Chronicle article under my byline. Neither I nor the TF received any comments or suggestions in response to the article.
The latest proposal of the selected vendor, version 9, of July 18, 2005, seemed to incorporate all the features that were deemed necessary and cost-effective by those involved in the project. At this point, all members of the TF and the CC agreed that the technical part of the project had been successfully completed and all that was left to do was for the Board and the Executive Director to negotiate and sign a contract with the selected vendor. We planned to hold the first CCE trial sittings at the Seattle Conference in November 2005 and regular sittings starting in early 2006.
While working with the prospective vendors on the technical specs of the CCE, the TF also pursued part b) of the project to secure sites for the exams. By the time of the April Board meeting, positive answers had been received from nine potential sites, mostly universities, from different regions of the U.S., with several further sites showing a high probability of agreeing to host the ATA's CCE. The CCE project seemed to be on track and on schedule.
The first RFP
However, a series of events between June and July 2005 put the immediate future of the CCE project in jeopardy. Former ATA president Muriel Jérôme O'Keefe, with whom I had had a disagreement unrelated to the CCE project earlier that year, contacted the Board impugning the reputation of the vendor selected by the TF. She also claimed that the selected vendor was her competitor and suggested that, if ATA chose this vendor (which happens to be an ATA corporate member), it had the legal obligation to also consult other corporate members. The Board did not consult the TF, which had received only positive references about the vendor, including those from a TF member who had had direct business dealings with it, and failed to check whether the former president's company was in fact engaged in computerized testing. (Its website clearly showed that it was not.) Instead, without informing the TF about the complaint or attempting to learn the facts the TF had collected, the Board accepted her allegations at face value. After the Board had acted upon those allegations, President Scott Brennan consulted ATA's legal counsel about the former president's claim of legal restrictions on the Association doing business with one of its members. The counsel's response was:
"I don't have any policies about doing business with members. I don't think such policies are necessary unless there is a need. I don't think this one situation requires ATA to adopt a policy."
The way the former president's complaint was handled is but one example of the lack of communication between the TF and the Board. ATA Secretary Alan Melby, who was a member of both bodies, instead of acting as the liaison between them, in his statements and votes on the Board rejected the proposals he had voted for in his capacity as TF member and actively hindered communications between the Board and the TF. When I asked one of the Directors about specific actions taken by the Board at its July 2005 meeting (which I, as an active ATA member would have been entitled to attend and the minutes of which I would be able to read even if I hadn't been involved in the CCE project), Melby, in a message posted to the Board Listserv on August 9, 2005, warned the Director against providing me with the requested information or discussing the merits of the issue:
"I think you are about to step into a black hole—there will be no end to the back and forth with Gabe—he is treating you as his source of secret [emphasis mine] info about what really happened in Denver [at the July 2005 Board meeting]."
As a result, the TF was largely kept in the dark about the Board's deliberations concerning the CCE. I received virtually all the information about the Board's actions, which I'm describing here, via informal back channels, sometimes weeks after the fact. Conversely, Board members apparently knew little about or decided to ignore the TF's work despite the reports I submitted to the January, April, and July 2005 Board meetings and two invitations I sent to each individual Board member to join the TF's Yahoo Group as observers. Only two of them accepted the invitation; however, it is unclear to what extent they familiarized themselves with the over 1000 messages exchanged from November 2004 through July 2005 or the documents archived in the Group's Files section. Decisions on important aspects of the CCE project were made without regard for the facts that would be apparent from the TF's discussions.
Unbeknownst to the TF and apparently triggered by the former president's complaint, other objections to the TF's recommendation were also raised before and during the July 2005 Board meeting. One of these objections reportedly stemmed from the concern that the solution proposed by the TF would not ensure "ownership" of the process by ATA, jeopardizing the program's survival in the event cooperation with the selected vendor became impossible for any reason. The arrangements agreed upon between the vendor and the TF to address this concern (see more about this issue below) were ignored. Focusing their attention on the CCE software, while losing sight of the complex back-room hosting, support, and maintenance services that must be provided by trained staff to run it efficiently, the Board was made to believe that having the software developed specifically for ATA or otherwise ensuring the "ownership" of the software would automatically guarantee its proper operation in an actual testing environment and the program's continuity in the event the relationship between the ATA and the software provider was severed for any reason.
Finally, the Board was made to believe that a Request for Proposals (RFP) process controlled by two individuals would ensure more "openness" in identifying suitable candidates and secure a better deal for the ATA than did the systematic and meticulous search and selection, followed by exhaustive negotiations between the TF and the selected vendor with each and every member of the TF and the CC having access to all information, including each vendor's proposal.
Overriding the unanimous approval of the TF's vendor selection by the CC, the Board, in its July meeting held in Denver, CO, decided to issue an RFP for the CCE. Neither the TF nor the CC were consulted in advance and were not immediately informed of the Board's decision after it had been made. The enormous amount of information collected by the TF during eight months of research and discussions, some of which would have directly refuted the arguments for the RFP, was ignored.
Upon learning about the RFP decision, I sent each ATA Board member and officer an 1100-word message on July 31 and another 1700-word message on August 17 explaining why the RFP would be a waste of ATA's resources and might jeopardize the entire CCE project. I also had a long telephone conversation with ATA President Scott Brennan and contacted individual Board members by phone and/or e-mail. To my great frustration, I found almost all Board members reluctant to express their personal opinions on the issue or to engage in a discussion about the decision made in Denver. What I didn't know then is that Board members were just complying with the misguided "the Board speaks with one voice" policy (see about this further below).
In my messages to the Board I emphasized the difficulty, well documented in the TF's Yahoo message archives, in finding providers willing and able to supply, at a reasonable cost, both the software and the associated service required for the CCE. I believed then, as I believe now, that putting up a fixed set of specifications for bid in an RFP might be the proper procedure for a commodity like nuts and bolts, where the specifications are well-known in advance and are not changed after the bids are in. This was obviously not the case with the CCE, as later events clearly showed. For a complex and specialized service, in which only a handful of organizations have proven experience, and where the specs are subject to change as the project progresses, an approach such as followed by the TF of identifying those qualified and experienced testing organizations and working with them to adapt their existing systems to the ATA's requirements is more likely to yield the desired results.
In my last message to the Board members I said:
"Contrary to some allegations voiced at the [July 2005] Board meeting, the software in question is much more than a mere secure data transfer program; it involves a complex back-room management system, which is not apparent at the time of the exam, but whose proper design is essential for the smooth, secure, and confidential operation of passage archiving, updates, candidate sign-up, identification, passage assignment, exam text processing, etc. Attempting to have such software developed from scratch by an outfit that may not be familiar with testing procedures or languages, rather than using a proven and commercially operational system managed by an experienced, language-knowledgeable staff may prove disastrous both financially and technically... The testing software, and possibly the server on which it runs, must be operated and maintained by specialized personnel, a task which, according to [the Certification Program Manager], the current ATA HQ staff is not prepared to undertake. Outsourcing software and hardware management to third parties would defeat the purpose of ownership..."
Ignoring my arguments, the Board proceeded with the RFP. This left me no choice but to resign from the chairmanship of the TF. I did not, however, resign from my position on the ATA Certification Committee at that time and continued to follow the progress of the project.
The Board appointed a two-member team (Secretary Alan Melby and Certification Program Manager Terry Hanlen) to prepare and pursue the RFP. By the end of August, this team had prepared a draft RFP and submitted it to the Certification Committee (CC) and the Board for comments and suggestions. The TF, which after my resignation continued without a Chair, was sidelined.
The RFP, with the slight modifications suggested by Board and CC members, was sent out on September 1 to corporate ATA members, posted on websites of IT professionals such as the Software Developers Forum (SDF), American Society for Association Executive's Technology Section, Guru.com, which claims to have 492,000 members, and the Women in IT listserv, sent to organizations such as the Association of Test Publishers, and several individual companies. Over a million individuals and companies had the opportunity to see ATA's RFP, of which a grand total of twelve (12) responded by the set deadline of October 10. Two of those were immediately rejected for not complying with the terms of the RFP. This disappointingly low response rate to the RFP did not, however, discourage the selection team. A final selection of a provider was to be presented and approved at the Board Meeting in Seattle on November 12 or 13.
The second RFP
The Melby-Hanlen selection team was unable to choose a vendor by the time of the Seattle Board meeting, which was presided over by a new President, with a newly elected President-elect and Treasurer, as well as three new Board members in attendance. At the meeting, a motion to fund the CCE failed. Nevertheless, it was decided that the selection process would continue with the final decision to be made by ATA's Executive Committee; the Board and the CC would be allowed to comment, but not to vote, on the decision.
The Board also approved some fundamental changes in the specifications as proposed by the selection team. The first one of these changes involved Macintosh compatibility, which had been discussed by the TF and found desirable, albeit not essential. The second change concerned security, which now was to be provided by "spyware" to be incorporated in a specially designed browser. Access to the Internet (although not to all resources of the computer) would still be blocked, at least initially. To compensate for the relaxed security, a bank of four sets of three passages each (a total of 12 passages) would be used instead of a single set of three passages for each language combination during the exam year. No consideration was given to the fact that the new system would increase the graders' workload and ATA's costs for passage selection, preparation of model translations and Passage-Specific Guidelines (PSG), considered essential to ensure consistency of grading. The third change was waiving a key requirement of the TF and of the first RFP that no software be installed on the hard drives of the third-party computers (a requirement of several computer labs) as a trade-off for leaving the CD-ROM drive free and thus making future use of selected electronic dictionaries and spelling checkers possible.
Of these changes, the requirement of Mac compatibility was later abandoned. Although the CD drive was made available, at the time of writing (February 2015), I'm not aware of any plans to allow the use of the candidates' electronic dictionaries. Although access to some on-line websites has been allowed, the roster of "whitelisted" sites for all 26 language combinations is far from complete.
The increase in the number of passages prompted some graders to resign and, at one point, threatened the entire program with collapse. The new RFP containing the modified specs was issued, only to the ten remaining candidates, to be responded to by December 1.
On December 5, the members of the Certification Committee were shown a CCE interface prepared by a graduate student at Brigham Young University and asked to comment on it. This interface was essentially identical to that of the vendor selected by the TF, which had been demonstrated eight months earlier at the Language Chairs' meeting. It was intended as a model and its source code was to be shared with the vendor to be selected. On December 7, in a teleconference, the Board approved the two finalists chosen by the two-member team. Neither the Board nor the CC were shown copies of any of the actual proposals.
The CC was never told how many of the ten candidates remaining after the first RFP responded to the second RFP. On December 22, the selection team sent an e-mail to the members of the Board and the CC, ranking two finalists by price and "comfort level"—with one candidate being presented as the clear favorite—and requesting comments by the following day, December 23, a Friday, and the day before Christmas Eve. The actual proposals were not attached to the e-mail, which contained no technical specifications, information about the experience or identity of either prospective vendor or their proposed timetable for implementing the project. No information whatsoever was given about the other proposals, if there were any, submitted in response to the second RFP. From this e-mail it became clear, however, that the price proposed by the vendor preferred by the two-person selection team ("Vendor A") was approximately three times as high as the latest offer of the vendor selected by the now defunct TF (see table).
Extensive discussion & unanimous approval by Task Force & Certification Committee; submitted to language chairs at April 2005 meeting and to Membership in June 2005 Chronicle article
2-person team for selection. No technical info supplied to Certification Committee, language chairs, or membership.
Ongoing fee / year
Tech Support weekdays
$ 0 (included)
$ 125 /hr
$ 500/4-hr sitting
$ 2,500/yr (est.)*
Tech Support weekends/holidays
$ 300/4-hr sitting
$ 4,500/yr (est.)*
$ 720/4-hr sitting
$ 10,800/yr (est.)*
Tech Support total / year
Total Est. Yearly Cost
(ongoing + tech support)
Discussed and approved by Task Force, Certification Committee, Language Chairs.
No info provided
In language testing business since 1996. Existing system in commercial operation, only lacks security feature required by ATA. Sample interface tested by TF, CC.
No info provided
32 weeks from software scope definition to field testing
No info provided
Source Code license
Source code deposited in escrow to ensure continuity
Non-exclusive source code license (meaning??)
* Based on an estimated 20 certification sittings/year, of which 5 are on weekdays and
15 on holidays/weekends.
I called the attention of President Greenfield, President-Elect Stejskal, and Treasurer Krawutschke to this fact by e-mailing them the above table. The response I received from the President was that the decision had already been made, i.e., the request for Board and the CC members' comments on the eve of a major holiday was a mere formality. The three-fold price increase, I was told, reflected the additional features, which made it "money well-spent." Neither the CC, which would have to live with the results of the decision for many years to come, nor the Board, which has ultimate responsibility for the Association's finances, had had a voice in the decision.
I was also told that the data of the vendor selected by the TF (the middle column of the table) was no longer relevant because that vendor, having found the ATA "too difficult to work with," had raised its price in response to the RFP. However, the table shows the price ATA would be actually paying had the Board heeded my urging to accept the vendor's July 18, 2005 proposal. Since that proposal also includes the extra cost of the security module, which is now being replaced by a "lite" version and substantially less expensive "spyware," a comparison of the two proposals based on the same specifications would be even more favorable to the TF's choice.
The Source Code issue
(Source code is the editable code of a computer program, which makes it possible to modify the program at the same level it was originally written.)
The Board was rightly concerned, just as the TF had been, about the certification program's continuity in the event cooperation with the vendor became impossible at any time in the future. It was also aware of the need for future updates of the software without depending on third parties, to accompany technological changes or changes in the ATA program.
In its latest offer, the vendor selected by the TF agreed to the TF's request not only to have the source code held in escrow by a third party, but also "to provide to ATA a non-exclusive copy of the system they develop/customize for us, as long as we do not give it to their competitors." No TF member, including Secretary Melby, objected to this arrangement with the potential vendor. However, when Melby, in his new capacity as a member of the two-person selection team and acting without consulting the TF or the CC, came out with new demands, this vendor (understandably) refused to grant ATA the right to have its proprietary software modified by third parties, which is what Melby meant by using the vague wording of the RFP, which called for a "non-exclusive source code license."
The reason I was given for the offer of the vendor selected by TF not having been accepted was that it would be difficult to extract the source code held in escrow by a third party if the need arose. However, this argument does not hold water, since the problem could have been solved by a suitable choice of the third party and by suitable contractual arrangements for the release of the source code. The very assumption that full control over the software, including its source code, can ensure the continuity of the certification program independently of an experienced provider of CCE services is questionable, since it overrates the importance of the software and underrates that of management and service.
I think it is regrettable that the company selected by the TF, which had worked with us for six months fine-tuning the system's specs and revising its proposal eight times in the process, and whose position did not seem to be too far from the Board's, was simply discarded on this issue, instead of an attempt being made to negotiate a mutually acceptable solution.
The second CCE Task Force
On February 14, 2006, a new Task Force was formed under the chairmanship of Jost Zetzsche, a computer expert, translator, and member of the first TF. The first TF was not formally dissolved, nor were its members dismissed. No new members, besides those of the first TF, were recruited. Membership in the new TF was to be by appointment by the Head of the Task Force, upon the Certification Committee Chair's recommendation and subject to approval by the ATA President. In other words, while the first TF was open to any ATA member willing to help, TF-II was to be "protected" by a triple filter against members who might ask embarrassing questions. I was not invited to join. One of the most computer-savvy TF-I members, who had also expressed doubts about the Board's RFP decision, was hastily invited only after President Greenfield had read the first version of this article.
The mandate of the new TF was defined as a technical advisory body to the Certification Committee, which in turn was to report to the Board of Directors. The TF was to deal with details of the interface and the keyboards and help in recruiting testing sites. Its mandate did not include discussions on the broader issues of the CCE, such as vendor evaluation, cost versus benefits of features, best ways to ensure security, or making the system acceptable to administrators of potential testing sites. A member's request for a copy of the winning proposal (which in the meantime became a contract) remained unheeded. Not surprisingly, TF-II was virtually inactive from the day of its establishment. During its 3-year existence, 49 messages were posted on the TF-II Yahoo website. Compare this figure with 1092 messages during the 9 months between November 2004 and July 2005 in the TF-I group.
On February 17, 2006, the Certification Program Manager informed the TF and the CC that a contract had been signed "to develop a secure, commercial version of the software needed to offer our exam via computers." The company selected for providing this service is a software development company, with some experience in e-learning and training. According to an ATA officer, it also claims experience in computerized testing, although their website gives no indication of such experience. It was never involved in foreign-language computing. It is not a corporate member of the ATA. The prices this company is charging the ATA are basically those shown in the above table for "Vendor A," except that the one-time fee was raised from $41,500 to $42,700. ATA received no price reduction for the interface developed by the Brigham Young graduate student. The selected company was to deliver the final version of the system for ATA's approval in five months (i.e., by July 2006), which "should allow simulated exam sittings and, hopefully, an actual computerized sitting by the end of the year ."
On March 12 I gave ATA president Marian Greenfield the URL of the original version of this article and told her that I intended to publish it in the April 2006 issue of the TJ. At the same time, I asked her for corrections of any error of fact and for comments, which I promised to publish in the same issue. She forwarded the URL of the article to the Board and other individuals who had been involved in the CCE project. While she pointed out some subjective opinions expressed in the article, which I then corrected, neither she nor the other individuals were able to pinpoint any error of fact. She refused to allow her comments to be published.
As mentioned above, at the last minute I decided to withhold the publication of the article.
On April 22 I was removed from the Certification Committee. A few days later I announced that, after 25 years of service, I was taking an indefinite leave of absence as a grader. At the same time, having decided that I should attempt to correct ATA's flawed communication and decision-making processes from the inside, I announced my candidacy to the Board of Directors.
On July 6, as the deadline for delivery of the CCE software was fast approaching, ATA members received an e-mail broadcast, signed by President Greenfield, admitting that the project "has turned out to be a much longer and more complicated process than we had at first envisioned." While the message assured the membership that the project "is progressing nicely," there was no more mention of any time frame for the start of actual exams, or even a demo at the 2006 New Orleans Conference, but only of "a session ... dedicated to the CertSoft project [as the CCE was now referred to], consisting of a status report followed by a question-and-answer period." The provider that was contracted, which had no previous foreign-language experience, has now "assigned a project manager who speaks several languages and has experience in software internationalization."
ATA members were given an URL with a demo version of the client-side software, which, for all practical purposes, was identical to the one shown TF members in February, which in turn was basically the same as that of the vendor selected by the first TF and which was demonstrated to Board Members and Language Chairs in April 2005. It was unclear whether the "new" software was developed by the selected vendor based on the BYU graduate student's code. Recruitment of test sites was apparently on hold.
The view from within
On November 2, 2006, at the annual Conference of the ATA in New Orleans, I was elected to the Board of Directors of ATA. On the same day, Conference attendees were shown a version of the CCE software, labeled "alpha version," as a PowerPoint demo, since no Internet connection was available in the room where the session was held.
At my first Board meeting on November 4, 2006, the question of "out-of-scope" charges by the software vendor, i.e., charges for changes in the specs with respect to the second RFP, came up. Apparently, not only had the specs of the software been changed after the first RFP had been issued and the first proposals received as mentioned above, but further changes were made in the specs of the second RFP after the contract had been signed, giving rise to additional expenses. To my question whether the contract contained a provision imposing late delivery penalties payable by the vendor, which could offset these "out-of-scope" charges, I received an evasive answer with the information promised for the next meeting. At the January 2007 Board meeting we were told that the "out-of-scope" expenses at that point amounted to slightly over $4000, "negotiated down from the $10,000 originally requested." Obviously, contrary to the proposal negotiated by the TF-I, the contract negotiated by the Melby-Hanlen team and ultimately signed contained no late delivery penalty provision.
After the May 2007 Board meeting, some features of the beta version of the software were demonstrated to the Board. Board members were also told that trial sittings might possibly be started around the annual Conference in October-November 2007. The progress report submitted to the July 2007 Board meeting, one year after the final version of the software was to be delivered, only stated that a minor problem noticed during the April demo had been solved. Negotiations were being conducted with several potential sites to hold the first computerized exam sitting during the San Francisco Conference. Since these negotiations ultimately turned out to be unsuccessful, the first actual exams were scheduled for January 17, 2008, in Fairfax, VA (George Mason University). This sitting, for which nine candidates had signed up, was a total failure, and the exams had to be retaken on paper at a later date.
In November 2009, I was elected Treasurer of ATA as a petitioned candidate, which allowed me to gain further insight into the real cost of the keyboarded certification exam project. After the failed sitting in 2008, a few more sittings were held with variable success, and the software continued to be modified. The latest version I have information about was described in Melby's report to the January 2011 Board meeting. According to this report, the exam would require three classes of computers, one of them a "virtual" computer for each candidate, one actual computer owned by the candidate, and a remote server. This system turned out to be so complicated that only one person, ATA's Certification Program Manager, Jonathan Mendoza, understood and was able to run it. Mendoza resigned for personal reasons effective January 2015, leaving the keyboarded certification project on hold.
By the time I completed my second term as Treasurer of ATA in November 2013, the capitalized expenses (i.e., the expenses that were booked as purchase of capital goods and thus depreciated over a period of three years) amounted to $90,000, which does not include the amounts spent and booked as operating expenses since 2005. These expenses were never labeled or reported separately as pertaining to the keyboarded exam project, and I can only estimate that they amounted to more than $50,000. Thus, while the total expenditure has reached eight times the estimated first-year cost of the system proposed by the first TF, certification candidates still cannot count on a working keyboarded system. I've been told of suggestions made to scrap the entire project, at least in its latest form. Whether or not this happens, the money spent by ATA and the work done by many volunteers and Headquarters personnel in the past 10 years will go to waste.
Let me start by stating that most members of ATA's Board of Directors are talented and well-meaning individuals who work selflessly for the Association. Their work over the years has contributed to making ATA truly "the Voice of Translators and Interpreters." This article is not intended to criticize those individuals, but the culture that made the disastrous decisions on the CCE project I've described here possible, and which changed little from one president to the other during the ten years this article covers. During my term on the Board, while I refrained from being involved in the keyboarded exam project, I attempted to address the broader communication and decision-making issues that were responsible for those poor decisions. Thus, I proposed and the Board approved the repeal of the infamous "the Board speaks with one voice" policy, which was never sanctioned by the Board as all policies must be under the Bylaws, and which prevented the Board from effectively communicating with other ATA bodies, and Board members from freely exchanging ideas with other ATA members. I also advocated and initiated free and informal discussions of controversial subjects on the Board's Listserv, where such discussions are not subject to the time constraints of formal Board meetings.
Unfortunately, the current culture on the Board values "congeniality" above performance. If you are on friendly terms with the President, you may be admitted to his or her "inner circle," from which the important appointments are made. Volunteers are not held accountable for actually doing the job they signed up for; they are expected to conform, rather than to perform. Those who show independence are given unmistakable signs of disapproval and are denied committee positions where they could make a significant contribution. Free discussion of issues on the Board's Listserv is discouraged. The Board meeting agenda is received by the Board members a week before the meeting, not giving them sufficient time to familiarize themselves with the issues and discuss them on the Listserv prior to the Board meeting. This results in the tendency I mentioned previously of the Board more often than not uncritically rubber-stamping any proposals coming from the President or the Executive Committee. It also explains how, in clear violation of the Bylaws, the President was able to set policies without the Board's consent and without being challenged by Board members. I succeeded in having at least two of these policies (that of a single candidate for the Officers' positions and "the Board speaks with one voice" policy) repealed. I also called the Board's attention to the stipulations of the Bylaws regarding the authority and duties of the Board and the President. It is my firm belief that, in order for the Board to properly serve the Association, the balance between the powers of the President and the Board, as set forth by the Bylaws, must be restored. Board members must be more assertive and show independence and initiative in fulfilling their duties of
"hav[ing] the power and authority to manage the Association's property and to regulate and govern its affairs; to determine policies and changes therein; to specify and review the work of the elected officers..." [emphasis mine].
ATA Bylaws, Art. 4, Section 2,
"[t]he President shall be responsible for carrying out the policies and directives adopted or approved by the Board of Directors."
ATA Bylaws, Art. 5, Section 2, a).
I sincerely hope that this article will contribute to making the governance of ATA and similar organizations more effective and more democratic with the involvement not only of a Board fully aware of its responsibilities and unafraid of introducing innovations in the members' interests, but also of an informed and engaged membership that should hold the association's elected or appointed office-holders accountable for performing the job they signed up for.