1. Background to the Introduction of External Evaluation
Since the importance of external evaluation was first advocated, The Institute of Social Science (ISS), University of Tokyo, has guided its administration on the premise that external evaluation should be conducted constantly. This concept of “ongoing external evaluation” implies that ISS’s research activities should be open to public scrutiny thereby incorporating criticisms and other feedback into the activities of ISS.
The Faculty Council of ISS on July 9 and 16 in 1998 further developed the concept of “ongoing external evaluation” into formal processes of evaluation and decided to implement a comprehensive evaluation by third parties for academic year 1999 (April 1999-March 2000). This new approach, “external evaluation as a process,” aims to “(1) declare ISS’s processes of self-evaluation (that is, the review of past achievements and the status of current programs, institutional reforms, clarification of the issues currently under debate and future direction) and submit them to evaluation by third parties; and (2) internalize the motifs of external evaluation in the course of its regular research activities.”
2. Outline of Implementation of External Evaluation
In July 1998, the Preparatory Committee for the Implementation of External Evaluation was set up within ISS and charged with establishing a plan for external evaluation and appointing suitable external reviewers. On November 26, 1998, the Faculty Council approved the committee’s proposed plan, as follows.
(1) Fundamental Philosophy and Aims
In pursuing its mission to promote comparative and coordinated research in the social sciences, the Institute of Social Science recognizes the great importance of keeping its overall program constantly open to public scrutiny, of improving its accountability to the Japanese public, and of actively soliciting critique of its activities and modus operandi. ISS has formalized this approach to the issue of evaluation under the principles of “ongoing external evaluation” and “external evaluation as a process.” “External evaluation as a process” represents an approach that aims to keep its activities constantly open to external scrutiny and critique as part of the evolving process of the activities themselves.
On the basis of the above mentioned “process approach,” the external evaluation currently being devised aims to solicit an intensive and comprehensive evaluation of ISS’s programs as a whole by third parties in accordance with ISS’s self-review and self-evaluation.
(2) Plan for Implementation
(a) Basic Method of Evaluation
- Two evaluative procedures shall be carried out, one for general evaluation and the other for specific evaluation.
- General evaluation shall assess the objectives, organization and activities of the Institute of Social Science as a whole.
- Specific evaluation shall assess institute-wide interdisciplinary Joint Research projects and the operations of the Information Center for Social Science Research on Japan.
- The ISS self-review and self-evaluation report (Shakai Kagaku Kenkyujo no genjo to kadai), which describes the status of ISS as a whole and the results of the specific evaluation mentioned in (iii) above, shall be submitted as the basic material for the general evaluation.
- For the evaluation of individual researchers’ research activities, in addition to the existing procedure for self-evaluation of performance, each researcher shall prepare a research progress report explaining his/her contribution as a staff member to overall ISS research, and how those research achievements have been shaped by virtue of the researcher’s being part of the ISS staff. These progress reports will be published in ISS self-review and self-evaluation reports.
- In the evaluation of individual researcher performance, self-review in accordance with the personnel administration system will be carried out parallel with external evaluation. Although such self-review does not itself constitute external evaluation, it will be included in the general evaluation as a feature of the personnel administration system. For all staff members who will have served for seven years or more as professors by the academic year 1999, performance will be assessed by external evaluation.
- Assessments of the Institute of Social Science will be solicited from overseas researchers, and these shall be used as secondary material for the general evaluation. A questionnaire survey will be conducted among overseas researchers who have worked at ISS to date as visiting scholars and associates so as to compile their assessments on the role ISS has played in their work. Overseas scholars who have been invited to date to ISS as visiting professors will also be requested to provide evaluations of ISS or opinions about the role it should play in the future.
(b) Organization of Evaluation Committees
- In the appointment of reviewers who constitute the Committee for General Evaluation, priority will be placed on ensuring that the Committee not only has in-depth knowledge of research in the social sciences but also can apply all-round expertise and discretion in gauging the direction of academic institutional change and the development of scholarship in Japan. Although overseas scholars may be appointed to the Committee, those who have spent time at the Institute of Social Science in the past as visiting professors shall not, in principle, be eligible for appointment to the Committee.
The Committee for General Evaluation (comprising up to 10 members) will make assessments and recommendations regarding all aspects of ISS activities and modes of operation, based on hearings and observations conducted at ISS and on documentary data consisting of ISS self-reviews and self-evaluation reports, the report by the two panels for specific evaluation, and the assessments and opinions of the overseas researchers surveyed.
- For the specific evaluations, two panels of experts will be organized to evaluate ISS-wide Joint Research projects and the operations of the Information Center for Social Science Research on Japan respectively. Membership to the expert panels will not be restricted to academic scholars.
(c) Composition of the Self-review and Self-evaluation Report
- The ISS self-evaluation report Shakai Kagaku Kenkyujo no genjo to kadai [Current Status and Tasks of the Institute of Social Science] elucidates the distinctive features, current problems and future vision of ISS.
(3) Other Measures
In accordance with the principle of “external evaluation as a process,” ISS will maintain a permanent committee for self-review and self-evaluation. In addition to reorganizing the existing system of self-review, ISS will review its overall nature and activities on the basis of the report submitted by the Committee for General Evaluation. ISS will further consider establishing a system whereby advice on management of its programs, future vision, and other issues may be gathered from scholars and opinion leaders outside ISS on an ongoing basis.
3. Organization of the Committee for External Evaluation
In accordance with the above plan, the ISS Preparatory Committee for the Implementation of External Evaluation appointed members to the Committee for External Evaluation, which was then formally established in April 1999 as the Committee for External Evaluation of the Institute of Social Science with the following eight members.
- Ishii Shiro, Professor, International Research Center for Japanese Studies (Law)
- Kitagawa Zentaro, Vice-director, Meijo University’s International Institute for Advanced Studies (Law)
- Takabatake Michitoshi, Professor, Surugadai University (Political Science)
- Teranishi Juro, Director, Institute of Economic Research, Hitotsubashi University (Economics)
- Yoshikawa Hiroyuki, President, University of the Air; Chair, Science Council of Japan (Engineering)
- Curtis, Gerald, Professor, Columbia University (Political Science)
- Kim, Young-ho, Dean, Graduate School of Business and Management, Kyungpook National University (Economics)
- Sako, Mari, Professor, University of Oxford (Management)
4. Composition of the Present Report
Since April 1999 ISS’s Committee for the Implementation of External Evaluation has carried out various practical tasks in accordance with its implementation plan. Its main task in that regard has been the compilation of the present report. The report, including appendices, shall be submitted to the Committee for External Evaluation as documentary material for reference in the evaluation process.
The report is made up of four parts, Part I being a broad overview, Part II providing more detailed information, Part III dealing with the research activities of ISS staff, and Part IV containing relevant reference materials. Under the headings “organization and administration,” “research activities,” “educational activities,” “international exchange activities” and the “Information Center for Social Science Research on Japan,” the report provides an account and analysis of the overall activities of the Institute of Social Science, primarily since its 1985 reorganization into divisions, and the direction of ISS development in the years ahead. Part I, which outlines the history of ISS and describes its current status and outlook, serves as a summary of the particulars provided in Part II. An overall understanding of ISS and its future goals may therefore be derived from Part I alone. Part III, on the research activities of ISS staff, contains the self-review reports by the individual researchers on their activities mainly over the past ten years.
As stipulated in the plan for the implementation of external evaluation, the following materials (two appendices and one appended report) will be submitted along with the present report to the Committee for External Evaluation:
- a summary of the findings of the expert panel organized to assess the institute-wide Joint Research projects;
- a summary of the results of the questionnaire survey of overseas researchers and a synopsis of data on overseas researchers who have conducted research at the Institute of Social Science; and
- a report by the expert panel organized to assess the operations of the Information Center for Social Science Research on Japan (to be submitted at a later date).