Skip to main content

This page has been translated using TexTra by NICT. Please note that the translation may not be completely accurate.
If you find any mistranslations, we appreciate your feedback on the "Request form for improving the automatic translation ".

The 2nd Expert Meeting on Policy Evaluation in Digital Agency in fiscal year 2022 (2022)

Overview

  • Date and Time: Thursday, July 21, 2022 (2022) from 10:00 to 11:46
  • Location: Online
  • Agenda:
    1. Opening
    2. Introduction of members
    3. Proceedings
      1. Cooperation between Policy Evaluation and Administrative Project Review
      2. Preparation of the Preliminary Analytical Table for Digital Agency Policy Evaluation for Fiscal 2022
      3. Comprehensive Evaluation Method
      4. Promoting EBPM in Digital Agency
      5. Other (future plans)
    4. Exchange of opinions
    5. Adjournment

Materials

References

Summary of proceedings

Date

Wednesday, July 21, 2022 (2022), from 10:00 to 11:46

Location

Held online

Attendees

Member

  • Naoko IWASAKI (Professor, local government Institute of E-Government, Waseda University)
  • Ryu Kanbayashi (Professor, Institute of Economics, Hitotsubashi University)
  • SASASHIMA Munehiko (Professor, Graduate School of Information Science, University of Hyogo)
  • ICHIRO SATO (Professor, National Institute of Informatics)
  • Junko Omogawa River (Professor, Faculty of education, Saitama University)

Digital Agency

  • Yasuichiro Tomiyasu (General Manager of Strategy & Organization Group Group)
  • Kazunori Yamamoto (Deputy Director of Strategy & Organization Group)
  • Hirohira Yoshida (Councilor Strategy & Organization Group)

Secretariat

  • Toshiyuki Ohira (Planning Officer of Strategy & Organization Group)
  • Eiji Amari (Deputy Director, Strategy & Organization Group)

Minutes

The main opinions are as follows.

  • In your opening remarks, you talked about moving away from the so-called myth of infallibility. Probably, such policy evaluation started from the idea of New Public Management, NPM. Basically, NPM is important in the administration in terms of how to handle fields that are quite similar to those of private sector companies. Of course, the part that overlaps with private sector could fail. Therefore, I think it would be good to create a new administrative idea here, a mechanism that does not stick to so-called infallibility, in which the system of policy evaluation itself could fail in administrative activities.
  • In order to conduct policy evaluation, it is important to determine whether or not the indicator is suitable for policy evaluation.
    For example, are the outputs and outcomes appropriate? I think that if we don't do that now, we will have a hard time later.
  • Regarding the management of unified information system training, the output indicators are good, but the number of people who completed the group training of the outcome indicator seems to be an output. As an outcome, it is probably not appropriate as an outcome indicator unless we say that people with such abilities will do such work.
  • With regard to the My Number, I believe that the spread of the My Number itself is very important. Although specific goals such as enhancing safety and reliability are included, public relations are not listed as a specific indicator. Could it be possible to include items such as trust in the My Number, recognition of what the My Number can do, how the people recognize it, and effective public relations to enhance it?
  • With regard to the unified information system training, I think that there is not only the number of people who have completed it, but also some goals that we are originally aiming for. I think that by setting goals such as the percentage of people who have reached this level of understanding, it can be made into an outcome indicator.
  • From the perspective of human resource development, whether or not a person has taken a course cannot be said to be an outcome.
  • The outcome is how the developed human resources are reflected in the production process and productivity. The most important point is that the unified training for information systems will provide time for development in Digital Agency, shorten the time, or create good things, and that the whole policy will move forward and the people will be happy. This is probably the largest series of policies.
  • One point is whether it can be incorporated into specific indicators, but I think the current situation in Issue is that this cannot be thought of immediately.
  • This time, the main measures of Policy Evaluation and the projects subject to Project Review in Digital Agency are basically one to-one. Originally, Policy Evaluation and Administrative Project Review have different purposes and viewpoints. Therefore, it is necessary to view the same items from different viewpoints.
  • The spread of My Number is quite difficult. The reason why it is difficult is that there are two stories, the spread of My Number itself and the spread of My Number Card. The biggest problem is that it is promoted in cooperation with other ministries and agencies. For example, it is correct for the government as a whole to list the spread of My Number and My Number Card as its goals. However, as an evaluation of Digital Agency's policies, it is a little different, or it is related to other ministries and agencies's policies. I do not deny that Digital Agency will list such overall goals, but at the same time, it is necessary to properly create indicators for Digital Agency's policies and initiatives.
  • Regarding the promotion of My Number System, I have heard that the issuance rate of cards has recently increased rapidly. It can be judged that the increase in the number of issues of this indicator this time has greatly contributed to the promotion policy, and the evaluation is high. As a future evaluation method, it is necessary to evaluate whether or not it is user-friendly and truly convenient for the people. Therefore, it is desirable to set indicators such as the convenience of Mynaportal, the utilization rate, and the ease of use by residents in the evaluation items of the Administrative Project Review Sheet, in addition to the number of administrative procedures to be performed using Mynaportal.
  • The Government of Japan is working on various measures to ensure that it reaches the entire population by the end of the next fiscal year. Related ministries and agencies are working together to implement Individual Number Card Point projects in Ministry of Internal Affairs and Communications and health insurance cards in the Ministry of Health, Labor and Welfare by the end of the current fiscal year.
  • It is better to organize in advance to what extent the policy objectives, outputs, and outcome indicators will be extended and to what extent they will be set. In other words, if we limit ourselves to what can be done in Digital Agency, the ultimate goal of the policy on My Number cannot be evaluated. This is already clear, so it is better not to manage the policy on My Number Card or My Number only in Digital Agency, or set the outcome and evaluate it there.
  • It would be misleading to draw a distinction between what is ultimately targeted by policy and what can be achieved as an effort in Digital Agency, and to state explicitly that policy evaluation in Digital Agency is the latter.
  • In light of the current situation in which digitalization is lagging behind other countries, if we realize a world-class digital society, it is recognized that the relationship between how much My Number Card is issued is already quite remote. If the final outcome and the interim outcome are determined in the overall government plan, Digital Agency will not touch on this part. If we say at the beginning that the outcome to which Digital Agency can contribute will be used as an outcome indicator for policy evaluation, and consider what can be used there, it will be possible to avoid a dogfight.
  • I am aware that Ministry of Internal Affairs and Communications has set a theme for such cross-sectional efforts every fiscal year. The theme of the My Number may be the spread rate of cards.
  • It is possible to assume that the outcome is actually like this, but there is a problem that there are no specific indicators to measure it. Therefore, I think that there may be cases where the outcome indicators are a little earlier stage, project plan figures, or blank.
  • Since the outcome indicator is the ultimate purpose of the policy, it does not come out from such operational data. How many people used it, how many times it was used, and how long it took are, in that sense, only indirect indicators of the policy goal. As a result, I think that whether people have become happy or economic activity has become active should be the outcome.
  • We may have to go and get the outcome indicators. It is not information that comes out on its own, so in the case of this administrative review, in that sense, I think it is basically the case that we look only at the output without sticking to the outcome, and run the PDCA cycle. I think it will be in the form of attaching a sensor and checking whether it is working well by looking at the numerical values from the sensor. How many people used it, how many times it was used, and how many minutes it took will be important outputs, but in terms of whether it is achieving the policy goals, I think the outcome will be to ask each and every user, whether the satisfaction level of the area with public service has increased, and whether the number of business operators who have entered the region has increased as a result.
  • We also have to think about the premise that we can't go and collect data from EBPM.
  • In the end, if it is only indicators that can be measured, it is very doubtful whether Digital Agency will be able to evaluate the measures that we will take in the future. However, there are no good ideas, and it is very difficult.
  • When looking at local measures for policy formation and evaluation of agile type, in many cases they have been advanced in a waterfall type. I think it will lead to a story of infallibility myth, but in order to make sure there is no mistake, I feel uneasy whether I will change the way of taking one step for sure, then taking another step after taking the whole confirmation to agile type and get used to it.
  • Digital Agency hopes to put EBPM into practice as one of the measures subject to policy evaluation this time.
  • It is necessary to review the current outcome indicators, but among these measures, I think that the unified information systems training is most suitable for EBPM. There are two reasons. One is that EBPM on human resource development is already being carried out in various countries around the world, and there is a framework that can be referred to to some extent. The other is that when measuring the outcome, I think that something can be done in Digital Agency. If we proceed in the form of design the logic model a little more properly, measuring the outcome properly, and determining the effect of the training, I think that evaluation by EBPM may be possible.

End