In fiscal 2023, we conducted a technology validation for the appropriate use of generative AI in Digital Agency and administration.

Published:: May 13, 2024
Last Updated:: May 17, 2024

Based on the Priority plan (decided by the Cabinet on June 7, 2022) for the realization of the digital society, Digital Agency is investigating the appropriate use of AI in the public and private sectors after understanding the actual situation and trends of AI and identifying risks and necessary measures.

Recent technological innovations such as generative AI have made it possible to obtain various advantages. While monitoring the trend of such technologies, the government has repeatedly discussed the business use of generative AI by related ministries and agencies, such as the 10th Council for the Promotion of a Digital Society Secretariat .

Based on the history of the discussions so far, Digital Agency has conducted a technology validation for the appropriate utilization of generative AI since December 2023. We will disclose the overall content of the validation results in fiscal 2023, a sample of specific input sentences to generative AI obtained in validation, and a testing case used for some validation.

Purpose of Technical validation

We conducted a technical generative AI to obtain the following specific knowledge for future use of validation in government information systems.

Acquisition of knowledge on how generative AI should be used for administrative work
Identification of services that can be improved through the use of generative AI
Estimation of likely business improvement effects

Overview of Technical validation

Period and Eligible Persons

Period: From December 2023 to March 2024
Subject: Some employees of central government agencies including Digital Agency and some local government

Validation Content

We utilized a text-based generative AI and conducted the following two main validation.

Business improvement validation by an administrative official himself
A chat interface-type web application that can select multiple large-scale language models was provided to administrative officials (some officials of central government agencies including Digital Agency and some local government), and validation was made to see if it could contribute to the business improvement of administrative officials themselves.
Use cases for each specific validation
We have made a validation on how to use generative AI for each specific use cases.

Data, etc.

Environmental Development of Technology generative AI for Appropriate Utilization of validation in Administration (PDF / 15,949 kb) (updated May 17, 2024)
This document summarizes the outline of this validation, the results of the questionnaire, and detailed reports on each specific use cases.
Post Prompt List (CSV / 67 kb)
This is a list of prompts (input sentences to the large-scale language model) posted by the help desk of the contractor during the validation period. This is a document for those who can already use the large-scale language model (LLM).
Example of testing using AI with reference to a specific document (CSV / 1,125 kb)
This is a material that extracted an example of a validation conducted when a specific use cases was testing. In the text-based validation used in the Technical generative AI this time, we are currently considering how the testing case should be made and how far the testing should be made. For reference in future discussions, we have disclosed it without processing as much as possible.
Digital Agency Tech Blog
The findings obtained from this validation are presented in three parts.