Background: Hereditary breast and ovarian cancer (HBOC) is a major type of hereditary cancer. Establishing effective screening to identify high-risk individuals for HBOC remains a challenge. We developed a prototype of a chatbot system that uses artificial intelligence (AI) for preliminary HBOC screening to determine whether individuals meet the National Comprehensive Cancer Network BRCA1/2 testing criteria. Objective: This study’s objective was to validate the feasibility of this chatbot in a clinical setting by using it on a patient population that visited a hospital. Methods: We validated the medical accuracy of the chatbot system by performing a test on patients who consecutively visited the Kanagawa Cancer Center. The participants completed a preoperation questionnaire to understand their background, including information technology literacy. After the operation, qualitative interviews were conducted to collect data on the usability and acceptability of the system and examine points needing improvement. Results: A total of 11 participants were enrolled between October and December 2020. All of the participants were women, and among them, 10 (91%) had cancer. According to the questionnaire, 6 (54%) participants had never heard of a chatbot, while 7 (64%) had never used one. All participants were able to complete the chatbot operation, and the average time required for the operation was 18.0 (SD 5.44) minutes. The determinations by the chatbot of whether the participants met the BRCA1/2 testing criteria based on their medical and family history were consistent with those by certified genetic counselors (CGCs). We compared the medical histories obtained from the participants by the CGCs with those by the chatbot. Of the 11 participants, 3 (27%) entered information different from that obtained by the CGCs. These discrepancies were caused by the participant’s omissions or communication errors with the chatbot. Regarding the family histories, the chatbot provided new information for 3 (27%) of the 11 participants and complemented information for the family members of 5 (45%) participants not interviewed by the CGCs. The chatbot could not obtain some information on the family history of 6 (54%) participants due to several reasons, such as being outside of the scope of the chatbot’s interview questions, the participant’s omissions, and communication errors with the chatbot. Interview data were classified into the following: (1) features, (2) appearance, (3) usability and preferences, (4) concerns, (5) benefits, and (6) implementation. Favorable comments on implementation feasibility and comments on improvements were also obtained. Conclusions: This study demonstrated that the preliminary screening system for HBOC using an AI chatbot was feasible for real patients.
