Within Artificial Intelligence, programmers use the terms "Black Box" or "White Box". Explainable AI (XAI) is a "white box" model, and is artificial intelligence in which the results of the solution can be understood by humans. It contrasts with the concept of the "black box" in machine learning where the inner workings of the models are harder to understand and don't provide an estimate of the importance of each feature on the model predictions, nor explain how the different features interact to arrive at a specific decision.
Medical Decision Support Systems (MDSS) are computer-based programs that analyse data within a patient's healthcare records to provide questions, prompts or reminders to assist clinicians at the point of care, for example, it may requests some of the patient's data (height, weight, age, sex etc.) or symptoms (headache, rash, dizziness, breathlessness) or current treatment (drug(s) and dosage) and in response, propose a set of appropriate diagnoses or treatment plans.