Artificial Intelligence (AI) is revolutionizing the healthcare industry by enhancing diagnostic accuracy, streamlining administrative processes, and improving patient outcomes. From predictive analytics to personalized treatment plans, AI is enabling healthcare professionals to make more informed decisions and deliver better care.

However, when we talk about AI use cases in healthcare, we can divide them into two broad categories:

1. Clinical Text-Based Tasks and General-Purpose Multimodal Reasoning
2. Non-text healthcare data that includes medical imaging, EMR, signal, genomic etc.

We have seen how Azure OpenAI LLM models are revolutionizing healthcare by automating tasks, enhancing decision-making, and improving patient engagement. These models are not just theoretical tools—they are already being deployed in real-world clinical environments to solve some of the most pressing challenges in the industry. Be it writing a discharge summary to a conversational appointment scheduling and also for a multilingual patient interaction, Azure OpenAI LLMs models fits all. Even for running a diabetes campaign you can use the multimodal which can create amazing campaign images.

While text-based AI models have shown tremendous promise, the healthcare industry also deals with non-text multimodal data, such as medical imaging and specialized medical text like longitudinal electronic medical records. Here the Azure OpenAI LLM models struggle to understand and finds it challenging to process these non-text modalities. To address these challenges, Microsoft has released three specialized healthcare AI models in the Azure AI Model Catalog: MedImageInsight, MedImageParse, and CXRReportGen.

These AI models are specifically designed for healthcare organizations to rapidly build and deploy AI solutions tailored to their specific needs, all while minimizing the extensive compute and data requirements typically associated with building multimodal models from scratch. With these healthcare AI models, healthcare professionals have the tools they need to ha
ess the full potential of AI to assist patient care.

1. MedImageInsight:

MedImageInsight is an embedding model designed for sophisticated image analysis, including classification and similarity search in medical imaging. It can be used in various modalities such as radiology, pathology, ophthalmology, and dermatology. For example, researchers can build tools to automatically route imaging scans to specialists or flag potential abnormalities for further review, improving efficiencies and patient outcomes.

2. MedImageParse:

MedImageParse is designed for precise image segmentation across various imaging modalities, including X-rays, CT scans, MRIs, ultrasounds, dermatology images, and pathology slides. It can be fine-tuned for specific applications such as tumor segmentation or organ delineation, aiding in targeted cancer detection, diagnostics, and treatment planning. MedImageParse unifies tasks such as segmentation, detection, and recognition of relevant objects, through image parsing, by jointly conducting segmentation, detection, and recognition across numerous object types and imaging modalities. By applying the interdependencies among these subtasks—such as the semantic labels of segmented objects—the model enhances accuracy and enables novel applications. For example, it allows users to segment all relevant objects in an image, by using a simple text prompt. This approach eliminates the need to manually specify bounding boxes for each object.

3. CXRReportGen:

Chest X-rays are the most common radiology procedure globally. CXRReportGen is a multimodal AI model that generates report findings from chest X-rays. By incorporating current and prior images along with key patient information, this model highlights AI-generated findings directly on the images, aligning with human-in-the-loop workflows. It has demonstrated exceptional performance on the industry-standard MIMIC-CXR benchmark, accelerating tu
around times and enhancing diagnostic precision. This multimodal AI model generates report findings from chest X-rays, highlighting AI-generated findings directly on the images to align with human-in-the-loop workflows.

Getting Started:

You will find all the above 3 models in Azure AI Foundry-> Model Catalog under the “Health and Life Sciences” Industry section. You have to choose Managed Compute to deploy one of these models.

Further you can use the GitHub repo which has a built-in notebook

azureml-examples/sdk/python/foundation-models/healthcare-ai/cxrreportgen/cxr-deploy.ipynb at main · Azure/azureml-examples · GitHub

You can also leverage Microsoft lea
with the step-by-step details on how to deploy and consume this model as an online endpoint for real-time inference.

How to deploy and use CXRReportGen healthcare AI model with Azure AI Foundry – Azure AI Foundry | Microsoft Lea

Conclusion:

So, to effectively apply AI in healthcare, it’s important to choose the right model based on the nature of the task. For clinical text-based tasks and general-purpose reasoning, Azure OpenAI GPT models are the ideal choice, offering powerful capabilities in summarization, communication, and automation. On the other hand, for non-text data such as medical imaging, EMR, and other multimodal healthcare data, Microsoft’s specialized healthcare AI models are better suited to handle the complexity and precision required. Aligning your use case with the appropriate model category ensures more accurate results and greater impact in real-world clinical settings.

Note: The healthcare AI models are intended for research and model development exploration. The models are not designed or intended to be deployed in clinical settings as-is nor for use in the diagnosis or treatment of any health or medical condition, and the individual models’ performances for such purposes have not been established. You bear sole responsibility and liability for any use of the healthcare AI models, including verification of outputs and incorporation into any product or service intended for a medical purpose or to inform clinical decision-making, compliance with applicable healthcare laws and regulations, and obtaining any necessary clearances or approvals.

About the Author

Jaideep Roy

Technical Specialist in Microsoft| Presales Expertise in Data, Analytics & AI| Enabling Intelligent Cloud Solutions for Enterprise

Reference:

Roy, J (2025). Demystifying Microsoft Healthcare AI Models in Azure AI Foundry and Their Actual Use-Cases. Available at: (5) Demystifying Microsoft Healthcare AI Models in Azure AI Foundry and Their Actual Use-Cases | LinkedIn [Accessed: 7th August 2025].