Azure LLM Access Process
Introduction
Purpose
This document details the process to enable collaborators of the enterprise AI/ML team to access LLMs for coding assistants, research, or other purposes approved by the enterprise AI/ML team or by another relevant group (e.g. IRB, Clinical AI Council).
Scope
This document only covers resources provisioned and managed by the enterprise AI/ML team and available within the Children's National Azure tenant.
Available Models
| Model Name | Deployment Name | Stage | Release Date | EoL |
|---|---|---|---|---|
| GPT-6.1 Sol | gpt-6.1-sol |
Devtest | 2026-09-30 | N/A |
| GPT-6 Sol | gpt-6-sol |
Devtest | 2026-09-23 | N/A |
| GPT-6 Luna | gpt-6-luna |
Devtest | 2026-09-23 | N/A |
| GPT-5 Mini | gpt-5-mini |
Devtest | 2026-01-23 | N/A |
| Text Embedding | text-embedding-3-large |
Devtest | 2026-01-29 | N/A |
Process Overview
Proccess Map
The process map below provides an overview for how collaborators gain access to Azure-based LLMs.
Factors Considered for Approval
The following factors are considered as part of the approval process:
- Type and volume of data to be brought to the LLM.
- Computing environment where LLM APIs will be called from.
- What model(s) will be used (e.g. GPT-5, GPT-6)?
- Estimated cost and cost center to assign utilization to.
- Frequency of API calls.
- Technical knowledge and skills of team members using the LLM.
Model Maintenance
Models are expected to be maintained until a minor or major version update is released. At this point, the AI/ML team will make a determination based on the above factors as to whether to deploy that model. Unless necessary for research or approved other purposes, the older version of the model will be deprecated to reduce cost and/or maintenance burden. This will be communicated clearly to affected individuals, allowing two weeks for comments and feedback before deprecation procedures will be initiated via RITM.
Note
If you have a need for a model outside of what is currently deployed, please communicate with the AI/ML team via email at AI_ML_Advanced_Analytics@childrensnational.org
Azure Components
Azure Resources Involved and Purpose
| Azure Resource Type | CN-specific Resource Name | Purpose |
|---|---|---|
| AI Foundry Hub | aif-aiml-dev |
Azure AI Foundry is Microsoft's platform for developing Generative AI (GenAI) applications. A hub is analogous to a computing environment: it provides a logical container to provision related Azure resources (e.g. Key Vaults) to multiple underlying projects. |
| AI Foundry Project | aiml |
A Foundry project is intended to be used for a single use case. For researcher LLM access only a single project is managed/maintained. |
| Azure Private Endpoint | *Multiple -- individual resources will typically have thier own endpoint | Private Endpoints are used to bring Azure platform-as-a-service resources into a virtual network and completely off of the public internet. All resources use a private endpoint to ensure there is no public internet ingress to APIs and to ensure that traffic to/from the resources is aligned with our enterprise networking architecture. |
Azure Identity and Access Management Groups
Access to APIs are controlled primarily using Active Directory and API Management. Users are assigned to the EDP_Developer_AIMLCollaborator AD group for general access or may have another AD group assignment that has elevated access. That group is granted API permissions to a set of operations in APIM, allowing them to use LLM APIs as long as they're on the CNH network and logged in with their Microsoft account. An alternative, API key-based approach is available upon request.
Security Considerations
How will users access LLMs?
Users will primarily interact with the LLMs by executing code on personal compute, a virtual machine, or by calling the APIs from compute within our Enterprise Data Platform (EDP).
These compute platforms are generally secured by enforcing login with valid CN credentials as well as network isolation through CN's private networks (e.g. Azure VNets and VPN).
User training and awareness
User training is not required to use LLM APIs, but is recommended. The AI/ML team can provide instructions on how to set up access via HTTP requests using different programming languages OR can walk through the setup process for a supported application (i.e. Codex, SSMS, OpenCode). Training can be scheduled via the AI/ML Consultation Request form. Please provide detail on the workflow involving the LLM and the level of experience necessary for the training.
Networking considerations
Users will be interacting with LLMs that are secured through multiple mechanisms.
First, all LLM API in the AI Hub and Azure OpenAI use Azure private endpoints, which effectively remove the endpoints from the public internet and force network traffic to route using Azure's private network backbone.
Second, private endpoint ingress is controlled and managed such that requests must come from a trusted IP range (e.g., a VNet range) for the API to respond.
IAM Management
Users will be instructed that their access to the EDP_Developer_AIMLCollaborators AD group is time-limited and they will be removed from the group once they no longer need access to any LLMs. The Enterprise AI/ML team will have admin/management priveleges to the AD group and will be able to remove users via SailPoint.
API Limits
Cost risks will be managed primarily by attributing limits to the APIs. Azure has robust features for managing LLM cost utilization, for example by limiting the number of tokens that can be requested or setting global budget limits. These budgets can be set per AD group via API Management. Higher budgets can be allocated depending on the use case involving the API.
Cost Management and Attributions
Projects expected to incur $50/month or less in costs and without any contractual requirements for cost attribution will be attributed to the Enterprise Data and Analytics cost center (21840).
Larger projects, or projects with contractual requirements will need to provide cost attribution information (e.g., grant number) and an agreed process for cost attribution in place before users will be granted access. The Enterprise AI/ML team will manage the process of monitoring and reporting cost, utilization, and alerting users when they are expected to incur overages or going outside the expected/anticipated utilization.
Known Risks and Mitigation Plans
Over-provisioned access
Azure KeyVault does not allow for fine-grained controls. Once a user has read access to a KeyVault, they can read and interact with any secrets in the vault. The same applies to AI Foundry models where once a model is deployed, it becomes available to all users unless API Management controls are put in place. This creates a challenge where users can be over-provisioned for their use case and could potentially access LLMs and APIs that were not deployed for them.
The Enterprise AI/ML team plans to mitigate this risk by not sharing endpoint names across teams, and instructing users they are only authorized/approved to use a single endpoint for their use case. This more informal, trust-based approach, to management is preferred to the alternative of indivudal KeyVaults for each use case which requires additional management and different risks.