SecOps Logging - Databricks

Note

Databricks takes up to two hours to populate inference tables from serving AI models: https://docs.databricks.com/aws/en/generative-ai/agent-evaluation/monitoring-agent-framework#limitations

Admin-view user logs: https://learn.microsoft.com/en-us/azure/databricks/admin/account-settings/audit-logs
Admin-view system tables: https://learn.microsoft.com/en-us/azure/databricks/admin/system-tables/

All log tables are restricted to administrators only unless indicated otherwise in the Notes column:

Location	Source	Log	Type	Description	Method	Notes
Databricks	REST API	Model Build	Conda package downloads	Logs from building the conda environment in which the model runs, including any installed packages.	Build Logs
Databricks	REST API	Model Service	Container serving information	Logs from the container hosting the model.	Serving Logs
Databricks	???	Model Events	Deployment status	The current availability of a given model endpoint to receive queries	\<Unknown>
Databricks	AI Gateway	Model Monitoring	User Request	The text of the query submitted by an individual user	Inference Tables	>2 hour delay (see above), available to permissioned users
Databricks	AI Gateway	Model Monitoring	User Request Time	The time when a user submitted a request to a model	Inference Tables	>2 hour delay (see above), available to permissioned users
Databricks	AI Gateway	Model Monitoring	Response Latency	The speed at which the model was able to respond to a request measured in milliseconds	Inference Tables	>2 hour delay (see above), available to permissioned users
Databricks	AI Gateway	Model Monitoring	Error Logging	Any issues encountered with performing inference on a given request are logged	Inference Tables	>2 hour delay (see above), available to permissioned users
Databricks	AI Gateway	Model Monitoring	Model Traces	The MLflow model trace represented in JSON (includes request, documents, response, and metadata) for a given model interaction	MLflow Trace Search	>2 hour delay (see above), available to permissioned users
Databricks	Audit	Network Access	Access Time	When the outbound network access queries occurred	System Tables - Outbound Network
Databricks	Audit	Network Access	Destination Name	Where outbound network access queries were destined	System Tables - Outbound Network
Databricks	Audit	Network Access	DNS (if destination was DNS)	Which DNS destination was used to perform an outbound network access query	System Tables - Outbound Network
Databricks	Audit	Network Access	Storage Location (if destination was Storage)	Which Storage destination was desired by an outbound network access query	System Tables - Outbound Network
Databricks	Audit	Job Events	Creation Time	The time at which a job was created	Audit Logs - Jobs
Databricks	Audit	Job Events	Deletion Time	The time at which a job was deleted	Audit Logs - Jobs
Databricks	Audit	Job Events	Created By	The user that created a given job	Audit Logs - Jobs
Databricks	Audit	Job Events	Deleted By	The user that deleted a given job	Audit Logs - Jobs
Databricks	Audit	Job Events	Start Time	The time at which a job was run	Audit Logs - Jobs
Databricks	Audit	Job Events	Job Status	The success or failure status of a job	Audit Logs - Jobs
Databricks	Audit	Job Events	Job Permissions	Triggered when a user attempts to change a job's permissions	Audit Logs - Jobs
Databricks	Audit	Groups Events	Add to group	Records when admins add a user to a group	Audit Logs - Groups
Databricks	Audit	Groups Events	Create group	Records when admins create a group	Audit Logs - Groups
Databricks	Audit	Groups Events	Remove group	Records when admins remove a group	Audit Logs - Groups
Databricks	Audit	Secrets Events	Create secret scope	Event where user creates a secret scope	Audit Logs - Secrets
Databricks	Audit	Secrets Events	Delete secret scope	Event where user deletes a secret scope	Audit Logs - Secrets
Databricks	Audit	Secrets Events	Get secret	Event when user requests value of a secret	Audit Logs - Secrets
Databricks	Audit	Secrets Events	Put secret	Records when user writes the value of a secret to a scope	Audit Logs - Secrets
Databricks	Audit	Model Serving Events	Create model endpoint	Logs creation of a model serving endpoint	Audit Logs - Real Time Serving
Databricks	Audit	Model Serving Events	Update model endpoint access	Shows the result of permissions changes to a given model serving endpoint	Audit Logs - Real Time Serving
Databricks	Audit	Model Serving Events	Delete model endpoint	Logs delete of a model serving endpoint	Audit Logs - Real Time Serving
Databricks	Audit	Model Serving Events	Enable model endpoint	Shows when a user enables a model serving endpoint for requests	Audit Logs - Real Time Serving
Databricks	Audit	Model Serving Events	Disable model endpoint	Shows when a user disables a model serving endpoint from receiving requests	Audit Logs - Real Time Serving
Databricks	Audit	Model Serving Events	Update model endpoint	Shows when a user updates a model serving endpoint to use a different model	Audit Logs - Real Time Serving