Azure Data Explorer is Microsoft's analytics service for very large volumes of machine-generated data — logs, metrics, traces, telemetry, and IoT events. It is built for questions asked over billions of rows: you write a query, and it comes back in seconds. The same engine powers Fabric Eventhouse, Azure Monitor, and Application Insights.
You query it with KQL (Kusto Query Language), a pipeline language that reads left to right. Start with a table, then pipe the rows through operators:
StormEvents| where StartTime > ago(7d) and State == "FLORIDA"| summarize Events = count() by EventType| top 10 by Events
Azure Data Explorer also has a second command family: management commands, which all start with a dot (.show tables, .create table, .ingest inline). Queries read data; management commands inspect and change the cluster itself.
In Sim, this integration gives your agents both halves:
Ask questions of your telemetry — turn a plain-English question into KQL, run it, and answer with real numbers instead of a guess
Discover the data model first — list databases, tables, and stored functions, read a table's schema, and check its size and row count, so a generated query references columns that actually exist and you know what it will scan
Push rows in — send small batches straight into a table, or materialize a query result into a rollup table with .set-or-append
Manage tables — create a table from a column schema, or drop one you no longer need
Debug the pipeline — list ingestion failures with their error codes and root causes, and check the state of a long-running operation
Run any management command — the escape hatch for policies, mappings, and anything else on the control plane
Authentication uses a Microsoft Entra service principal (an app registration with a tenant ID, client ID, and client secret) rather than an interactive sign-in, so scheduled and unattended workflows keep working without anyone logging in. Grant that principal access to the database with .add database <DATABASE> viewers ('aadapp=<clientId>;<tenantId>') — use viewers for read-only agents, and ingestors or users only when a workflow needs to write.
A few things worth knowing before you build:
Enable Read-only on the Run Query operation whenever an agent writes its own KQL. It sends the x-ms-readonly header, and the cluster then refuses anything that would change data — a cheap guardrail against a generated query doing more than you intended.
Results are capped at 10,000 rows. Every result reports rowCount, totalRowCount, and truncated, so a query that returned more than the cap says so rather than quietly looking complete. Aggregate with summarize or bound the query with take instead of pulling raw rows.
Ingest Rows Inline is for small batches. It is ideal for tens or hundreds of rows from a workflow run. For continuous or high-volume loading, use Azure Data Explorer's queued or streaming ingestion instead.
Ingest From Query defaults to set-or-append, which adds to an existing table. set-or-replace discards everything already in the target table — pick it only when you mean to rebuild the rollup from scratch. For a large backfill, turn on the background option and poll Show Operations with the operation ID it returns.
Ingest From Query matches columns by position, not by name. Kusto aligns the query result to the target table on column type and order, so a query that projects the right columns in the wrong order ingests data into the wrong columns without erroring. End the query with an explicit project in the table's column order, and confirm with Show Table Schema first.
Drop Table is permanent. It deletes the table and its data. Give an agent the viewers role rather than admins unless a workflow genuinely needs to change schema.
Run Kusto Query Language queries against Azure Data Explorer and Fabric Eventhouse clusters, discover databases, tables, and schemas, push small batches of rows inline, and run management commands. Authenticates with a Microsoft Entra service principal using client credentials, so no interactive sign-in is needed.
Run an Azure Data Explorer management command (a control command starting with ".") such as .show, .create, .alter, or .drop. Write commands change cluster state permanently; use the Query operation for reads.
Read the column schema of an Azure Data Explorer table in CSL form (e.g., "Timestamp:datetime,Level:string"). Use this before writing a KQL query against an unfamiliar table.
Read size, row count, hot-cache footprint, and effective policies for a table — or for every table in the database when no table is given. Use it to see how much data a table actually holds before querying it.
List the stored functions in an Azure Data Explorer database, with their parameters and bodies, so an agent can reuse existing logic instead of rewriting it.
Push rows directly into an Azure Data Explorer table with .ingest inline. Data is parsed as CSV against the table schema unless an ingestion property says otherwise. Intended for small batches — use queued or streaming ingestion for production volumes.
Materialize the result of a KQL query into a table with .set, .append, .set-or-append, or .set-or-replace. Use this to build rollup or summary tables instead of pushing rows from a workflow. Kusto matches the query result to the target table by column type and position, NOT by column name, so project the columns in exactly the table's order or the data lands in the wrong columns.
Microsoft Entra tenant ID hosting the service principal
clientId
string
Yes
Microsoft Entra application (client) ID
clientSecret
string
Yes
Microsoft Entra application client secret
resource
string
No
Token audience override. Defaults to the cluster URI itself
database
string
Yes
Database containing the target table
table
string
Yes
Table to ingest the query result into
mode
string
No
set (create, fail if it exists), append (add to an existing table), set-or-append (default), or set-or-replace (replace all data)
sourceQuery
string
Yes
KQL query whose result becomes the ingested data (e.g., LogsTable | where Level == "Error" | where Timestamp > ago(1h)). Project the columns in the target table's order — matching is positional, not by name
async
boolean
No
Return immediately with an OperationId and keep ingesting in the background. Check progress with Show Operations
ingestionProperties
string
No
Optional ingestion properties clause contents, e.g. distributed=true, tags='["daily"]'
Create a table in an Azure Data Explorer database from a CSL column schema. Succeeds without changing anything if a table of the same name already exists.
List ingestion failures recorded for a database, with the failing table, error code, root cause detail, and whether the failure is permanent or transient. Failures are retained for 14 days.
Check the state of administrative operations on a cluster, such as an async ingestion. Given an operation ID it returns that operation latest update; with no ID it returns the operations from the last two weeks.