Unlock deep visibility and understanding of your AI workflows with LangSmith – a powerful platform for tracing, debugging, and monitoring LLM-powered applications and automations. Integrate LangSmith into your processes to capture detailed execution traces, log input/output data, attach metadata, and optimize your workflows through data-driven observability.
With the LangSmith integration, you can:
- Trace and debug runs: Forward workflow, tool, or model runs to LangSmith, record hierarchical execution details, and pinpoint bottlenecks or failures quickly.
- Attach rich metadata: Enrich your traces by logging inputs, outputs, tags, custom metadata, reasons for failure, and more for in-depth insight and analytics.
- Monitor workflow performance: Visualize executions, monitor error rates, durations, and success metrics over time to improve reliability and efficiency.
- Collaborate and audit: Enable team-based debugging and track changes, enabling transparent auditing and rapid iteration on chained LLM workflows.
- Automate observability: Seamlessly connect LangSmith traces to your workflow automations for always-on, effortless monitoring without manual instrumentation.
LangSmith empowers engineers, data scientists, and product teams to iterate faster, catch issues earlier, and build more robust LLM-based applications—whether you’re orchestrating agents, chains, or end-to-end workflows.
Drive better observability, actionable insights, and higher product quality by integrating LangSmith into your automated processes today.
Usage Instructions
Send run data to LangSmith to trace executions, attach metadata, and monitor workflow performance.
Actions
langsmith_create_run
Forward a single run to LangSmith for ingestion.
Input
| Parameter | Type | Required | Description |
|---|---|---|---|
apiKey | string | Yes | LangSmith API key |
id | string | No | Unique run identifier |
name | string | Yes | Run name |
run_type | string | Yes | Run type (tool, chain, llm, retriever, embedding, prompt, parser) |
start_time | string | No | Run start time in ISO-8601 format |
end_time | string | No | Run end time in ISO-8601 format |
inputs | json | No | Inputs payload |
run_outputs | json | No | Outputs payload |
extra | json | No | Additional metadata (extra) |
tags | json | No | Array of tag strings |
parent_run_id | string | No | Parent run ID |
trace_id | string | No | Trace ID |
session_id | string | No | Session ID |
session_name | string | No | Session name |
status | string | No | Run status |
error | string | No | Error details |
dotted_order | string | No | Dotted order string |
events | json | No | Structured events array |
Output
| Parameter | Type | Description |
|---|---|---|
accepted | boolean | Whether the run was accepted for ingestion |
runId | string | Run identifier provided in the request |
message | string | Response message from LangSmith |
langsmith_create_runs_batch
Forward multiple runs to LangSmith in a single batch.
Input
| Parameter | Type | Required | Description |
|---|---|---|---|
apiKey | string | Yes | LangSmith API key |
post | json | No | Array of new runs to ingest |
patch | json | No | Array of runs to update/patch |
Output
| Parameter | Type | Description |
|---|---|---|
accepted | boolean | Whether the batch was accepted for ingestion |
runIds | array | Run identifiers provided in the request |
message | string | Response message from LangSmith |
messages | array | Per-run response messages, when provided |
langsmith_update_run
Patch an existing LangSmith run with outputs, status, or timing once it completes.
Input
| Parameter | Type | Required | Description |
|---|---|---|---|
apiKey | string | Yes | LangSmith API key |
runId | string | Yes | ID of the run to update |
name | string | No | Corrected run name |
Output
This tool does not produce any outputs.
langsmith_get_run
Retrieve a single LangSmith run by ID.
Input
| Parameter | Type | Required | Description |
|---|---|---|---|
apiKey | string | Yes | LangSmith API key |
runId | string | Yes | ID of the run to retrieve |
Output
| Parameter | Type | Description |
|---|---|---|
id | string | Run ID |
runId | string | Run ID (alias of id, for consistency with other operations) |
name | string | Run name |
runType | string | Run type (tool, chain, llm, retriever, embedding, prompt, parser) |
status | string | Run status |
startTime | string | Run start time (ISO) |
endTime | string | Run end time (ISO) |
inputs | json | Run inputs payload |
outputs | json | Run outputs payload |
error | string | Error details, if the run failed |
tags | array | Tags attached to the run |
sessionId | string | Project (session) ID the run belongs to |
traceId | string | Trace ID |
parentRunId | string | Parent run ID |
totalTokens | number | Total tokens consumed by the run |
totalCost | string | Total cost of the run |
langsmith_create_feedback
Attach a score, correction, or comment to a LangSmith run.
Input
| Parameter | Type | Required | Description |
|---|---|---|---|
apiKey | string | Yes | LangSmith API key |
runId | string | Yes | ID of the run to attach feedback to |
key | string | Yes | Feedback metric name (e.g. "correctness", "user_score") |
score | number | No | Numeric score for the feedback metric |
value | string | No | Categorical value for the feedback metric |
comment | string | No | Free-text comment explaining the feedback |
correction | json | No | Corrected output for the run |
feedbackSourceType | string | No | Origin of the feedback (api, app, or model) |
Output
| Parameter | Type | Description |
|---|---|---|
id | string | Feedback ID |
key | string | Feedback metric name |
runId | string | ID of the run the feedback was attached to |
score | number | Score recorded for the feedback |
value | string | Categorical value recorded for the feedback |
comment | string | Comment recorded for the feedback |
createdAt | string | When the feedback was created (ISO) |