Azure AI Projects client library for JavaScript - version 2.11.0

The AI Projects client library is part of the Microsoft Foundry SDK, and provides easy access to resources in your Microsoft Foundry Project. Use it to:

  • Create and run Agents using the .agents property on the client.
  • Enhance Agents with specialized tools:
    • Agent Memory Search (Preview)
    • Agent-to-Agent (A2A) (Preview)
    • Azure AI Search
    • Bing Custom Search (Preview)
    • Bing Grounding
    • Browser Automation
    • Code Interpreter
    • Computer Use
    • File Search
    • Function Tool
    • Image Generation
    • Microsoft Fabric (Preview)
    • Model Context Protocol (MCP)
    • OpenAPI
    • Microsoft SharePoint (Preview)
    • Web Search (Preview)
    • Tool Search
  • Get an OpenAI client using the .getOpenAIClient. method to run Responses, Conversations, Evals and FineTuning operations with your Agent.
  • Manage hosted agent sessions and files using the .agents operations on a deployment that supports sessions.
  • Manage skills (preview) for reusable agent capabilities, using the .beta.skills operations.
  • Manage toolboxes for grouping tools into reusable collections, using the .toolboxes operations.
  • Manage memory stores (preview) for Agent conversations, using the .beta.memoryStores operations.
  • Manage routines (preview) for scheduling and dispatching automated workflows, using the .beta.routines operations.
  • Manage model versions (preview) for creating, updating, and managing custom model versions, using the .beta.models operations.
  • Explore additional evaluation tools (some in preview) to assess the performance of your generative AI application, using the .evaluationRules, .beta.evaluationTaxonomies, .beta.evaluators, .beta.insights, and .beta.schedules operations.
  • Run Red Team scans (preview) to identify risks associated with your generative AI application, using the .beta.redTeams operations.
  • Fine tune AI Models on your data.
  • Enumerate AI Models deployed to your Foundry Project using the .deployments operations.
  • Enumerate connected Azure resources in your Foundry project using the .connections operations.
  • Upload documents and create Datasets to reference them using the .datasets operations.
  • Create and enumerate Search Indexes using the .indexes operations.

The client library uses version v1 of the Microsoft Foundry data plane REST APIs.

Product documentation | Samples | Package (npm) | API reference documentation | SDK source code

Table of contents

Getting started

Prerequisite

Authorization

  • An Entra ID is needed to authenticate the client. Your application needs an object that implements the TokenCredential interface. Code samples here use DefaultAzureCredential. To get that working, you will need:
    • An appropriate role assignment. see Role-based access control in Microsoft Foundry portal. Role assigned can be done via the "Access Control (IAM)" tab of your Azure AI Project resource in the Azure portal.
    • Azure CLI installed.
    • You are logged into your Azure account by running az login.
    • Note that if you have multiple Azure subscriptions, the subscription that contains your Azure AI Project resource must be your default subscription. Run az account list --output table to list all your subscription and see which one is the default. Run az account set --subscription "Your Subscription ID or Name" to change your default subscription.

Install the package

npm install @azure/ai-projects dotenv

Key concepts

Create and authenticate the client with Entra ID

Entra ID is the only authentication method supported at the moment by the client.

To construct an AIProjectsClient, the projectEndpoint can be fetched from projectEndpoint. Below we will assume the environment variable FOUNDRY_PROJECT_ENDPOINT was defined to hold this value:

import { AIProjectClient } from "@azure/ai-projects";
import { DefaultAzureCredential } from "@azure/identity";

const projectEndpoint = process.env["FOUNDRY_PROJECT_ENDPOINT"] || "<project endpoint string>";
project = new AIProjectClient(projectEndpoint, new DefaultAzureCredential());

Preview operation groups and opt-in feature flags

Some preview operations require an explicit foundryFeatures opt-in flag. For example:

await project.agents.createVersion(
  "preview-agent",
  {
    kind: "workflow",
  },
  { foundryFeatures: "WorkflowAgents=V1Preview" },
);
for await (const rule of project.evaluationRules.list()) {
  console.log(rule.id);
}

Preview operation groups include .beta.voiceAgents, .beta.skills, .beta.memoryStores, .beta.routines, .beta.models, .beta.evaluationTaxonomies, .beta.evaluators, .beta.insights, .beta.schedules, and .beta.redTeams. Voice Agents have both GA and preview features; see Voice Agent operations for their opt-in requirements.

Use .agentInsightMonitors to list the monitors that analyze your agents:

for await (const monitor of project.agentInsightMonitors.list()) {
  console.log(`${monitor.id}: ${monitor.agent_name}`);
}

Examples

Performing Responses operations using OpenAI client

Your Microsoft Foundry project may have one or more AI models deployed. These could be OpenAI models, Microsoft models, or models from other providers. Use the code below to get an authenticated OpenAI from the openai package, and execute a chat completions call.

Run the code below. Here we assume deploymentName (str) is defined. It's the deployment name of an AI model in your Foundry Project. As shown in the "Models + endpoints" tab, under the "Name" column.

See the "responses" folder in the package samples for additional samples, including streaming responses.

const openAIClient = project.getOpenAIClient();
const response = await openAIClient.responses.create({
  model: deploymentName,
  input: "What is the size of France in square miles?",
});
console.log("response = ", JSON.stringify(response, null, 2));
const detailResponse = await openAIClient.responses.create({
  model: deploymentName,
  input: "And what is the capital city?",
  previous_response_id: response.id,
});
console.log("detailed response = ", JSON.stringify(detailResponse, null, 2));

Performing Agent operations

The .agents property on the AIProjectsClient gives you access to all Agent operations. Agents use an extension of the OpenAI Responses protocol, so you will likely need to get an OpenAI client to do Agent operations, as shown in the example below.

For setup, see Quickstart: Set up Microsoft Foundry resources.

const openAIClient = project.getOpenAIClient();
const agent = await project.agents.createVersion("my-agent-basic", {
  kind: "prompt",
  model: deploymentName,
  instructions: "You are a helpful assistant that answers general questions",
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);
const conversation = await openAIClient.conversations.create({
  items: [
    { type: "message", role: "user", content: "What is the size of France in square miles?" },
  ],
});
console.log(`Created conversation with initial user message (id: ${conversation.id})`);
// Generate response using the agent
console.log("\nGenerating response...");
const response = await openAIClient.responses.create(
  {
    conversation: conversation.id,
  },
  {
    body: { agent: { name: agent.name, type: "agent_reference" } },
  },
);
console.log(`Response output: ${response.output_text}`);
// Add a second user message to the conversation
console.log("\nAdding a second user message to the conversation...");
await openAIClient.conversations.items.create(conversation.id, {
  items: [{ type: "message", role: "user", content: "And what is the capital city?" }],
});
console.log("Added a second user message to the conversation");
// Generate second response
console.log("\nGenerating second response...");
const response2 = await openAIClient.responses.create(
  {
    conversation: conversation.id,
  },
  {
    body: { agent: { name: agent.name, type: "agent_reference" } },
  },
);
console.log(`Response output: ${response2.output_text}`);
// Clean up
console.log("\nCleaning up resources...");
await openAIClient.conversations.delete(conversation.id);
console.log("Conversation deleted");
await project.agents.deleteVersion(agent.name, agent.version);
console.log("Agent deleted");

Voice Agent operations

Use project.agents to create and manage voice agents. Choose the connection client that matches your agent:

  • project.voiceAgents.realtime for Realtime and cascaded voice agents.
  • project.beta.voiceAgents.live for GPT Live (preview).

The old project.beta.voiceAgents.realtime path is a deprecated compatibility alias for the same client. Use the root path in new code. Ordinary Realtime connections are GA and do not send a VoiceAgents preview opt-in by default.

Scope API or feature VoiceAgents=V1Preview
GA Realtime/cascaded definitions, delegate subagents, and phone system tools through project.agents Not required
GA project.agents.createFromPrompt (agents:generate) Do not send
GA All 14 stored Realtime conversation operations on project.voiceAgents.conversations; Realtime protocol and OpenAI Realtime models Not required
GA TPE/Twilio telephony and call-job list/get operations on project.voiceAgents.telephony Not required
Conditional Realtime connections to an agent using inline handoff, SIP transfer destinations, or post_call Pass foundryFeatures to connect()
Conditional Telephony binding/transfer/call writes and call-job create/cancel/delete on project.voiceAgents.telephony Pass foundryFeatures only when using SIP
Preview GPT Live definitions, delegation, sessions, stored conversations, and models Required; native Live and project.beta.voiceAgents.liveConversations supply it internally
Preview Inline handoff configuration/state/events, SIP models, and post_call configuration/types Required for these preview members; use the relevant operation's opt-in
Preview All seven campaign operations on project.beta.voiceAgents.telephony Required; supplied internally

A GA operation can accept a union with preview-only members. Opt in when selecting Live, inline-handoff, or SIP members, or when configuring post_call; the presence of those members does not make the entire union or operation preview.

For telephony writes, use the connection, binding, call, or call-job provider to determine SIP usage, not just the address type: a SIP connection can use phone_number source and destination addresses. The REST tracing sample accepts only an existing TPE/Twilio connection for its optional outbound call; it does not demonstrate SIP calls. Separately, transfer.require_handoff is a GA transfer-summary setting; it does not by itself enable preview inline handoff.

When migrating older samples, replace project.beta.agents.createFromPrompt with project.agents.createFromPrompt, project.beta.voiceAgents.conversations with project.voiceAgents.conversations, and non-campaign uses of project.beta.voiceAgents.telephony with project.voiceAgents.telephony. Replace their retired Beta*OptionalParams and conversation-download response types with the corresponding non-beta types. Keep campaign and GPT Live operations under project.beta.voiceAgents; use the root Realtime client listed above.

This Node.js example sends text to an existing Realtime agent and prints the reply. Use an agent without an automatic greeting so the first response is the answer to your message.

// Ordinary Realtime sessions are GA and do not require a preview opt-in.
const connection = await project.voiceAgents.realtime.connect("<voice agent name>");
try {
  await connection.sendText("Hello. Please introduce yourself briefly.");
  for await (const event of connection) {
    if (
      event.type === "response.output_text.delta" ||
      event.type === "response.output_audio_transcript.delta"
    ) {
      process.stdout.write(event.delta);
    } else if (event.type === "error") {
      throw new Error(event.error.message);
    } else if (event.type === "response.done") {
      if (event.response.status !== "completed") {
        throw new Error(`Voice response ${event.response.status}.`);
      }
      break;
    }
  }
} finally {
  await connection.close();
}

Read events with one for await loop. Leaving the loop closes the connection. Unknown event types arrive as type: "unknown" without disconnecting the session; their original type is in eventType. Validate rawEvent before using its fields. Malformed events throw VoiceAgentProtocolError. Handle service error events as shown above.

To keep conversations, set store: true on the agent or on connect(), then use project.voiceAgents.conversations to read Realtime conversations. Connecting to a disabled agent fails with 409 Conflict and error code agent_disabled.

The inline-handoff sample shows the explicit opt-in for a Realtime agent that uses preview configuration:

const connection = await project.voiceAgents.realtime.connect("<inline-handoff agent name>", {
  foundryFeatures: "VoiceAgents=V1Preview",
});
await connection.close();

In Node.js, this option sends the foundry-features header. Browsers and React Native send the equivalent foundry_features WebSocket query parameter. Use the typed option, not the reserved generic query parameter. Default Realtime samples use GA agent configurations; for an existing agent with inline handoff, SIP transfer destinations, or post_call, opt in as above.

Choose a complete sample for the task you want:

Task Node.js sample
Create and read an agent definition Configure a voice agent
Generate an agent from a goal Generate a voice agent
Send speech and save the audio reply Realtime audio
Send text and handle a function call Realtime text and tools
Inspect bindings and call history without placing calls Telephony
Route a turn to an inline specialist (preview) Inline handoff
Inspect Realtime (GA) or Live (preview) recordings Conversation recordings
Inspect preview campaigns and GA outbound call jobs Read-only campaigns

The preview handoff sample opts in when creating the definition and connecting. It creates two inline Realtime specialists, observes the correlated handoff events and the target's completed response, then deletes its sample agent. If creation is interrupted, an aborted HTTP request does not cancel server-side creation: use the logged unique agent name to check for a resource that finishes later. The tracing samples are separate instrumentation walkthroughs; the basic generation, Realtime, and GPT Live examples do not require tracing setup. The configuration sample is GA by default; setting either optional post-call destination enables preview transcript delivery and adds the definition opt-in. Post-call delivery also supports optional merged-recording URLs; the sample does not request caller audio.

Stored conversations and recordings

The conversation-recordings sample is read-only and supports both persisted API families. Set FOUNDRY_AGENT_NAME and choose FOUNDRY_VOICE_CONVERSATION_PROTOCOL=realtime or live. Realtime uses the GA project.voiceAgents.conversations client without a preview header; Live uses project.beta.voiceAgents.liveConversations with the required preview header. It reads one page and can select an exact conversation or item using the variables documented in sample.env. Use IDs returned by the REST collections, not the native Realtime WebSocket item or response IDs. Create the recording first by connecting with store: true and finalize the session before downloading audio. The sample never deletes a stored conversation or prints its caller transcripts.

Set FOUNDRY_VOICE_RECORDING_OUTPUT_FILE to opt into downloading a WAV file. With FOUNDRY_VOICE_AUDIO_ITEM_ID, this downloads that item's heard audio; otherwise it downloads the merged stereo recording. These WAV files are different from the raw PCM files written by the streaming samples. Keep downloaded caller media private. Bring-your-own-storage recordings must be read from blob_uri with your own storage credentials; this sample does not fetch arbitrary storage URLs.

Realtime additionally exposes response inspection and generated-but-unheard item audio. The generated-audio variables require an interrupted or truncated item where the service produced audio beyond the heard segment; cancellation alone does not guarantee that artifact exists. Missing audio returns 404. An unfinished merged recording can return 409 recording_not_ready; follow its Retry-After guidance instead of treating that response as an empty recording.

For stored GPT Live sessions, use project.beta.voiceAgents.liveConversations. Connect with project.beta.voiceAgents.live.connect() and store: true to persist a Live session; project.voiceAgents.realtime.connect() uses the Realtime protocol and is not a Live connection. The following Node.js example reads a completed Live conversation, lists its stored items without printing transcripts, and saves its Foundry-managed merged WAV recording:

import { AIProjectClient } from "@azure/ai-projects";
import { DefaultAzureCredential } from "@azure/identity";
import { pipeline } from "node:stream/promises";
import { createWriteStream } from "node:fs";

const project = new AIProjectClient("<project endpoint>", new DefaultAzureCredential());
const agentName = "<GPT Live agent name>";
const conversationId = "<stored Live conversation ID>";
const conversations = project.beta.voiceAgents.liveConversations;
const conversation = await conversations.get(agentName, conversationId);
console.log("Conversation:", conversation.id, conversation.status);
for await (const item of conversations.listItems(agentName, conversationId)) {
  console.log("Stored item:", item.id, item.type);
}
const recording = await conversations.getAudio(agentName, conversationId);
if (recording.blob_uri) {
  throw new Error("Retrieve BYOS audio from blob_uri using your own storage credentials.");
}
const audio = await conversations.downloadAudio(agentName, conversationId);
if (!audio.readableStreamBody) {
  throw new Error("Expected a Node.js audio stream.");
}
await pipeline(audio.readableStreamBody, createWriteStream("live-conversation.wav"));

Campaign operations use the preview project.beta.voiceAgents.telephony client. Use deleteCampaign() to delete a campaign; deleteTelephony() is a deprecated compatibility alias for that operation. Call-job list/get operations use the GA project.voiceAgents.telephony client, including jobs created by a preview campaign. The campaign sample only reads campaign preparation, validation errors, operation monitors, and call jobs. The campaign reads require a deployment with those preview routes enabled. It never creates or cancels a call: campaign creation can authorize dialing before its preparation poller returns. An empty validation-error page during preparation does not establish that the campaign is valid or ready.

Custom query parameters

Pass query to add URL parameters for one connection. Use names supported by your service or gateway; the names below are examples:

const connection = await project.voiceAgents.realtime.connect("my-voice-agent", {
  query: {
    custom_key: "custom value",
    custom_flag: true,
    // Explicit URL transport: only use non-sensitive prompt variables.
    structured_inputs: JSON.stringify({ persona: "guide" }),
  },
});
await connection.close();

Values can be strings, finite numbers, or booleans. The SDK URL-encodes them and captures them when connect() is called. A value replaces a matching endpoint parameter; undefined removes it. These values are not reused for later connections.

Use typed options such as store, agentVersionOverride, and Realtime's foundryFeatures instead of putting SDK settings in query. SDK/authentication names, noncanonical structured-input names, and names starting with h- remain reserved. Realtime explicitly permits the exact structured_inputs key with a JSON-object string, as shown above; do not combine it with structuredInputs. Other custom query values do not become prompt variables.

Realtime structuredInputs preserves header-based transport in Node.js. Browsers use a URL query because they cannot send arbitrary WebSocket upgrade headers. If your service requires the query form, opt in explicitly through query.structured_inputs instead of structuredInputs. Never put secrets or sensitive prompt variables in query parameters: server and proxy access logs can retain URLs.

GPT Live voice agents

Use project.beta.voiceAgents.live for GPT Live agents. When creating the agent, set sub_protocol: "live", model_type: "self_deployed", and model to your GPT Live deployment name. Creation requires VoiceAgents=V1Preview. Pass { foundryFeatures: "VoiceAgents=V1Preview" } to project.agents.create() or createVersion(). Native Live connections always send the required preview flag internally; callers do not need a flag on live.connect().

This short example requires an existing agent configured for mono PCM16LE audio at 24 kHz. Live advances on an audio clock even when adding text instructions, so the example sends silence until the correlated acknowledgement arrives, then closes the session. Use the complete samples below to send text or speech and read the reply.

import {
  AIProjectClient,
  LiveServerEventLiveInstructionsAppended,
  LiveServerEventLiveErrorEvent,
} from "@azure/ai-projects";
import { DefaultAzureCredential } from "@azure/identity";

const project = new AIProjectClient("<project endpoint>", new DefaultAzureCredential());
const connection = await project.beta.voiceAgents.live.connect("<GPT Live agent name>", {
  store: false,
});
const eventId = "brief-instructions";
// Requires an agent configured for mono PCM16LE at 24 kHz.
const silence = new Uint8Array(4800); // 100 ms of audio.
let correlatedAck = false;
let serviceError: Error | undefined;
const onAppended = (event: LiveServerEventLiveInstructionsAppended): void => {
  if (event.client_event_id === eventId) correlatedAck = true;
};
const onError = (event: LiveServerEventLiveErrorEvent): void => {
  serviceError = new Error(event.error.message);
};
connection.on("session.instructions.appended", onAppended);
connection.on("error", onError);
try {
  await connection.appendInstructions("Keep your answers brief.", { eventId });
  const deadline = Date.now() + 10000;
  while (!correlatedAck) {
    if (serviceError) throw serviceError;
    if (connection.state !== "connected") {
      throw new Error(`Live connection became ${connection.state} before the instructions ACK.`);
    }
    if (Date.now() >= deadline) {
      throw new Error("Timed out waiting for the correlated instructions ACK.");
    }
    await connection.sendAudio(silence);
    await new Promise((resolve) => setTimeout(resolve, 100));
  }
  if (serviceError) throw serviceError;
} finally {
  connection.off("session.instructions.appended", onAppended);
  connection.off("error", onError);
  await connection.close();
}

Run a GPT Live sample

Set FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_LIVE_MODEL in .env. Use a deployment name, not a model catalog ID. The function sample also needs FOUNDRY_LIVE_BACKEND_MODEL. See sample.env for these settings and optional audio file paths.

Task Node.js sample
Send a greeting and print the spoken reply as text GPT Live text
Send a speech file and save the audio reply GPT Live audio
Run a local function and let the agent speak the result GPT Live function calling

From a built SDK checkout, run a sample with Node.js 22.18 or later:

node samples-dev/agents/agentVoiceLiveText.ts

Each sample creates and deletes its own agent. It prints captions but does not play sound through your speakers. The audio sample uses a recorded file, not a microphone, and reads and saves raw mono PCM16LE at 24 kHz, without a WAV or other file header.

Press Enter after the reply, or Ctrl+C, to stop. Usage is billable until the session closes. Live needs continuous audio input, even for text and tools, so the samples send silence while idle. Live has no speech-finished event; a completed backend response does not mean the agent has finished speaking.

The text sample appends session instructions and waits for their correlated acknowledgement before adding commentary. That acknowledgement confirms acceptance, not completed speech; keep the session open until the expected reply arrives.

Session controls and events

connect() waits for session.started. Read the current settings from connection.session, including the selected managed agent and version when applicable. Read events with one for await loop or use on, off, and once listeners for subsequent events. For listener-only consumption, supply onEvent when connecting to receive startup events and initial audio without waiting for connect() to return. The original startup event, including the persisted conversation ID when provided by the service, remains available through connection.sessionStarted. An iterator started after an onEvent callback was configured receives subsequent events, not past events.

import { AIProjectClient } from "@azure/ai-projects";
import { DefaultAzureCredential } from "@azure/identity";

const project = new AIProjectClient("<project endpoint>", new DefaultAzureCredential());
const connection = await project.beta.voiceAgents.live.connect("<GPT Live agent name>", {
  store: true,
  onEvent: (event) => {
    if (event.type === "session.started") {
      console.log("Conversation:", event.conversation_id);
    } else if (event.type === "session.output_audio.delta") {
      console.log("Received an audio chunk.");
    }
  },
});
try {
  console.log("Session:", connection.sessionStarted.session.id);
} finally {
  await connection.close();
}
Task API or event
Send raw audio sendAudio(); Live detects speech boundaries, so no audio-buffer commit is needed
Read transcripts session.input_transcript.delta, session.output_transcript.delta
Receive audio session.output_audio.delta; decode the base64 data before saving or playing it
Add text context, rather than a user message appendInstructions(), appendThinking(), appendCommentary()
Pause or resume audio processing muteInputAudio(), unmuteInputAudio()
Update backend settings and transcript triggers configureSession(); the initial model, audio format, and delegation owner cannot change
Close the session close(); waits for session.closed and rejects if finalization is not confirmed

close() and dispose() also reject an unconfirmed shutdown when the socket has already disconnected. Inspect connection.closed for the service's finalization event and any connection error.

With Responses delegation, backend events arrive inside response.event. Use sendResponseItem() to add a user message and requestResponse() to ask for a reply. For a function call, wait for the calling backend response to finish, then use sendToolOutput() to return the result and request continuation. These methods require Responses delegation. For client-handled delegation, use appendThinking() or appendCommentary() and pass the delegation ID as delegationId.

Decision context items support user messages, function calls and results, and custom tool calls and results. Send them with decision.item.create; received decision.item.created events retain the corresponding item fields and stored ID. Decision commands require support from the service deployment.

Live does not support Realtime's response.cancel. An abort signal cancels an SDK connection or send operation, not speech already being generated. To store Live conversations, enable store and use project.beta.voiceAgents.liveConversations; the service deployment must support persistence. Set liveOptions on AIProjectClient to change connection, session-start, and close timeouts. Use the stored-conversation sample with FOUNDRY_VOICE_CONVERSATION_PROTOCOL=live to inspect persisted items or download their WAV audio.

Browser authentication

Use a browser credential such as InteractiveBrowserCredential. The SDK uses native WebSocket with live and short-lived bearer-token subprotocols, including in Web Workers. Node.js uses a bearer header instead. Do not supply realtime as a Live subprotocol, put tokens in URLs, or embed API keys, client secrets, or long-lived tokens in browser code. If direct browser authentication is unavailable, supply an authenticated backend relay through webSocketFactory.

Realtime connections keep structuredInputs in the x-ms-voice-structured-inputs header in Node.js and use the structured_inputs URL query in browsers and Workers. Explicit Realtime query.structured_inputs opts into URL transport on any host. GPT Live continues to use the URL query on all hosts and has no header-based alternative. Proxy and server access logs can retain URLs: do not put credentials, personal data, or other sensitive information in URL-carried inputs.

Using Agent tools

Agents can be enhanced with specialized tools for various capabilities. Tools are organized by their connection requirements:

Built-in Tools

These tools work immediately without requiring external connections.

Code Interpreter

Write and run Javascript code in a sandboxed environment, process files and work with diverse data formats. OpenAI Documentation

const openAIClient = project.getOpenAIClient();
const response = await openAIClient.responses.create({
  model: deploymentName,
  input: "I need to solve the equation 3x + 11 = 14. Can you help me?",
  tools: [{ type: "code_interpreter", container: { type: "auto" } }],
});
console.log(`Response output: ${response.output_text}`);

See the full sample code in agentCodeInterpreter.ts.

File Search

Built-in RAG (Retrieval-Augmented Generation) tool to process and search through documents using vector stores for knowledge retrieval. OpenAI Documentation

const openAIClient = project.getOpenAIClient();
const assetFilePath = path.join(
  __dirname,
  "..",
  "samples-dev",
  "agents",
  "assets",
  "product_info.txt",
);
const vectorStore = await openAIClient.vectorStores.create({
  name: "ProductInfoStreamStore",
});
console.log(`Vector store created (id: ${vectorStore.id})`);
// Upload file to vector store
const fileStream = fs.createReadStream(assetFilePath);
const uploadedFile = await openAIClient.vectorStores.files.uploadAndPoll(
  vectorStore.id,
  fileStream,
);
console.log(`File uploaded to vector store (id: ${uploadedFile.id})`);
// Create agent with file search tool
const agent = await project.agents.createVersion("StreamingFileSearchAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions:
    "You are a helpful assistant that can search through product information and provide detailed responses. Use the file search tool to find relevant information before answering.",
  tools: [
    {
      type: "file_search",
      vector_store_ids: [vectorStore.id],
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentFileSearchStream.ts.

Image Generation

Generate images based on text prompts with customizable resolution, quality, and style settings:

const agent = await project.agents.createVersion("agent-image-generation", {
  kind: "prompt",
  model: deploymentName,
  instructions: "Generate images based on user prompts",
  tools: [
    {
      type: "image_generation",
      quality: "low",
      size: "1024x1024",
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

After calling responses.create(), you can download file using the returned response:

import { fileURLToPath } from "url";

const openAIClient = project.getOpenAIClient();
const agent = await project.agents.createVersion("agent-image-generation", {
  kind: "prompt",
  model: deploymentName,
  instructions: "Generate images based on user prompts",
  tools: [
    {
      type: "image_generation",
      quality: "low",
      size: "1024x1024",
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);
const response = await openAIClient.responses.create(
  {
    input: "Generate an image of Microsoft logo.",
  },
  {
    body: { agent: { name: agent.name, type: "agent_reference" } },
  },
);
console.log(`Response created: ${response.id}`);
const imageData = response.output?.filter((output) => output.type === "image_generation_call");
if (imageData && imageData.length > 0 && imageData[0].result) {
  console.log("Downloading generated image...");
  const __filename = fileURLToPath(import.meta.url);
  const __dirname = path.dirname(__filename);
  const filename = "microsoft.png";
  const filePath = path.join(__dirname, filename);
  // Decode base64 and save to file
  const imageBuffer = Buffer.from(imageData[0].result, "base64");
  fs.writeFileSync(filePath, imageBuffer);
  console.log(`Image downloaded and saved to: ${path.resolve(filePath)}`);
} else {
  console.log("No image data found in the response.");
}

Web Search (Preview)

Perform general web searches to retrieve current information from the internet. OpenAI Documentation

const openAIClient = project.getOpenAIClient();
// Create Agent with web search tool
const agent = await project.agents.createVersion("agent-web-search", {
  kind: "prompt",
  model: deploymentName,
  instructions: "You are a helpful assistant that can search the web",
  tools: [
    {
      type: "web_search_preview",
      user_location: {
        type: "approximate",
        country: "GB",
        city: "London",
        region: "London",
      },
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);
// Create a conversation for the agent interaction
const conversation = await openAIClient.conversations.create();
console.log(`Created conversation (id: ${conversation.id})`);
// Send a query to search the web
console.log("\nSending web search query...");
const response = await openAIClient.responses.create(
  {
    conversation: conversation.id,
    input: "Show me the latest London Underground service updates",
  },
  {
    body: { agent: { name: agent.name, type: "agent_reference" } },
  },
);
console.log(`Response: ${response.output_text}`);

See the full sample code in agentWebSearch.ts.

Computer Use

Enable agents to interact directly with computer systems for task automation and system operations (requires a model deployment that supports the computer tool, such as gpt-5.4; for computer-use-preview deployments, use computer_use_preview):

const agent = await project.agents.createVersion("ComputerUseAgent", {
  kind: "prompt" as const,
  model: deploymentName,
  instructions: `
You are a computer automation assistant.

Be direct and efficient. When you reach the search results page, read and describe the actual search result titles and descriptions you can see.
    `.trim(),
  tools: [
    {
      type: "computer",
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

After calling responses.create(), process the response in an interaction loop. Handle computer_call output items and provide screenshots as computer_call_output with computer_screenshot type to continue the interaction.

See the full sample code in agentComputerUse.ts.

Model Context Protocol (MCP)

Integrate MCP servers to extend agent capabilities with standardized tools and resources. OpenAI Documentation

const openAIClient = project.getOpenAIClient();
const agent = await project.agents.createVersion("agent-mcp", {
  kind: "prompt",
  model: deploymentName,
  instructions:
    "You are a helpful agent that can use MCP tools to assist users. Use the available MCP tools to answer questions and perform tasks.",
  tools: [
    {
      type: "mcp",
      server_label: "api-specs",
      server_url: "https://gitmcp.io/Azure/azure-rest-api-specs",
      require_approval: "always",
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);
// Create a conversation thread to maintain context across multiple interactions
console.log("\nCreating conversation...");
const conversation = await openAIClient.conversations.create();
console.log(`Created conversation (id: ${conversation.id})`);
// Send initial request that will trigger the MCP tool to access Azure REST API specs
// This will generate an approval request since requireApproval="always"
console.log("\nSending request that will trigger MCP approval...");
const response = await openAIClient.responses.create(
  {
    conversation: conversation.id,
    input: "Please summarize the Azure REST API specifications Readme",
  },
  {
    body: { agent: { name: agent.name, type: "agent_reference" } },
  },
);

After calling responses.create(), check for mcp_approval_request items in the response output. Send back McpApprovalResponse with your approval decision to allow the agent to continue its work.

See the full sample code in agentMcp.ts.

OpenAPI

Call external APIs defined by OpenAPI specifications without additional client-side code. OpenAI Documentation

const weatherSpecPath = path.resolve(__dirname, "../assets", "weather_openapi.json");
const agent = await project.agents.createVersion("MyOpenApiAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions:
    "You are a helpful assistant that can call external APIs defined by OpenAPI specs to answer user questions. When calling the weather tool, always include the query parameter format=j1.",
  tools: [
    {
      type: "openapi",
      openapi: {
        name: "get_weather",
        description: "Retrieve weather information for a location using wttr.in",
        spec: weatherSpecPath,
        auth: { type: "anonymous" },
      },
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentOpenApi.ts.

Function Tool

Define custom functions that allow agents to interact with external APIs, databases, or application logic. OpenAI Documentation

/**
 * Define a function tool for the model to use
 */
const funcTool = {
  type: "function" as const,
  function: {
    name: "get_horoscope",
    description: "Get today's horoscope for an astrological sign.",
    strict: true,
    parameters: {
      type: "object",
      properties: {
        sign: {
          type: "string",
          description: "An astrological sign like Taurus or Aquarius",
        },
      },
      required: ["sign"],
      additional_properties: false,
    },
  },
};
const agent = await project.agents.createVersion("function-tool-agent", {
  kind: "prompt",
  model: deploymentName,
  instructions: "You are a helpful assistant that can use function tools.",
  tools: [funcTool],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

After calling responses.create(), process function_call items from response output, execute your function logic with the provided arguments, and send back FunctionCallOutput with the results.

See the full sample code in agentFunctionTool.ts.

  • Memory Search Tool (Preview)

The Memory Store Tool adds Memory to an Agent, allowing the Agent's AI model to search for past information related to the current user prompt.

The embeddingModelDeployment is the name of the model used to create vector embeddings for storing and searching memories.

const memoryStoreName = "AgentMemoryStore";
const embeddingModelDeployment =
  process.env["MEMORY_STORE_EMBEDDING_MODEL_DEPLOYMENT_NAME"] || "<embedding model>";
const scope = "user_123";
const memoryStore = await project.beta.memoryStores.create(
  memoryStoreName,
  {
    kind: "default",
    chat_model: deploymentName,
    embedding_model: embeddingModelDeployment,
    options: {
      user_profile_enabled: true,
      chat_summary_enabled: true,
    },
  },
  {
    description: "Memory store for agent conversations",
  },
);
console.log(
  `Created memory store: ${memoryStore.name} (${memoryStore.id}) using chat model '${deploymentName}'`,
);
// Create an agent that will use the Memory Search tool
const agent = await project.agents.createVersion("MemorySearchAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions:
    "You are a helpful assistant that remembers user preferences using the memory search tool.",
  tools: [
    {
      type: "memory_search_preview",
      memory_store_name: memoryStore.name,
      scope,
      update_delay: 1, // wait briefly after conversation inactivity before updating memories
    },
  ],
});

See the full sample code in agentMemorySearch.ts.

Connection-Based Tools

These tools require configuring connections in your AI Foundry project and use projectConnectionId.

Azure AI Search

Integrate with Azure AI Search indexes for powerful knowledge retrieval and semantic search capabilities:

const aiSearchConnectionId = process.env["AI_SEARCH_CONNECTION_ID"] || "";
const aiSearchIndexName = process.env["AI_SEARCH_INDEX_NAME"] || "";
const agent = await project.agents.createVersion("MyAISearchAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions:
    "You are a helpful assistant. You must always provide citations for answers using the tool and render them as: `[message_idx:search_idx†source]`.",
  tools: [
    {
      type: "azure_ai_search",
      azure_ai_search: {
        indexes: [
          {
            project_connection_id: aiSearchConnectionId,
            index_name: aiSearchIndexName,
            query_type: "simple",
          },
        ],
      },
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentAiSearch.ts.

Bing Grounding

Ground agent responses with real-time web search results from Bing to provide up-to-date information:

const bingProjectConnectionId = process.env["BING_GROUNDING_CONNECTION_ID"] || "";
const agent = await project.agents.createVersion("MyBingGroundingAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions: "You are a helpful assistant.",
  tools: [
    {
      type: "bing_grounding",
      bing_grounding: {
        search_configurations: [
          {
            project_connection_id: bingProjectConnectionId,
          },
        ],
      },
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentBingGrounding.ts.

Bing Custom Search (Preview)

Use custom-configured Bing search instances for domain-specific or filtered web search results:

const bingCustomSearchProjectConnectionId = process.env["BING_CUSTOM_SEARCH_CONNECTION_ID"] || "";
const bingCustomSearchInstanceName = process.env["BING_CUSTOM_SEARCH_INSTANCE_NAME"] || "";
const agent = await project.agents.createVersion("MyAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions:
    "You are a helpful agent that can use Bing Custom Search tools to assist users. Use the available Bing Custom Search tools to answer questions and perform tasks.",
  tools: [
    {
      type: "bing_custom_search_preview",
      bing_custom_search_preview: {
        search_configurations: [
          {
            project_connection_id: bingCustomSearchProjectConnectionId,
            instance_name: bingCustomSearchInstanceName,
          },
        ],
      },
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentBingCustomSearch.ts.

Microsoft Fabric (Preview)

Connect to and query Microsoft Fabric:

const fabricProjectConnectionId = process.env["FABRIC_PROJECT_CONNECTION_ID"] || "";
const agent = await project.agents.createVersion("MyFabricAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions: "You are a helpful assistant.",
  tools: [
    {
      type: "fabric_dataagent_preview",
      fabric_dataagent_preview: {
        project_connections: [
          {
            project_connection_id: fabricProjectConnectionId,
          },
        ],
      },
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentFabric.ts.

Microsoft SharePoint (Preview)

Access and search SharePoint documents, lists, and sites for enterprise knowledge integration:

const sharepointProjectConnectionId = process.env["SHAREPOINT_PROJECT_CONNECTION_ID"] || "";
const agent = await project.agents.createVersion("MyAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions:
    "You are a helpful agent that can use SharePoint tools to assist users. Use the available SharePoint tools to answer questions and perform tasks.",
  // Define SharePoint tool that searches SharePoint content
  tools: [
    {
      type: "sharepoint_grounding_preview",
      sharepoint_grounding_preview: {
        project_connections: [
          {
            project_connection_id: sharepointProjectConnectionId,
          },
        ],
      },
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentSharepoint.ts.

Browser Automation

Automate browser interactions for web scraping, testing, and interaction with web applications:

const browserAutomationProjectConnectionId =
  process.env["BROWSER_AUTOMATION_PROJECT_CONNECTION_ID"] || "";
const agent = await project.agents.createVersion("MyAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions: `You are an Agent helping with browser automation tasks.
              You can answer questions, provide information, and assist with various tasks
              related to web browsing using the Browser Automation tool available to you.`,
  // Define Browser Automation tool
  tools: [
    {
      type: "browser_automation",
      browser_automation: {
        connection: {
          project_connection_id: browserAutomationProjectConnectionId,
        },
      },
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentBrowserAutomation.ts.

Set FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL_NAME, and BROWSER_AUTOMATION_PROJECT_CONNECTION_ID (the project connection resource ID for your Azure Playwright resource) before running the sample. It uses the GA browser_automation tool to visit https://example.com and report the page title and visible description based on browser results. Optionally set FOUNDRY_AGENT_NAME to choose the agent name; it defaults to MyAgent.

The streaming sample prints each completed text message once, including the browser live-view URL, and logs remote function action names, call IDs, and arguments without browserSessionId. Treat the live-view URL as sensitive: do not share it or commit it to source control. Failed or incomplete responses exit with a nonzero status instead of reporting success. The sample deletes the agent version it created in a finally block, whether response processing succeeds or fails.

MCP with Project Connection

MCP integration using project-specific connections for accessing connected MCP servers:

const mcpProjectConnectionId = process.env["MCP_PROJECT_CONNECTION_ID"] || "";
const agent = await project.agents.createVersion("agent-mcp-connection-auth", {
  kind: "prompt",
  model: deploymentName,
  instructions: "Use MCP tools as needed",
  tools: [
    {
      type: "mcp",
      server_label: "api-specs",
      server_url: "https://api.githubcopilot.com/mcp",
      require_approval: "always",
      project_connection_id: mcpProjectConnectionId,
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentMcpConnectionAuth.ts.

Agent-to-Agent (A2A) (Preview)

Enable multi-agent collaboration where agents can communicate and delegate tasks to other specialized agents:

const a2aProjectConnectionId = process.env["A2A_PROJECT_CONNECTION_ID"] || "";
const agent = await project.agents.createVersion("MyA2AAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions: "You are a helpful assistant.",
  // Define A2A tool for agent-to-agent communication
  tools: [
    {
      type: "a2a_preview",
      project_connection_id: a2aProjectConnectionId,
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentAgentToAgent.ts.

OpenAPI with Project Connection

Call external APIs defined by OpenAPI specifications using project connection authentication:

const tripAdvisorProjectConnectionId = process.env["TRIPADVISOR_PROJECT_CONNECTION_ID"] || "";
function loadOpenApiSpec(specPath: string): unknown {
  if (!fs.existsSync(specPath)) {
    throw new Error(`OpenAPI specification not found at: ${specPath}`);
  }
  try {
    const data = fs.readFileSync(specPath, "utf-8");
    return JSON.parse(data);
  } catch (error) {
    throw new Error(`Failed to read or parse OpenAPI specification at ${specPath}: ${error}`);
  }
}
const tripAdvisorSpecPath = path.resolve(__dirname, "../assets", "tripadvisor_openapi.json");
const tripAdvisorSpec = loadOpenApiSpec(tripAdvisorSpecPath);
const agent = await project.agents.createVersion("MyOpenApiConnectionAgent", {
  kind: "prompt",
  model: deploymentName,
  instructions:
    "You are a travel assistant that consults the TripAdvisor Content API via project connection to answer user questions about locations.",
  tools: [
    {
      type: "openapi",
      openapi: {
        name: "get_tripadvisor_location_details",
        description:
          "Fetch TripAdvisor location details, reviews, or photos using the Content API via project connection auth.",
        spec: tripAdvisorSpec,
        auth: {
          type: "project_connection",
          security_scheme: {
            project_connection_id: tripAdvisorProjectConnectionId,
          },
        },
      },
    },
  ],
});
console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

See the full sample code in agentOpenApiConnectionAuth.ts.

For complete working examples of all tools, see the samples-dev directory.

Evaluation

Evaluation in Azure AI Project client library provides quantitative, AI-assisted quality and safety metrics to asses performance and Evaluate LLM Models, GenAI Application and Agents. Metrics are defined as evaluators. Built-in or custom evaluators can provide comprehensive evaluation insights.

The code below shows some evaluation operations. Full list of sample can be found under "evaluations" folder in the package samples

const openAIClient = project.getOpenAIClient();
const dataSourceConfig = {
  type: "custom" as const,
  item_schema: {
    type: "object",
    properties: { query: { type: "string" } },
    required: ["query"],
  },
  include_sample_schema: true,
};
const evalObject = await openAIClient.evals.create({
  name: "Agent Evaluation",
  data_source_config: dataSourceConfig,
  testing_criteria: [
    {
      type: "azure_ai_evaluator",
      name: "violence_detection",
      evaluator_name: "builtin.violence",
      data_mapping: { query: "{{item.query}}", response: "{{item.response}}" },
    } as any,
  ],
});
console.log(`Evaluation created (id: ${evalObject.id}, name: ${evalObject.name})`);

See the full sample code in agentEvaluation.ts.

Deployments operations

The code below shows some Deployments operations, which allow you to enumerate the AI models deployed to your Microsoft Foundry Projects. These models can be seen in the "Models + endpoints" tab in your Microsoft Foundry Project. Full samples can be found under the "deployment" folder in the package samples.

import { ModelDeployment } from "@azure/ai-projects";

const modelPublisher = process.env["MODEL_PUBLISHER"] || "<model publisher>";
console.log("List all deployments:");
const deployments: ModelDeployment[] = [];
const properties: Array<Record<string, string>> = [];

for await (const deployment of project.deployments.list()) {
  // Check if this is a ModelDeployment (has the required properties)
  if (
    deployment.type === "ModelDeployment" &&
    "modelName" in deployment &&
    "modelPublisher" in deployment &&
    "modelVersion" in deployment
  ) {
    deployments.push(deployment);
    properties.push({
      name: deployment.name,
      modelPublisher: deployment.modelPublisher,
      modelName: deployment.modelName,
    });
  }
}
console.log(`Retrieved deployments: ${JSON.stringify(properties, null, 2)}`);

// List all deployments by a specific model publisher (assuming we have one from the list)
console.log(`List all deployments by the model publisher '${modelPublisher}':`);
const filteredDeployments: ModelDeployment[] = [];
for await (const deployment of project.deployments.list({
  modelPublisher,
})) {
  // Check if this is a ModelDeployment
  if (
    deployment.type === "ModelDeployment" &&
    "modelName" in deployment &&
    "modelPublisher" in deployment &&
    "modelVersion" in deployment
  ) {
    filteredDeployments.push(deployment);
  }
}
console.log(
  `Retrieved ${filteredDeployments.length} deployments from model publisher '${modelPublisher}'`,
);

// Get a single deployment by name
if (deployments.length > 0) {
  const deploymentName = deployments[0].name;
  console.log(`Get a single deployment named '${deploymentName}':`);
  const singleDeployment = await project.deployments.get(deploymentName);
  console.log(`Retrieved deployment: ${JSON.stringify(singleDeployment, null, 2)}`);
}

Connections operations

The code below shows some Connection operations, which allow you to enumerate the Azure Resources connected to your Microsoft Foundry Projects. These connections can be seen in the "Management Center", in the "Connected resources" tab in your Microsoft Foundry Project. Full samples can be found under the "connections" folder in the package samples.

import { Connection } from "@azure/ai-projects";

// List the details of all the connections
const connections: Connection[] = [];
const connectionNames: string[] = [];
for await (const connection of project.connections.list()) {
  connections.push(connection);
  connectionNames.push(connection.name);
}
console.log(`Retrieved connections: ${connectionNames}`);

// Get the details of a connection, without credentials
const connectionName = connections[0].name;
const connection = await project.connections.get(connectionName);
console.log(`Retrieved connection ${JSON.stringify(connection, null, 2)}`);

const connectionWithCredentials = await project.connections.getWithCredentials(connectionName);
console.log(
  `Retrieved connection with credentials ${JSON.stringify(connectionWithCredentials, null, 2)}`,
);

// List all connections of a specific type
const azureAIConnections: Connection[] = [];
for await (const azureOpenAIConnection of project.connections.list({
  connectionType: "AzureOpenAI",
  defaultConnection: true,
})) {
  azureAIConnections.push(azureOpenAIConnection);
}
console.log(`Retrieved ${azureAIConnections.length} Azure OpenAI connections`);

// Get the details of a default connection
const defaultConnection = await project.connections.getDefault("AzureOpenAI", {
  includeCredentials: true,
});
console.log(`Retrieved default connection ${JSON.stringify(defaultConnection, null, 2)}`);

Dataset operations

The code below shows some Dataset operations. Full samples can be found under the "datasets" folder in the package samples.

Local filesystem uploads (project.datasets.uploadFile, project.datasets.uploadFolder, and project.beta.models.create) are only supported in Node.js. Browser applications can import AIProjectClient and VoiceAgentRealtimeClient from @azure/ai-projects and use the dataset and model REST operations without Node.js filesystem polyfills.

import { DatasetVersionUnion } from "@azure/ai-projects";

const VERSION1 = "1.0";
const VERSION2 = "2.0";
const VERSION3 = "3.0";

// sample files to use in the demonstration
const sampleFolder = "sample_folder";
// Create a unique dataset name for this sample run
const datasetName = `sample-dataset-basic`;
console.log("Upload a single file and create a new Dataset to reference the file.");
console.log("Here we explicitly specify the dataset version.");

const dataset1 = await project.datasets.uploadFile(
  datasetName,
  VERSION1,
  path.join(__dirname, sampleFolder, "sample_file1.txt"),
);
console.log("Dataset1 created:", JSON.stringify(dataset1, null, 2));

const credential = project.datasets.getCredentials(dataset1.name, dataset1.version, {});
console.log("Credential for the dataset:", credential);
console.log(
  "Upload all files in a folder (including subfolders) to the existing Dataset to reference the folder.",
);
console.log("Here again we explicitly specify a new dataset version");
const dataset2 = await project.datasets.uploadFolder(
  datasetName,
  VERSION2,
  path.join(__dirname, sampleFolder),
);
console.log("Dataset2 created:", JSON.stringify(dataset2, null, 2));
console.log(
  "Upload a single file to the existing dataset, while letting the service increment the version",
);
const dataset3 = await project.datasets.uploadFile(
  datasetName,
  VERSION3,
  path.join(__dirname, sampleFolder, "sample_file2.txt"),
);
console.log("Dataset3 created:", JSON.stringify(dataset3, null, 2));

console.log("Get an existing Dataset version `1`:");
const datasetVersion1 = await project.datasets.get(datasetName, VERSION1);
console.log("Dataset version 1:", JSON.stringify(datasetVersion1, null, 2));
console.log(`Listing all versions of the Dataset named '${datasetName}':`);
const datasetVersions = project.datasets.listVersions(datasetName);
for await (const version of datasetVersions) {
  console.log("List versions:", version);
}
console.log("List latest versions of all Datasets:");
const latestDatasets = project.datasets.list();
for await (const dataset of latestDatasets) {
  console.log("List datasets:", dataset);
}
// List the details of all the datasets
const datasets = project.datasets.listVersions(datasetName);
const allDatasets: DatasetVersionUnion[] = [];
for await (const dataset of datasets) {
  allDatasets.push(dataset);
}
console.log(`Retrieved ${allDatasets.length} datasets`);
console.log("Delete all Datasets created above:");
await project.datasets.delete(datasetName, VERSION1);
await project.datasets.delete(datasetName, VERSION2);
await project.datasets.delete(datasetName, dataset3.version);
console.log("All specified Datasets have been deleted.");

Files operations

The code below shows some Files operations using the OpenAI client, which allow you to upload, retrieve, list, and delete files. These operations are useful for working with files that can be used for fine-tuning and other AI model operations. Full samples can be found under the "files" folder in the package samples.

const openAIClient = project.getOpenAIClient();
console.log("Uploading file");
const created = await openAIClient.files.create({
  file: fs.createReadStream(filePath),
  purpose: "fine-tune",
});
console.log(`Uploaded file with ID: ${created.id}`);
const uploadedFile = await openAIClient.files.retrieve(created.id);
console.log("Processed file metadata:\n", JSON.stringify(uploadedFile, null, 2));
console.log(`Retrieving file content with ID: ${uploadedFile.id}`);
const contentResponse = await openAIClient.files.content(uploadedFile.id);
const buf = Buffer.from(await contentResponse.arrayBuffer());
console.log(buf.toString("utf-8"));
// 4) List all files
console.log("Listing all files:");
const filesList = await openAIClient.files.list();
for (const f of filesList.data ?? []) {
  console.log(JSON.stringify(f));
}
// 5) Delete the file
console.log(`Deleting file with ID: ${uploadedFile.id}`);
const deleted = await openAIClient.files.delete(uploadedFile.id);
console.log(
  `Successfully deleted file: ${deleted?.id || uploadedFile.id}, deleted=${String(deleted?.deleted ?? true)}`,
);

Indexes operations

The code below shows some Indexes operations. Full samples can be found under the "indexes" folder in the package samples.

import { AzureAISearchIndex } from "@azure/ai-projects";

const indexName = "sample-index";
const version = "1";
const azureAIConnectionConfig: AzureAISearchIndex = {
  name: indexName,
  type: "AzureSearch",
  version,
  indexName,
  connectionName: "sample-connection",
};

// Create a new Index
const newIndex = await project.indexes.createOrUpdate(indexName, version, azureAIConnectionConfig);
console.log("Created a new Index:", newIndex);
console.log(`Get an existing Index version '${version}':`);
const index = await project.indexes.get(indexName, version);
console.log(index);
console.log(`Listing all versions of the Index named '${indexName}':`);
const indexVersions = project.indexes.listVersions(indexName);
for await (const indexVersion of indexVersions) {
  console.log(indexVersion);
}
console.log("List all Indexes:");
const allIndexes = project.indexes.list();
for await (const i of allIndexes) {
  console.log("Index:", i);
}
console.log("Delete the Index versions created above:");
await project.indexes.delete(indexName, version);

Fine-tuning operations

The code below shows how to create fine-tuning jobs using the OpenAI client. These operations support various fine-tuning techniques like Supervised Fine-Tuning (SFT), Reinforcement Fine-Tuning (RFT), and Direct Performance Optimization (DPO). Full samples can be found under the "finetuning" folder in the package samples.

import { JobCreateParams } from "openai/resources/fine-tuning/jobs";

const trainingFilePath = "training_data_path.jsonl";
const validationFilePath = "validation_data_path.jsonl";
const openAIClient = project.getOpenAIClient();
// 1) Create the training and validation files
const trainingFile = await openAIClient.files.create({
  file: fs.createReadStream(trainingFilePath),
  purpose: "fine-tune",
});
console.log(`Uploaded file with ID: ${trainingFile.id}`);
const validationFile = await openAIClient.files.create({
  file: fs.createReadStream(validationFilePath),
  purpose: "fine-tune",
});
console.log(`Uploaded file with ID: ${validationFile.id}`);
// 2) Wait for the files to be processed
await openAIClient.files.waitForProcessing(trainingFile.id);
await openAIClient.files.waitForProcessing(validationFile.id);
console.log("Files processed.");
// 3) Create a supervised fine-tuning job
const fineTuningJob = await openAIClient.fineTuning.jobs.create({} as JobCreateParams, {
  body: {
    trainingType: "Standard",
    training_file: trainingFile.id,
    validation_file: validationFile.id,
    model: deploymentName,
    method: {
      type: "supervised",
      supervised: {
        hyperparameters: {
          n_epochs: 3,
          batch_size: 1,
          learning_rate_multiplier: 1.0,
        },
      },
    },
  },
});
console.log("Created fine-tuning job:\n", JSON.stringify(fineTuningJob));

Agent session operations

Use project.agents to manage sessions and session files for hosted agents. Sessions provide isolated sandbox environments for agent interactions. This example requires a hosted agent deployment that supports session operations.

import { VersionRefIndicator } from "@azure/ai-projects";

const agentName = "MySessionAgent";
// Create a session for the agent
const versionIndicator: VersionRefIndicator = {
  type: "version_ref",
  agent_version: "1.0",
};
const session = await project.agents.createSession(agentName, versionIndicator);
console.log(`Session created: ${session.agent_session_id}`);
// Upload a file to the session sandbox
const filePath = "/sandbox/hello.txt";
const fileContent = new TextEncoder().encode("Hello from the agent sessions sample!");
const uploadResult = await project.agents.uploadSessionFile(
  agentName,
  session.agent_session_id,
  filePath,
  fileContent,
);
console.log(`Uploaded file: ${uploadResult.path} (${uploadResult.bytes_written} bytes)`);

See the full sample code in agentSessions.ts and the matching notebook.

Skills operations (preview)

The .beta.skills operations let you create and manage reusable skills that define agent capabilities.

const skillName = "sample-skill";
// Create a new skill
const created = await project.beta.skills.create(skillName, {
  inlineContent: {
    description: "Example skill created by the @azure/ai-projects sample.",
    instructions: "You are a helpful assistant that answers questions concisely.",
    metadata: { owner: "sample" },
  },
});
console.log(`Skill created: ${created.name} (id: ${created.skill_id})`);
// Retrieve the skill
const fetched = await project.beta.skills.get(skillName);
console.log(`Retrieved skill: ${fetched.name} (id: ${fetched.id})`);

See the full sample code in skillBasic.ts.

Toolboxes operations (preview)

The .toolboxes operations let you create and manage toolboxes — reusable collections of tools that can be shared across agents.

import { ToolUnion, MCPTool } from "@azure/ai-projects";

const toolboxName = "mcp";
// Define tools for the toolbox
const tools: ToolUnion[] = [
  {
    type: "mcp",
    server_label: "api_specs",
    server_url: "https://github.com/Azure/azure-rest-api-specs",
    require_approval: "never",
  } satisfies MCPTool,
];
// Create a new toolbox version
const created = await project.toolboxes.createVersion(toolboxName, tools, {
  description: "Example toolbox created by the @azure/ai-projects sample.",
  metadata: { status: "created" },
});
console.log(`Toolbox: ${created.name} (tools: ${created.tools.length})`);
// Retrieve the toolbox
const fetched = await project.toolboxes.get(toolboxName);
console.log(`Retrieved toolbox: ${fetched.name} (${fetched.id})`);

See the full sample code in toolboxesCrud.ts.

Tracing

Experimental feature gate

Important: GenAI tracing instrumentation is an experimental preview feature. Spans, attributes, and events may be modified in future versions. To use it, you must explicitly opt in by passing tracingOptions with experimental: true when constructing the AIProjectClient, or by setting the environment variable:

AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING=true

Only enable this feature after reviewing your requirements and understanding that the tracing behavior may change in future versions.

Getting started with tracing

You can add an Application Insights Azure resource to your Microsoft Foundry project. If one was enabled, you can get the Application Insights connection string, configure your AI Projects client, and observe traces in Azure Monitor. Typically, you might want to start tracing before you create a client or Agent.

Installation

To send traces to Azure Monitor:

npm install @azure/monitor-opentelemetry @opentelemetry/api

To print traces to the console (useful for local development):

npm install @opentelemetry/sdk-trace-node @opentelemetry/api

How to enable tracing

Tracing is enabled by passing tracingOptions to the AIProjectClient constructor. If tracingOptions is not provided, GenAI tracing is completely disabled and no GenAI spans or metrics are emitted. This setting controls only GenAI-specific tracing (semantic convention spans for model calls, agent operations, token usage, etc.). The tracing configuration applies to all operations performed through that specific client instance, including agent operations and any OpenAI client obtained via project.getOpenAIClient(). Different client instances can have independent tracing configurations.

import { DefaultAzureCredential } from "@azure/identity";
import { AIProjectClient } from "@azure/ai-projects";

const endpoint = process.env["FOUNDRY_PROJECT_ENDPOINT"] || "<project endpoint>";
const credential = new DefaultAzureCredential();

// Tracing enabled
const project = new AIProjectClient(endpoint, credential, {
  tracingOptions: { experimental: true },
});

// Tracing disabled (default — no tracingOptions passed)
const projectNoTrace = new AIProjectClient(endpoint, credential);

console.log(project, projectNoTrace);

Passing tracingOptions: {} (an empty object) also enables tracing — in that case, all individual settings are resolved from their corresponding environment variables.

The tracingOptions object accepts the following properties:

Option Environment Variable Default Description
experimental AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING false Acknowledge the experimental nature of this feature (required to emit spans)
contentRecording OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT false Capture prompt and completion content in traces
traceContextPropagation AZURE_TRACING_GEN_AI_ENABLE_TRACE_CONTEXT_PROPAGATION true Inject W3C trace context headers into requests

Each option is resolved using the following precedence order:

  1. Explicit value in tracingOptions (highest priority)
  2. Environment variable (checked when the option is omitted/undefined)
  3. Default value (used when neither is set or the environment variable cannot be read)

Note: Because all tracing features are currently experimental, experimental: true must also be set (either explicitly or via the environment variable) for any spans to be emitted.

Voice Agents tracing

Register an OpenTelemetry provider and exporter, then set tracingOptions: { experimental: true } on AIProjectClient. This traces voice-agent generation, telephony changes, conversation deletion, and Realtime sessions and responses. No separate Voice Agents tracing flag is needed.

import { AIProjectClient } from "@azure/ai-projects";
import { DefaultAzureCredential } from "@azure/identity";

const project = new AIProjectClient("<project endpoint>", new DefaultAzureCredential(), {
  tracingOptions: {
    experimental: true,
    contentRecording: false,
  },
});
const connection = await project.voiceAgents.realtime.connect("<voice agent name>");
try {
  await connection.sendText("Hello.");
  for await (const event of connection) {
    if (event.type === "error") {
      throw new Error(event.error.message);
    }
    if (event.type === "response.done") {
      if (event.response.status !== "completed") {
        throw new Error(`Voice response ${event.response.status}.`);
      }
      break;
    }
  }
} finally {
  await connection.close();
}

Realtime records a session span, a connection span, and an invoke_agent span per response, not per audio chunk. These include status, token usage, and audio byte counts. First-audio latency is measured from response creation to receipt, not from user speech or speaker playback. Service IDs appear in traces, not metric dimensions.

REST and project.voiceAgents.realtime share the project's tracing settings. A separately constructed VoiceAgentRealtimeClient does not inherit them. Input summaries describe the latest input, not the full conversation history.

Content recording is off by default. Enable contentRecording: true only if you intend to record text, transcripts, and tool arguments/results, which can contain sensitive data. Raw media, credentials, query values, telephone numbers, SDP, and raw error/close messages are not recorded. Read/list operations use the existing HTTP pipeline tracing.

See realtime tracing sample and REST tracing sample.

Azure Monitor tracing

Here is a code sample that shows how to enable Azure Monitor tracing:

import { useAzureMonitor } from "@azure/monitor-opentelemetry";
import { AIProjectClient } from "@azure/ai-projects";
import { DefaultAzureCredential } from "@azure/identity";

const projectEndpoint = process.env["FOUNDRY_PROJECT_ENDPOINT"] || "<project endpoint>";
// Configure Azure Monitor tracing (must be done before creating the client)
const connectionString =
  process.env["APPLICATIONINSIGHTS_CONNECTION_STRING"] || "<connection string>";
useAzureMonitor({
  azureMonitorExporterOptions: { connectionString },
  samplingRatio: 1,
  tracesPerSecond: 0,
});
// Create client with tracing enabled (experimental)
const project = new AIProjectClient(projectEndpoint, new DefaultAzureCredential(), {
  tracingOptions: {
    experimental: true,
    contentRecording: false,
    traceContextPropagation: true,
  },
});

You may also want to create a span for your scenario:

import { trace, context } from "@opentelemetry/api";

const tracer = trace.getTracer("MyScenario");
const span = tracer.startSpan("myOperation");
const ctx = trace.setSpan(context.active(), span);
await context.with(ctx, async () => {
  // Your agent operations here
});
span.end();

See the full sample code in agentBasicWithAzureMonitorTracing.ts.

Console tracing

For local development, you can print traces to the console:

import {
  NodeTracerProvider,
  SimpleSpanProcessor,
  ConsoleSpanExporter,
} from "@opentelemetry/sdk-trace-node";
import { AIProjectClient } from "@azure/ai-projects";
import { DefaultAzureCredential } from "@azure/identity";

// Set up OpenTelemetry with a console exporter (must be done before creating the client)
const provider = new NodeTracerProvider({
  spanProcessors: [new SimpleSpanProcessor(new ConsoleSpanExporter())],
});
provider.register();
// Create client with tracing enabled (experimental)
const projectEndpoint = process.env["FOUNDRY_PROJECT_ENDPOINT"] || "<project endpoint>";
const project = new AIProjectClient(projectEndpoint, new DefaultAzureCredential(), {
  tracingOptions: {
    experimental: true,
    contentRecording: false,
    traceContextPropagation: true,
  },
});

See the full sample code in agentBasicWithConsoleTracing.ts.

Enabling content recording

Content recording controls whether message contents and tool call details (such as parameters and return values) are captured in traces. This data may include sensitive user information.

To enable content recording, pass contentRecording: true in tracingOptions when constructing the client, or set the environment variable OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT to true. Content recording defaults to false.

Enabling trace context propagation

Trace context propagation allows client-side spans to be correlated with server-side spans from Azure OpenAI and other Azure services. When enabled, the SDK automatically injects W3C Trace Context headers (traceparent and tracestate) into HTTP requests made by OpenAI clients obtained via project.getOpenAIClient().

This ensures that all operations within a distributed trace share the same trace ID, providing end-to-end visibility across your application and Azure services in your observability backend.

Trace context propagation is enabled by default. To disable it, pass traceContextPropagation: false in tracingOptions when constructing the client, or set the environment variable AZURE_TRACING_GEN_AI_ENABLE_TRACE_CONTEXT_PROPAGATION to false.

Important security and privacy considerations:

  • Trace IDs: When trace context propagation is enabled, trace IDs are sent to Azure OpenAI and other external services.
  • Request correlation: Trace IDs allow Azure services to correlate requests from the same session or user across multiple API calls, which may have privacy implications depending on your use case.

Only enable trace context propagation after carefully reviewing your observability, privacy, and security requirements.

Troubleshooting

Exceptions

Client methods that make service calls raise an RestError for a non-success HTTP status code response from the service. The exception's code will hold the HTTP response status code. The exception's error.message contains a detailed message that may be helpful in diagnosing the issue:

import { isRestError } from "@azure/core-rest-pipeline";

try {
  const result = await project.connections.list();
} catch (e) {
  if (isRestError(e)) {
    console.log(`Status code: ${e.code}`);
    console.log(e.message);
  } else {
    console.error(e);
  }
}

For example, when you provide wrong credentials:

Status code: 401 (Unauthorized)
Operation returned an invalid status 'Unauthorized'

Reporting issues

To report issues with the client library, or request additional features, please open a GitHub issue here

Next steps

Have a look at the package samples folder, containing fully runnable code.

Regenerating from TypeSpec (maintainers)

This package is regenerated from the TypeSpec spec in Azure/azure-rest-api-specs. The six regeneration stages and their issue-assigned orchestrator are encoded as skills under .github/skills/ and can be driven end-to-end by a GitHub Copilot coding-agent task.

The repository-level ai-projects-regen custom agent is only a GitHub discovery entry point. The issue-assigned workflow is owned by the package-local run-issue-regeneration skill.

To start the workflow from an issue, include exactly one value for each required label in the issue description:

TypeSpec commit: <40-character-lowercase-SHA>
Base branch: <branch>

Assign the issue to Copilot, select the ai-projects-regen custom agent, and select the same starting branch named in the description. The agent stops before making changes when either value is missing or invalid.

To dispatch a regen as a cloud agent task, run from this directory:

pwsh -NoProfile -File ./scripts/start-cloud-regen.ps1                     # latest commit on feature/foundry-release
pwsh -NoProfile -File ./scripts/start-cloud-regen.ps1 -TspCommit <sha>    # pin a specific commit
pwsh -NoProfile -File ./scripts/start-cloud-regen.ps1 -DryRun             # render the prompt locally, do not dispatch
pwsh -NoProfile -File ./scripts/start-cloud-regen.ps1 -Repo myuser/azure-sdk-for-js -Follow   # smoke-test on a fork

On Windows, invoking the script directly (e.g. ./scripts/start-cloud-regen.ps1) may be blocked by the default Restricted execution policy. The pwsh -NoProfile -File ... form above sidesteps that. Alternatively, run Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned once to allow local scripts.

Prerequisites:

  • gh CLI installed and authenticated (gh auth login) against the target repo. The agent-task subcommand is in preview and requires a recent gh.
  • Membership in an organization with the GitHub Copilot coding agent enabled for the target repo.
  • Push access to the target repo (the cloud agent uses its own GitHub App identity to push and open the draft PR).

Caveat: the dispatched prompt runs pnpm install --filter "@azure/ai-projects..." and pnpm turbo build --filter="@azure/ai-projects..." --token 1 inline at the start of the task. If the cloud agent's sandbox blocks those network calls, the task will fail at setup; in that case run the skills locally, or coordinate with the SDK build team to add a centrally-managed copilot-setup-steps.yml workflow at the repo root.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.