This topic was automatically generated from Slack. You can find the original thread here.
Hello! I am migrating the SDK to the new version, and I am finding a few issues along the way which I’ll explain in this thread. Could I get some help?
Create Frontend Client
First, for instantiating the frontend client, as instructed here
It provides with 2 options:
• Option 1: Using PipedreamClient with token callback (for dynamic token management)
• Option 2: Using createFrontendClient with connect token (for simple token-based auth)
I’m trying to use option 2, as it’s the one that fits my current structure the best.
In this option, there is the following example:
// `tokenCallback` is also supported here
const client: PipedreamClient = createFrontendClient({
token: 'connect-token',
externalUserId: 'user-123'
});
Here, only token and externalUserId are provided, although it is stated that tokenCallback is also supported.
However, in my case, tokenCallback is REQUIRED, at least from the createFrontendClient args type in the SDK’s version 2.0.5
Listing Apps
Before the migration, I obtained a list of apps using this method:
const page = await pipedream.getApps(searchProps)
In this “page” result object, I could obtain not only the list of apps in the current page, using page.data, but also page.page_info.total_count and page.page_info.end_cursor for paginated queries.
Now, it has changed to this:
const page = await pipedream.apps.list(searchProps)
However, the result object now only has data, getNextPage and hasNextPage, which I cannot use because I stored the end_cursor value to perform paginated queries asyncronously
Run Actions
Now, pipedream.actions.run(...) will return this object, which is kind of useless:
export interface RunActionResponse {
exports?: unknown;
os?: unknown;
ret?: unknown;
stashId?: Pipedream.StashId | undefined;
}
How do I know how to obtain the result from a RunActionResponse?
pls help
I’m sorry you’re running into issues with the SDK migration. I’ll address each of your questions step by step: — ### 1. Frontend Client Instantiation (createFrontendClient and tokenCallback required) You noticed that tokenCallback is required in createFrontendClient in SDK v2.0.5, even though the docs suggest you can just pass a static token. What’s happening: This is likely a breaking change or a documentation mismatch in the SDK. In v2.0.5, the type definition for createFrontendClient requires tokenCallback, even if you want to use a static token. What to do: - If you only have a static token, you can wrap it in a callback: const client = createFrontendClient({ tokenCallback: async () => "your-static-connect-token", externalUserId: "user-123" }); - This satisfies the type requirement and works the same as passing a static token. — ### 2. Listing Apps: Pagination Info (end_cursor, total_count) Previously, you accessed page.page_info.total_count and page.page_info.end_cursor. Now, the SDK only exposes data, getNextPage, and hasNextPage. What’s happening: The SDK now abstracts pagination, encouraging you to use getNextPage/hasNextPage for sequential fetching. Direct access to cursors and counts may not be exposed. Workarounds: - If you need to store cursors for async pagination, you may need to: - Check if the SDK exposes the raw response (sometimes as raw or similar). - If not, you may need to request this feature or use the REST API directly for advanced pagination needs. - For now, you can chain getNextPage() calls, but you can’t manually jump to a cursor. — ### 3. Run Actions: Getting Results from RunActionResponse You get a response like: ts export interface RunActionResponse { exports?: unknown; os?: unknown; ret?: unknown; stashId?: Pipedream.StashId | undefined; } How to get the result: - The main result of the action is usually in the ret property. - If the action step in Pipedream uses return or $.export("result", value), check both ret and exports. Example: ```
const response = await pipedream.actions.run(…); const result = response.ret ?? response.exports;
For the RunActionResponse I have done this:
function getResultFromActionResponse(
response: RunActionResponse,
): TypedResult<unknown, Error> {
// Expected return statement when no error occurs
if (response.ret) return Result.ok(response.ret)
// When an error occurs, the error is usually logged
if (Array.isArray(response.os)) {
// The error is not always in the first log
for (const log of response.os) {
const output = log as {
ts?: number
k?: 'error'
err?: { name: string; message: string; stack: string }
}
if (output.k === 'error' && output.err) {
return Result.error(new Error(output.err.message))
}
}
}
// Some additional data is returned as"exports"
if (typeof response.exports === 'object' && response.exports !== null) {
// Returned value is automatically exported here
if ('$return_value' in response.exports) {
return Result.ok(response.exports['$return_value'])
}
// Some tools may return a summary of the result
if ('$summary' in response.exports) {
return Result.ok(response.exports['$summary'])
}
}
return Result.error(new Error('Tool did not return a value'))
}
What do you think? Would you change this in any way to improve it? I want to know when the tool worked, what did it return, or what error it threw if any
Your approach to extracting the result from a RunActionResponse is solid and aligns well with Pipedream’s documented response structure: - ret: the main return value (if any) - exports: named exports, including $return_value and $summary - os: logs and errors (with errors as { k: 'error', err: { ... } }) What you’re doing well: - You check ret first, which is the primary return value. - You scan os for errors, which is important since errors may be logged there. - You check exports for $return_value and $summary, which covers most cases for exported data. - You provide a fallback error if nothing is found. Possible improvements: 1. Check for explicit errors first. If the action failed, it’s likely that os will contain an error, and ret/exports may be empty or misleading. Consider checking os for errors before returning ret or exports. 2. Return all error details. If you find an error in os, you might want to include the error name and stack in your returned error for better debugging. 3. Handle empty ret values. If ret is null or undefined, you might want to treat that as “no value returned” unless you expect null to be a valid result. 4. Consider returning more context. Sometimes, it’s helpful to return the full response (or at least the os logs) alongside the result or error for debugging. Suggested revision: ```
typescript function getResultFromActionResponse( response: RunActionResponse, ): TypedResult<unknown, Error> { // 1. Check for errors in logs first if (Array.isArray(response.os)) { for (const log of response.os) { const output = log as { ts?: number k?: ‘error’ err?: { name: string; message: string; stack: string } } if (output.k === ‘error’ && output.err) { const err = output.err; const error = new Error(err.message); error.name = err.name; // Optionally attach stack if needed // (error as any).stack = err.stack; return Result.error(error); } } } // 2. Check for a return value if (typeof response.ret !== “undefined” && response.ret !== null) { return Result.ok(response.ret); } // 3. Check for exports if (typeof response.exports === ‘object’ && response.exports !== null) { if (‘$return_value’ in response.exports) { return Result.ok(response.exports[‘$return_value’]); } if (‘$summary’ in response.exports) { return Result.ok(response.exports[‘$summary’]); } } // 4. Fallback error return Result.error(new Error(‘Tool did not return a value’)); }
ConfigurableProp Types
I had the following method that would translate a ConfigurableProp object to a JSONSchema:
function propToJSONSchema(prop: ConfigurableProp): JSONSchema7 | undefined {
const base: Partial<JSONSchema7> = {
title: prop.name,
description: prop.description,
...('default' in prop ? { default: prop.default } : {}),
}
switch (prop.type) {
case 'boolean':
return { ...base, type: 'boolean' }
case 'integer':
return {
...base,
type: 'integer',
minimum: prop.min,
maximum: prop.max,
}
case 'integer[]':
return {
...base,
type: 'array',
items: {
type: 'integer',
minimum: prop.min,
maximum: prop.max,
},
}
case 'string':
case 'sql': // treat as string
case '$.airtable.baseId':
case '$.airtable.tableId':
case '$.airtable.viewId':
case '$.airtable.fieldId':
case '$.discord.channel':
return {
...base,
type: 'string',
enum: getOptions<string>(prop),
}
case 'string[]':
case '$.discord.channel[]':
return {
...base,
type: 'array',
items: {
type: 'string',
enum: getOptions<string>(prop),
},
}
case 'object':
case '$.service.db':
case '$.interface.http':
case '$.interface.apphook':
// No structural definition is provided; expose generic object
return {
...base,
type: 'object',
additionalProperties: true,
}
case '$.interface.timer': {
// Two allowed shapes: { intervalSeconds: number } or { cron: string }
const intervalSchema: JSONSchema7 = {
type: 'object',
properties: {
intervalSeconds: { type: 'integer', minimum: 1 },
},
required: ['intervalSeconds'],
additionalProperties: false,
}
const cronSchema: JSONSchema7 = {
type: 'object',
properties: {
cron: { type: 'string' },
},
required: ['cron'],
additionalProperties: false,
}
if (prop.static) {
// Fixed value
return { ...base, const: prop.static }
}
return {
...base,
oneOf: [intervalSchema, cronSchema],
}
}
default:
// Fallback: return nothing
return undefined
}
}
However, since the new SDK version, the ConfigurableProp type doesn’t seem to have many of the specific attributes, like min and max for type “integer”, or options attribute for type “string”. So everything (or at least typescript’s types) fails.
How do I do it now ? Is there a better way to approach this?
You’re correct: the new Pipedream SDK’s ConfigurableProp type is more generic and does not guarantee the presence of fields like min, max, or options on all prop types. These fields are now optional and only present when relevant for a given prop. This means your old code, which assumed their presence, will break or cause TypeScript errors. ## Why did this change? Pipedream props are now more flexible and their shape is determined by the type field and the optional presence of other fields. For example, min and max only make sense for integer props, and options is only present if the prop is a dropdown. ## How should you handle this now? You need to check for the presence of these fields before using them. This means updating your code to use type guards or optional chaining. ### Example: Handling min and max for integer props typescript case 'integer': return { ...base, type: 'integer', ...(typeof prop.min === "number" ? { minimum: prop.min } : {}), ...(typeof prop.max === "number" ? { maximum: prop.max } : {}), } ### Example: Handling options for string props typescript case 'string': return { ...base, type: 'string', ...(Array.isArray(prop.options) ? { enum: prop.options.map(opt => typeof opt === "string" ? opt : opt.value) } : {}), } If options is a function (for dynamic options), you may not be able to statically convert it to JSON Schema. ### Generalized Approach You should narrow the prop type based on the type field, and only access fields that are valid for that type, checking for their existence: typescript function propToJSONSchema(prop: ConfigurableProp): JSONSchema7 | undefined { const base: Partial<JSONSchema7> = { title: prop.label ?? prop.name, description: prop.description, ...(prop.default !== undefined ? { default: prop.default } : {}), }; switch (prop.type) { case "boolean": return { ...base, type: "boolean" }; case "integer": return { ...base, type: "integer", ...(typeof prop.min === "number" ? { minimum: prop.min } : {}), ...(typeof prop.max === "number" ? { maximum: prop.max } : {}), }; case "integer[]": return { ...base, type: "array", items: { type: "integer", ...(typeof prop.min === "number" ? { minimum: prop.min } : {}), ...(typeof prop.max === "number" ? { maximum: prop.max } : {}), }, }; case "string": return { ...base, type: "string", ...(Array.isArray(prop.options) ? { enum: prop.options.map(opt => typeof opt === "string" ? opt : opt.value) } : {}), }; case "string[]": return { ...base, type: "array", items: { type: "string", ...(Array.isArray(prop.options) ? { enum: prop.options.map(opt => typeof opt === "string" ? opt : opt.value) } : {}), }, }; case "object": return { ...base, type: "object", additionalProperties: true, }; // ...handle other types as needed... default: return undefined; } } ## Summary - Check for the presence of optional fields before using them. - Narrow the prop type using the type field. - Use optional chaining or type guards to avoid TypeScript errors. - Dynamic options (functions) can’t be represented in JSON Schema—you may need to skip or handle these differently. This approach is now necessary due to the more flexible, generic prop typing in the new SDK. — References: - Pipedream Prop Docs (see the table for which fields are valid for which types) - Pipedream SDK Example (see how ConfigurableProps are used) If you have a specific prop type or error, please share the type definition or error message for more targeted help!
Great. But I’m still having this issue:
Listing Apps
Before the migration, I obtained a list of apps using this method:
const page = await pipedream.getApps(searchProps)
In this “page” result object, I could obtain not only the list of apps in the current page, using page.data, but also page.page_info.total_count and page.page_info.end_cursor for paginated queries.
Now, it has changed to this:
const page = await pipedream.apps.list(searchProps)
However, the result object now only has data, getNextPage and hasNextPage, which I cannot use because I stored the end_cursor value to perform paginated queries asyncronously
can this be fixed?
Thank you for explaining your issue in detail. I understand the frustration—pagination is a common need, and losing access to cursors like end_cursor can break existing async workflows. I’m sorry for the trouble this change has caused. If you need more help or want to talk to a human, please visit Support - Pipedream. ### What Changed? Previously, the result from pipedream.getApps(searchProps) included a page_info object with properties like total_count and end_cursor, which you could use to manage pagination manually and asynchronously. Now, with pipedream.apps.list(searchProps), you only get: - data (the current page of results) - getNextPage (a function to fetch the next page) - hasNextPage (a boolean indicating if more pages exist) The new API abstracts pagination, encouraging you to use getNextPage for synchronous, sequential fetching, but it no longer exposes the raw cursor values you need for custom async pagination. ### Can This Be Fixed? Currently, with the new SDK, you cannot directly access end_cursor or total_count from the result. The SDK intentionally hides these details to simplify usage for most users, but this breaks advanced use cases like yours. #### Workarounds 1. Use the REST API Directly - The REST API docs show that the raw API response still includes page_info with end_cursor and total_count. - You can call the API directly with axios or fetch to get these values: import { axios } from "@pipedream/platform"; export default defineComponent({ async run({ steps, $ }) { const response = await axios($, { url: "https://api.pipedream.com/v1/apps", headers: { Authorization: `Bearer ${this.pipedream.$auth.api_key}`, }, params: { // your searchProps here } }); // response.page_info.end_cursor, response.page_info.total_count, etc. return response; } }); 2. Request SDK Enhancement - If you prefer using the SDK, you can request that Pipedream exposes pagination cursors and metadata in the SDK response. Please submit this feedback at Support - Pipedream. #### Summary - Short-term: Use the REST API directly to access end_cursor and total_count. - Long-term: Request SDK support for exposing pagination metadata. Again, sorry for the disruption. If you need more help or want to discuss this with the Pipedream team, please visit Support - Pipedream.