This topic was automatically generated from Slack. You can find the original thread here.
I’m having some more trouble with data stores. I have a set of returned values [{“id”:6580398817466,“sku”:“79647”},{“id”:6580755857594,“sku”:“14113934”},{“id”:6631157399738,“sku”:“12570949”},{“id”:6580010975418,“sku”:“9918763”},{“id”:6580011040954,“sku”:“9941252”}…
That I want to enter into a Data Store as key/value pairs. I hadnt been able to enter them directly so I built an extra code step that formatted them into the correct format (I think) {“7593892020470”:“22269763”,“6625223114938”:“18400579”,“6643999572154”:“18409612”,“7713365524726”:“18400579”,“6618209386682”:“11806755”…
Unfortunately because I’m working with large sets the code step that converts them into key/value pairs is crashing. It always felt like an unnecessary step bourne of my lack of understanding. So is there a more direct way to put the first return into a Data Store as a set of key/value pairs?
Hello , may I ask for implementation detail of the code step that’s crashing?
As Pipedream datastore is still in preview, I think maybe you can consider using another data storage so that you won’t need to workaround data store current limitation?
Hi the workflow is actually still locked up (crashed) but I made a copy and the specific step it was running out of memory for was as follow:
exportdefault defineComponent({
name: “Convert Paths to Key-Value Pairs”,
description: “Converts multiple paths to key-value pairs”,
key: “convert_paths_to_key_value_pairs”,
version: “0.0.1”,
type: “action”,
props: {
customRequest: {
type: “object”,
label: “Custom Request”,
description: “The steps.custom_request object”,
},
idDataSource: {
type: “string”,
label: “ID Data source”,
description: “Enter the final part of the path to the data you’re trying to access for the ID values, e.g. ‘styles’ instead of steps.custom_request.$return_value.styles”, default: “skus”
},
idDataProperty: {
type: “string”,
label: “ID Data property”,
description: “The property to extract ID data from (optional). Enter ‘NA’ if not applicable.”, default: “id”
},
skuDataSource: {
type: “string”,
label: “SKU Data source”,
description: “Enter the final part of the path to the data you’re trying to access for the SKU values, e.g. ‘styles’ instead of steps.custom_request.$return_value.styles”, default: “skus”
},
skuDataProperty: {
type: “string”,
label: “SKU Data property”,
description: “The property to extract SKU data from (optional). Enter ‘NA’ if not applicable.”, default: “sku”
}
}, async run({ $ }) { const idData = this.customRequest.$return_value[this.idDataSource]; const skuData = this.customRequest.$return_value[this.skuDataSource];
**let** extractedData = {};
**for** (**let** i = 0; i < idData.length; i++) {
extractedData[idData[i][**this**.idDataProperty].toString()] = skuData[i][**this**.skuDataProperty];
}
**return** extractedData;
@U02SX7ETBFB Just FYI, it seems to be specifically changing the memory settings that cause the lock up. I made a copy and that ran fine handling less data for a bit, with some difficulty but got through it. So I deployed and set the trigger for 12am and put the memory up (to make sure it could get through unsupervised). Putting the memory up immeiately caused white screen of death. not a massive deal for my work, but I thought you’d be interested. I don’t know if it will run at 12am but the builder is locked up anyway.
