CLI returns 500 when trying to publish/deploy a source

Help
,
1

I’m trying to deploy a custom source but the CLI just errors out:

[error] component create api call responded with status: 500, body: {"error":"unexpected error, please contact [support@pipedream.com](mailto:support@pipedream.com)"}

Code:

const axios = require('axios@~0.21.1');

const NUMBER_OF_ITEMS = 10;
const DEFAULT_INTERVAL_SECS = 60 * 60; // 1h

module.exports = {
  name: 'Magic the Gathering Arena EN News',
  description: 'Emits the latest MtG:A EN News from https://magic.wizards.com/en/mtgarena',
  key: 'chrigi-mtga-new-news-item-on-page',
  version: '1.0.0',
  dedupe: 'unique',
  props: {
    cfMtGANewsEndpoint: {
      type: 'string',
      label: 'Contentful Endpoint MtG:A News Endpoint',
      description:
        'Extract from API call on "https://magic.wizards.com/en/mtgarena" when pressing "LOAD MORE" in news section',
      optional: true,
      default: 'https://graphql.contentful.com/content/v1/spaces/s5n2t79q9icq/environments/master',
    },
    cfBearerToken: {
      type: 'string',
      label: 'Contentful Bearer Token',
      description:
        'Input without the "Bearer" prefix; use only the "CPET-..." part. Extract the token from the "Authorization" header of the API call to "https://graphql.contentful.com/content/v1/spaces/..." on "https://magic.wizards.com/en/mtgarena"',
      optional: true,
      default: 'CPET-V...',
    },
    timer: {
      type: '$.interface.timer',
      default: {
        intervalSeconds: DEFAULT_INTERVAL_SECS,
      },
    },
  },
  methods: {
    getAuthHeader() {
      return `Bearer ${this.cfBearerToken}`;
    },
    getRequestBody(numberOfItems) {
      return {
        operationName: 'newsCardCollection',
        variables: {
          locale: 'en',
          limit: numberOfItems,
          skip: 0,
          category: '',
          scoped: true,
          preview: false,
          scopedTo: 'mtgarena',
        },
        query: `fragment NewsCardCollectionFields on NewsCardCollection {\n  total\n  items {\n    sys {\n      id\n      __typename\n    }\n    category {\n      value\n      __typename\n    }\n    media {\n      url\n      __typename\n    }\n    title\n    excerpt\n    link\n    callToAction {\n      ...CtaFields\n      __typename\n    }\n    __typename\n  }\n  __typename\n}\n\nquery newsCardCollection($locale: String = \"en\", $preview: Boolean, $limit: Int = ${numberOfItems}, $skip: Int = 0, $category: String = \"\", $scoped: Boolean! = false, $scopedTo: String) {\n  newsCardCollection(locale: $locale, preview: $preview, limit: $limit, skip: $skip, order: publishedDateTime_DESC, where: {category: {value_contains: $category}}) @skip(if: $scoped) {\n    ...NewsCardCollectionFields\n    __typename\n  }\n  scopedNewsCardCollection: newsCardCollection(locale: $locale, preview: $preview, limit: $limit, skip: $skip, order: publishedDateTime_DESC, where: {category: {value_contains: $category}, associatedProduct: $scopedTo}) @include(if: $scoped) {\n    ...NewsCardCollectionFields\n    __typename\n  }\n}\n\nfragment CtaFields on Cta {\n  sys {\n    id\n    __typename\n  }\n  identifier\n  link\n  linkOut\n  target\n  label\n  openModal\n  modalCopy\n  type\n  style\n  size\n  shape\n  leading\n  trailing\n  eventCategory\n  eventAction\n  eventLabel\n  isGameClientDownload\n  pcClient\n  pcClientEventLabel\n  macClient\n  macClientEventLabel\n  androidClient\n  androidClientEventLabel\n  iOsClient\n  iOsClientEventLabel\n  supportingText\n  labelColorOverwrite\n  buttonColorOverwrite\n  borderColorOverwrite\n  __typename\n}\n`,
      };
    },
  },
  async run(event) {
    const magicResponse = await axios.post(this.cfMtGANewsEndpoint, this.getRequestBody(NUMBER_OF_ITEMS), {
      headers: {
        'Content-Type': 'application/json',
        Authorization: this.getAuthHeader(),
      },
    });

    if (magicResponse.status !== 200) {
      throw new Error(`Could not load MtG:A news; HTTP Code: ${magicResponse.status}`);
    }

    const rawNewsItems = magicResponse.data?.data?.scopedNewsCardCollection?.items ?? [];
    rawNewsItems
      .map((item) => {
        return {
          id: item.sys.id,
          title: item.title,
          description: item.excerpt,
          image: {
            url: item.media.url,
          },
          url: item.link,
        };
      })
      .forEach((item) => {
        this.$emit(item, {
          id: item.id,
          summary: item.title,
          ts: event.timestamp,
        });
      });
  },
};
2

@chrigi Event sources still run under a Node 12 runtime, so the error was related to the fact that you were using the optional chaining operator here (only available on Node 14+):

const rawNewsItems = magicResponse.data?.data?.scopedNewsCardCollection?.items ?? [];

I tried modifying that to use lodash/get and it deployed OK:

const get = require("lodash/get");
const rawNewsItems = get(
  magicResponse,
  ["data", "data", "scopedNewsCardCollection", "items"],
  []
);

Would that work for you? We need to address the runtime inconsistency in runtimes so this isn’t an issue moving forward - sorry that affected you!

3

Thanks a lot, sure that works for now. I just assumed everything was on Node 14 now.

Would still be great to get better error messages or even better, a linter in the CLI so I could check the code to conform to the pipedream specs before deploying it.

1 Like
4

Were you running pd deploy to deploy the source, or were you developing with pd dev?

5

I tried both. “pd deploy” just gave the error and quit while “pd dev” would error the same way but I had to manually cancel the stuck process afterwards.

Currently I’ve just configured my eslint to error on these features so I won’t be running into it anymore hopefully.

For anyone in the future running into the same problem, here is a minimal eslint config for that:

module.exports = {
  env: {
    commonjs: true,
    es2020: true,
    node: true,
  },
  extends: 'eslint:recommended',
  plugins: ['es'],
  parserOptions: {
    ecmaVersion: 11,
  },
  rules: {
    'es/no-nullish-coalescing-operators': 'error',
    'es/no-optional-chaining': 'error',
  },
};

using eslint and eslint-plugin-es. (rules is missing es/no-string-prototype-matchall as that was not yet part of the plugin version I was using at the time of writing)

6

That’s a great call. We actually just added an .eslintrc to our main repo, and added logic to raise errors when using these features when you’re developing app files and event sources, where they wouldn’t be supported.

Check out the linting docs here, and if you have any feedback on our .eslintrc config, definitely let us know.