How to Determine the Model Used in Google Palm Step Compared to MakerSuite?

1

This topic was automatically generated from Slack. You can find the original thread here.

im using the built in google palm step and seeing different results when i compare to makersuite. is there a way to see what model is being used? i dont seem to see it

2

Are you referring to the Chat action?

Here’s the source code:

3

In the google_palm_api.app.mjs file, you can see the API call includes a hardcoded model argument:

Looks like the model is set to models/chat-bison-001.

Maybe at the time when this component was developed, there was only one model available.

4

interesting

5

im fixing to test w/ custom nodejs code, to see if it reflects the issue in seeing. basically, PD’s step is returning A on a prompt and MakerSuite, and local Node code, returns B with the same prompt

6

Can you configure the model with MakerSuite?

7

Or it could just be the nature of the AI, not all prompt answers are deterministic 1:1 every time

8

nod, but im just using defaults there

9

sure. im going to look at tempetature and other settings too

10

wow

11

yeah… its just… genai being genai :wink:

12

context: im asking it for blackjack advice and telling it my hand. for example, “I’ve got a QUEEN of SPADES and a 8 of DIAMONDS.” - and in the response, it says my hand trotals 19, not 18

13

so on one hand, it gets bj, on the other hand, not so much

14

Ah yes, gen AI and math aren’t always so strong. From what I remember it wasn’t until GPT v3.5 that it could count effectively. We instructed Pi to ignore all credit calculation questions because it was just terrible at math.

15

ive noticed (in makersuite) if i lowercase my suits, it kinda helps

16

i may also consider seeing if it handles “eight” versus 8 better

17

Interesting, that’s good to know.

18

nope, it still thinks i have 19 sometimes

19

well, im fine with it - its kinda cool when it works.

20

go home google your drunk: “I would recommend hitting. With a total of 18, you are only one point away from 21, which is the winning hand in blackjack. The dealer’s seven of hearts gives them a total of 17, which is below the 21 threshold. Therefore, there is a good chance that the dealer will bust if they hit, which would give you the win. However, if you stay, you will not be able to improve your hand and will have to hope that the dealer busts.”