CRM Financial Services Expert (26% accuracy GPT-3.5-Turbo)

[zestor]

Thank you for contributing an eval! ♥️

🚨 Please make sure your PR follows these guidelines, failure to follow the guidelines below will result in the PR being closed automatically. Note that even if the criteria are met, that does not guarantee the PR will be merged nor GPT-4 access granted. 🚨

PLEASE READ THIS:

In order for a PR to be merged, it must fail on GPT-4. We are aware that right now, users do not have access, so you will not be able to tell if the eval fails or not. Please run your eval with GPT-3.5-Turbo, but keep in mind as we run the eval, if GPT-4 gets higher than 90% on the eval, we will likely reject since GPT-4 is already capable of completing the task.

We plan to roll out a way for users submitting evals to see the eval performance on GPT-4 soon. Stay tuned! Until then, you will not be able to see the eval performance on GPT-4. We encourage partial PR's with ~5-10 example that we can then run the evals on and share the results with you so you know how your eval does with GPT-4 before writing all 100 examples.

Eval details 📑

Eval name

crm-fins-expert

Eval description

Runs a set of questions that an expert in Salesforce and Financial Services should be able to answer.

What makes this a useful eval?

Building chatbots with professional services domain expertise. Specifically, advisory services for expertise in customer relationship management software such as Salesforce with domain expertise in Financial Services usually require a human and are very expensive. The goal is to provide a lower-cost alternative to service a larger segment of the industry at a similar quality level.

Criteria for a good eval ✅

Below are some of the criteria we look for in a good eval. In general, we are seeking cases where the model does not do a good job despite being capable of generating a good response (note that there are some things large language models cannot do, so those would not make good evals).

Your eval should be:

• [X] Thematically consistent: The eval should be thematically consistent. We'd like to see a number of prompts all demonstrating some particular failure mode. For example, we can create an eval on cases where the model fails to reason about the physical world.
• [X] Contains failures where a human can do the task, but either GPT-4 or GPT-3.5-Turbo could not.
• [X] Includes good signal around what is the right behavior. This means either a correct answer for Basic evals or the Fact Model-graded eval, or an exhaustive rubric for evaluating answers for the Criteria Model-graded eval.
• [X] Include at least 100 high quality examples (it is okay to only contribute 5-10 meaningful examples and have us test them with GPT-4 before adding all 100)

If there is anything else that makes your eval worth including, please document it below.

Unique eval value

Insert what makes your eval high quality that was not mentioned above. (Not required)

Eval structure 🏗️

Your eval should

• [X] Check that your data is in evals/registry/data/{name}
• [X] Check that your yaml is registered at evals/registry/evals/{name}.jsonl
• [X] Ensure you have the right to use the data you submit via this eval

(For now, we will only be approving evals that use one of the existing eval classes. You may still write custom eval classes for your own cases, and we may consider merging them in the future.)

Final checklist 👀

Submission agreement

By contributing to Evals, you are agreeing to make your evaluation logic and data under the same MIT license as this repository. You must have adequate rights to upload any data used in an Eval. OpenAI reserves the right to use this data in future service improvements to our product. Contributions to OpenAI Evals will be subject to our usual Usage Policies (https://platform.openai.com/docs/usage-policies).

• [X] I agree that my submission will be made available under an MIT license and complies with OpenAI's usage policies.

Email address validation

If your submission is accepted, we will be granting GPT-4 access to a limited number of contributors. Access will be given to the email address associated with the merged pull request.

• [X] I acknowledge that GPT-4 access will only be granted, if applicable, to the email address used for my merged pull request.

Limited availability acknowledgement

We know that you might be excited to contribute to OpenAI's mission, help improve our models, and gain access to GPT-4. However, due to the requirements mentioned above and high volume of submissions, we will not be able to accept all submissions and thus not grant everyone who opens a PR GPT-4 access. We know this is disappointing, but we hope to set the right expectation before you open this PR.

• [X] I understand that opening a PR, even if it meets the requirements above, does not guarantee the PR will be merged nor GPT-4 access granted.

Submit eval

• [X] I have filled out all required fields in the evals PR form
• [X] (Ignore if not submitting code) I have run pip install pre-commit; pre-commit install and have verified that black, isort, and autoflake are running when I commit and push

Failure to fill out all required fields will result in the PR being closed.

Eval JSON data

Since we are using Git LFS, we are asking eval submitters to add in as many Eval Samples (at least 5) from their contribution here:

View evals in JSON

Eval

{"sample_id":"1","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"What  feature should I use within Salesforce to create a hierarchy, assign accounts to nodes within the hierarchy, and assign the same user to multiple nodes within the hierarchy?"}],"positive": ["territory management"],"negative":["role hierarchy", "account team", "sharing rule"]}
{"sample_id":"2","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"What single feature of Financial Services Cloud should I use to take detailed meeting notes, specify the confidentiality level of the notes, and add action items or next steps without revealing the details of those meetings with my manager through the role hierarchy?"}],"positive":["interaction summary", "interaction summaries"],"negative":["log a call", "private contacts"]}
{"sample_id":"3","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"What single feature of Financial Services Cloud includes OmniScripts, DataRaptors, and FlexCards?"}],"positive":["omnistudio"],"negative":["flow", "einstein"]}
{"sample_id":"4","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"What single feature of Financial Services Cloud should be used to capture a template that defines the tasks and other items needed to complete a business process with each item assigned to a user or queue and given a completion date?"}],"positive":["action plan"],"negative":["process builder", "milestone tracker", "actions framework", "workflow rules", "approval processes", "lightning flow", "flow"]}
{"sample_id":"5","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"What single feature of Financial Services Cloud allows you to view your customers’ relationships in custom relationship graphs?"}],"positive":["arc", "actionable relationship center"],"negative":["relationship map"]}
{"sample_id":"6","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"What single feature of Financial Services Cloud provides a data model, console app, and Record Association Builder that supports branch managers and administrators to track the work output of branches and employees?"}],"positive":["branch management"],"negative":[]} 
{"sample_id":"7","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We are creating a brand new Financial Services Cloud installation and we are getting ready to model individuals. What model should I use to model individuals in FSC?"}],"positive":["person account"],"negative":["individual model"]}
{"sample_id":"8","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC we are using individual records to track opt-in compliance on user, contact, and lead. There is singularly one other object we are forgetting what is it?"}],"positive":["person account"],"negative":[]}
{"sample_id":"9","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"Answering yes or no. We are starting a new FSC program and should we use the individual model to model persons?"}],"positive":["no"],"negative":["yes"]}
{"sample_id":"10","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC we need an overall view of members by rolling up their information in a household account, what single place do go to map our account record types to group record type for rollups?"}],"positive":["custom metadata"],"negative":["affiliation mapping"]}
{"sample_id":"11","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC we need an overall view of members by rolling up their information into a household account, what single place do we go to map our account record types to group record type for rollups?"}],"positive":["custom metadata"],"negative":["affiliation mapping"]}
{"sample_id":"12","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC how many primary groups can each person have?"}],"positive":["one","1"],"negative":[]}
{"sample_id":"13","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC for groups that have client (direct) and business (indirect) members. You are limited to only answering with one of two possible answers a2a or a2c. How are these group memberships modeled either a2c or a2a?"}],"positive":["a2c"],"negative":["a2a"]}
{"sample_id":"14","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"You are limited to only answering with one of these possible answers a2a,a2c,c2c, or recip. Which do you use to model relationships between businesses, institutions, and groups?"}],"positive":["a2a"],"negative":["a2c","c2c","recip"]} 
{"sample_id":"15","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"You are limited to only answering with one of these possible answers a2a,a2c,c2c, or recip. Which do you use to model relationships between a person and a business or other account?"}],"positive":["a2c"],"negative":["a2a","c2c","recip"]} 
{"sample_id":"16","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"You are limited to only answering with one of these possible answers a2a,a2c,c2c, or recip. Which do you use to model relationships between two people?"}],"positive":["c2c"],"negative":["a2a","a2c","recip"]} 
{"sample_id":"17","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"You are limited to only answering with one of these possible answers a2a,a2c,c2c, or recip. Which do you use to model the nature of the relationship between a person and another person, business, or other account?"}],"positive":["recip"],"negative":["a2a","a2c","c2c"]}
{"sample_id":"18","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model assets, such as real estate or collectibles, and liabilities, such as a mortgage, that are not otherwise represented in the financial account?"}],"positive":["assets and liabilities"],"negative":["financial holding"]} 
{"sample_id":"19","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model a person’s financial goal, such as retirement or home purchase?"}],"positive":["financial goal"],"negative":[]} 
{"sample_id":"20","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model details of documents used to verify the identity of a person?"}],"positive":["identification document","identity document"],"negative":[]}
{"sample_id":"21","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model revenue generated from operating a person’s financial account or by offering advisory services?"}],"positive":["revenue"],"negative":["financial account"]} 
{"sample_id":"22","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model a financial account, such as a brokerage account or bank account?"}],"positive":["financial account"],"negative":[]} 
{"sample_id":"23","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model charges and fees taken for servicing a financial account?"}],"positive":["charges and fees"],"negative":["fee object"]} 
{"sample_id":"24","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model the role occupied by a person or organizational entity that is involved with a financial account, such as a beneficiary or trustee?"}],"positive":["financial account role"],"negative":[]} 
{"sample_id":"25","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model the transactions that have taken place on a financial account?"}],"positive":["financial account transaction"],"negative":[]} 
{"sample_id":"26","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model a periodic summary of purchases, deposits, interest, fees, and charges against a financial account?"}],"positive":["billing statement"],"negative":["account transaction summary"]} 
{"sample_id":"27","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model the cards to access funds in a financial account?"}],"positive":["card"],"negative":[]} 
{"sample_id":"28","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model a holding in an investment account, such as 10,000 shares of Microsoft (MSFT)?"}],"positive":["financial holding"],"negative":[]} 
{"sample_id":"29","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC what single object is used to model an asset that is part of a financial holding, such as a stock?"}],"positive":["securities"],"negative":["financial account object"]} 
{"sample_id":"30","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"One of my FSC admin users can't see branch management objects what permission set license do they need?"}],"positive":["financial services cloud extension"],"negative":["financial services cloud plus"]} 
{"sample_id":"31","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"One of my FSC admin users can't use CDS what permission set license do they need?"}],"positive":["financial services cloud extension"],"negative":[]} 
{"sample_id":"32","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"One of my FSC admin users can't use ARC what permission set license do they need?"}],"positive":["financial services cloud extension"],"negative":[]} 
{"sample_id":"33","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"One of my FSC admin users can't use record alerts what permission set license do they need?"}],"positive":["financial services cloud foundations"],"negative":[]} 
{"sample_id":"34","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we need to turn on rollup summary calculations where should we go?"}],"positive":["custom settings"],"negative":["nonprofit success pack"]} 
{"sample_id":"35","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, which setting group do I need to turn on rollup summary calculations?"}],"positive":["wealth application config"],"negative":["rollup settings feature"]} 
{"sample_id":"36","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, to search for and launch quick actions, OmniScripts, Screen flows, Field Service Mobile flows, and Autolaunched flows which component should my service agents use?"}],"positive":["action launcher"],"negative":["lightning flow component"]} 
{"sample_id":"37","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we want to capture repeatable tasks in templates and then automate the task sequences. For example, enhance collaboration and productivity by automatically assigning task owners and deadlines for specific client engagement, such as account openings, loan approvals, and claims processing. What feature should we use?"}],"positive":["action plan"],"negative":["process builder","flow builder"]} 
{"sample_id":"38","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we want to segment similar client profiles, curate them, and design timely and personalized client outreach programs. What feature should we use?"}],"positive":["actionable segmentation"],"negative":["einstein","tableau","marketing cloud","journey builder"]} 
{"sample_id":"39","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we would like to view our customers’ relationships in an easy-to-navigate custom relationship graph. What feature should we use?"}],"positive":["actionable relationship center","arc"],"negative":["relationship map"]} 
{"sample_id":"40","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we need to represent coverage team for accounts and need to support a hierarchy and external colleagues who are not Salesforce users, what single FSC feature can we use?"}],"positive":["branch management"],"negative":[]} 
{"sample_id":"41","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we need to track private meetings with customers related to material non-public information, it's important that only the participants of the meeting see the notes, what single FSC feature can we use?"}],"positive":["interaction summary"],"negative":["private events","private contact points"]} 
{"sample_id":"42","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we need to track opportunities with customers related to material non-public information, it's important that only the participants of the opportunity are able to see it, what single FSC feature can we use?"}],"positive":["financial deal"],"negative":["opportunity","private sharing"]} 
{"sample_id":"43","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we are using branch management what single Salesforce feature can we use to limit what accounts are shown in reports and dashboards based on the user picking a branch from the branch selector?"}],"positive":["scoping rules"],"negative":["row-level security", "sharing rules", "sharing settings"]} 
{"sample_id":"44","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we need to relate accounts to branches, what single object should we use?"}],"positive":["branch unit customer"],"negative":[]} 
{"sample_id":"45","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we need to relate users to branches, what single object should we use?"}],"positive":["branch unit business member"],"negative":[]} 
{"sample_id":"46","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we need to relate external colleagues to branches and they are not Salesforce users, what single object should we associate with branch unit business member?"}],"positive":["banker"],"negative":["external business member"]} 
{"sample_id":"47","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"In FSC, we need to relate custom object records to branches, what single object should we use?"}],"positive":["branch unit related record"],"negative":[]} 
{"sample_id":"48","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We have a KYC process that must be completed before we begin any transaction, what ready made single FSC feature should we use?"}],"positive":["caller verification","identity verification"],"negative":[]} 
{"sample_id":"49","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We have a need to share records with the same group of users on accounts and opportunities, the group must be maintained by an admin, what FSC feature should we use?"}],"positive":["compliant data sharing","CDS"],"negative":["sharing set", "public group", "sharing rule"]} 
{"sample_id":"50","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We have a need to share records with the same group of users on accounts, opportunities, interaction, interaction summary, and financial deal, the group must be maintained by an admin, what FSC feature should we use?"}],"positive":["compliant data sharing","CDS"],"negative":["sharing set", "public group", "sharing rule"]} 
{"sample_id":"51","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We are using compliant data sharing, what object defines the role given to a CDS user or group for the parent object and the level of access?"}],"positive":["participant role"],"negative":["sharing set"]} 
{"sample_id":"52","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We are using opportunities, we have private OWD for opportunities, it's important that only the CDS partipants of an opportunity see the opportunity. We turned on the system this morning and found that managers in the role hierarchy can see opportunities shared with their subordinates. What single feature is the issue?"}],"positive":["grant access using hierarchies"],"negative":[]} 
{"sample_id":"53","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"With FSC, we are using OmniStudio and need to create digital forms to collect and validate data and avoid time-consuming and error-prone manual methods, what single feature should we use?"}],"positive":["discovery framework"],"negative":[]} 
{"sample_id":"54","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"With FSC, we are using OmniStudio and need to generate contracts, proposals, quotes, reports, non-disclosure agreements, service agreements, what single feature should we use?"}],"positive":["document generation"],"negative":[]} 
{"sample_id":"55","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"With FSC, we need to show life events for a person account or contact record and business milestones for an account record, what component should we use on record pages?"}],"positive":["events or milestones"],"negative":["timeline"]} 
{"sample_id":"56","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"With FSC, we need to Manage consent and disclosure documents, generate authorization request forms, and track user responses, what single feature should we use?"}],"positive":["intelligent document management"],"negative":["docusign","adobe sign","conga","process builder"]} 
{"sample_id":"57","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"With FSC, we need a referral management workflow that helps source referrals internally and externally across lines of business, what single feature should we use?"}],"positive":["intelligent need-based referral"],"negative":["lightning flow"]} 
{"sample_id":"58","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"With FSC, we need to group similar interests into categories so that we can easily create reports and dashboards showing common themes and interests across clients, what single feature should we use?"}],"positive":["interest tag","topics"],"negative":[]} 
{"sample_id":"59","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"With FSC, we need to create custom UI through configuration and not code, what feature should we use?"}],"positive":["omnistudio"],"negative":[]} 
{"sample_id":"60","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We have an external master customer system for all customer profiles, when customer profiles are changed, we need to alert users to changes in the account record, what single FSC feature should we use?"}],"positive":["record alert"],"negative":[]} 
{"sample_id":"61","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We need to automatically link new accounts to a branch, what single FSC feature should we use?"}],"positive":["record association builder"],"negative":[]} 
{"sample_id":"62","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We need a comprehensive and chronological view of customer interactions, showing key events relating to a person or a resource in one place, what single FSC feature should we use?"}],"positive":["timeline"],"negative":[]} 
{"sample_id":"63","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We need to encrypt some fields which may contain PII, what Salesforce feature should we use?"}],"positive":["shield"],"negative":[]} 
{"sample_id":"64","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We need to upload about 10 million account records, what Salesforce API should we use?"}],"positive":["bulk"],"negative":[]} 
{"sample_id":"65","input":[{"role":"system","content":"As an experienced Salesforce Certified Technical Architect who has delivered many successful Sales Cloud, Service Cloud, and Financial Services Cloud projects for Financial Services Customers and Banks."},{"role":"user","content":"We need to create a high performance logging framework that is async with a lazy writer to a custom object, what single Salesforce async feature should we use?"}],"positive":["platform event"],"negative":[]}