1.
Mr. Singh is working as on optimization consultant for a leading financial institution where he had setup an A/B test to validate two different versions of the campaign landing page. He had shared the QA links internally with the marketing & UX leads to validate the user experience. One of the leads discovered an issue which doesn’t comply with organization UX policy & hence wanted Mr. Singh to make appropriate corrections. Which of the access level would suffice to make the required corrections to the A/B test? (Select all that applies)
Correct Answer(s)
A. Editor
B. Approver
Explanation
The correct answer is Editor and Approver. Mr. Singh, as an optimization consultant, would need the Editor access level to make the required corrections to the A/B test. This access level allows him to edit and modify the campaign landing page. Additionally, the Approver access level is necessary as Mr. Singh needs to get approval for the changes made before implementing them.
2.
Mr. Singh has setup an A/B test for users who are not part of any other test within the same domain. In which part of the audience section can you create this segment?
Correct Answer
B. Visitor Profile
Explanation
In the Visitor Profile section, you can create a segment for users who are not part of any other test within the same domain. This section allows you to define specific criteria and conditions based on user behavior and attributes, such as their location, device type, or engagement level. By creating a segment in the Visitor Profile, Mr. Singh can target and analyze the A/B test specifically for this group of users who are not part of any other test.
3.
Mr. Singh has setup an A/B test on the home page of a leading e-Commerce site to validate two different versions of a page element. Also there is global test running across the site which is not a key test for the top management as compared to the test setup on home page. What needs to be done by the optimization consultant to ensure that there is no conflict in the delivery of the content of both the tests? Select all that applies
Correct Answer(s)
B. Validate selector (elements on the page) collision
C. Validate priority levels for the test & ensure the home page test has priority over global test
D. Validate audience criteria
Explanation
The optimization consultant needs to validate the selector collision to ensure that the elements on the page do not conflict with each other in both tests. They also need to validate the priority levels for the tests and ensure that the home page test has priority over the global test. Additionally, they need to validate the audience criteria to make sure that the right audience is being targeted in both tests.
4.
Mr. Singh has to setup a test to validate changes to two different elements of the home page which is receiving lots of traffic. Element A – Hero BannerElement B – Header Link In total, each of the elements had two versions including the default control version. Key stakeholders from the marketing team want to understand whether change in hero banner had a greater impact on the conversions than changing the header link? In this situation, which of the following test would be appropriate to identify the impact of elements.
Correct Answer
C. MVT
Explanation
MVT (Multivariate Testing) would be the appropriate test to identify the impact of changes to both Element A (Hero Banner) and Element B (Header Link) on the conversions. MVT allows testing multiple variations of different elements simultaneously, which is suitable for this situation where changes to both elements need to be evaluated. It will help determine the individual impact of each element on conversions and also any potential interaction effects between the two elements.
5.
An A/B test was setup by Mr. Singh which was planned to run for three weeks with sample size 60,000 for each experience. These numbers were derived using Adobe recommended sample size calculator. After 10 days of the activity going live, below results were observed. What should be the recommendation to key stakeholders? VisitsLiftConfidenceControl (Exp A)35,000 Exp B38,000-21%99%
Correct Answer
A. Continue the test until estimated sample size is reached for each experience.
Explanation
The recommendation to key stakeholders is to continue the test until the estimated sample size is reached for each experience. This is because the test has only been running for 10 days, which may not be enough time to gather sufficient data for accurate results. By continuing the test until the estimated sample size is reached, it allows for a more reliable and statistically significant conclusion to be drawn from the experiment.
6.
There are two campaign banners set up on the home page of a leading e-Commerce site which also has Adobe Analytics configured on the site. Mr. Singh has setup an A/B test to validate the campaign performance where adding product to cart is the primary goal of the test. Key stakeholders want to understand users from which campaign contributed more to the conversions? How can you setup the goal to understand the expected behavior?
Correct Answer
B. Add campaign click as dependency in advanced settings while creating the activity
Explanation
By adding campaign click as a dependency in the advanced settings while creating the activity, you can set up the goal to understand the expected behavior of users from each campaign. This means that the goal will only be triggered when a user clicks on a campaign banner, allowing you to track and measure the conversions specifically from each campaign. This will provide insights into which campaign contributed more to the conversions and help the key stakeholders understand the performance of each campaign.
7.
An AP test was setup by Mr. Singh on the home page of a leading financial institution. The test audience included users from geo location North America & the configuration included variations across five different elements i.e. CTA, Hero Banner, Header Link, Footer Link, & Banner Message. After setting up the test Mr. Singh had shared the Preview URL links with leading marketing folks across each region to get a sign-off before going live with the activity. One of the stakeholders intended to validate the activity to understand whether someone from North America was actually qualifying for the test. What needs to be done in this case to meet the stakeholder expectation?
Correct Answer
D. Set the activity to live & use multiple audiences which includes QA & actual audience to validate.
Explanation
To meet the stakeholder expectation of validating whether someone from North America is qualifying for the test, the activity should be set to live and multiple audiences should be used. This includes both the QA audience and the actual audience. By doing this, the test can be validated with real users from North America as well as with the QA audience, ensuring that the activity is functioning correctly and reaching the intended target audience.
8.
Mr. Singh has setup an experience targeting test which has four different experiences. The activity went live last week and based on the observation from one of the key marketing folks, there was an issue where he was qualifying incorrectly. Below is the audience criteria along with the experience. How should the order of the experience-audience pair be configured while setting up the test?Experience AUsers coming from “Gurgaon” locationExperience BUsers coming from “Haryana”Experience CUsers coming from “India”
Correct Answer
D. Exp. A | Exp. B | Exp. C
Explanation
The correct order of the experience-audience pair should be Exp. A, Exp. B, Exp. C. This is because the experiences are targeted based on the location of the users. Users from "Gurgaon" should be targeted first (Exp. A), followed by users from "Haryana" (Exp. B), and finally users from "India" (Exp. C). This ensures that the experiences are shown in a hierarchical manner, with more specific locations being targeted first before moving to broader locations.
9.
Mr. Singh had created an MVT test on the home page of a leading cruise lines site, where the stakeholders requested to have three variations each across four different page elements. Post configuration, the preview URL links were distributed to UX & marketing lead for validation from user experience standpoint. Based on the guidelines from UX team, some of the combinations were impacting & recommended to be excluded from the activity. How can Mr. Singh achieve that?
Correct Answer
A. Under preview mode, select the combination to be excluded
Explanation
Mr. Singh can achieve excluding certain combinations from the MVT test by selecting the combination to be excluded under preview mode. This allows him to identify and remove the combinations that are impacting the user experience and do not meet the guidelines provided by the UX team. By selecting the combination to be excluded, Mr. Singh can ensure that only the desired variations are included in the test, improving the overall user experience on the cruise lines site.
10.
Based on the calculations derived using sample size calculator, Mr. Singh figured out that to run an A/B test on the campaign landing page to validate two different versions of CTA text would take 21 days. The activity was configured and was set to live. After two weeks it was assessed that the test duration is actually increasing and it would take more time than initially estimated. What are the possible reasons for increase in the test duration?
Correct Answer
B. Over estimation of traffic numbers
Explanation
The possible reason for the increase in the test duration is overestimation of traffic numbers. This means that Mr. Singh initially estimated a higher number of visitors to the campaign landing page than what was actually observed during the test. As a result, it took longer than expected to gather enough data and reach a statistically significant conclusion.
11.
A test was configured to show the experience for a new users who started his website journey from home.html with tablet device. Below are 4 journey of 4 users, please choose who will be qualified for the experience?1. Paul entered on website via home.html using mobile phone and added products to the cart in past but never made a purchase.2. Simon is a new visitor and currently on home.html page of the website using Tablet devices and used paid search while landed at product.html page.3. Rose is using a tablet device and visited few videos of the brand on youtube in the past. She started her journey from home.html4. Charlie never visited the website in the past and started her journey on website first time from home.html using a mobile device.
Correct Answer
B. Rose
Explanation
Rose is qualified for the experience because she is a new user who started her journey from home.html with a tablet device. She has also shown interest in the brand by visiting videos on YouTube in the past.
12.
Boss at Travel company wants to increase the booking numbers. A product manager suggesting that we should have the Flight and Hotel booking module on Hero Banner, however, UX manager is saying that this will impact the user experience of the customers and will increase the bounce rate from the page.Which test type you would recommend in this scenario?
Correct Answer
B. A/B Test
Explanation
In this scenario, the boss at the travel company wants to increase booking numbers, and the product manager suggests having the Flight and Hotel booking module on the Hero Banner. However, the UX manager believes that this will negatively impact the user experience and increase the bounce rate. To determine the best course of action, an A/B test would be recommended. This test would involve creating two versions of the website - one with the Flight and Hotel booking module on the Hero Banner and one without. By comparing the performance of both versions, the company can gather data and make an informed decision on whether to implement the module or not.
13.
Which type of test you should use for the below?Product manager, UX manager and Marketing manager was brainstorming for improving the performance for one of the key landing pages. Product manger has researched and found that competitors use an image which suggest users which product they are about to purchase as compared to company’s own brand. At the same time UX manager says that images doesn’t help in decision making but Text on CTA and colour of CTA has the highest impact on user’s decision. Marketing manager says we are using different copy on our advertisements on TV and other offline channels and this is why we are not getting orders.
Correct Answer
A. MVT
Explanation
The question describes a scenario where different stakeholders have different opinions on what factors contribute to improving the performance of a landing page. The product manager suggests using an image, the UX manager believes that text and color on the call-to-action (CTA) have the highest impact, and the marketing manager believes that the use of different copy on advertisements is affecting orders. In order to test the effectiveness of these different factors, a Multivariate Test (MVT) should be used. MVT allows for testing multiple variables simultaneously to determine the most effective combination for improving performance.
14.
A retail website is considering running two A/B Test on their checkout funnel, attempting to optimize the conversion rates. Test 1 is a bottom-of-funnel A/B test projected to take 6-8 weeks to reach significance with a minimum detectable lift of 3%. Test 2 is a top-of-funnel test that is estimated take only 4 weeks with an expected lift of 12 %.Which is the best recommendation for this scenario?
Correct Answer
C. Run #2 first and then test #1
Explanation
Running Test 2 first and then Test 1 is the best recommendation for this scenario. Test 2 is estimated to reach significance in 4 weeks with an expected lift of 12%, while Test 1 is projected to take 6-8 weeks with a minimum detectable lift of 3%. By running Test 2 first, the retail website can potentially achieve a significant lift in conversion rates within a shorter time frame. Once Test 2 is completed, they can then proceed to run Test 1 and further optimize their conversion rates. This approach allows for quicker results and the opportunity to implement successful changes sooner.
15.
The optimization team for a very hip and trend fashion blog want to improve the depth of the visit and page consumption of the site. Bounce rate of the website is very high and stakeholders are asking for improvements that will encourage visitors to read more articles, view additional slideshows, and watch more videos.As an optimization consultant, what will you suggest from below?
Correct Answer
C. Run a recommendation Test
Explanation
To improve the depth of visit and page consumption on the fashion blog, running a recommendation test would be an effective suggestion. This test would involve recommending related articles, slideshows, and videos to the visitors based on their browsing behavior and interests. By providing personalized recommendations, it can encourage visitors to explore more content on the site, reducing the bounce rate and increasing engagement.
16.
You configured a test and running this test for a week time. First weekly report was sent to stakeholders and they found that visitors number in the report are lower then the expected numbers and saying the configuration of test is not correct. what you will do to ensure that everything is correct? Check that all apply.
Correct Answer(s)
A. Review the activity url and template rules
B. Validate the Audience setup
D. Will do a browser functionality testing
E. Validate data with Analytics for the same dimension/segment
Explanation
To ensure that everything is correct, you should review the activity URL and template rules to check if there are any errors or issues that may be affecting the visitor numbers. Additionally, validating the audience setup will help confirm if the targeting is accurate. Performing browser functionality testing will help identify any technical issues that may be causing the lower numbers. Lastly, comparing the data with Analytics for the same dimension/segment will provide a cross-reference to validate the accuracy of the reported numbers.
17.
As an optimization consultant, which experience will be a winner recommend as a winner in a test which was configured to improve the RPV. Which of the below is a winner?Assume all other required parameters are achieved.
Correct Answer
E. Variant 2
Explanation
Variant 2 is recommended as the winner in this test to improve the RPV (Revenue Per Visitor). The question implies that all other required parameters have been achieved, meaning that Variant 2 has shown the highest RPV compared to the control and other variants. This suggests that Variant 2 has performed better in generating revenue per visitor, making it the recommended winner in the test.
18.
Optimization consultant wants to setup to deliver 4 different retail promo banners to individual groups of people that came through a specific email campaigns. Which of the activity is best suited in this scenario?
Correct Answer
B. Experience targeting
Explanation
Experience targeting is the best-suited activity in this scenario because it allows the optimization consultant to deliver different retail promo banners to specific groups of people who came through specific email campaigns. Experience targeting enables the consultant to create personalized experiences for different segments of the audience based on their characteristics or behavior, ensuring that each group receives the most relevant and effective promotional banners. This helps to maximize the impact of the campaign and improve overall conversion rates.
19.
Optimization consultant has set up an A/B test with Target as a reporting source & circulated the activity details to get a sign-off from UX & marketing leads. One of the leads proposed to use A4T reporting source. What is the possible reasons to use A4T over Target reporting source?
Correct Answer
A. Stakeholders want to create more than one custom metrics in Target
Explanation
The possible reason to use A4T over Target reporting source is that stakeholders want to create more than one custom metrics in Target. A4T allows for the creation of multiple custom metrics, which can provide more detailed and specific insights for analysis.
20.
An A/B/C test is running since two weeks and the stakeholders realized that experience C is unable to generate the required conversions and want to stop the experience for all the new & returning visitors. What should be done by the BP to address the issue?
Correct Answer
D. D) Deactivate the activity, create a new activity with only A & B experience & set that to live.
Explanation
The stakeholders have identified that experience C is not generating the desired conversions. In order to address this issue, the BP should deactivate the activity associated with experience C. They should then create a new activity that includes only experiences A and B, and set that new activity to live. This way, all the traffic will be diverted to experiences A and B, and experience C will no longer be active. This allows for a focused approach on the experiences that are generating the desired conversions.
21.
BP has created an AP test and is setting up the required audience segment for the activity. Audience criteria is dependent on Analytics variable eVar66 which has the bookingStatus of the user(booked/Non-booked). BP has created the audience with the below configuration. The expected content is not getting delivered on the page. What are the possible concerns?
Correct Answer
A. Mbox parameter is not firing with the required Target global mbox request.
Explanation
The possible concern is that the Mbox parameter is not firing with the required Target global mbox request. This means that the necessary information is not being sent to Target for the audience segmentation to work properly. As a result, the expected content is not being delivered on the page.
22.
BP has setup an A/B test, where the user lands on the test page from a specific display campaign. The primary goal of the activity is to measure the increased lift in revenue. The volume of transactions is very high and hence the stakeholders want to know the number of orders which has value greater than $200 in the reporting interface. How should the BP setup the test to achieve this?
Correct Answer
B. Add reporting segment to include visitors with order value > $200
Explanation
To measure the increased lift in revenue and track the number of orders with a value greater than $200, BP should add a reporting segment to include visitors with order value > $200. This will allow them to filter the reporting data and focus specifically on the desired metric. By using a reporting segment, BP can easily analyze the impact of the A/B test on high-value orders and determine if there is a significant increase in revenue from this specific segment of visitors.
23.
Mr. Smith, an optimization consultant was trying to rearrange a header nav. element with other header nav. element, but was unable to find the default VEC capability to make the required changes to the test. Which of the following activity type is he using to build the test case? choose all that applies
Correct Answer(s)
A. Automated Personalization
C. MVT
Explanation
Mr. Smith, as an optimization consultant, is trying to rearrange a header nav. element with another header nav. element. However, he is unable to find the default VEC capability to make the required changes to the test. This suggests that he is using Automated Personalization to build the test case, as it allows for personalized experiences based on user behavior and preferences. Additionally, he is also using MVT (Multivariate Testing) to test multiple variations of the header nav. elements simultaneously.
24.
Which activity has a similar interface like MVT type activity?
Correct Answer
D. Automated Personalization
Explanation
Automated Personalization has a similar interface to the MVT (Multivariate Testing) type activity. Both activities involve automatically personalizing content based on user behavior and preferences. This means that the system automatically selects and displays the most relevant content to each individual user, optimizing the user experience and increasing engagement. Both activities also require minimal manual intervention, as the system handles the personalization process automatically.
25.
BP had setup an MVT test and the activity is live since two weeks. Based on the initial calculations, it was derived that the test would require two weeks to reach significance. In this case, the significance was not reached and hence the test was inconclusive. When BP did deeper analysis, he realized that a particular experience is performing better and showing significance when validated against a group of users coming from a YouTube platform campaign. What should be the next steps by BP?
Correct Answer
C. Create an experience targeting activity for users coming from YouTube
Explanation
Based on the deeper analysis, BP has identified that a particular experience is performing better and showing significance when validated against a group of users coming from a YouTube platform campaign. To capitalize on this finding, the next step would be to create an experience targeting activity specifically for users coming from YouTube. This would allow BP to further optimize and personalize the experience for this segment, potentially improving the overall results.
26.
BP wants to setup an A/B test across all the product detail pages. Sample URL for a product detail page is www.abc.com/products/men/123.html. How should the test be setup so that the correct audience qualifies for the test & meet the configuration requirements?
Correct Answer
A. Create an audience ->current page URL contains www.abc.com/products
Explanation
To set up an A/B test across all the product detail pages, an audience needs to be created based on the current page URL containing "www.abc.com/products". This ensures that the correct audience, i.e., the users who are on the product detail pages, qualifies for the test. The other options mentioned are not suitable for this purpose as they either target the landing page or specific URLs, which may not cover all the product detail pages. Additionally, it is mentioned that an A/B test can only be set up on a single page, further supporting the need to create an audience based on the current page URL.
27.
BP should circulate the preview experience URL’s, so that stakeholders can validateMr. Singh has setup and A/B test to validate two different versions of campaign landing page on a leading cruise lines site. The audience criteria to qualify for this test is if the user is returning visitor with a prior booking on the site. The activity is planned to be live next week on PROD site and the stakeholders want to validate if the activity actually delivers the expected experience to the booked users. What needs to be done by BP to address the requirement?
Correct Answer
C. BP should setup QA audience combined with AND returning booked guest audience to validate the content delivery on PROD
Explanation
The correct answer is BP should setup QA audience combined with AND returning booked guest audience to validate the content delivery on PROD. This is because the stakeholders want to validate if the activity delivers the expected experience to the booked users. By setting up a QA audience combined with the returning booked guest audience, BP can ensure that the content delivery on the PROD site is validated specifically for the target audience of returning visitors with prior bookings. This will help to ensure that the activity delivers the expected experience to the intended audience.
28.
On the home page of a leading e-commerce site, BP want to validate the changes on a CTA & Hero Banner. There are two versions of CTA & Hero banner, with a total of 4 combinations. Along with this, BP wants to perform actions like changing the CTA position & adding an element prior to the carousel hero banner. Which of the activity type is recommended in this case
Correct Answer
A. A/B Activity
Explanation
MVT test are recommended for testing offers and not layout.
29.
An A/B test is live since a week now. One of the stake holders mentioned that he is unable to see a particular experience and wants the BP to address the scenario. What is the quickest way for BP to verify if a particular experience is actually getting delivered?
Correct Answer
C. Validate the reports to check if that particular experience has any data against it
Explanation
The quickest way for the BP to verify if a particular experience is actually getting delivered is to validate the reports and check if that particular experience has any data against it. By analyzing the reports, the BP can determine if the experience is being delivered and if there is any data associated with it. This method provides concrete evidence of whether the experience is being delivered or not.
30.
What are the key parameters required/recommended to implement the order confirmation mbox on the confirmation page.
Correct Answer
A. 1) orderId, orderTotal, productPurchaseId
Explanation
The key parameters required/recommended to implement the order confirmation mbox on the confirmation page are orderId, orderTotal, and productPurchaseId. These parameters are essential for tracking and confirming the order details. The orderId helps identify the specific order, the orderTotal represents the total amount of the order, and the productPurchaseId is used to track the specific product purchased.
31.
Choose the right answer based on the results below results.
Correct Answer
C. Variant 4 is best performing compared to control with 96% confidence.
Explanation
Based on the given results, Variant 4 is consistently mentioned as the best performing compared to control. The confidence level associated with this statement is 96%. This means that there is a high level of certainty (96%) that Variant 4 outperforms the control.
32.
Our client XYZ.com ran a campaign on website to improve their order conversion with 95% of the confidence. For the next year, client had estimated total # of orders 10,000. Based on the results of the existing test (Table below) they have decided to go ahead with winner. As a Target BP you are tasked to calculate the incremental number of orders for next year.
Correct Answer
B. 3,000
Explanation
The answer of 3,000 is the incremental number of orders for next year. This can be determined based on the results of the existing test mentioned in the table. Since the client ran a campaign to improve order conversion with 95% confidence, it can be assumed that the results of the test showed a significant increase in order conversion. Therefore, the client decided to go ahead with the winner, indicating that the winner resulted in an additional 3,000 orders compared to the previous year's estimate of 10,000 orders.
33.
A retail website has been running an AB test with four variants for the past few days. Stakeholders have examined the results using the reporting console and observed that Variant #3 is demonstrating notably lower conversion rates, which is causing concern due to its impact on the primary metric. It has been calculated that a sample size of 30,000 per variant is necessary to attain the desired confidence level. Currently, all variants have only reached about 30% of this required sample size and will need several more weeks to reach the target. As a Target consultant, what recommendations would you offer to address this situation to the stakeholders?
Correct Answer
D. Wait for the test to provide conclusive results as per defined sample size.
Explanation
It is recommended to wait for the test to provide conclusive results attaining minimum sample size. Reason: The test has been running for only a few days and all the variants have reached around 30% of the sample size. It would be premature to make any decisions based on incomplete data. Waiting for the test to reach the required sample size will provide more reliable and accurate results, allowing stakeholders to make informed decisions about the impact of Variant 3 on the business metric.
34.
You have configured an AP activity on a website with a "Random Forest" algorithm selected. This test has been running for quite a few days. Ecommerce improvement is the key success criteria for this. One of the experiences is receiving maximum traffic and has very low conversion, stakeholder asked you look into the issue and fix that. What you will do in that situation.
Correct Answer
B. You will let stakeholders know that Adobe Target is currently learning and let it run.
Explanation
In this situation, the correct answer is to let stakeholders know that Adobe Target is currently learning and let it run. This is because the AP activity is using the "Random Forest" algorithm, which means that Adobe Target is continuously learning and optimizing the experiences based on the data it collects over time. By letting it run, Adobe Target will gather more data and improve the conversion rate for the experience receiving maximum traffic. Changing the configuration to stop traffic or excluding the experience may hinder the learning process and prevent Adobe Target from making accurate optimizations.
35.
You are currently running one MVT test where you have different offer combinations or discount which you would like to show to users at different paces on the page. You have configured and activated the campaign. One of your stakeholders saw one experience where users were seeing 2 contradicting offers. He raised this issue with your, what you will do or forget to do while configuring the test.
Correct Answer
C. You forgot to exclude the experience from 'preview section'
Explanation
The stakeholder saw one experience where users were seeing 2 contradicting offers, indicating that the stakeholder was able to view the test before it was fully configured and activated. This suggests that the experience was not properly excluded from the 'preview section' of the test, allowing stakeholders to see unfinished or conflicting content.
36.
Your client is very advance in testing cycle and have been running multiple tests/section which are receiving high traffic. He submitted a test request that to be configured for a section where 3 more tests are currently running. He wanted to show the newly configured test for only customers who have not been exposed to any other test. What you should do to show this new test for anybody who hasn't seen the previous test?
Correct Answer
C. You will create an audience with "not in other test"
Explanation
To show the new test to customers who have not been exposed to any other test, you should create an audience with the condition "not in other test". This audience will include customers who have not been included in any other test, ensuring that they are eligible to be shown the new test. This will allow you to target specifically those customers who have not seen any previous tests and provide them with a unique experience.
37.
Automated personlization activity is recommended only for the ecommerce websites as this is heavily dependent on conversion data.
Correct Answer
B. False
Explanation
Automated personalization activity is not only recommended for ecommerce websites, but can also be beneficial for other types of websites. While conversion data may be helpful in determining the effectiveness of personalization efforts, it is not the only factor to consider. Personalization can enhance user experience and engagement on any website, regardless of its purpose. Therefore, the statement that automated personalization is recommended only for ecommerce websites is false.
38.
Mr. John, an optimization practice lead had setup an A/B test 2 weeks back, which had an estimated test duration of three weeks. One of the experience reached confidence 95%+. Stakeholders approved to stop the test and analyze the results. The test showed a 10% increase in overall orders. The sample size was reached, which was set to prove up to a 20% difference in revenue per visitor (RPV) The following effects were observed:• 10% increase in order rate• 3% increase in page views per visit• 3% increase in average order size• 10% increase in product views A stakeholder asks the consultant to annualize the impact on overall revenue projected through the next year to present as program ROI. How should the optimization consultant proceed?
Correct Answer
C. Inform the stakeholder that the projected revenue cannot be determined as the test was designed to prove RPV
Explanation
The optimization consultant should inform the stakeholder that the projected revenue cannot be determined as the test was designed to prove RPV. The A/B test was specifically set up to measure the difference in revenue per visitor (RPV), and not overall revenue. While the test showed a 10% increase in overall orders, this does not directly translate to a specific percentage increase in revenue. To accurately determine the impact on overall revenue, a separate test or analysis specifically focused on RPV would need to be conducted.
39.
Below are the results from an A/B/C/D test, which was run for the duration of 3 weeks. BP is asked to interpret the results. Which of the statements is true based on the below result. ExpVisitorsRPVRevenuep-valueControl25111.9$30000.5Exp B24416.3$40000.47Exp C24920.8$50000.95Exp D25323.7$60000.04
Correct Answer
C. Exp. D is better than control with an RPV of 23.7 with greater than 95% confidence
Explanation
Exp. D is better than control with an RPV of 23.7 with greater than 95% confidence. This means that the revenue per visitor (RPV) for Experiment D is higher than that of the control group, and the confidence level is greater than 95%. In other words, there is a high level of certainty that the difference in RPV between Experiment D and the control group is not due to chance.
40.
Mr. Adam is an optimization consultant for a leading financial institution, who had setup an A/B test two weeks back. The test has reached significance which was designed to prove up to a 10% difference in orders at 95% confidence. The sample size was 50,000 visitors per variation. Based on the below results, what should Adam recommend keeping the primary KPI in mind. Experience Visitors Orders Order Rate Lift Confidence Control23,0758463.666% Experience B23,08510004.331%18.3%96%
Correct Answer
C. Continue running the activity to set sample size because the results are inconclusive.
Explanation
The results of the A/B test are inconclusive because the observed lift in orders (18.3%) is not close to the estimated lift (10%). Therefore, it is recommended to continue running the activity to collect more data and determine the appropriate sample size. This will help to obtain more reliable results and make a confident decision based on the primary KPI.
41.
Mr. Smith, an optimization consultant had setup an A/B auto-allocate test three weeks back, which was planned to be run for 2 weeks. Due to over estimation of traffic during the initial phase, an additional week was required to reach the test completion based on pre-defined significance and sample size. The test showed a 10% increase in RPV, which was set to prove up to a 20% difference in number of orders The following effects were observed:- 10% increase in order rate
- 3% increase in page views per visit
- 20% increase in number of orders
- 3% increase in average order size
- 10% increase in product views
A stakeholder asks the consultant to annualize the impact on projected orders for year 2018, which is estimated to be 160000 . How should the consultant proceed?
Correct Answer
C. Inform the stakeholder that the projected orders for 2018 would be 192000
Explanation
The consultant should inform the stakeholder that the projected orders for 2018 would be 192,000. This is because the test showed a 10% increase in RPV, which was set to prove up to a 20% difference in the number of orders. Since the increase in RPV is greater than the estimated lift for the number of orders, it can be concluded that there will be a proportional increase in the number of orders for the entire year. Therefore, the projected orders for 2018 would be 192,000.
42.
A/B Auto Allocate activity uses enhanced version of the alogirithm used by Automated Personalization activity type.
Correct Answer
B. False
Explanation
The statement is false because A/B Auto Allocate activity does not use an enhanced version of the algorithm used by the Automated Personalization activity type. These two activities use different algorithms for their respective purposes.
43.
In which of the activity type, offer level targeting can be acheived?
Correct Answer
B. Automated Personalization
Explanation
Automated Personalization is the activity type in which offer level targeting can be achieved. This means that through Automated Personalization, specific offers can be targeted to individual users based on their preferences, behavior, or other data. This allows for a more personalized and targeted approach to marketing, increasing the likelihood of conversion and customer satisfaction.
44.
While using Adobe Target Sample Size calculator, the optimization consultant derived that the expected traffic would be 10,000 visitors per experience per day. But when observed in the ongoing activity this wasn’t the case and the traffic numbers were 50% less than the expected numbers. What is the first activity that should be performed by the consultant?
Correct Answer
C. Re-validate the activity setup, metrics & ensure that data is getting into reports
Explanation
The consultant should first re-validate the activity setup, metrics, and ensure that data is getting into reports. This is because the observed traffic numbers were significantly lower than the expected numbers, indicating a potential issue with the activity setup or data collection. By re-validating these aspects, the consultant can identify any potential errors or discrepancies and make the necessary adjustments to ensure accurate reporting and analysis.
45.
"Recently viewed content", "Recently watched videos", "Most viewed Items across the site" & "Top selling products from across the site" represent an example of which of the following?
Correct Answer
B. Recommendations Criteria
Explanation
The options "Recently viewed content", "Recently watched videos", "Most viewed Items across the site" & "Top selling products from across the site" represent examples of Recommendations Criteria. These criteria are used to determine what content or products to recommend to users based on their recent activities, popular items, and trends.