🪟🧪 [Experiment] Incentivize speedy first connection (aka activation) #17593

letiescanciano · 2022-10-05T06:24:58Z

What

We know that users that activate, converts better. We want to test if incentivizing users to create their first connection faster (activate) will improve our conversion rate.

To incentivize this behavior, we will be showing a banner with a 24 hour countdown, which will be visible until the user has set up their first connection.

If the user is in the experiment, sees the banner and create her first connection, we'll add (manually for now) 100 credits to their trial

How

I've created a new folder where I will be adding experiments, each experiment having its own folder to try to make it easier to remove if the experiment is not successful.
We have <SpeedyConnectionBanner /> responsible to render the banner itself and a couple more components for the banner content, especially the countdown.

There is one missing piece, adding orb trial credits when the action is completed. Due to security issues, we can't have that request made directly from the FE as any user would be able to add their own credits in cloud.
For now, we'll do this manually (adding credits on Orb UI) but, since gamification is something we want to invest on, we are drafting a proposal on how this could be automated in the future.

letiescanciano · 2022-10-05T06:25:56Z

airbyte-webapp/src/hooks/services/Experiment/experiments.ts

@@ -15,4 +15,5 @@ export interface Experiments {
  "authPage.oauth.github": boolean;
  "authPage.oauth.google.signUpPage": boolean;
  "authPage.oauth.github.signUpPage": boolean;
+  "onbarding.speedyConnection": boolean;


Launch Darkly FF

letiescanciano · 2022-10-05T06:26:27Z

airbyte-webapp/src/packages/cloud/cloudRoutes.tsx

@@ -92,7 +95,7 @@ const MainRoutes: React.FC = () => {
        <Route path={`${RoutePaths.Settings}/*`} element={<CloudSettingsPage />} />
        <Route path={CloudRoutes.Credits} element={<CreditsPage />} />

-        {workspace.displaySetupWizard && (
+        {(workspace.displaySetupWizard || isExperimentVariant) && (


We'll redirect users to the onboarding flow if they click on the banner CTA, therefore we need this route accessible

airbyte-webapp/src/packages/cloud/services/auth/AuthService.tsx

letiescanciano · 2022-10-05T06:29:49Z

airbyte-webapp/src/pages/OnboardingPage/OnboardingPage.tsx

  useTrackPage(PageTrackingCodes.ONBOARDING);

  const navigate = useNavigate();

-  useEffectOnce(() => {


Not related, but this was duplicated, page view was being tracked by line 64.

letiescanciano · 2022-10-05T06:31:00Z

airbyte-webapp/src/pages/OnboardingPage/useStepsConfig.tsx

@@ -13,7 +14,14 @@ const useStepsConfig = (
  setCurrentStep: (step: StepType) => void;
  steps: Array<{ name: JSX.Element; id: StepType }>;
 } => {
+  // exp-speedy-connection
+  const location = useLocation() as unknown as ILocationState<{ step: StepType }>;


Need to type location state, as typescript will complain otherwise.
remix-run/history#930

letiescanciano · 2022-10-05T06:31:59Z

...rc/components/experiments/SpeedyConnection/SpeedyConnectionBanner/SpeedyConnectionBanner.tsx

+            [styles.textDecorationNone]: location.pathname.includes("onboarding"),
+          })}
+          to={RoutePaths.Onboarding}
+          state={{


Jump into create source step in the onboarding flow

timroes · 2022-10-05T20:18:42Z

It would be useful to have more details on the PR description, what this is actually adding. You described why we're doing this, but it would be useful having ideally some screenshots but at least some description like "This will add a banner, before activation, showing a countdown till .... and so on". Right now only the code tells what this PR actually adds.

timroes · 2022-10-05T20:11:19Z

...components/experiments/SpeedyConnection/hooks/use-experiment-speedy-connection-experiment.ts

+  const isTrial = Boolean(cloudWorkspace.trialExpiryTimestamp);
+  const timestamp = localStorage.getItem("exp-speedy-connection-timestamp");
+
+  const expiredOfferDate = timestamp ? JSON.parse(timestamp) : 0;


Suggested change

const expiredOfferDate = timestamp ? JSON.parse(timestamp) : 0;

const expiredOfferDate = timestamp ? Number(timestamp) : 0;

We should try to avoid JSON.parse and JSON.strnigify as much as possible. Those methods are really slow, and reduce type-safteness since the result could in theory be anything. If we know we want to parse a string to a number here, using the Number constructor is the better option.

timroes · 2022-10-05T20:12:22Z

airbyte-webapp/src/packages/cloud/services/auth/AuthService.tsx

+          // exp-speedy-connection
+          localStorage.setItem(
+            "exp-speedy-connection-timestamp",
+            JSON.stringify(new Date(new Date().getTime() + _24_HOURS))


Similar to above, just String(...) instead of JSON.stringify.

timroes · 2022-10-05T20:19:18Z

airbyte-webapp/src/packages/cloud/services/auth/AuthService.tsx

@@ -65,6 +65,8 @@ interface AuthContextApi {
  logout: AuthLogout;
 }

+const _24_HOURS = 24 * 60 * 60 * 1000;


I think we should just inline this instead of adding a _ prefixed variable here.

I extracted it to make it more descriptive, but can inline instead, no problem!

airbyte-webapp/src/packages/cloud/services/auth/AuthService.tsx

timroes · 2022-10-05T20:21:58Z

...rc/components/experiments/SpeedyConnection/SpeedyConnectionBanner/SpeedyConnectionBanner.tsx

+          <FormattedMessage id="experiment.speedyConnection.next" defaultMessage="in the next" />{" "}
+        </Text>
+
+        <CountDownTimer {...{ expiredOfferDate }} />


🎨 codestyle

Suggested change

<CountDownTimer {...{ expiredOfferDate }} />

<CountDownTimer expiredOfferDate={expiredOfferDate} />

Curious: is this the case always? Or do we use the first codestyle when there are lots of props? I've found it more useful and faster to code in the past!

timroes · 2022-10-05T20:27:59Z

...components/experiments/SpeedyConnection/hooks/use-experiment-speedy-connection-experiment.ts

@@ -0,0 +1,18 @@
+import { useExperiment } from "hooks/services/Experiment";


We're using camelCase for file names, this file name is hyphened.

airbyte-webapp/src/components/experiments/SpeedyConnection/CountDownTimer/use-countdown.ts

timroes · 2022-10-05T20:44:34Z

...rc/components/experiments/SpeedyConnection/SpeedyConnectionBanner/SpeedyConnectionBanner.tsx

+            <FormattedMessage
+              id="experiment.speedyConnection.firstConnection"
+              defaultMessage="Set up your first connection "
+            />{" "}


Messages that contain of several parts, need to be combined in one message (with placeholders) to properly be internationalizationable. I'm happy to sync on that to run you through that.

Happy to learn about the right way to do it, but I think for this case we can go with this version, as message is probably something to be iterated on something to be

timroes · 2022-10-05T20:49:48Z

The banner currently overlaps also all button on the top. If the banner is shown we need to make sure that the content is pushed below, which happens for the other banner in the div with styles.content.

…vation * master: (26 commits) supply a source id for schema discovery in connector integration tests (#17662) Source Iterable: Add permission check for stream (#17602) Moving TrackingClientSingleton.initialize into the bean itself (#17631) Handle null workspace IDs in tracking/reporting methods gracefully (#17641) Bump Airbyte version from 0.40.11 to 0.40.12 (#17653) Revert "Do not wait the end of a reset to return an update (#17591)" (#17640) Standardize HttpRequester's url_base and path format (#17524) Create geography_type enum and add geography column in connection and workspace table (#16818) airbyte-cron: update connector definitions from remote (#16438) Do not wait the end of a reset to return an update (#17591) Remove redundant title labels from connector specs (#17544) Updated GA4 status support large schema discovery (#17394) 🪟 🐛 Fixes connector checks not properly ending their loading state (#17620) 🪟🧪 [Experiment] add hideOnboarding experiment (#17605) Source Recharge: change releaseStage to GA (#17606) Source Recharge: skip stream if 403 received (#17608) remove sonar-scan workflow (#17609) Mark/tables should be full width on all pages (#17401) Auto fail all workfow if there is a Versioning issue (#17562) ...

letiescanciano · 2022-10-06T13:37:26Z

@timroes ready for re-review

airbyte-webapp/src/hooks/services/Experiment/experiments.ts

timroes

Reviewed. Left cleanup comments for later. Tested locally seems to work. One issue outstanding that at the moment the banner might show but the route isn't there, which we'll need to address in one of the mentioned ways, after that it looks good to merge.

timroes · 2022-10-06T17:42:29Z

...webapp/src/components/experiments/SpeedyConnection/CountDownTimer/CountDownTimer.module.scss

+@use "../../../../scss/variables";
+@use "../../../../scss/colors";
+
+.flex {


🧹 We usually tend to not name CSS rules by their content but the element they are attached too, e.g. this would rather be .container and the one below .countdownDigits.

timroes · 2022-10-06T17:49:31Z

airbyte-webapp/src/components/experiments/SpeedyConnection/CountDownTimer/CountDownTimer.tsx

+
+  return (
+    <div className={styles.flex}>
+      <Text as="h2" className={styles.textColorOrange}>


🧹 Those should usually not be h2, since they are not really semantically a header to anything in the document. So this currently breaks accessibility.

You are right, changing now.
It was a heading in the Figma, but I should have been more careful

timroes · 2022-10-06T17:49:42Z

airbyte-webapp/src/components/experiments/SpeedyConnection/CountDownTimer/CountDownTimer.tsx

+        {hours.toString().padStart(2, "0")}h
+      </Text>
+      <Text as="h2" className={styles.textColorOrange}>
+        {minutes.toString().padStart(2, "0")}m{" "}


Additional space only behind the m but not the others?

timroes · 2022-10-06T17:50:28Z

airbyte-webapp/src/components/experiments/SpeedyConnection/CountDownTimer/useCountdown.ts

+export const useCountdown = (targetDate: string) => {
+  const countDownDate = new Date(targetDate).getTime();
+
+  const [countDown, setCountDown] = useState(countDownDate - new Date().getTime());


🧹 Date.now() is usually the preferred one (doesn't require instantiation of an object).

timroes · 2022-10-06T18:01:37Z

...rc/components/experiments/SpeedyConnection/SpeedyConnectionBanner/SpeedyConnectionBanner.tsx

+                className={classNames(styles.linkCta, {
+                  [styles.textDecorationNone]: location.pathname.includes("onboarding"),
+                })}
+                to={RoutePaths.Onboarding}


🧹 We might want to link later to the Connection page here, depending on how our experiment goes with disabling the onboarding page.

✅ Addressed, so both experiments can live together

timroes · 2022-10-06T18:20:10Z

...te-webapp/src/components/experiments/SpeedyConnection/hooks/useExperimentSpeedyConnection.ts

+  const isVariantEnabled = useExperiment("onbarding.speedyConnection", false);
+
+  const isTrial = Boolean(cloudWorkspace.trialExpiryTimestamp);
+  const timestamp = localStorage.getItem("exp-speedy-connection-timestamp");


🧹 We talked about the approach with local storage already offline, so just putting that here to remember.
Also this comes currently with all the drawbacks that this isn't working cross machines and you might even give wrong users credits since we're not resetting that when logging out.

Also we'd usually try to put the key into an exported const some place so we don't need to copy it all over the place.

airbyte-webapp/src/packages/cloud/cloudRoutes.tsx

timroes · 2022-10-07T09:24:18Z

airbyte-webapp/src/packages/cloud/views/layout/MainView/MainView.module.scss

@@ -25,5 +25,13 @@ $banner-height: 30px;
        height: calc(100% - #{$banner-height});
      }
    }
+
+    &.speedyConnectionBanner {
+      margin-top: 50px;


🧹 We'd try to extract that in a variable, same as above.

airbyte-webapp/src/packages/cloud/views/layout/MainView/MainView.tsx

timroes · 2022-10-07T09:26:53Z

airbyte-webapp/src/pages/OnboardingPage/useStepsConfig.tsx

@@ -13,7 +14,14 @@ const useStepsConfig = (
  setCurrentStep: (step: StepType) => void;
  steps: Array<{ name: JSX.Element; id: StepType }>;
 } => {
+  // exp-speedy-connection
+  const location = useLocation() as unknown as ILocationState<{ step: StepType }>;


🧹 We'd usually encapsulate this into a separate hook useLocationState that just takes a generic, so we don't need to cast in the consuming place to unknown and back, but have that ugly cast limitd to the one place inside the hook.

…vation * master: (32 commits) fixed octavia position and z-index on onboarding page (#17708) Revert "Revert "Do not wait the end of a reset to return an update (#17591)" (#17640)" (#17669) source-google-analytics-v4: use hits metric for check (#17717) Source linkedin-ads: retry 429/5xx when refreshing access token (#17724) 🐛 Source Mixpanel: solve cursor field none expected array (#17699) 🎉 8890 Source MySql: Fix large table issue by fetch size (#17236) Test e2e testing tool commands (#17722) fixed escape character i18n error (#17706) Docs: adds missing " in transformations-with-airbyte.md (#17723) Change Osano token to new project (#17720) Source Github: improve 502 handling for `comments` stream (#17715) #17506 source snapchat marketing: retry failed request for refreshing access token (#17596) MongoDb Source: Increase performance of discover (#17614) Testing tool commands for run scenarios (#17550) Kustomize: Missing NORMALIZATION_JOB_* environment variables in stable-with-resource-limits overlays (#17713) Fix console errors (#17696) Revert: #17047 Airbyte CDK: Improve error for returning non-iterable from connectors parse_response (#17707) #17047 Airbyte CDK: Improve error for returning non-iterable from connectors parse_response (#17626) 📝 Postgres source: document occasional full refresh under cdc mode (#17705) Bump Airbyte version from 0.40.12 to 0.40.13 (#17682) ...

timroes

Code LGTM. Left cleanup comments. Testes (the 2nd previous version) locally. Requiring still a green CI, but looks fine for me.

…vation * master: (22 commits) Update full-refresh-append.md (#17784) Update full-refresh-overwrite.md (#17783) Update incremental-append.md (#17785) Update incremental-deduped-history.md (#17786) Update cdc.md (#17787) 🪟 🔧 Ignore classnames during jest snapshot comparison (#17773) feat: replace openjdk with amazoncorretto:17.0.4 on connectors for seсurity compliance (#17511) Start testing buildpulse. (#17712) Add missing types to the registry (#17763) jobs db descriptions (#16543) config db data catalog (#16427) Update lowcode docs (#17752) db migrations to support new webhook operations (#17671) Bump Airbyte version from 0.40.13 to 0.40.14 (#17762) September Release Notes (#17754) Revert "Use java-datadog-tracer-base image (#17625)" (#17759) Add connection migrations for schema changes (#17651) Connection Form Refactor - Part Two (#16748) Improve E2E testing around the Connection Form (#17577) Bump strict encrypt version (#17747) ...

…vation * master: (98 commits) 🐛 Source Bing Ads - Fix Campaigns stream misses Audience and Shopping (#17873) Source S3 - fix schema inference (#17991) 🎉 JDBC sources: store cursor record count in db state (#15535) Introduce webhook configs into workspace api and persistence (#17950) ci: upload test results to github for analysis (#17953) Trigger the connectors build if there are worker changes. (#17976) Add additional sync timing information (#17643) Use page_token_option instead of page_token (#17892) capture metrics around json messages size (#17973) 🐛 Correct kube annotations variable as per the docs. (#17972) 🪟 🎉 Add /connector-builder page with embedded YAML editor (#17482) fix `est_num_metrics_emitted_by_reporter` not being emitted (#17929) Update schema dumps (#17960) Remove the bump in the value.yml (#17959) Ensure database initialization in test container (#17697) Remove typo line from incremental reads docs (#17920) DocS: Update authentication.md (#17931) Use MessageMigration for Source Connection Check. (#17656) fixed links (#17949) remove usages of YamlSeedConfigPersistence (#17895) ...

…irbytehq#17593) * 🪟🧪 [Experiment] Incentivize speedy first connection (aka activation)

🪟🧪 [Experiment] Incentivize speedy first connection (aka activation)

749d577

letiescanciano requested a review from a team as a code owner October 5, 2022 06:24

letiescanciano commented Oct 5, 2022

View reviewed changes

remove duplicated timestamp creation

880709e

letiescanciano commented Oct 5, 2022

View reviewed changes

airbyte-webapp/src/packages/cloud/services/auth/AuthService.tsx Show resolved Hide resolved

remove duplicated unneeded changes

fa73103

letiescanciano commented Oct 5, 2022

View reviewed changes

letiescanciano requested a review from timroes October 5, 2022 06:32

github-actions bot added area/platform issues related to the platform area/frontend Related to the Airbyte webapp labels Oct 5, 2022

timroes requested changes Oct 5, 2022

View reviewed changes

letiescanciano added 2 commits October 6, 2022 11:31

PR comments

f62fcc4

remove unneeded code

bd71ec7

letiescanciano requested a review from timroes October 6, 2022 09:32

letiescanciano added 3 commits October 6, 2022 15:21

WIP

56881b1

fix content top margin

0397b53

letiescanciano added 3 commits October 6, 2022 16:28

fix import

7eb04fe

improve formatted message

42bab80

cleanup

d33417f

josephkmh reviewed Oct 6, 2022

View reviewed changes

airbyte-webapp/src/hooks/services/Experiment/experiments.ts Outdated Show resolved Hide resolved

letiescanciano added 3 commits October 7, 2022 08:38

fix typo

4129a01

add metadata to create connection event

6d1f9a4

add trialExpiryTimestamp to event

76c069b

timroes reviewed Oct 7, 2022

View reviewed changes

letiescanciano added 2 commits October 7, 2022 12:21

move hasConnections check to custom hook

3062b21

final changes

3ce4f84

letiescanciano requested review from timroes and josephkmh October 7, 2022 15:36

timroes approved these changes Oct 10, 2022

View reviewed changes

letiescanciano added 8 commits October 10, 2022 10:21

add useExperimentSpeedyConnection mock

ad3f302

add useExperimentSpeedyConnection mock

c2950f9

increase timeout for e2e

2002658

remove cloud vs oss dependencies

e2d34a6

move experiments/ to packages/cloud

cd6f24e

remove unneccesary mock

0a3fff5

letiescanciano merged commit 5466a68 into master Oct 20, 2022

letiescanciano deleted the leti/experiment-speedy-connection-for-activation branch October 20, 2022 06:56

octavia-squidington-iii mentioned this pull request Oct 20, 2022

Bump Airbyte version from 0.40.15 to 0.40.16 #18268

Merged

jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022

🪟🧪 [Experiment] Incentivize speedy first connection (aka activation) (a…

b4635e8

…irbytehq#17593) * 🪟🧪 [Experiment] Incentivize speedy first connection (aka activation)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🪟🧪 [Experiment] Incentivize speedy first connection (aka activation) #17593

🪟🧪 [Experiment] Incentivize speedy first connection (aka activation) #17593

letiescanciano commented Oct 5, 2022 •

edited

Loading

letiescanciano Oct 5, 2022

letiescanciano Oct 5, 2022

letiescanciano Oct 5, 2022

letiescanciano Oct 5, 2022

letiescanciano Oct 5, 2022

timroes commented Oct 5, 2022

timroes Oct 5, 2022

timroes Oct 5, 2022

timroes Oct 5, 2022

letiescanciano Oct 6, 2022

timroes Oct 5, 2022

letiescanciano Oct 6, 2022

timroes Oct 5, 2022

timroes Oct 5, 2022

letiescanciano Oct 6, 2022

timroes commented Oct 5, 2022

letiescanciano commented Oct 6, 2022

timroes left a comment

timroes Oct 6, 2022

timroes Oct 6, 2022

letiescanciano Oct 7, 2022

timroes Oct 6, 2022

timroes Oct 6, 2022

timroes Oct 6, 2022

letiescanciano Oct 7, 2022

timroes Oct 6, 2022

timroes Oct 7, 2022

timroes Oct 7, 2022

timroes left a comment •

edited

Loading

	const expiredOfferDate = timestamp ? JSON.parse(timestamp) : 0;
	const expiredOfferDate = timestamp ? Number(timestamp) : 0;

	<CountDownTimer {...{ expiredOfferDate }} />
	<CountDownTimer expiredOfferDate={expiredOfferDate} />

		@@ -0,0 +1,18 @@
		import { useExperiment } from "hooks/services/Experiment";

🪟🧪 [Experiment] Incentivize speedy first connection (aka activation) #17593

🪟🧪 [Experiment] Incentivize speedy first connection (aka activation) #17593

Conversation

letiescanciano commented Oct 5, 2022 • edited Loading

What

How

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timroes commented Oct 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timroes commented Oct 5, 2022

letiescanciano commented Oct 6, 2022

timroes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timroes left a comment • edited Loading

Choose a reason for hiding this comment

letiescanciano commented Oct 5, 2022 •

edited

Loading

timroes left a comment •

edited

Loading