Overview

‘The Funnel’ is the most important thing in a startup. Yes, arguably more important than the app itself since you can build a funnel to gauge interest in an idea even before you start to build. (Some would even say that you ‘should’ rather than ‘can.’) Marketing and Sales activities are going to start next month for Events in Plain Sight so this week I had the goal of instrumenting things so I can see where I have problems. Of course, some design decisions around how to sign up and create a new Community are going to complicate my life, but what else is new.

There are actually ish 2 funnels for EiPS; one for getting users to register, and one for creating a new Community. Users have to register in order to create Events and so I don’t care too to much about the ‘registration’ route, but I do care a lot about what converts to a new Community since that is what is monetized and will need all the insights I can get. Do users who convert first interact with another Community? Do they come from a lead magnet? Where do they come from? Etc. That’s where analytics come in.

Here is what the conversion funnel looks like;

[logs into the application]
goes to their Dashboard
clicks the ‘Create Community’ button
fills out the form (successfully, no validation errors) which takes them back to the Dashbord
clicks on the ‘Billing’ button next to their Community
on that page, they click on the Subscription they want
which redirects them to Stripe to do the billing, and then it redirects back to the billing page

Is this great? Absolutely not. I’ve not even started to optimize this. Am I going to offer a trial of some sort? Wizard-ize the new Community form? Unknown. But that’s a problem for another time.

Google Tag Manager (the ‘easy’ part)

I’m using Google Tag Manager (GTM) with Google Analytics (GA) do this instrumentation. Which gets me 80% of the way as everything is in a browser.

The first thing that needed to be down was getting the GTM code loaded and reporting home. I used Easily setup and send data to Google Tag Manager in Laravel apps (then unused, then used again) which Just Worked[tm].

And there was a detour around ‘what the heck is GTM?!?!’ – What Is Google Tag Manager & How Does It Work? was what finally made the pieces click for me. Essentially, GTM injects other bits of JS into the browser, the most obvious is GA. But can integrate with a number of CRM/marketing platforms.

I appreciate the problem it solves and lets you muck with analytics / tracking without needing development or deploys after the initial one. But good gravy it is it not straight forward at times. For example, to fire an event from a button click (like ‘Create Community’) you have to create a Trigger (what is happening in the browser) and tie it to an Event. Again, I get it. The power is unlocked at scaled, but it feels like unnecessary steps.

Google Analytics Measurement Protocol (the ‘not easy’ part)

My Laravel Cashier checkout route redirects to the same place (the Dashboard) on success or failure. Could I have likely made my life a little easier if I redirected to a page that had the GTM stuff on it which fired and immediately sent them on their way to the Dashboard? Yes. I likely could have. But I still wanted an event in the stream when successfully creating a new Community into the database. There is no browser involved in that.

Obviously this is not a unique requirement, so Google provides a way for offline creation of events through the Measurement Protocol. Which was actually pretty to use. Once I had everything sorted out, which meant stashing, retrieving and parsing the GA cookies.

Cookies

Cookies live in the browser. Which means they are not available to the backend. Care to guess where GA stores the information you need? That’s right, in cookies. Specifically _ga and _ga_MEASUREMENT-ID. Well, it is actually the Measurement ID minus the ‘G-‘ at the beginning. So if your Measurement ID is G-1213456 the cookie you want is _ga_1213456.

Now that we know what cookies we need, we need to intercept them. Which is actually ‘pretty easy’ in the Laravel Request object.

$request->cookie('_ga')
$request->cookie('_ga_'.substr(config('services.google.ga4.measurement-id'), 2))

(Obviously I’m not hard coding various ids so I can switch them around whether I’m local or production, etc. Why not just store it without the G-? Because we need the full thing later. So six of one, half dozen of another.)

However, the above code will return empty cookie values due to the default configuration of Laravel which encrypts 3rd party cookies. Fortunately, they provide a way to change this behaviour. Unfortunately it doesn’t work if you are storing the cookie names in config files because env() and config() aren’t available because the app isn’t booted yet.

You can get around this by adding this too the AppServiceProvider in the boot().

EncryptCookies::except([
    '_ga',
    '_ga_'.substr(env('GOOGLE_ANALYTICS_MEASUREMENT_ID'), 2),
]);

Now the content of the cookie will show up when fetched from $request.

Storage

I’m choosing to store the cookie contents in the cache as it is fast, is available to jobs, and technically could scale should I need it to. I also only [currently] need this information when someone is on the Dashboard so I could have put it in the Controller, but in order to not box myself into a corner I made a Route Middleware that I added to the route.

Some notes;

config('googletagmanager.enabled') is the toggle from the Spatie package mentioned above.
I’m keying the cache entry by the user id as that is who it is associated with. That way if they clear the cookies, etc. the new one will be picked up next request.
A week seemed like a reasonable time period to keep things in cache.
I thought about using the new Cache::flexible functionality, but I don’t think it really provides any value here

<?php

namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Auth;
use Illuminate\Support\Facades\Cache;

class GoogleAnalyticsMiddleware
{
    public function handle(Request $request, Closure $next)
    {
        if (config('googletagmanager.enabled')) {
            if ($request->hasCookie('_ga_'.substr(config('services.google.ga4.measurement-id'), 2))) {
                Cache::remember('ga-'.Auth::id(), 86400, function () use ($request) {
                    $payload = [
                        '_ga' => $request->cookie('_ga'),
                        '_ga_'.substr(config('services.google.ga4.measurement-id'), 2) => $request->cookie('_ga_'.substr(config('services.google.ga4.measurement-id'), 2)),
                    ];

                    return $payload;
                });
            }
        }

        return $next($request);
    }
}

Now that we have the values in the cache we need to use them. But first, a side quest.

Both cookies we stored have information encoded in them that we need to parse out. The _ga cookie is the easiest.

GA1.1.109778539.1768248623

GA1: This is a GA4 cookie
1: Domain level. 1 is the root. 2 is the first subdomain, 3 is the second subdomain, etc.
109778539: Random number
1768248623: Creation timestamp
109778539.1768248623: The ‘client id’ (this is needed later)

_ga_MEASUREMENT-ID is more complex. So I made a helper following this pattern I made a helper function to do this parsing as I’ll need it anywhere I’m trying to fire events from the backend.

<?php

use Illuminate\Support\Str;

if (! function_exists('cookieParser')) {
    function cookieParser($raw): array
    {
        //        see https://www.tink.ca/en/insights/gs1-gs2-silent-transformation-session-cookies-google-analytics-4
        //        example: GS2.1.s1768595837$o5$g1$t1768595874$j23$l0$h0
        $crumbs = [];

        $crumbs['version'] = Str::substr($raw, 0, 5);

        $parts = explode('$', Str::substr($raw, 6));
        foreach ($parts as $key => $value) {
            match ($value[0]) {
                's' => $crumbs['session_id'] = Str::substr($value, 1),
                'o' => $crumbs['session_number'] = Str::substr($value, 1),
                'g' => $crumbs['engagement_level'] = Str::substr($value, 1),
                't' => $crumbs['last_hit_timestamp'] = Str::substr($value, 1),
                'j' => $crumbs['countdown'] = Str::substr($value, 1),
                'l' => $crumbs['connection_status'] = Str::substr($value, 1),
                'h' => $crumbs['encrypted_identifier'] = Str::substr($value, 1),
                'd' => $crumbs['join_id'] = Str::substr($value, 1),
            };
        }

        return $crumbs;
    }
}

Firing Events

As mentioned, I want to fire a GA event when the Community is created. (Side note, in the code a Community is a ‘Niche’ for legacy reasons.) That pattern is pretty easy; tell the model to fire an Event on ‘created’ and then hang a Listener on it. Because this is going to a 3rd party, I’m going the extra step of making it queued so they can be re-run if necessary.

<?php

namespace App\Listeners\GoogleAnalytics;

use App\Events\NicheCreatedEvent;
use Carbon\Carbon;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Cache;
use Illuminate\Support\Facades\Http;
use Log;

class NicheCreatedListener implements ShouldQueue
{
    use SerializesModels;

    public function __construct() {}

    public function shouldQueue(NicheCreatedEvent $event)
    {
        return config('googletagmanager.enabled');
    }

    public function handle(NicheCreatedEvent $event): void
    {
        $cookies = Cache::get('ga-'.$event->niche->owner_id);

        if (! $cookies) {
            Log::warning('Could not get client_id from cache.');
            //            have trackers turned off or such.
            return;
        }

        $parts = cookieParser($cookies['_ga_'.substr(config('services.google.ga4.measurement-id'), 2)]);

        $payload = [
            'client_id' => substr($cookies['_ga'], 6),
            'timestamp_micros' => Carbon::now()->format('Uu'),
            'events' => [
                [
                    'name' => 'niche_created',
                    'params' => [
                        'niche_id' => $event->niche->id,
                        'session_id' => $parts['session_id'],
                    ],
                ],
            ],
        ];

        $response = Http::post(
            'https://www.google-analytics.com/mp/collect'.
            '?measurement_id='.config('services.google.ga4.measurement-id').
            '&api_secret='.config('services.google.ga4.api-key'),
            $payload
        );

        if (! $response->successful()) {
            //            should never fire as endpoint explicitly does not return response codes other than 200
            $response->throw();
        }
    }
}

The docs on the Measurement Protocol are pretty decent. Here are the ones you will need for sure;

Send Measurement Protocol events to Google Analytics
Payload – format of the JSON sent over the wire
Validating Events – explains how to debug things since the endpoint /always/ returns 200 even if you mess up the format of the payload.

Most of that should be self-explanatory. The only interesting thing is that session_id is optional. By including it, your event will show up in the ‘Real-time’ reports.

I did similar for the ‘customer.subscription.created’ event from Cashier/Stripe to capture that a subscription was created. (Which, again, I wouldn’t have had to do if I had a different return route for the Stripe redirect.)

Funnel Exploration

With both the browser and non-browser parts of my Funnel working, I can put these events into my GA reporting as parting of a ‘Funnel Exploration.’ Currently it looks like this;

(browser) First open/visit
(browser) Session start
(browser) New Community Start
(backend) New Community Finish
(backend) Subscribed

You can have 10 Events tracked in each Funnel. So if I do a lead magnet or such, I can add it in, or make a new exploration.

Conclusion

I had penciled this in for a single day’s worth of work, but thanks to some non-trivial rabbit holes, it took all week. (In my defense, that week does include half a week of not working on EiPS at all.)

Learning Tag Manager is now on my list of tactical reccomendations to people building apps. And to recognize that a full analytics solution extends beyond the browser. Hopefully the rabbit holes won’t be as deep for the next person who finds this.

Laravel [Cashier] and Google Tag Manager / Google Analytics 4