Universal Analytics Plugin Online Hackathon – Dual tracking

I’ve been thinking about doing a Google Analytics related hackaton for a long time. Some months ago, I started to take a look about how Universal Analytics Plugins work and I decided that coding a plugin to all the data to a secondary property using just a plugin would be a real nice example.

For years now, I’ve sharing a lot of code that I’ve worked on, some tracking ideas too, but still I don’t consider myself a developer, if i must say it, I really think that I really suck at programming even if I can do some stuff myself.

So here I am trying to organize an online Universal Analytics Hackaton. I hope this can turn on a great change to learn from other people, and understand how plugins work!!!

Of course you may be asking what’s a “Hackathon” (don’t be shy about asking). Let’s quote the Wikipedia:

A hackathon (also known as a hack day, hackfest or codefest) is an event in which computer programmers and others involved in software development and hardware development, including graphic designers, interface designers and project managers, collaborate intensively on software projects. Occasionally, there is a hardware component as well. Hackathons typically last between a day and a week. Some hackathons are intended simply for educational or social purposes, although in many cases the goal is to create usable software. Hackathons tend to have a specific focus, which can include the programming language used, the operating system, an application, an API, or the subject and the demographic group of the programmers. In other cases, there is no restriction on the type of software being created.

GitHub Repository:

https://github.com/thyngster/universal-analytics-dual-tracking-plugin

For now I’ve pushed to the repository  with some “core” code, that “already” works.

How to load the plugin:

ga('create', 'UA-286304-123', 'auto');
ga('require', 'dualtracking', 'http://www.yourdomain.com/js/dualtracking.js', {
    property: 'UA-123123123213-11',
    debug: true,
    transport: 'image'
});
ga('dualtracking:doDualTracking');
ga('send', 'pageview');

Some stuff you need to take in mind when loading a plugin in Google Analytics:

  • The plugin needs to be hosted within your domain
  • It needs to be “initialized” AFTER the “create” method call and BEFORE the “pageview” method.
  • If for some reason the plugin crashes it may affect your data collection, please don’t use this in production before it has been fully tested.

Still it needs to be improved, for example:

  1. We don’t want to use global variables
  2. Payload size check, and based on the results send a POST or GET request
  3. Add XHR transport method
  4. Code cleanup/Best practises
  5. Plugin option to send a local copy for the hits
  6. Better debug messages
  7. Name convention improvement
  8. Any other idea?

Anyone is welcome to push code, add ideas, give testing feedback, through the Github repository or the comments on this blog post.

 

 

 

 

Getting super clean content reports in Google Analytics using GTM

In Google Analytics the urls are case sensitive, therefore in our content reports /index.html will be different to /Index.html, and querystring values will make Google Analytics to think that even if it’s the same page it will recorded as a new one, /index.html?cache=off and /index.html?cache=on will be recorded as 2 different pageviews for example.

The first problem its easily fixable with a lowercase filter within the views, but the querystring parameters it’s going to be a problem … I bet you’re saying that you can just add them to the Exclude URL Query Parameters list within your view configuration page and Yes! that’s right, but I’m pretty sure that you’re likely going to end having some marketing campaigns adding new parameters, or IT adding some parameters to get some funcionality switched on (like enabling some caching feature or whatever).

So today, we’ll be using Google Tag Manager to solve this problem of having all our content reports fragmented due the unexpected querystring parameters in our pages. So let’s think about it, wouldnt be easier to identify the real parameters and getting ride of the rest that are not expected for the page functionality?, If you think about it, it’s likely a better way to do it, we can know which parameters will be used in our site, but we cannot think on unexpected ones.

To achive this, we’re going to make use of just one single variable in Google Tag Manager, yeah that’s it, just one single Custom Javascript variable.

We’ll just need to configure the paramList array on the code top, and add there all the querystring parameters that we want to keep. Any other parameter that is not listed in our array will be removed from the querystring value that is going to be recorded by Google Analytics

function(){
  try{
    
        // We'll need to defined the QS values we want to keep in our reports         
        var paramsList = ["two","one","three"];

        // CrossBrowser inArray polyfill 
        if (!Array.prototype.indexOf) {  
            Array.prototype.indexOf = function (searchElement /*, fromIndex */ ) {  
                "use strict";  
                if (this == null) {  
                    throw new TypeError();  
                }  
                var t = Object(this);  
                var len = t.length >>> 0;  
                if (len === 0) {  
                    return -1;  
                }  
                var n = 0;  
                if (arguments.length > 0) {  
                    n = Number(arguments[1]);  
                    if (n != n) { // shortcut for verifying if it's NaN  
                        n = 0;  
                    } else if (n != 0 && n != Infinity && n != -Infinity) {  
                        n = (n > 0 || -1) * Math.floor(Math.abs(n));  
                    }  
                }  
                if (n >= len) {  
                    return -1;  
                }  
                var k = n >= 0 ? n : Math.max(len - Math.abs(n), 0);  
                for (; k < len; k++) {  
                    if (k in t && t[k] === searchElement) {  
                        return k;  
                    }  
                }  
                return -1;  
            }  
        }  
        var qsParamsSanitizer= function(qs,permitted_parameters){
        var pairs = qs.slice(1).split('&');
        var result = {};
        pairs.forEach(function(pair) {
            pair = pair.split('=');
            result[pair[0]] = decodeURIComponent(pair[1] || '');
        });

        var qsParamsObject = JSON.parse(JSON.stringify(result));
        for (var p in qsParamsObject){
            if(permitted_parameters.indexOf(p)==-1)
                delete qsParamsObject[p];
        }
        var rw_qs = '?' + 
                Object.keys(qsParamsObject).map(function(key) {
                    return encodeURIComponent(key) + '=' +
                        encodeURIComponent(qsParamsObject[key]);
                }).join('&');
        if(rw_qs=="?") rw_qs="";
        return rw_qs;
     }        
     return qsParamsSanitizer(document.location.search,paramsList);
    }catch(e){
       // let's let GA to use the current location.href if
       // for some reason our code fails.
       return undefined;
    }
}

Now, we only need to set our pageview tag “page” parameter so Google Analytics uses the new sanitized array instead of the one that it’s on the url.

We’re done!. Let’s see how it works with a screenshot

Now you just need to sit down, and wait some hours to start seeing your reports in a clean way and with no fragmentation. Happy analyzing!


 

 

Tricks to deal with customized Google Analytics and Google Tag Manager integrations

In the past years, I worked on implementing Google Analytlics and Google Tag Manager in a lot of differences scenearios and almost any kind of integrations. So I’m gonna show you some code snippets that may help you integrating Google Tag Manager and Google Analytics into your libraries in the best way possible.

Checking is Universal Analytics is available

Most integrations I’ve seen does check for the ga object within the DOM before trying to push something to it. This is really nice, and a check that should be done ALWAYS, but Universal Analytics allows you to change the default name for the “ga” object, so under those situations our code may not work as we’re expecting.

Luckly for us, Universal Analytics creates another variable, named window.GoogleAnalyticsObject, that will allow us to grab the current object name being used by Universal Analytics.

So using the following code will allow us to check is Google Analytics is really available even if the current name has been changed:

if(window.GoogleAnalyticsObject && typeof(window[GoogleAnalyticsObject])=="function"){ 
   // Code Here
}

Checking for current dataLayer object name for Google Tag Manager

Google Tag Manager users the “dataLayer” name by default, but it may changed to use any other name.

This piece of code will allow us to find the right dataLayer name dinamically:

var gtm_datalayer_names=[];
if(typeof(google_tag_manager)!="undefined")
    for(i in google_tag_manager){
        if(google_tag_manager[i].gtmDom==true){
            gtm_datalayer_names.push(i);        
        }        
}

The we could do something like:

window[gtm_datalayer_names[0]].push({});

We’ll even be able to send pushes to all the different dataLayer available on the current page:

if(gtm_datalayer_names.length>0){
        var i = window.gtm_datalayer_names.length;
        while(i--){
           var dlName = gtm_datalayer_names[i];
           window[gtm_datalayer_names] = window[gtm_datalayer_names] || [];                 
           window[gtm_datalayer_names[i]].push({
              'event': 'my-event-name',
              'dataLayerName': gtm_datalayer_names[i]
           });                                           
        }    
}

Debugging and monitoring GTM Variables for errors

Google Tag Manager does not throw any error when the code in a variable fails. This is not bad per se as it would save us from having our sites failing if something is not coded as it should, but it will blind us agains some minor errors that may not be noticiables, for example if our Enhanced Ecommerce is failing for a certain product, or from some specific browser.

Thanksfully we can use try{}catch(e){} to capture those errors :). And we could use it to send an event to GA so we can monitor and fix the errors easily, having also all the info about where did the error happened, which browsers, location, etc.

The main problem of sending an event within a variable error is that we may end having dozens of duplicated events for each error as variables got executed several times and not just once . The following piece of code will loop thru all the dataLayer pushes to find out if the current error has been already pushed to avoid this problem.

try{
   // JS Variable Code Goes Here
}catch(e){
    if(dataLayer.filter(function (obj) { return obj.errorMsg === e.message; }).length==0){
        dataLayer.push({'event':'variable error','errorMsg': e.message});
    }
}

Now just set an event that fires on the custom event named “variable error”, and that reads errorMsg variable value from the dataLayer.
You can easily personalize this to your needs.

Let’s see an small report for an Enhanced Ecommerce Implementation that is using Variables instead of the dataLayer values to populate the info:

debug_gtm_var_01

Those errors won’t prevent the hit to be sent, but ecommerce data will be missing on them.

But this is not where it ends, now you can use Real Time reporting to monitor your recently published container and be able to fix things in almost no time:

debug_gtm_var_02

 

Enabling Optimizely native Google Analytics Integration within Google Tag Manager

As you may already know, Optimizely has an in-built Google Analytics integration, which will allow us to track the experiments and variations being showed to a client using custom dimensions.

For enabling them, you’ll need to put the following lines after the create command and the pageview one:

window.optimizely = window.optimizely || [];
window.optimizely.push("activateUniversalAnalytics");

This is a bit tricky when using Google Tag Manager native tags, since it doesn’t allow you to execute anything after the tracker creation command.

To achive this we’ll be using a little hack ( using a Custom HTML tag ), and the tag secuencing that was released some months ago in Google Tag Manager.

Code
<script>
 
  window['ga']=window['ga']||function(){
  (window['ga'].q=window['ga'].q||[]).push(arguments);
  if(arguments[0]=="create")
  {
    window.optimizely = window.optimizely || [];
    window.optimizely.push("activateUniversalAnalytics");
  }
}
</script>

Now we’ll need to set a tracker name for our pageview tag, since Optimizely will try to set the custom dimension data to a tracker name and Google Tag Manager created a random tracker name for each fired Universal Analytics Tag.

After this we’ll need to enable Google Analytics within our experiment and define to which custom dimension index we want to send the data:

#Tip – Finding Legacy GA code after migrating to Universal Analytics

 

You may have already migrated your Google Analytics Implementation to Universal Analytics. It may happen that you still have some legacy ga snippet lying around in some pages without having noticed it, for example in some landing pages that are not belong the default CMS system/templates.

But we can easily check this out just looking at Google Analytics data. It’s as easy as crossing your pageview/event reports with the “Data Source” dimension.

This dimension will be only set when using Universal Analytics endpoint. The hits sent to __utm.gif will have that dimension set as “(not set)” while Universal Analytics hit will force that dimension to be “web”.

 

So, that is!. It couldn’t be more easy, and I’d suggest to include this comprobation in your Universal Analytics migration checklist. 

In almost no time, you’ll be aware of :

  • Pages with an old tracking code.
  • Pages that may be missing the new code (if we did not remove the old one, it’s right to think that we may not tag them with the new one).
  • Pages with duplicate codes (GA + UA)

 

 

 

Finding pages missing Google Tag Manager snippet within Google Tag Manager

This time, we’ll be using Google Tag Manager itself, to find pages in our site that may be missing Google Tag Manager. Ok, it may sound strange, but was not able to find any other way to say it .

Basically we’re going to use a custom html tag, to detect is the previous page had loaded Google Tag Manager code.

To achive this we’ll be using a single Custom HTML tag ,a cookie and the onbeforeunload event.

The following flow chart will show you the logic that we’re going to follow to detect the pages that are missing the Google Tag Manager snipper for our site.

So we’ll create a new custom HTML tag (that will be fired for ALL pages default trigger), and then add the following code to it, remember to update the “hostName” variable value to your domain root:

<script>
  (function(){
    // Change this to your root domain name
  var hostName = 'thyngster.com';
    // On before Unload 
  window.onbeforeunload = function(e) {
  	expire = new Date();
  	expire.setTime(new Date()*1+10*1000);
  	document.cookie = "__ttt="+escape(1) + ";expires="+expire.toGMTString()+ ";domain=."+hostName+";path=/";
   	};  

    // If user's coming from elsewhere but our domain, exit.  
    if(location.hostname.split('.')[location.hostname.split('.').length -2].length==2)
      var rootDomainName = location.hostname.split('.').slice(-3).join('.');
    else
      var rootDomainName = location.hostname.split('.').slice(-2).join('.');
    if(document.referrer.indexOf(rootDomainName)==-1)
      {return;}


    function isCookiePresent(name) {
      match = document.cookie.match(new RegExp(name + '=([^;]+)'));
      if (match) return true; else return null;
    }

    if(!isCookiePresent('_ttt')){
      dataLayer.push({
        'event':'missingtag',
        'referrer': document.referrer
      });
      document.cookie = '__ttt=; expires=Thu, 2 Aug 2001 20:47:11 UTC; domain=.'+hostName+';path=/';
    }  
  })();  
</script>

Ok, that’s all! 

Now we could create a new Event tag that fires when the event equals to “missingtag” to send that data to Google Analytics.

If you’re struggling with some hits being sent from hardcoded tags, I suggest you to take a look to this great tip from @fastbloke  for being able to find them, and you could look to this other old post

If you’re sending the data to Google Analytics, remember to set the event to a nonInteractional hit!

 

Any feedback, comment or improvement will be really appreciated.

 

 

Tracking the Hover Intent with Google Tag Manager

At the moment, Google Tag Manager listeners are limited to Form Submits, Link Clicks and Clicks (for any DOM element). We are going to write a custom listener for hover intents by users. This means that we’re not only to do something when the user pass the mouse pointer over an element, but instead we’re waiting a certain time with the mouse over the element before submitting the action to GTM.

For this we’re going to use the mouseenter and the mouseleave javascript events.

The mouseenter event is fired when the mouse is moved over a DOM element, e mouseleave  in the other side is fired when the mouse leaves the DOM Element.

So we are going to start a time, when the mouseenter event occurs, and then the mouseleave event to disable the previous time. This way, if the pointer is not over an element for the set time, nothing will be sent to Google Tag Manager.

Instead of pushing our data as we usually do, this time we’ll do it the same way Google Tag Manager does with it’s built-in listeners, so we’ll be pushing a gtm.hover  event to the dataLayer that will look this way:

We’ll have the current gtm.element  info, and the gtm.elementClasses , gtm.elementId as we have with the others listeners in Google Tag Manager to allow us to set our triggers.

We’ll need to create a new tag, let’s call it for example : “TOOL – Hover Intents Listener” and let’s put the following code into that tag:

// Tracking Top Links Hover Menus
  function trackHoverIntent(selector, time) {
    var timeoutId;
    var elements = window.document.querySelectorAll(selector);
    for (var i = 0; i < elements.length; i++) {
      elements[i].addEventListener('mouseenter', function(event) {
        var targetElement = event.target || event.srcElement;
        var classes = targetElement.className;
        var id = targetElement.id;
        if (!timeoutId) {
          timeoutId = window.setTimeout(function() {
            dataLayer.push({
              'event': 'gtm.hover',
              'gtm.element': targetElement,
              'gtm.elementClasses': classes,
              'gtm.elementId': id,
              'gtm.elementHoverTime': time
            });
            timeoutId = null;
          }, time);
        }
      });
    }

    for (var i = 0; i < elements.length; i++) {
      elements[i].addEventListener('mouseleave', function() {
        if (timeoutId) {
          window.clearTimeout(timeoutId);
          timeoutId = null;
        }
      });
    }
  }

#Tip: Remember to put the code above betweentags if you don’t wanna inject a bunch of code into your visible HTML code.

Then we’ll need to add the listeners to the elements we want this way:

trackHoverIntent('img.thumbnail', 1500);

For example this code above will send a gtm.hover event everytime the visitor moves his mouse over an image with the class “thumbnail” for more than 1.5 seconds.

 

Track alert() pop ups automatically in Google Tag Manager

Did you ever faced a site that still used the old-fashined alert popups for example for showing a form validation messages? and you were wondering how to track that form alert popups without needing to change anything on the site (as it is likely going to be an old one). And yep,they still exist. Hopefully if we are already using Google Tag Manager (or we can tell the client to add some little piece of code to their pages) we could track them easily using what is called a Proxy Pattern.

Sadly this won’t be supported by IE8 for the alerts as it seems it’s not a function but an object, therefore we’re adding a check for pre IE9 browsers.

In essence, we’re saving the current window[“alert”] function in a new variable, and the redefining the same function, but returning the passed arguments to our original saved function, preserving the original function behaviour.

We’ll just need to create a Custom Html Tag that will fire either in all pages or just for the ones that we want to track alerts on.

// Let's check if the browser is <=IE8
if(!(document.all && !document.addEventListener))
{ 
 // Just in case check, to avoid loops
 if(!window.proxied_alert)
 {
   window.proxied_alert = window.alert; // Preserve the original function
   window.alert = function() { 
     var message = (!arguments[0]) ? 'null': arguments[0];
     dataLayer.push({'event': 'alert_showed','alert_message': message});
     return proxied_alert.apply(this, arguments);
   }
 }
}

In this example we’re pushing the info to the dataLayer, but we could directly do anything like firing an event using the native ga/_gaq functions or pushing the info to any other tool we want.

As we’re modifying the original window.alert function, I suggest  to deeply debug/test the site functionality after implementing this code, as it may happen that other script is doing the same thing to the alert function. (some modals scripts plays with this to automatically convert the alert calls to a nicer popups).

Tips to track an ajax based website using GTM and Universal Analytics

I’ve been dealing with some Ajax based website tracking these past months (mostly AngularJS based ones), and I’ve learn a lot of stuff , mostly some important points about how Google Tag Manager does work.

Usually tracking an Ajax based website is a PITA, since mostly we don’t have page reloads as all the content is loaded asynchronously. So either you ask the developers to push you the content loaded info into the dataLayer or you play with the history location.

Still we need to have some points in mind while tracking those sites with Google Tag Manager (and maybe natively or using any other tag management system).

1. Use a fixed tracker name

The way Google Tag Manager works, everytime a Google Analytics tags is fired it creates a new tracker with a name (“GTM”+Current timeStamp ). This is not a problem for most tracking implementations, but we need to take in mind that websites based in Ajax usually changes the current URI without even needing to reload the page, so it may happen that if we’re using the campaign tagging we may ending having a lots of unneeded visit respawn and referrals.

Let me explain this with a simple example:

https://www.domainlinkingtous.com has a link to our site and it’s using the campaign tagging to allow us to track their visits to us.

http://www.ourdomain.com?utm_source=test_source&utm_medium=test_medium&utm_campaign=test_campaign

So the visit will end in our page with the following values:

URL: http://www.ourdomain.com?utm_source=test_source&utm_medium=test_medium&utm_campaign=test_campaign
Referrer: http://www.domainlinkingtous.com

Let’s map this to Universal Analytics parameters:

dl: http://www.ourdomain.com?utm_source=test_source&utm_medium=test_medium&utm_campaign=test_campaign
dr: http://www.domainlinkingtous.com

Universal Analytics uses the “dl” parameter  (document.location) and dr(document.referrer), to calculate the visit attribution. It will search for gclid, utm_ parameters in the “dl” parameter value (in that order of preference) and if it doesn’t find them it will try to calculate the attribution from the “dr” string.

As we have the the utm parameters in our landing page, the referrer won’t be taken in mind to calculate the visit attribution, so we’re ok at this point.

source: test_source
medium: test_medium
campaign: test_campaign

But if the visitor decides to go to another page, and like we said in this case there will be no page reload  Google Tag Manager will spawn a new tracker and we’ll end having those values.

dl: http://www.ourdomain.com/secondPage.html
dr: http://www.domainlinkingtous.com

As we said before if there’re no utm or gclid parameters in the document.location Google analytics in going to look to the document.referral string and it will find a different domain than the current loading one, so it will track a new visit:

source: domainlinkingtous.com
medium: referral
campaign: /

D’oh. This is wrong, we’ll miss the original visit attribution, plus we’ll have some extra ghost sessions in our data.

Best way to do this (even if it’s not recommended) to set a fixed tracker name in our tags.

ajax_sites-01

This way, the same tracker instance will be used everytime and the original dl parameter will be kept.

2. Force the Non-Interaction Flag to false in your pageviews

This is another important point to have in mind. Google Tag Manager uses the set command to set the values instead of setting them directly to hit. So if we fire and non-interactive event tag, the subsecuent hits will inherit that flag making the subsecuente pageviews to have the “ni” parameter attached to them.

Let’s how Google Tag Manager is going to translate a single Event tag:

ga("create", "UA-40180485-1", {"cookieDomain": "auto","name": "myTracker"});
ga("myTracker.set", "&gtm", "GTM-MW3DNR");
ga("myTracker.set", "nonInteraction", true);
ga("myTracker.send", {"eventAction": "test","eventCategory": "test", "eventLabel": "test","eventValue": undefined,"hitType": "event"});

As you can see the nonInteraction is globaly for the tracker, and this will affect our next pageviews (remember that we don’t have any page reload so the nonInteraction flag will be kept till we disable it.

An easy going fix for this without needing to deal with hitCallback/eventCallback is to automatically remove the “ni” parameter for all pageviews as it’s not likely going to need it (ever used a non-interactional pageview?).

Let’s see how can we achive this. We can’t use undefined string on GTM input boxes as it will be treated as a string, therefore we’re going to create a new Variable that is going to return us the real undefined value.

ajax_sites-02
N
ow we can safely force the “ni” parameter to undefined in our pageview tags this way:

ajax_sites-03

Did you fance any other problems when tracking ajax based sites?, share them in a comment 🙂

Greets: Thanks fly one more time to Yehoshua Coren from Analytics Ninja, for reviewing these points with me.