Skip to main content
  1. Blog Post/

Universal Analytics Migration Library - Custom task

12 min · 4944 words
Table of Contents

When Google Announced Google Analytics 4, I started to work on this library, with the main aim to be used as a customTask.

Now the current deadline is 1 year-ish for all free Universal Analytics users and 3 extra months ( thanks? ) for paid users. In my very humble opinion, this is very short notice for such a big work. In my experience, a big implementation will take at least 6 months on average, for defining everything, write up the specs, have it team implement it, the GTM ( or any other TMS ) setup and then doing the QAing a dozen times until finally all the data is coming as expected and with the proper values.

The current help Google is offering in this situation is allowing us to turn on a switch on the interface that will autogenerate some GA4 events and pageviews from the Universal Analytics hits. If you wonder how they're doing this, they're adding a listener for the window.ga('send') calls.

At a first look, it may look like good help, but I really feel that's all against how Google Analytics 4 is supposed to be implemented. I really see a lot of issues coming from this:

    Another issue with the automatic migration tool Google offers is that an Event Action seems to be converted to an Event Name. That's a terrible thing, most of the clients I worked on, will end up having thousands of different events. ( I wonder if this is the reason for having a "non-written" no limit for the events coming from the website ).

    In any case, if you're migrating your setup to Google Analytics 4, there's a no better chance to fix whatever is not right with your setup. One of the most important implementation steps in GA4 is the data model definition, that's gonna define your current data quality and it's gonna limit you in the future ( event parameter limits, user properties limits, etc ). Don't take the lazy path and regret it later.

    Universal Analytics Payload Parser

    Now, the main post point, I've released a Universal Analytics Payload Parser library. This library is able of taking a Univeral Analytics Hit and converting them into some properly modeled dataLayer pushes.

    This library can be used as a customTask in Universal Analytics and it will push the hit contained data into a Google Tag Manager (dataLayer.push), Tealium IQ (utag.link) or Adobe Launch (_satellite.track) pushes.

    Instead of having someone send some predefined data into your GA4 property, this tool will push the data to your TMS so you can send the data to GA4 in the format, you want or pass the data to ANY other tool you want if at some point you want to migrate to any other analytics solution ( Snowplow, Matomo, Amplitude, Adobe Analytics, etc ), or even to any internal data solution you have built in your company.

    Features

    This library accepts a Universal Analytics Payload and parses it to return a formated and standardized object that can be used to pass the current info to any Tag Management System. At the time of writing this post, the library supports the output for Google Tag Manager, Tealium IQ, or Adobe Launch, adding a callback of support for sending the data to any external source should be pretty straightforward.

    This library will take all the UA hit types, and will create an object with all data contained on that hit.

      Supported Hit Types / Parameters

      Next, you can find the list of supported hit types and all the data that will be shipped with the generated object push.

      Hit TypeStatusDefault Parameters
      pageviewSupportedpage_title,page_location, page_path
      eventSupportedcategory, action, label, value, nonInt
      timingSupportedtiming_category, timing_value, timing_time, timing_label
      socialSupportedsocial_action, social_network, social_target
      exceptionSupportedexception_description, exception_is_fatal
      transactionSupportedtransaction_id, transaction_affiliation, transaction_revenue, transaction_shipping, transaction_tax, currency
      itemSupportedtransaction_id, item_id, item_name, item_price, item_quantity, item_variation. currency

      Ecommerce

      As we mentioned before the Universal Analytics Payload Parser the library will also take care of parsing the Enhanced Ecommerce data contained within the hits and will generate the following standalone Ecommerce Pushes.

      EventParameters
      view_item_list[item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, index,item_list_name]
      select_item[item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, index, item_list_name]
      view_item[item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price]
      add_to_cart[item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity]
      view_cart[item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity]
      being_checkout[item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity]
      remove_from_cart[item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity]
      purchase[item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity], transaction_id, value, tax, shipping, affiliation, coupon
      refund[item_id, item_name, quantity, item_brand, item_variant, item_category, item_category2, item_category3, item_category4, item_category5, price, quantity], transaction_id
      view_promotion[promotion_id, promotion_name, creative_name, creative_slot]
      select_promotion[promotion_id, promotion_name, creative_name, creative_slot]

      Setup / Configuration

      The library expects a configuration variable, like this one.

      var parserConfig = {
          tms: 'log',
          eventsName: 'ga_event',
          ecommerceEventsEnabled: true,
          skipTransportEvent: false,
          mapping: {
              keys: {
                  cd1: 'client_id',
                  cd3: 'logged_in',
                  cd7: 'is_in_stock',
                  cd19: 'referrer',
                  cd20: 'full_url',
                  // Metrics
                  cm201: 'votes',
                  // content Grouping
                  cg1: 'content_group_1',
                  cg3: 'page_type'
              },
              events: {
                  'checkout': {
                      '1' : 'view_cart',
                      '2' : 'begin_checkout'
                  }            
              }
          }
      }
                                      
      Option
      tmsgtm, tealiumiq, launch, log
      eventsNameThe event name to be used for the events.
      ecommerceEventsEnabledWhether or not to parse the e-commerce data and push it
      skipTransportEventIf we set this to true, the current hit/event will be skipped and only the EEC event will be generated
      mapping.keysOur dimensions, metrics, and content_grouping definition, this will convert the “cd1” payload key into something more readable like “client_id” in the pushes.
      mapping.events.checkoutGA4 not longer accepts checkout_steps, this will let you define which current Checkout Steps should be considered the new “begin_checkout” and “view_cart” events

      Examples

      pageview hit ( no dimensions )

      v=1&_v=j96&a=609235607&t=pageview&_s=1&dl=https%3A%2F%2Fwww.thyngster.com%2F&ul=es-es&de=UTF-8&dt=Web%20Analyst%20and%20Sr.%20Implementation%20Consultant%20-%20David%20Vallejo&sd=24-bit&sr=1920x1080&vp=432x1009&je=0&fl=Service%20Provider%7CNetwork%20Domain&_u=QACAAAABAAAAAC~&jid=2095545038&gjid=1011316425&cid=353185870.1650295583&tid=UA-286304-9&_gid=1709590183.1650826121&_r=1&gtm=2wg4k09LNT&cd1=353185870.1650295583&z=637815381

      Pageview Hit Payload

      Generated Object Push

      {
          "event": "page_view",
          "eventData": {
              "page_title": "Web Analyst and Sr. Implementation Consultant - David Vallejo",
              "page_location": "https://www.thyngster.com/"
          }
      }
                                      

      pageview hit ( with dimensions, metrics, content_groupings )

      v=1&_v=j96&a=609235607&t=pageview&_s=1&dl=https%3A%2F%2Fwww.thyngster.com%2F&ul=es-es&de=UTF-8&dt=Web%20Analyst%20and%20Sr.%20Implementation%20Consultant%20-%20David%20Vallejo&sd=24-bit&sr=1920x1080&vp=432x1009&je=0&&cd1=123123123123.2321231232132&cd2=homepage&cg1=homepage&cm3=1&fl=Service%20Provider%7CNetwork%20Domain&_u=QACAAAABAAAAAC~&jid=2095545038&gjid=1011316425&cid=353185870.1650295583&tid=UA-286304-9&_gid=1709590183.1650826121&_r=1&gtm=2wg4k09LNT&cd1=353185870.1650295583&z=637815381

      Generated Object Push

      {
          "event": "page_view",
          "eventData": {
              "client_id": "353185870.1650295583",
              "dimension_2": "homepage",
              "content_group_1": "homepage",
              "metric_3": "1",
              "page_title": "Web Analyst and Sr. Implementation Consultant - David Vallejo",
              "page_location": "https://www.thyngster.com/"
          }
      }
                                      

      event hit ( with dimensions, metrics, content_groupings and add_to_cart e-commerce details )

      v=1&_v=j96&a=136223288&t=event&cu=USD&_s=5&dl=https%3A%2F%2Fenhancedecommerce.appspot.com%2Fitem%2Fb55da&ul=es-es&de=UTF-8&dt=Product%20View&sd=24-bit&sr=1920x1080&vp=683x992&je=0&ec=ecommerce&ea=add_to_cart&ev=16&_u=SCCAAUALAAAAAC~&jid=1374636135&gjid=489822857&cid=625473581.1650821355&tid=UA-41425441-17&_gid=335165655.1650821355&_r=1&gtm=2ou4k0&pa=add&pr1id=b55da&pr1nm=Flexigen%20T-Shirt&pr1pr=16.00&pr1qt=1&pr1br=Flexigen&pr1ca=T-Shirts&pr1va=red&z=494832007

      Generated Object Pushes

      {
          "event": "ga_event",
          "eventData": {
              "category": "ecommerce",
              "action": "add_to_cart",
              "value": "16",
              "nonInt": true
          }
      }
                                      
      {
          "event": "add_to_cart",
          "items": [
              {
                  "item_id": "b55da",
                  "item_name": "Flexigen T-Shirt",
                  "price": "16.00",
                  "quantity": "1",
                  "item_brand": "Flexigen",
                  "item_category": "T-Shirts",
                  "item_variant": "red"
              }
          ]
      }
                                      

      event - including purchase info

      v=1&_v=j96&a=1751633849&t=event&cu=USD&_s=13&dl=https%3A%2F%2Fenhancedecommerce.appspot.com%2Fcheckout&ul=es-es&de=UTF-8&dt=Checkout&sd=24-bit&sr=1920x1080&vp=967x1009&je=0&ec=ecommerce&ea=purchase&ev=115&_u=SCCAAUALAAAAAC~&jid=&gjid=&cid=625473581.1650821355&tid=UA-41425441-17&_gid=335165655.1650821355&gtm=2ou4k0&pa=purchase&pr1id=b55da&pr1nm=Flexigen%20T-Shirt&pr1pr=16.00&pr1qt=3&pr1br=Flexigen&pr1ca=T-Shirts&pr1va=red&pr2id=8835a&pr2nm=Isoternia%20T-Shirt&pr2pr=57.00&pr2qt=1&pr2br=Isoternia&pr2ca=T-Shirts&pr2va=red&pr2ps=1&tr=115&tt=5.00&ts=5.00&z=1549501619

      Generated Object Pushes

      {
          "event": "purchase",
          "items": [
              {
                  "item_id": "b55da",
                  "item_name": "Flexigen T-Shirt",
                  "price": "16.00",
                  "quantity": "3",
                  "item_brand": "Flexigen",
                  "item_category": "T-Shirts",
                  "item_variant": "red"
              },
              {
                  "item_id": "8835a",
                  "item_name": "Isoternia T-Shirt",
                  "price": "57.00",
                  "quantity": "1",
                  "item_brand": "Isoternia",
                  "item_category": "T-Shirts",
                  "item_variant": "red",
                  "index": "1"
              }
          ],
          "value": "115",
          "tax": "5.00",
          "shipping": "5.00"
      }
                                      

      hit - impressions

      v=1&_v=j96&a=1653659752&t=event&ni=1&_s=2&dl=https%3A%2F%2Fenhancedecommerce.appspot.com%2F&dr=https%3A%2F%2Fwww.google.com%2F&ul=es-es&de=UTF-8&dt=Home&sd=24-bit&sr=2560x1440&vp=570x1321&je=0&ec=engagement&ea=view_item_list&_u=SCCAAUAL~&jid=&gjid=&cid=625473581.1650821355&tid=UA-41425441-17&_gid=335165655.1650821355&gtm=2ou4k0&il1nm=homepage&il1pi1id=9bdd2&il1pi1nm=Compton%20T-Shirt&il1pi1pr=44.00&il1pi1br=Compton&il1pi1ca=T-Shirts&il1pi2id=f6be8&il1pi2nm=Comverges%20T-Shirt&il1pi2pr=33.00&il1pi2br=Comverges&il1pi2ca=T-Shirts&il1pi2ps=1&il1pi3id=b55da&il1pi3nm=Flexigen%20T-Shirt&il1pi3pr=16.00&il1pi3br=Flexigen&il1pi3ca=T-Shirts&il1pi3ps=2&il1pi4id=bc823&il1pi4nm=Fuelworks%20T-Shirt&il1pi4pr=92.00&il1pi4br=Fuelworks&il1pi4ca=T-Shirts&il1pi4ps=3&il1pi5id=035f0&il1pi5nm=Futuris%20T-Shirt&il1pi5pr=55.00&il1pi5br=Futuris&il1pi5ca=T-Shirts&il1pi5ps=4&il1pi6id=8835a&il1pi6nm=Isoternia%20T-Shirt&il1pi6pr=57.00&il1pi6br=Isoternia&il1pi6ca=T-Shirts&il1pi6ps=5&il1pi7id=57b9d&il1pi7nm=Kiosk%20T-Shirt&il1pi7pr=55.00&il1pi7br=Kiosk&il1pi7ca=T-Shirts&il1pi7ps=6&il1pi8id=dc646&il1pi8nm=Lunchpod%20T-Shirt&il1pi8pr=90.00&il1pi8br=Lunchpod&il1pi8ca=T-Shirts&il1pi8ps=7&il1pi9id=7w9e0&il1pi9nm=Masons%20T-Shirt&il1pi9pr=31.00&il1pi9br=Masons&il1pi9ca=T-Shirts&il1pi9ps=8&il1pi10id=239b5&il1pi10nm=Pigzart%20T-Shirt&il1pi10pr=82.00&il1pi10br=Pigzart&il1pi10ca=T-Shirts&il1pi10ps=9&il1pi11id=6d9b0&il1pi11nm=Poyo%20T-Shirt&il1pi11pr=62.00&il1pi11br=Poyo&il1pi11ca=T-Shirts&il1pi11ps=10&il1pi12id=6c3b0&il1pi12nm=Zappix%20T-Shirt&il1pi12pr=99.00&il1pi12br=Zappix&il1pi12ca=T-Shirts&il1pi12ps=11&il2nm=shirts%20you%20may%20like&il2pi1id=6c3b0&il2pi1nm=Zappix%20T-Shirt&il2pi1pr=99.00&il2pi1br=Zappix&il2pi1ca=T-Shirts&il2pi2id=8835a&il2pi2nm=Isoternia%20T-Shirt&il2pi2pr=57.00&il2pi2br=Isoternia&il2pi2ca=T-Shirts&il2pi2ps=1&il2pi3id=035f0&il2pi3nm=Futuris%20T-Shirt&il2pi3pr=55.00&il2pi3br=Futuris&il2pi3ca=T-Shirts&il2pi3ps=2&il2pi4id=239b5&il2pi4nm=Pigzart%20T-Shirt&il2pi4pr=82.00&il2pi4br=Pigzart&il2pi4ca=T-Shirts&il2pi4ps=3&z=819126443

      Generated Object Pushes

      {
          "event": "ga_event",
          "eventData": {
              "category": "engagement",
              "action": "view_item_list",
              "nonInt": false
          }
      }
                                      
      {
          "event": "view_item_list",
          "items": [
              {
                  "item_list_name": "shirts you may like",
                  "item_id": "6c3b0",
                  "item_name": "Zappix T-Shirt",
                  "price": "99.00",
                  "item_brand": "Zappix",
                  "item_category": "T-Shirts"
              },
              {
                  "item_list_name": "shirts you may like",
                  "item_id": "8835a",
                  "item_name": "Isoternia T-Shirt",
                  "price": "57.00",
                  "item_brand": "Isoternia",
                  "item_category": "T-Shirts",
                  "index": "1"
              },
              {
                  "item_list_name": "shirts you may like",
                  "item_id": "035f0",
                  "item_name": "Futuris T-Shirt",
                  "price": "55.00",
                  "item_brand": "Futuris",
                  "item_category": "T-Shirts",
                  "index": "2"
              },
              {
                  "item_list_name": "shirts you may like",
                  "item_id": "239b5",
                  "item_name": "Pigzart T-Shirt",
                  "price": "82.00",
                  "item_brand": "Pigzart",
                  "item_category": "T-Shirts",
                  "index": "3"
              },
              {
                  "item_list_name": "homepage",
                  "item_id": "035f0",
                  "item_name": "Futuris T-Shirt",
                  "price": "55.00",
                  "item_brand": "Futuris",
                  "item_category": "T-Shirts",
                  "index": "4"
              },
              {
                  "item_list_name": "homepage",
                  "item_id": "8835a",
                  "item_name": "Isoternia T-Shirt",
                  "price": "57.00",
                  "item_brand": "Isoternia",
                  "item_category": "T-Shirts",
                  "index": "5"
              },
              {
                  "item_list_name": "homepage",
                  "item_id": "57b9d",
                  "item_name": "Kiosk T-Shirt",
                  "price": "55.00",
                  "item_brand": "Kiosk",
                  "item_category": "T-Shirts",
                  "index": "6"
              },
              {
                  "item_list_name": "homepage",
                  "item_id": "dc646",
                  "item_name": "Lunchpod T-Shirt",
                  "price": "90.00",
                  "item_brand": "Lunchpod",
                  "item_category": "T-Shirts",
                  "index": "7"
              },
              {
                  "item_list_name": "homepage",
                  "item_id": "7w9e0",
                  "item_name": "Masons T-Shirt",
                  "price": "31.00",
                  "item_brand": "Masons",
                  "item_category": "T-Shirts",
                  "index": "8"
              },
              {
                  "item_list_name": "homepage",
                  "item_id": "239b5",
                  "item_name": "Pigzart T-Shirt",
                  "price": "82.00",
                  "item_brand": "Pigzart",
                  "item_category": "T-Shirts",
                  "index": "9"
              },
              {
                  "item_list_name": "homepage",
                  "item_id": "6d9b0",
                  "item_name": "Poyo T-Shirt",
                  "price": "62.00",
                  "item_brand": "Poyo",
                  "item_category": "T-Shirts",
                  "index": "10"
              },
              {
                  "item_list_name": "homepage",
                  "item_id": "6c3b0",
                  "item_name": "Zappix T-Shirt",
                  "price": "99.00",
                  "item_brand": "Zappix",
                  "item_category": "T-Shirts",
                  "index": "11"
              }
          ]
      }
                                      

      CustomTask Setup

      As usual, we will need to set up a customTask variable for our tags. I'm attaching an example for Google Tag Manager

      function() {
          return function(customTaskModel) {
              var originalSendHitTask = customTaskModel.get('sendHitTask');
              customTaskModel.set('sendHitTask', function(model) {
                  try {
                      var parserConfig = {
                          tms: 'gtm',
                          eventsName: 'ga_event',
                          ecommerceEventsEnabled: true,
                          skipTransportEvent: false,
                          mapping: {
                              keys: {
                                  cd1: 'client_id',
                                  cd3: 'logged_in',
                                  cd7: 'is_in_stock',
                                  cd19: 'referrer',
                                  cd20: 'full_url',
                                  // Metrics
                                  cm201: 'votes',
                                  // content Grouping
                                  cg1: 'content_group_1',
                                  cg3: 'page_type'
                              },
                              events: {
                                  'checkout': {
                                      '1': 'view_cart',
                                      '2': 'begin_checkout'
                                  }
                              }
                          }
                      };
        
                     [[ADD CODE FROM GITHUB]]
      
                      uaPayloadParser(parserConfig, model.get('hitPayload'));
                      originalSendHitTask(model);
                  } catch (e) {
                      originalSendHitTask(model);
                  }
              });
          }
      }
                                      

      In order to have updated content, please replace the [[ADD CODE FROM GITHUB]] with the code from the build/ folder in the GitHub project.

      https://github.com/thyngster/universal-analytics-payload-parser/blob/main/build/uaPayloadParser.min.js

      Recap

      So, that's all, as you can see rather than sending data to Google Analytics 4, this library allows sending the data to the dataLayer ( or anywhere else ) instead so you can work with the data in any way you want, you could define different event names, you could sanitize the values, you can block/skip the events you don't need.

      The best point is that you'll have full control over how the data ends in your Google Analytics 4 account, and this will work even if you're using GTAG, or even if you have a hardcoded implementation, just adding the custom task in your main snippet should allow you to have the data anywhere else.

      GITHUB PROJECT LINK: https://github.com/thyngster/universal-analytics-payload-parser