For our November 2024 event, we brought Josh Silverbauer in from Philly to talk about behavior analytics (in the form of MS Clarity) and marketing analytics (in the form of Piwik PRO) and when you might want to use each one.

Since Josh is well-known for writing parody songs to introduce speakers, here at CBUSDAW we flipped the script on him and opened the event with a surprise parody song about Josh.

We present “It’s Josh Silverbauer” to the tune of “In the Midnight Hour” by Wilson Pickett, sung by Jim Gianoglio and featuring Jason Packer on kazoo.

While Josh is a fan of both Clarity and Piwik PRO, he’s pointed out that he’s not paid anything by either organization… so he’s free to tell it like it is. And “how it is” is that both tools are great additions to any analyst’s arsenal — and with a generous free tier for Piwik PRO and a totally free product with Clarity there’s not much barrier to entry.

Josh pointed out how the two tools can easily used to supplement each other. For example, one could use Piwik PRO to find a particular aggregate group of users that aren’t converting well, and then review those users’ entire sessions with the session recording feature in Clarity. Or check the heatmap of the landing page for that same group.

If you’ve used session recording tools in the past you know that it can be pretty tedious to watch the recordings one-by-one. It’s like, “geez, just click the button already user #23341, it’s RIGHT THERE”.

Microsoft has recently integrated CoPilot into Clarity so it can now help save you from watching a ton of videos and instead can summarize and do some basic analysis for you.

Josh described Piwik PRO as “what Google’s Universal Analytics 2.0 could have been if GA4 didn’t exist”. If you were a serious user of our dearly departed UA you’ll feel right at home in Piwik PRO, and you’ll be pleased to see how well thought-out the platform is.

Josh’s slides:

Ok, so Clarity and Piwik PRO are both pretty cool tools, but what about rock operas?

Josh (and Jason) are releasing Volume 2 of Josh’s epic analytics rock opera entitled “User Journey Volume 2: The Custom Dimension” on November 18th.

You can listen to the first volume, “Universal Sunset” now on Spotify and most other streaming platforms.

Finally, don’t forget to join us next month for our yearly holiday event at the Grandview Theater. No speakers, but there will be a movie this year and you can vote on what you’d like it to be!

Disclaimer: Piwik PRO is a sponsor of CBUSDAW — but they only pay for the monthly pizza, not our (or Josh’s) endorsements.

As analysts, we love to optimize everything that we possibly can — so when we have a speaker that gives us a new way to think about testing we are here for it!

For our October event we had Sanjay Tamrakar talk to us about doing geo-testing. Sanjay covered basic methods like traditional pilot testing, to difference in differences, all the way to the current state-of-the-art with Geolift.

Back in the pre-digital era, Columbus was considered to be a top location in the US to pilot test new products, since its demographics closely matched the country as a whole. This allowed companies to try out new products, but only gave marketers an idea of how well a new product might do nationwide, not how much incremental lift different product variations might give or how different Columbus might perform vs. Chicago or Charleston. This kind of granularity requires more powerful tools.

These days there are much more expedient and statistically rigorous methods to test things, like Geolift. Geolift is a an open-source package from Meta that allows the creation of artificial control groups which we can use to test against treatment groups without having to worry as much about building control groups using user-specific information and the privacy issues which that can bring. Geolift’s synthetic control methods can create control groups by amalgamating different untreated areas whose performance was expected to match the treated areas.

There was even some R code showing up on the big screen, which sadly Tim Wilson missed!

Sanjay was also kind enough to provide us with his slides:

Fresh from another successful DataConnect Conference, Lauren Burke-McCarthy led our July session of Data & Analytics Wednesday talking about how to survive and succeed as a solo practitioner of data science.

Being a “solo practitioner” could mean being the only data scientist on your team, being siloed in some way, or even being a freelance contractor. The strategies that Lauren presented were focused on how to best communicate and set expectations with stakeholders. We’ve all been there when a project has gone off the rails because what a practitioner implemented didn’t match at all what a stakeholder had envisioned. Let’s nip these misalignments in the bud as best we can before they can blossom into fully grown issues.

In fact it turns out many (perhaps most!) of these techniques could work for us in any data-related role we were in. What after all even is a data scientist? Lauren also took a crack at answering that age-old question off the top of her head. To paraphrase her answer, a Data Scientist focuses on models and experiments to make future-looking prediction — vs a Data Analyst works on analysis of current and historical data to identify trends and develop insights. If those two things seem to blur into each other at times, that just shows how Lauren’s advice on processes and communication works for both! Perhaps even those of us who have now added “AI” in our job titles? Could well be…

Looking to learn more about these techniques? Lauren was kind enough to provide us with our slides so you can take a look for yourself:

And, of course, pictures!

Please join us next month when the ever-delightful Matt Gershoff will be in town to discuss how to think purposely about data as we move towards privacy by design.

Our June 2024 meetup featured Dr. Maria Copot from OSU delving into some of the underlying theories behind our favorite A/B testing platforms. Though before we get into the fun math part (yes, it’s fun, don’t look at me like that) — we need to all remember that there needs to be a question behind your experiment. If you don’t have a hypothesis you’re trying to validate, then what’s the point of testing something? Once you’ve got something you want to test, then you can test it, but testing just for the sake of saying how many A/B tests your department ran last year isn’t going to get you where you want to be.

A lot of us have been asked, “is this result statistically significant?” And maybe we’ve even said, “well, the P-value is <0.05 so it’s significant”… But what exactly is a P-value and why is 0.05 the number a big deal? Dr. Copot explained the basics of P-values, including that 0.05 is an arbitrary benchmark, and that it can’t tell you anything about the size of an effect, its validity, or reason behind it. If that still sounds a bit confusing, it’s time to queue the memes about scientists being unable to explain P-values in an intuitive way. We think Dr. Copot’s explanation would be in the top quantile of that distribution at any rate. Even if math is fun, it isn’t always intuitive.

Dr. Copot also talked about sample sizes and power analysis (one such online calculator I’ve used many times here: https://www.evanmiller.org/ab-testing/sample-size.html), but then moved on to talking about Bayesian methods. Traditional A/B tools (like Google Optimize, RIP) have typically used Frequentists methods like we’ve been talking about with P-values. Newer tools have folded in some Bayesian methods, which thankfully are a little more intuitive, if perhaps more mathematically & computationally expensive.

Finally, we talked about how privacy regulations, sampling, and cookie limitations can make doing these kinds of experiments more difficult. One way around these limitations is to use paid platforms like Prolific where you can make your own sample group and run a group of fully consented users through an experiment of your choosing.

Please join us next month when Lauren Burke-McCarthy will talk about how to succeed as a solo data scientist.

Our March event featured Bill Balderaz from Futurety getting us up-to-date on the impending demise of third party cookies. As most of us have heard, third party cookies in Chrome are scheduled to be turned off later this year. What we’re less sure about, is what the heck is coming in to replace these cookies?

Love them or hate them, third party cookies have been a fundamental building block of the $600B+ digital ads ecosystem for years — and this turndown will be one of the biggest changes in the history of of the industry. While browsers such as Safari and Firefox have blocked third party cookies for years, it’s Chrome’s impending Q3 2024 turndown of these cookies that has caught the industry’s attention due to Chrome’s dominant marketshare (64%).

So what comes next? Surely we’re not going back to completely un-targeted “punch the monkey” style ads… One answer is first party data, where instead of relying on a third party cookies tracking our cross-site activity, we rely upon our own customer data. What does that mean in practical terms?

Bill gave us an example of how first party data can be used to build audiences with Futurety’s own product HUCKLE.

Per usual, the CBUSDAW audience had plenty of good questions. As we move towards this (third-party) cookieless future, we’re all  still collectively figuring out what exactly this means and how to balance user privacy with effective marketing solutions.

And, action shots from the event. The main author of this recap neglected to note that he had some fun with the event by doing a LinkedIn poll as a competition between local (real) cookie purveyors. Lion Cub’s won the poll and, as a result, were provided at the event, along with a “cookie consent” form. The event photographer had both a picture of that consent form and edit access to this recap, so he snuck this paragraph in when he added the photo gallery below!

Please join us next month when we have a panel of AI aficionados on tap for an audience-driven talk on the future of data science and AI.

Lots of analysts (the author of this recap included) once believed that multi-touch attribution was the long-awaited answer to John Wanamaker’s famous question about which half of our ad spend was wasted. Except even before we started losing lots of data, there were some serious problems with MTA that we’d been turning a blind eye to. In this new era of cookie restrictions and data privacy regulations, MTA has become even more problematic.

Enter MMM. Jim Gianoglio, founder of Cauzle Analytics, gave us in introduction to marketing mix modeling (MMM) — another way to look at answering this same eternal marketing question about what channels are and aren’t working. MMM has been around for a long time (it was so popular in the ’90s that it was immortalized in multiple songs), but the methods and practices have matured and it can really be useful for a high-level understanding of performance for many organizations.

Using MMM doesn’t mean you have to stop using MTA either! MTA can still give you information on a more granular level, and then MMM can help with overall strategy.

If you’re looking to learn more about MMM, check out Jim’s mailing list, podcast, and Slack at mmmhub.org.

https://www.slideshare.net/JasonPacker/cbusdaw-october-23-marketing-mix-modeling

Google’s search results pages (SERPs) are the lifeblood of site traffic for most companies. Organic plus paid search provides 68% of site traffic on average (with organic being the large majority of that), but Google (and Bing’s) experiments with included generative AI-powered search results may change things up!

Lincoln Rinehart, Sr. Director of SEO at Adept Marketing, helped provide some reassurance to those of us worried about how Google’s inclusion of AI into these SERPs might affect our traffic.

Even though Bing beat Google to the punch on this, and Google seriously flubbed their initial response, Google still dominates search engine market share. Any changes Google makes to their search experience can change the kind of traffic your site receives in a big way.

This kind of change seems pretty concerning, but Lincoln pointed out a number of ways in which it could be good for SEO and good for your site (even if you get less traffic).

• Changes the focus of SEO away from trying to “game the system” and towards how to benefit the users that do come to your site.
• SEO changes from a giant checklist approach to a holistic cross-disciplinary practice.
• Could decrease the amount of junky content created solely for SEO.
• Traffic that does come will be much more qualified.

Additionally, Google says AI-generated content is ok, as long as it’s not made explicitly for the purpose of manipulating rankings.

Lincoln also was careful to point out that this change may not even happen! This is still very much an experiment on Google’s side and they have some significant issues to work out before they decide if (and how) they might release this more widely.

Lincoln has also provided his slides! We promised not to dump all of the text out of the slides as white-on-white text at the bottom of the page.

And, just in case Google decides the presence of images makes a site more authoritative, we’ve got pics, too:

April’s meetup saw Jeewan Singh and Tomy Rhymond of Slalom Columbus give us an introduction into the concept of DataOps.

We all deal with data on a regular basis, though we usually are focused on only a small part of the whole lifecycle process of that data. Thinking more broadly about the whole journey of data, from ingest to insight, requires broadening our horizons past the details of analytical tactics towards a more holistic approach.

How do we better deal with managing that whole lifecycle? That’s where we need to change our mindsets and move towards the kind of agile processes that define DataOps.

The traditional way of dealing with data starts from a fundamental assumption that the process and business requirements are static. A naive idea that once we figure out how to ETL the data and get the business the reporting it needs, that we’re done. Of course we all know that’s not how things work in the real world, thus we need to have methods to deal with this constant change and evolution. We’re still allowed to complain when the business stakeholders change the requirements just when we’re almost done with the project (that’s a basic human right), but perhaps if we’ve built more flexibility into our process from the start that won’t mess us up quite as much.

Automation, continuous integration and delivery, test cases right from the start, infrastructure as code rather than clickops — these are the kinds of things that are at the heart of DataOps. If you’ve ever worked within DevOps you’ll find the approach to be similar, but tweaked and extended for the specific needs of the data world.

In case you missed it, Tomy and Jeewan have also generously provided their slide deck:

https://www.slideshare.net/JasonPacker/dataops-cbusdaw-april-23

Tony Zara, founder of ecommerce advertising agency Iron Pulley, gave us a glimpse into the programmatic advertising ecosystem and how one might succeed as an advertiser within this system. Tony pointed out that of the many ways to target potential customers, contextual targeting is one that is both least affected by privacy restrictions and least subject to fraud.

So what is contextual advertising? Fundamentally it’s a simple concept: showing ads that are relevant to the page they appear on. This classic method has been losing out to user-based targeting for several reasons, but one dysfunctional reason is that contextual doesn’t maximize revenue for the ad networks as well as user-based targeting does.

When we talk about display ad platforms, the 800 pound gorilla in the room is of course Google. In order to place a contextual ad, you need to have an ad tag on a website that is topically related to what you want to advertise for. Google currently controls 90%+ of this inventory, so if they want to make it hard to do contextual advertising they can (and have). Tony recommended reading the DOJ’s complaint against Google for a good overview of the situation.

So how does one do contextual advertising? Tony’s method is to find the sites that are relevant through human research. Advanced Google searches, SEM/SEO tools like Ahrefs, industry knowledge, etc. Finding the legitimate high-quality sites and doing placements on those sites rather than letting algorithms do “optimization” for you.

Tony has generously provided his slides:

And, some pictures from the event:

Please join us in March when we will have Lea Pica talking about Data Storytelling Secrets!

After having several presentations on GA4 migration, the haters finally got a night of their own.

Except Jason Packer, our presenter of these GA alternatives (and author of this post), actually thinks GA4 is pretty good. Jason encouraged everyone to primarily not see GA4 as the default choice, but one possible choice in a world of many different options.

And boy are there a lot of options. We took a list of 120+ and narrowed it down to about 15 products based upon existing installation base and talked about some of these products strengths and weaknesses. Then we talked about how to frame up some questions that can help us pick a few different options to evaluate.

Then we did a live consultation with a couple of volunteers to try and narrow down what options could make sense for their particular needs.

Find info on Jason’s book on Google Analytics Alternatives here.