Ask DN: Analytics software for UI?

1 month ago from , UI Designer @ hyperion.co

I was recently looking into product analytics software for a company project I am working on and I have found most of the ones I looked at (Mixpanel, Hotjar, FullStory), to be geared towards marketing / ecommerce, not product UI.

For example with neither Mixpanel or FullStory can you track the time between two actions for a single session. You can just see how many people drop out of a 'funnel' and the average time between steps for all of them.

Hotjar can't even be implemented on apps that don't use links to differentiate between pages.

Is there something better? Am I looking in the wrong place or asking the wrong questions?

Thanks.

14 comments

  • Jake FlemingJake Fleming, 1 month ago

    What are your goals for event tracking? What types of events are you wanting to track that these tools are failing at?

    2 points
    • Cristian Moisei, 1 month ago

      I already explained this, it's not that they cannot track specific events, but rather that they only track them once.

      Say I'd like to know in a process with 5 steps (A - B - C - D - E) how much time is spent between them, so that I can learn if for example the time between C and D is longer than usual, that there could be a problem there.

      Mixpanel and fullstory however will only record the time of the first session. This means if a user goes theough these steps 20 times a day for the entire month (as is the case of software), mixpanel can only tell me that one user went through these steps (now how many times), and that the first time he took x seconds between each steps (not am average of all sessions).

      1 point
      • Jake FlemingJake Fleming, 1 month ago

        Checkout Heap. https://heapanalytics.com/features/users

        The reason I asked what your goal is because I'm curious what you'd be using that information for. If the time between C and D took longer than normal for this specific user, what action are you going to take? And how will you know that person isn't just talking with a coworker or taking a shit this time around?

        Anyway not trying to be a troll. Good luck.

        2 points
        • Cristian Moisei, 1 month ago

          No, those are valid points. I would look at the average time between two points which should normalise the results, and if I notice a problem look into it: see if there is any feedback about it with our support team and A/B test alternatives to see which performs better.

          2 points
          • Jake FlemingJake Fleming, 1 month ago

            Cool! Would be interested to hear what you end up using and what sort of insights you were able to arrive at in the future.

            1 point
            • Cristian Moisei, 1 month ago

              I'm looking at using Amplitude at the moment - I'll shoot you a message about what I learn later on.

              2 points
      • Clay WClay W, 1 month ago

        This is pretty straight forward with vanilla javascript (or jquery).

        0 points
  • James FutheyJames Futhey, 1 month ago

    (I worked for a company which made an Analytics product & have worked as a data science consultant).

    What you're talking about is almost too advanced to fall under the category of what people today call “analytics”. Measuring timing, frequency, and recency is something that I've really only seen done by “Data Scientists” or companies with more advanced data teams (usually professionally-trained analysts. These tools are designed for the marketer or casual user).

    Typically, the tools these teams would use would pull events into a SQL database (usually Redshift), and then they would write a few very simple SQL queries to get the information you desire.

    If you want to try your hand at this using commodity tools on the market, I would recommend signing up for Segment.com (similar implementation flow as Mixpanel), and then exporting the data to a “data warehouse” (they give you one for free). It doesn't have to be Redshift or a real data warehouse, whatever SQL database your company already uses (and knows how to maintain) will do. Any developer on your team can help you write the kind of queries you need to answer the simplest questions you have about timing and frequency.

    There are a ton of tools out there for companies looking to set up a “real” data warehouse and a real data analysis project (the above flow is not what you want). But, bottom line, don't worry about this unless you're looking to hire a data scientist / build a data team.

    1 point
    • Cristian Moisei, 1 month ago

      Thank you James, this is great to know and I appreciate your insider view. We do not however have the resources for anything that advanced - is a custom solution feeding into Segment what the larger companies are using?

      And lastly, would there be another way (other than tracking time betwern steps) to figure out how efficient an interface element is?

      Thanks again.

      0 points
      • James FutheyJames Futhey, 1 month ago

        Segment is the free -> cheap way to get data into SQL (or pretty much any database you have) by writing tracking code pretty much identical to Mixpanel. A lot of companies are using it, and once you get data into SQL there's no limit to what you can do with it. It's not really as sophisticated as what the bigger companies are using to build data warehouses, but it's the exact same methodology, and probably better for your use-case.

        I think the thesis of your analysis is correct. For a short interaction in a (mostly) linear interaction path, if you derive the mean time between each pair of sequential steps over a large enough cohort, “cognitive load” for each step should be roughly correlative to “mean time on task”. You might also be able to find similar answers using eye tracking, but it certainly won't be easier (and your method using a larger sample should be less noisy).

        Without SQL, I'm not sure I know of a product that is going to get you this information. Analytics is usually descriptive data, and something like mean time between steps is derived data. I might be overlooking something (there are literally thousands of companies in this space), but based on recent usage of the market leaders, they don't really offer features like these (most of their users aren't doing things sophisticated enough to request them, and almost all advanced users want to “own” their data, which usually means SQL).

        0 points
  • nurkan kirkan, 1 month ago

    Hey Christian! Great question indeed. We have a product which calls Rakam. It is a full-stack analytics platform that allows companies to create analytics services based on their needs. Rakam is more like a data lake solution for all type of companies, it can collect real-time event data from websites, mobile platforms, third-party services and even remote databases and store it in a data lake so that companies can create custom reports and dashboards for their needs. It's on-premise (github.com/rakam-io/rakam) and open-source (github.com/rakam-io/rakam), it's modular and transparent so that companies can integrate their custom ETL rules, integrate their custom data warehouse and customize their reports. Basically all your data in one place and customize it based on your needs. Because while your business growth, however, your analytics solution wouldn't. It fills the gap between Analytics-as-a-service and in-house analytics stacks. The main points are i do believe either from both sides: Full control over your data and Cost-effective.

    let me know if you have any questions on your mind regards rakam. check our website: http://rakam.io

    best

    0 points
  • Alex PriceAlex Price, 1 month ago

    Going through the same process right now, found exactly the same shortcomings as you. I think we're going to go with Fullstory because it's about as close as you can get (with the playback feature they have).

    Another option, just to solve the "time between two actions" could be to use something like Keen IO, and just diff timestamps. ¯_(ツ)_/¯

    Depends on how large your development team is, but I know we'll continue to look for a more product-oriented analytics tool.

    0 points
  • V. MV. M, 1 month ago

    Hey Christian, are you looking for how much time user spent between actions and where they dropped off in between? You should check out user testing in CanvasFlip - https://www.canvasflip.com/user-testing.php

    Includes conversion analysis, heatmap, user videos and more. Works in web browser, no plugin or app needed. Let me know your thoughts.

    0 points